Concerns over AI Chat Privacy

Voice of the DBA

One of the major concerns for using GenAI tools is who is reading the data you submit as a prompt, and will this data be used in future training of the model? In other words, could someone using a future model access the data I put in a GenAI chat?

It’s a valid concern, and not just because of the vendors. There is a lawsuit over the use of data by OpenAI, and a court has ordered all chats to be retained, including deleted ones. Since this is a lawsuit, there is always a chance that some of the data retained gets entered into a court document or even that it might be read aloud in court and captured in a transcript.

This is a thorny data privacy issue that collides with the need for courts to maintain evidence for legal proceedings. I honestly don’t know what the answer here is, but I think this should be handled similarly to how code and other IP/proprietary items are handled as evidence. The challenge here is that there can be a lot of evidence, and I am not sure many legal organizations are really set up to handle and manage this much data.

I’m also not sure who I think should pay for this. If two parties engage in a lawsuit, but the data is actually held in custody by a third party. Do we need to have some nominal charge rate for keeping data that some need to prove their case? Is there a need to force companies like OpenAI and others to hold data in chats for a certain period of time? Something like the 7 or 10 years that a lot of governments require for tax records?

Our legal systems are outdated and ill-equipped to handle many ways in which the digital world differs from the analogue one. On one hand, I wish that lawmakers would work with advocates and technologists to update laws to make them more relevant, or perhaps, only applicable to digital data.

On the other, I worry about adding new laws that create overhead, perhaps stifle innovation, or will be out of date as soon as they are passed.

In 2025, we all should know that anything we send across the internet to another person or service could potentially be disclosed and abused. If that really bothers you, then beware of all free services and be cautious of the paid ones. Read the EULAs and decide if you can accept the risk.

Because once you send it, your data is out of your control.

Steve Jones

Listen to the podcast at Libsyn, Spotify, or iTunes.

Note, podcasts are only available for a limited time online.