ChatGPT has taken the world by storm. Within two months of its release it reached 100 million active users, making it the fastest-growing consumer application ever launched. Users are attracted to the tool’s advanced capabilities – and concerned by its potential to cause disruption in various sectors.

A much less discussed implication is the privacy risks ChatGPT poses to each and every one of us. Just yesterday, Google unveiled its own conversational AI called Bard, and others will surely follow. Technology companies working on AI have well and truly entered an arms race.

The problem is it’s fuelled by our personal data.

300 billion words. How many are yours?

ChatGPT is underpinned by a large language model that requires massive amounts of data to function and improve. The more data the model is trained on, the better it gets at detecting patterns, anticipating what will come next and generating plausible text.

OpenAI, the company behind ChatGPT, fed the tool some 300 billion words systematically scraped from the internet: books, articles, websites and posts – including personal information obtained without consent.

If you’ve ever written a blog post or product review, or commented on an article online, there’s a good chance this information was consumed by ChatGPT.

So why is that an issue?

The data collection used to train ChatGPT is problematic for several reasons.

First, none of us were asked whether OpenAI could use our data. This is a clear violation of privacy, especially when data are sensitive and can be used to identify us, our family members, or our location.

Even when data are publicly available their use can breach what we call textual integrity. This is a fundamental principle in legal discussions of privacy. It requires that individuals’ information is not revealed outside of the context in which it was originally produced.

Also, OpenAI offers no procedures for individuals to check whether the company stores their personal information, or to request it be deleted. This is a guaranteed right in accordance with the European General Data Protection Regulation (GDPR) – although it’s still under debate whether ChatGPT is compliant with GDPR requirements.

This “right to be forgotten” is particularly important in cases where the information is inaccurate or misleading, which seems to be a regular occurrence with ChatGPT.

Moreover, the scraped data ChatGPT was trained on can be proprietary or copyrighted. For instance, when I prompted it, the tool produced the first few paragraphs of Peter Carey’s novel “True History of the Kelly Gang” – a copyrighted text.

Finally, OpenAI did not pay for the data it scraped from the internet. The individuals, website owners and companies that produced it were not compensated. This is particularly noteworthy considering OpenAI was recently valued at US$29 billion, more than double its value in 2021.

OpenAI has also just announced ChatGPT Plus, a paid subscription plan that will offer customers ongoing access to the tool, faster response times and priority access to new features. This plan will contribute to expected revenue of $1 billion by 2024.

None of this would have been possible without data – our data – collected and used without our permission.

A flimsy privacy policy

Another privacy risk involves the data provided to ChatGPT in the form of user prompts. When we ask the tool to answer questions or perform tasks, we may inadvertently hand over sensitive information and put it in the public domain.

For instance, an attorney may prompt the tool to review a draft divorce agreement, or a programmer may ask it to check a piece of code. The agreement and code, in addition to the outputted essays, are now part of ChatGPT’s database. This means they can be used to further train the tool, and be included in responses to other people’s prompts.

Beyond this, OpenAI gathers a broad scope of other user information. According to the company’s privacy policy, it collects users’ IP address, browser type and settings, and data on users’ interactions with the site – including the type of content users engage with, features they use and actions they take.

It also collects information about users’ browsing activities over time and across websites. Alarmingly, OpenAI states it may share users’ personal information with unspecified third parties, without informing them, to meet their business objectives.

Time to rein it in?

Some experts believe ChatGPT is a tipping point for AI – a realisation of technological development that can revolutionise the way we work, learn, write and even think. Its potential benefits notwithstanding, we must remember OpenAI is a private, for-profit company whose interests and commercial imperatives do not necessarily align with greater societal needs.

The privacy risks that come attached to ChatGPT should sound a warning. And as consumers of a growing number of AI technologies, we should be extremely careful about what information we share with such tools.

The Conversation reached out to OpenAI for comment, but they didn’t respond by deadline.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The recent questioning of the heads of Amazon, Facebook, Google and Apple in the US Congress has highlighted the threat their practices pose to our privacy and democracy.

However these big four companies are only part of a vast, sophisticated system of mass surveillance.

In this network are thousands of data brokers, ad agencies and technology companies – some of them Australian. They harvest data from millions of people, often without their explicit consent or knowledge.

Currently, this includes data related to the COVID-19 pandemic. For instance, data giant Palantir has provided lab test results and emergency department statuses to the US Centres for Disease Control and Prevention.

How much do they know?

Data companies gather data about our online activity, location, DNA, health and even how we use our mouse. They use a range of techniques, such as:

• web-trackers planted on almost every page on the internet, which follow our browsing activity
• “smart” home devices leaking details of our usage habits and location
• millions of mobile apps sending our data to unknown third parties, including sensitive information such as when we last had sex
• millions of retailers tracking our purchasing habits and in-store movements.

This expansive tracking generates billions of data points that can reveal every facet of our lives including our family status, income, political affiliation, interests, friendships and sexual orientation.

Data companies use this information to compile detailed individual consumer profiles. These are used for purposes such as targeting us with ads, determining our eligibility for loans and assessing the riskiness of our lives.

The data industry in Australia

Some of the world’s largest data companies operate in Australia. Quantium is an Australian data analytics firm that acquires data from various partners including NAB, Qantas, Woolworths (which owns 50% of the company) and Foxtel.

These partnerships allow Quantium to “tap into the consumer data ecosystem with an unrivalled picture of the behaviours of more than 80% of Australian households, spanning banking, household and retail transactions”.

A company spokesperson told The Conversation most of its work is “data science and AI (artificial intelligence) work with first-party de-identified data supplied by the client”. From this, Quantium delivers “insights and AI/decision support tools” for clients.

Anonymised or “de-identified” data can still be accurately re-identified. Even if a person’s details are de-identified by being converted to an alphanumeric code, the conversion method is identical across most companies.

Therefore, each code is unique to an individual and can be used to identify them within the digital data ecosystem.

A lack of transparency

With a revenue of more than US$110 million last year, the insights from Quantium’s data seem to be proving valuable.

From this revenue, more than A$61 million between 2012 and 2020 came from projects commissioned by the Australian government. This includes two 2020 engagements:

• a “COVID-19 Data Analytics” project worth more than A$10 million with a contract period from March 17, 2020 to December 31, 2020

• a “Quantium Health Data Analytics” project valued at more than A$7.4 million with a contract period from July 1, 2020 to June 30, 2021.

Quantium’s spokesperson said they could not discuss the details of the contracts without government approval.

In the past decade, the Australian government has commissioned dozens of projects to other data analytics firms worth more than A$200 million.

These include a A$13.8 million Debt Recovery Service project with Dun & Bradstreet and a A$3.3 million National Police Checks project with Equifax – both started in 2016. It’s unclear what and how much data has been shared for these projects.

Last year, Quantium was one of several larger companies put on notice by Australia’s consumer watchdog for sharing data with third parties without consumers’ knowledge or consent.

How do they work?

Data companies largely operate in the shadows. We rarely know who has collected information about us, how they use it, who they give it to, whether it’s correct, or how much money is being made from it.

LiveRamp (formerly Acxiom) is a US-based company partnered with Australia’s Nine Entertainment Co. This partnership allows the Nine Network to give marketers access to online and offline data to target consumers across Nine’s digital network.

This data may include the Australian electoral roll, to which LiveRamp gained access last year.

Similarly, Optum is a US-based health data company that collects information from hospital records, electronic health records and insurance claims.

It has data on more than 216 million people and used this to develop a predictive algorithm that was shown to discriminate against black patients.

Compromising our democracy

The prevalence, scope and stealth of the abovementioned data practices are not congruent with the basic principles of a liberal democracy.

According to philosopher Isaiah Berlin, liberal democracies can only thrive if they have autonomous citizens with two types of freedoms:

1. freedom to freely speak, choose and protest
2. freedom from undue inspection and intervention.

Our data-driven world signals an extreme diminishing of both these freedoms. Our freedom of choice is harmed when our informational environments are doctored to nudge us towards behaviours that benefit other parties.

Our private space is all but gone in a digital environment where everything we do is recorded, processed and used by commercial and governmental entities.

How can we protect ourselves?

Although our ability to disconnect from the digital world and control our data is eroding rapidly, there are still steps we can take to protect our privacy.

We should focus on implementing legislation to protect our civil liberties. The Australian Consumer Data Right and Privacy Act stop short of ensuring the appropriate data protections. The Australian Competition and Consumer Commission highlighted this in its 2019 report.

In 2014, the US Federal Trade Commission recommended legislation to allow consumers to identify which brokers have data about them – and that they be able to access it.

It also recommended:

• brokers be required to reveal their data sources
• retailers disclose to consumers that they share their data with brokers
• consumers be allowed to opt out.

If we care about our freedoms, we should try to ensure similar legislation is introduced in Australia.

This article is republished from The Conversation under a Creative Commons license. Read the original article.