Data Privacy Issues With AI

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

By next year we will be producing as much data every 15 minutes as all of human civilisation did up to the year 2003. Data might be the new oil, but it’s unrefined. AI companies are the new oil refineries. Many companies are quietly changing their Terms and Privacy Policies to allow them use this data for machine learning, and the FTC weighed in on this in a blog post last week. This suggests that organisations reviewing their policies and documentation when it comes to AI and data protection in particular, and more broadly - T&Cs and contracts, need to be mindful about how AI is addressed. In their recent blog on the subject, the FTC says: “It may be unfair or deceptive for a company to adopt more permissive data practices—for example, to start sharing consumers’ data with third parties or using that data for AI training—and to only inform consumers of this change through a surreptitious, retroactive amendment to its terms of service or privacy policy.” The temptation for companies to unilaterally amend their privacy policies for broader data utilisation is palpable, driven by the dual forces of business incentive and technological evolution. However, such surreptitious alterations, aimed at circumventing user backlash, tread dangerously close to legal and ethical boundaries. We have already seen major companies fall foul of consumer backlash when they attempted to change their terms along these lines. Historically, the FTC in the US has taken a firm stance against what they deem deceptive practices. Cases like Gateway Learning Corporation and a notable genetic testing company underscore the legal repercussions that await businesses reneging on their privacy commitments. These precedents serve as a stark reminder of the legal imperatives that bind companies to their original user agreements. The EU context is also worth considering. The GDPR's implications for AI and technology companies are significant, particularly in its requirements for transparent data processing, the necessity of informed consent, and the rights of data subjects to object to data processing. For companies, this means navigating a labyrinth of legal obligations that mandate not only the protection of user data but also ensure that any changes to privacy policies are communicated clearly. The intersection of GDPR with the FTC's stance on privacy policy amendments seems to highlight a consensus on the importance of data protection and the rights of consumers in the digital marketplace. This synergy between the U.S. and EU approach creates a formidable legal landscape that AI companies must navigate with caution and respect for user privacy. The path forward for AI companies is clear: transparency is a key element in AI Governance upon which AI and data policies are built. It is arguably the most important element in the AI Act, and it is emerging as a key component in global legislation as jurisdications develop their own AI regulations.

This new white paper by Stanford Institute for Human-Centered Artificial Intelligence (HAI) titled "Rethinking Privacy in the AI Era" addresses the intersection of data privacy and AI development, highlighting the challenges and proposing solutions for mitigating privacy risks. It outlines the current data protection landscape, including the Fair Information Practice Principles, GDPR, and U.S. state privacy laws, and discusses the distinction and regulatory implications between predictive and generative AI. The paper argues that AI's reliance on extensive data collection presents unique privacy risks at both individual and societal levels, noting that existing laws are inadequate for the emerging challenges posed by AI systems, because they don't fully tackle the shortcomings of the Fair Information Practice Principles (FIPs) framework or concentrate adequately on the comprehensive data governance measures necessary for regulating data used in AI development. According to the paper, FIPs are outdated and not well-suited for modern data and AI complexities, because: - They do not address the power imbalance between data collectors and individuals. - FIPs fail to enforce data minimization and purpose limitation effectively. - The framework places too much responsibility on individuals for privacy management. - Allows for data collection by default, putting the onus on individuals to opt out. - Focuses on procedural rather than substantive protections. - Struggles with the concepts of consent and legitimate interest, complicating privacy management. It emphasizes the need for new regulatory approaches that go beyond current privacy legislation to effectively manage the risks associated with AI-driven data acquisition and processing. The paper suggests three key strategies to mitigate the privacy harms of AI: 1.) Denormalize Data Collection by Default: Shift from opt-out to opt-in data collection models to facilitate true data minimization. This approach emphasizes "privacy by default" and the need for technical standards and infrastructure that enable meaningful consent mechanisms. 2.) Focus on the AI Data Supply Chain: Enhance privacy and data protection by ensuring dataset transparency and accountability throughout the entire lifecycle of data. This includes a call for regulatory frameworks that address data privacy comprehensively across the data supply chain. 3.) Flip the Script on Personal Data Management: Encourage the development of new governance mechanisms and technical infrastructures, such as data intermediaries and data permissioning systems, to automate and support the exercise of individual data rights and preferences. This strategy aims to empower individuals by facilitating easier management and control of their personal data in the context of AI. by Dr. Jennifer King Caroline Meinhardt Link: https://lnkd.in/dniktn3V

The next big data privacy scandal in 2026 is not surveillance. It is surveillance pricing. Two people can buy the same thing on the same day and pay different prices because their data told the system they would tolerate it. This is the part more people need to understand. The next privacy battle is not only about: “Who has my data?” It is also about: “What are they doing with it?” Because once companies know your location, device type, browsing behaviour, repeat visits, urgency signals, and purchase history, privacy becomes a pricing issue. We are already seeing signals of this. Uber openly calls it surge pricing. Airbnb has Smart Pricing. Amazon lets sellers automate price changes in real time. Hotels and airlines have used dynamic pricing for years. In 2025, India’s consumer affairs ministry sent notices to Ola and Uber after allegations that identical rides were being priced differently on Apple and Android phones. So, what changes the privacy conversation is when dynamic pricing stops reacting only to market demand and starts learning from the customer in front of it. This is why I think the most important privacy question in 2026 is no longer: “Was my data leaked?” It is: “Is my data being used to influence the price, urgency, ranking, or offer I see?” Think about everyday Indian internet behaviour: You check a flight 4 times from the same laptop. You open a hotel app from a premium phone. You try booking a cab during rain, from a high-income pin code, late at night. You revisit the same product after showing clear buying intent. You may still call it convenience. But increasingly, it can also become behavioural exploitation. Because the moment customers feel the system knows them well enough to charge them more, trust collapses. And once trust collapses, growth gets expensive. My view is simple: Data privacy in 2026 is not just about protecting people from theft. It is about protecting people from invisible disadvantage. That is the conversation more founders, platforms, and regulators need to have now. Whats your surveillance pricing case you faced? Seqrite #DataPrivacy #DynamicPricing #AI #ConsumerRights #DigitalEconomy #Privacy #TechPolicy #StartupIndia #CyberSecurity #TrustInTechnology

Google's cookies announcement isn't the week's big news; Oracle's $115 million privacy settlement is. 👇🏼 This week's most important news headline is: "Oracle's $115 million privacy settlement could change industry data collection methods." Every marketer and media leader should understand the allegations in the complaint and execute a review of their data strategy, policies, processes, and protocols, especially as they pertain to third-party data. While we've been talking and fretting about cookie deprecation for four years, we've missed the plot on data permission and usage. It's time to get our priorities straight. Article in the comments section and Industry reaction from legal and data experts below. Jason Barnes, partner at the Simmons Hanly Conroy law firm: "This case is groundbreaking. The allegations in the complaint were that Oracle was building detailed dossiers about consumers with whom it had no first-party relationship. Rather than face a jury, Oracle agreed to a significant monetary settlement and also announced it was getting out of the business," Barnes said. "The big takeaway is that surveillance tech companies that lack a first-party relationship with consumers have a significant problem: no American has actually consented to having their personal information surveilled everywhere they go by a company they've never heard of, packaged into a commoditized dossier, and then monetized and sold without their knowledge." Debbie Reynolds, Founder, Chief Executive Officer, and Chief Data Privacy Officer at Debbie Reynolds Consulting, LLC: "Oracle's privacy case settlement is a significant precedent and highlights that privacy risks are now recognized as business risks, with reduced profits, increased regulatory pressure, and higher consumer expectations impacting organizations' bottom lines," Reynolds said. "One of the most important features of this settlement is Oracle's agreement to stop collecting user-generated information from external URLs and online forms, which is a significant concession in how they do business. Other businesses should take note." #marketing #data #media Ketch super{set}

𝗕𝗜𝗢𝗠𝗘𝗧𝗥𝗜𝗖 𝗦𝗘𝗖𝗨𝗥𝗜𝗧𝗬 & 𝗣𝗥𝗜𝗩𝗔𝗖𝗬 𝗜𝗡 𝗧𝗛𝗘 𝗕𝗜𝗢-𝗗𝗜𝗚𝗜𝗧𝗔𝗟 𝗔𝗚𝗘: 𝗧𝗛𝗘 𝗙𝗨𝗧𝗨𝗥𝗘 𝗢𝗙 𝗜𝗗𝗘𝗡𝗧𝗜𝗧𝗬 🔒🔬 In a world where technology is merging with biology, 𝗯𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆 has become essential for protecting personal identity. Fingerprints, facial recognition, iris scans, and even DNA are now used for authentication. But as digital and physical worlds merge, how secure is this data, and what are the implications for 𝗽𝗲𝗿𝘀𝗼𝗻𝗮𝗹 𝗽𝗿𝗶𝘃𝗮𝗰𝘆? Biometrics offer a secure alternative to traditional passwords, allowing access through a glance or touch. However, 𝗯𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰 𝗱𝗮𝘁𝗮 𝗶𝘀 𝗽𝗲𝗿𝗺𝗮𝗻𝗲𝗻𝘁 unlike passwords, it can’t be changed once compromised. A hacked fingerprint or facial scan could have severe consequences. 𝗛𝗼𝘄 𝘀𝗲𝗰𝘂𝗿𝗲 𝗮𝗿𝗲 𝗯𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰𝘀 𝗶𝗻 𝗮 𝗵𝗮𝗰𝗸𝗶𝗻𝗴-𝗽𝗿𝗼𝗻𝗲 𝘄𝗼𝗿𝗹𝗱? Companies like 𝗖𝗹𝗲𝗮𝗿𝘃𝗶𝗲𝘄 𝗔𝗜 have raised concerns about how biometric data is collected and stored. Additionally, 𝗯𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰 𝘀𝘂𝗿𝘃𝗲𝗶𝗹𝗹𝗮𝗻𝗰𝗲 is being used by governments to monitor populations, especially in countries like China, where facial recognition is used for citizen control. This blurs the boundaries between security and privacy. As biometrics continue to shape our digital identity, the need for 𝗿𝗼𝗯𝘂𝘀𝘁 𝗹𝗲𝗴𝗮𝗹 𝗳𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 becomes urgent. The 𝗚𝗲𝗻𝗲𝗿𝗮𝗹 𝗗𝗮𝘁𝗮 𝗣𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗼𝗻 𝗥𝗲𝗴𝘂𝗹𝗮𝘁𝗶𝗼𝗻 (GDPR) sets strict rules for personal data in the EU, but other regions need to catch up. A key issue is how 𝗯𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰𝘀 𝗶𝗻𝘁𝗲𝗿𝘀𝗲𝗰𝘁 𝘄𝗶𝘁𝗵 𝗔𝗜. AI systems use biometric data for decision-making, but we must ensure that biases don’t infiltrate these systems. 𝗕𝗶𝗼𝗺𝗲𝘁𝗿𝗶𝗰-𝗯𝗮𝘀𝗲𝗱 𝗱𝗶𝘀𝗰𝗿𝗶𝗺𝗶𝗻𝗮𝘁𝗶𝗼𝗻 could become widespread if not addressed properly. As we embrace the bio-digital age, balancing security and privacy will be a challenge. The responsibility lies with both governments and corporations to safeguard biometric data and prevent misuse. 𝗔𝗿𝗲 𝘄𝗲 𝗿𝗲𝗮𝗱𝘆 𝗳𝗼𝗿 𝗮 𝗳𝘂𝘁𝘂𝗿𝗲 𝘄𝗵𝗲𝗿𝗲 𝗼𝘂𝗿 𝗯𝗶𝗼𝗹𝗼𝗴𝗶𝗰𝗮𝗹 𝘁𝗿𝗮𝗶𝘁𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝘂𝗹𝘁𝗶𝗺𝗮𝘁𝗲 𝗸𝗲𝘆 𝘁𝗼 𝘀𝗲𝗰𝘂𝗿𝗶𝘁𝘆? And more importantly, how can we protect them from being exploited? Stay tuned. Next, we’ll explore 𝗧𝗵𝗲 𝗥𝗼𝗹𝗲 𝗼𝗳 𝗤𝘂𝗮𝗻𝘁𝘂𝗺 𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝗶𝗻 𝘁𝗵𝗲 𝗣𝗼𝘀𝘁-𝗗𝗶𝗴𝗶𝘁𝗮𝗹 𝗘𝗿𝗮, and how quantum advancements could redefine data security. #BiometricSecurity #BioDigitalAge #DataPrivacy #TechEthics #ClearviewAI #GDPR #CosmosRevisits

The TikTok privacy debate did not end with the US agreement. It has escalated. TikTok has updated its US Privacy Policy. It is now one of the most aggressive data collection regimes of any mainstream consumer platform. It explicitly acknowledges the collection and processing of sensitive personal information under US state privacy laws. Named directly: • Racial or ethnic origin. • Religious or philosophical beliefs. • Mental and physical health data. • Sexual orientation. • Transgender or nonbinary status. • Citizenship or immigration status. • Precise location data. The policy goes further. TikTok is collecting far more than what users consciously share. Under the updated policy, it gathers what you provide, what it observes automatically, and what it receives from third parties. That includes account details and identity verification documents, private messages, drafts and unpublished content, AI prompts and interactions, clipboard content, purchase and payment data, contact lists and social graphs, and an extensive set of technical signals such as device identifiers, keystroke patterns, battery state, audio configurations, and activity tracked across devices. This is not incidental data leakage. It is formalized, permitted, and documented. Images and video are treated as analyzable environments. TikTok states that it "identifies objects and scenery, detects faces and other body parts, extracts spoken words, and collects metadata describing how, when, where, and by whom content was created." Post a photo near the Golden Gate Bridge and you are not just sharing a moment. You are generating structured data about place, time, environment, and your body, or body parts. Photos and videos are not just content. They are raw material for computer vision, biometric analysis, and location inference. Tik Tok will use all of the collected data, and maintains the right to sell all of it to interested third parties, from vendors to the federal government. Leaders must act on this immdiately. Privacy policies are not background reading. They are power documents. When they change, accountability shifts with them. If you are a user, a parent, a school, a youth facing organization, nonprofits, and public institutions that use TikTok as a communications channel, the update changes the governance calculus. Engagement is not a neutral act. It carries serious legal and ethical obligations tied to data protection, duty of care, and institutional risk. The new policy deserves close reading. At this stage of platform power, and scale of data collection, policy literacy is a governance responsibility, not a personal preference. Read the policy here: https://lnkd.in/ejbm8THx

The recent $95 million settlement by Apple over allegations of Siri-enabled privacy breaches underscores a pivotal moment for tech professionals navigating the delicate balance between innovation and user trust. As voice assistants become integral to our daily lives, this case illuminates the risks of unintentional data collection and the potential fallout—financial, reputational, and ethical—when consumer privacy is perceived as compromised. For engineers, developers, and business leaders, this serves as a critical reminder: robust privacy safeguards and transparent practices aren’t optional—they’re fundamental to maintaining user loyalty in an increasingly data-sensitive world. This moment invites the tech community to reimagine AI solutions that are not only cutting-edge but also deeply rooted in trust and accountability. How can we, as innovators, ensure that technology enhances lives while respecting the privacy and trust of its users? #TechNews #Innovation #Privacy #Apple

Was compensation for breaching and violating a patient's privacy worth $10? This week, an unsettling development unfolded as BetterHelp, a widely-used teletherapy platform now owned by Teladoc Health, settled with the FTC for $7.8 million over serious breaches of user privacy. According to the Federal Trade Commission, BetterHelp sold highly sensitive user data—including IP and email addresses and even answers to mental health questions—to social media giants like Facebook and Snap Inc. The repercussions of such actions are profound. Clients of BetterHelp received notifications about the settlement, only to learn that the financial compensation offered is: ~$10. This token amount seems a slap in the face, trivializing the potential damage to those affected. Here's the problem: The bond between therapist/clinician and patient is sacrosanct, grounded in the assurance of confidentiality. When this trust is compromised, especially in such a blatant manner, it not only damages individual therapist-client relationships but could also deter people from seeking essential mental health services online. The potential for harm here is incalculable. It undermines individual trust in telehealth services and casts a long shadow over the promise and potential of leveraging technology in healthcare and mental healthcare. While it’s unclear whether these actions constitute a HIPAA violation—as it’s uncertain if the shared information was directly linked to identifiable patient health records—the breach of confidentiality remains a critical issue. More concerning is that the FTC reported that Betterhelp misrepresented that they were HIPAA compliant. As we continue to embrace telehealth and innovations in healthcare, it is imperative that we prioritize strong, enforceable protections for patient data. Technology can greatly enhance healthcare delivery, but it must not do so at the cost of patient safety and privacy. #telehealth #digitalhealth #privacy #FTC #BetterHelp #healthcare #HIPAA Genevieve Friedman UNC Kenan-Flagler Business School MBA@UNC ABIG Health

In an era where privacy is the ultimate luxury, Apple—a company renowned for its strong stance on user privacy—has found itself at the center of a massive controversy. The tech giant has agreed to pay $95 million (₹814 crores) in a lawsuit that accused Siri, its voice assistant, of recording private conversations without user consent and sharing them with third parties. The Allegations The case stemmed from claims that Siri was being inadvertently activated by users, leading to the recording of highly personal conversations. Even more troubling, these recordings were allegedly sent to third-party contractors for evaluation without user knowledge. This scandal was first exposed in 2019 by The Guardian, which reported that Apple’s contractors listened to sensitive discussions, including: - Medical consultations, - Private business meetings, and - Intimate personal exchanges. While Apple denied any wrongdoing, this case highlights a glaring gap between privacy promises and actual practices. The Settlement Here’s what it entails: - Payout to Users: Thousands of affected users will receive compensation of $24 (₹1,700) per device. - Legal Fees: A significant portion of the settlement—up to 30%—will go to the attorneys involved. - Apple’s Stance: The company maintains it did not violate user trust, but settled to avoid prolonged litigation. The Bigger Picture This incident is not just about Apple. It’s a wake-up call for every company operating in the digital age: 1. Transparency is Non-Negotiable: Users have the right to know how their data is collected, stored, and used. 2. Trust is Fragile: Even giants like Apple can face reputational damage if user privacy is compromised. 3. Accountability Must Follow Innovation: Companies can no longer prioritize profits over ethics. For Consumers - Be Informed: Read privacy policies, however tedious they may seem. - Be Proactive: Use device settings to limit data sharing and disable features like voice assistants when not in use. - Advocate for Stricter Regulations: Governments must enforce stringent data protection laws to safeguard user rights. Apple's Future Steps Since the controversy, Apple has taken steps to rebuild user trust, including: - Disabling human grading of Siri recordings, - Allowing users to opt out of sharing their data, and - Strengthening their privacy policies. However, this lawsuit serves as a stark reminder: Even the most trusted brands must remain under constant scrutiny. What’s Next? As users, we need to push for digital ethics and ensure companies treat our data with the respect it deserves. Should stricter penalties be imposed for such violations? Are current privacy laws sufficient in protecting us? Let’s discuss! #DataPrivacy #AppleLawsuit #TechnologyEthics #DigitalSecurity #Siri #ConsumerRights #TransparencyMatters #EthicalTech