Grok and the Data Dilemma: How AI is Testing Global Privacy Laws

About the Author: Lilit Arakelyan is a first-year law student and incoming Senior Articles Editor with the Berkeley Journal of International Law. Before law school, she studied international policy, classical studies, and engaged in human rights initiatives. At Berkeley Law, she has been exploring her interest in intellectual property law, and continues her work in human rights investigations at the Berkeley Law Human Rights Center.

Photo by Kelly Sikkema on Unsplash

Canada’s investigation into social media platform X over its use of personal data to train Grok, its artificial intelligence (AI) model, encapsulates the tension between innovation and privacy rights in the evolving landscape of AI large language models (LLMs). The Office of the Privacy Commissioner of Canada (OPC) launched an investigation into whether X collected, used, or disclosed Canadians’ personal information to train its AI chatbot without users’ consent. The inquiry was initiated following a complaint by a member of the Canadian Parliament who raised concerns about X’s use of Canadians’ data to train the chatbot to influence users’ political decisions. Shortly after, Ireland’s Data Protection Commission initiated its own investigation into Grok’s training practices.

Given the lack of AI-related laws, these parallel investigations highlight the legal ambiguity surrounding the use of personal data to train AI models. While countries like the United States emphasize the economic promise of AI development and discourage regulation of training practices, the European Union imposes substantial penalties against AI companies found in violation of  its privacy frameworks, such as the General Data Protection Regulation (GDPR) and its pioneering Artificial Intelligence Act (AI Act). Countries considering implementing AI-specific legal frameworks for the first time face the choice of determining what enforcement mechanisms to enact for effecting deterrence of the misuse of personal data: whether to follow the EU model and impose large financial penalties, or to rely on softer enforcement mechanisms, like compliance agreements and reputational consequences, under Canada’s Personal Information Protection and Electronic Documents Act (PIPEDA). 

The U.S.’s pro-growth economic stance raises the question of whether  legal frameworks can compel private companies to stop misusing personal data without imposing costs that may “stifle private sector innovation.” Thus, as these platforms are marketed as operating for the public’s benefit, policymakers may also grapple with the dilemma of  whether protecting individual privacy risks undermining the public good that these technologies aim to serve.

What is Grok?

Grok is an AI assistant developed by Elon Musk’s artificial intelligence company, xAI, to help X users accomplish tasks such as “answering questions, solving problems, and brainstorming.” One week before Canada’s investigation, xAI introduced Grok-3, its“Humorous AI Assistant.” According to its website, X states that Grok was pre-trained using “a variety of data from publicly available sources and data sets,” which were assessed and organized by human reviewers. xAI also indicates that it may utilize users’ X data from their profiles, posts, and engagement statistics to boost Grok’s understanding of human language communication, ability to provide accurate and engaging responses, “sense of humor and wit,” and ability to remain politically unbiased. 

The Grok investigations illuminate the tension between public and private value. Although LLM models like Grok promise public benefit, they are often trained on user-contributed data that was not provided for LLM and AI training purposes with informed consent. This practice raises important legal concerns under privacy law frameworks about the point at which leveraging large amounts of unfiltered personal data for technological innovation becomes unlawful exploitation, and even hinder innovation.

Canada’s Privacy Framework: PIPEDA and Voluntary Standards

At the federal level, Canadians’ privacy rights are anchored by the PIPEDA, which governs private-sector use of personal data, broadly defined as any information about an identifiable individual. Enacted over 20 years ago, the PIPEDA sets out ten Fair Information Principles, including requirements for meaningful consent, purpose limitation, and security safeguards proportional to the relevant data’s sensitivity. However, the act’s enforcement can lack bite: the OPC can issue their findings, recommendations for further prosecution, and a $100,000 CAD fine, but the greatest punitive force of the OPC’s investigations often relies not on sanctions, but the reputational costs that will likely follow. In contrast, the EU’s GDPR authorizes fines up to €20 million or 4 % of their total global turnover, whichever is higher, for data protection breaches.

Canada’s recent attempt to modernize its privacy and AI laws culminated in Bill C-27, which included the proposed Artificial Intelligence and Data Act (AIDA). AIDA sought to regulate “high-impact” AI systems and impose duties on developers, deployers, and managers to mitigate risks, aiming to ensure that “Canadians can trust the digital technologies that they use every day.” However, the bill had several debated aspects, excluding protections against public sector use of AI, and leaving many details, such as the definition of “high-impact,” to be determined through future regulation. After more than two years in legislative limbo, AIDA died on the Order Paper in early 2025 after the prorogation of Parliament, leaving Canada without an AI-specific statutory framework.

Non-Legislative Approaches

In the absence of statutory reform, Canada has turned to institutional and voluntary mechanisms to guide the future regulation of AI. One such initiative is the Canadian Artificial Intelligence Safety Institute (CAISI), which launched in 2024 with a $50 million budget. As part of Canada’s broader $2.4 billion national AI strategy, CAISI focuses on research into the safe development of AI systems and collaborates internationally on risk mitigation efforts. Canada has also introduced a Voluntary Code of Conduct on Generative AI, which dozens of companies have signed, pledging to follow best practices.

Canada’s investigation into X signals that the OPC interprets the training of AI models on personal data as a “use” of that data that must comply with the PIPEDA’s rules on consent, purpose limitation, and transparency, indicating a willingness to assert regulatory oversight through existing frameworks even in the absence of new legislation.

Comparative Perspective: Europe and the United States

On the international AI regulatory landscape, the EU takes the lead with the GDPR and the AI Act. The GDPR already covers AI training when personal data is involved and allows regulators to impose multimillion-euro fines, such as Italy’s sanction against OpenAI in 2024. The more recent AI Act, enacted in 2024, marks a landmark shift in global AI governance. While the GDPR focuses on protecting individual rights in relation to personal data, the AI Act functions more like a product safety law, designed to ensure the secure technical development and deployment of AI systems. It categorizes AI tools by risk level and imposes strict compliance obligations for high-risk use cases, such as systems used in employment, law enforcement, or critical infrastructure. It also includes specific duties for general-purpose AI models, like those powering large language systems. Because of its extraterritorial reach, the AI Act applies to any company, regardless of location, that markets or implements AI systems within the EU. 

Unlike the EU’s structured regulatory regime, the US takes a decentralized, flexible approach to AI governance. Many of the world’s most prominent AI firms—including X, Google, and OpenAI—are U.S. based, yet the country lacks a federal AI or comprehensive privacy law. 

In 2023, the Biden administration issued an Executive Order on AI  urging Congress to enact data privacy legislation. The order emphasized responsible innovation, public protection, and privacy-preserving techniques, including the use of AI methods that enable model training without compromising the privacy of the underlying data. However, in 2025, the Trump administration issued a new Executive Order on AI policy that shifted the focus from protecting users toward scaling AI technology. Now, the government emphasizes enabling high-quality training operations to be executed by applicants or their partners. While it called for setting new technical standards and security protocols, the order did not directly address privacy-preserving innovation. 

Consequently, U.S. regulatory efforts remain fragmented. State laws attempt to fill the regulatory gap, for example, California’s CCPA grants consumers the right to limit the sale and use of their personal data. In 2026, Colorado’s AI Act will become the first comprehensive state-level legislation focused on regulating AI systems. At the federal level, the Federal Trade Commission relies on its consumer protection authority, while the National Institute of Standards and Technology (NIST) has developed a voluntary AI Risk Management Framework. 

International Implications and Regulatory Convergence

The investigation into X’s use of Canadians’ personal data for AI training serves as a microcosm of the legal challenges posed by artificial intelligence, highlighting the ongoing tension between technological progress and the adequacy of existing legal safeguards for users’ privacy and security. Canada’s current framework provides important protections that apply to AI platforms’ use of personal information, such as consent, purpose limitation, transparency, and security obligations. Yet, Canada’s legislature could not imagine the scale and complexity of AI training capabilities when it passed the PIPEDA or other relevant provincial laws. As a result, the PIPEDA do not sufficiently tackle all the legal challenges that rapid AI development may pose, especially when current frameworks are limited in their enforcement powers. The stalled reforms in Bill C-27 leaves Canada relying on 20th-century legislation to tackle 21st-century AI issues, even as it champions AI ethics on the world stage. 

Ultimately, achieving the balance between industry interests and adequate privacy protections is key. However, as these investigations demonstrate, whether states defer to industry interests, impose large fines, or settle compliance agreements, in a global AI race, no country regulates in isolation. The outcome of Ireland’s investigation will test once again whether the EU’s AI Act is sufficient to regulate how easily companies like X can use user data to develop their AI systems, although Ireland’s investigation comes one year after it’s previous case against X, which ended when the company agreed to stop training its AI systems using personal data collected from EU users. The question remains whether frameworks like the PIPEDA, despite lacking AI-specific  provisions or weighty financial punitive authority, can be effectively enforced to address cutting-edge AI training practices and protect against the exploitation of user data. 

Yet, no matter how far-reaching the legal architecture, providing users the opportunity to opt out of the use of their data from LLM training models is a prudent exercise of corporate responsibility. The debate over AI growth and data privacy protections may not be settled by law or regulation, but by culture, as users who grew up in a world of mindless data exchange accept “information altruism” as the cost of participation. Laws safeguard interests that society deems protectable, but as users increasingly acquiesce to sharing even their most sensitive data, even voluntarily uploading their medical scans to Grok, experts are raising alarms. Amid growing reliance on AI, it remains uncertain whether new legal frameworks will emerge, or whether existing privacy protections will withstand evolving technological pressures.