Millions of students turn to AI chatbots every day for study help, emotional support and personal advice, often unaware of where their conversations and other personal information end up.
How this data is used—and the role schools play in protecting it—is a growing concern for educators, parents and children as artificial intelligence becomes a ubiquitous part of the school day, built into many of the classroom tools students are expected to use.
As the school year began in the United States, the country’s two largest school districts, New York and Los Angeles, announced they would suspend student use of AI for one year as they study how best to incorporate the technology into education.
In response to the growing backlash against AI and screen time, Microsoft announced last week that it had agreed to adopt safeguards and privacy standards for its AI in schools, as agreed with the American Federation of Teachers, the second-largest teachers union in the country.
The agreement’s security provisions make sense, but the standards will only be effective if all AI companies, especially tech giants, sign on, said Josh Golin, executive director of online security nonprofit Fairplay. The agreement sidesteps the question of whether artificial intelligence products are appropriate in classrooms and at what age they should be introduced, he said.
“I’m worried that a high-profile framework like this will be misunderstood and schools will think, ‘If they’re doing a good job with privacy and data security, then that means we should use this tool,’” Golin said.
The AFT said the agreement, which is legally binding and requires third-party verification of compliance, could set an industry standard. Two other leading artificial intelligence companies, OpenAI and Anthropic, said they were also discussing security and privacy agreements with AFT. But Google, the dominant provider of education technology to U.S. schools, has not said whether it plans to offer similar protection.
The agreement places restrictions on the use of student data for AI training
As part of the sweeping agreement, Microsoft says it will not use student or teacher data to train artificial intelligence systems, except for a narrow exception for data that can help protect students. It states that the data collected cannot be sold or used for advertising or product development.
The agreement prohibits any use of AI companions or features “intended to promote emotional attachment or dependence” and to keep students engaged longer than required by the learning task. It also includes a series of auditing and transparency measures that require the company to disclose to families in plain language how its tools work.
“This standard sets a high bar for children’s privacy and AI security, and we will extend this agreement to every school district across the country,” Brad Smith, vice chairman and president of Microsoft, said in a statement. The standards will apply to all schools with which Microsoft has contracts starting November 1, he said.
AFT President Randi Weingarten called the agreement “the first of its kind” and an important step toward establishing an industry standard in the absence of federal and state legislation on artificial intelligence in schools.
“We want parents to control their children’s education. We want teachers to control their children’s education, not educational technology. That’s what this agreement is all about,” Weingarten said.
She spoke at a joint news conference with Smith at the headquarters of the United Federation of Teachers, the New York affiliate of the AFT. The headquarters is home to the AFT National Academy for Teacher Training in Artificial Intelligence, which opened a year ago with $23 million in funding from Microsoft, OpenAI and Anthropic.
Along with a year-long moratorium on AI, school systems in New York and Los Angeles are planning sweeping reviews of their contracts for education technology and AI tools, with a focus on data privacy, transparency and accountability. New York’s ban applies to all elementary and middle school students, while Los Angeles’ ban applies to high school students.
The teachers union has also reached out to Google and artificial intelligence companies.
Weingarten said she contacted Google directly to inform them of the agreement before making it public in hopes that the company would become a party to it.
The company did not respond to a request for comment from The Associated Press on whether it plans to offer similar protection.
“Is Google going to adopt this framework? That’s a huge question,” said Golin, who runs an online safety nonprofit. He noted that schools also use thousands of artificial intelligence-based educational technologies produced by small companies.
Google found itself at the center of controversy over artificial intelligence in schools when it announced in August that it was enabling its Gemini chatbot for many K-12 students using Google Classroom, its learning management system. The company says Google Classroom is used by 150 million teachers and students worldwide.
Critics have denounced Google’s move as an escalating race between tech giants to penetrate schools and win over the next generation of AI users. The addition of AI to Classroom only affects schools that provide students with access to the Gemini app or Gemini notebook, but it marks a major shift. Until August, Google only allowed students 18 and older to turn on Gemini.
Weingarten said she is also in talks with OpenAI and Anthropic, which have publicly praised the agreement.
OpenAI, the creator of ChatGPT, called it “an important milestone in ensuring AI security and privacy in schools.” The company is partnering with AFT, the company said in a statement, “and we look forward to entering into our own agreement.” Anthropic, the creator of “Claude,” said in a statement: “We welcome all efforts to encourage the responsible use of AI in education.” The company said it is working with the AFT to create a “gold standard for security and privacy.”
Standards shift the burden of privacy onto companies
Data privacy experts say the new standards could help shape the conversation schools have with AI providers, whether the school is a Microsoft customer or not.
“What I hope is that this will take some of the pressure off individual school districts when negotiating with big tech companies,” said Elizabeth Laird, director of the Center for Democracy and Technology. Until now, schools’ bargaining power to secure privacy protections from tech companies has often depended on financial clout. “I think this standard goes a long way toward eliminating these types of inequities in school districts.”
Laird praised the agreement for going beyond what is required by law for vendors working with schools. For example, Microsoft has committed to not using sensitive data that many companies use, such as information about gender, race, age, zip code, which can be used to identify a person, even if their name is removed.
But until others sign on, the agreement underscores how difficult it is to set AI standards in the absence of legislation, Laird said. It also emphasizes that the burden of protecting student privacy should fall on technology companies, not parents, students or schools.
Microsoft’s Smith noted that tech companies were late to the game.
“We all know that there are important discussions and debates going on across the country about when AI should be used in schools, how it should be used and for what purposes,” Smith said. “Our role, indeed our responsibility, is to ensure that AI is delivered in a way that teachers, children and parents can trust.”
Published September 15, 2026