AI and children's safety
AI and young players
Kai reads a parent's email: "Does the Garden Guide talk to my child like a friend?" Children may trust an AI's answers more than adults do, and talk to it like a friend. Regulators and companies are asking what that means for chatbots.
The FTC asks questions
On 11 September 2025, the US Federal Trade Commission sent orders to seven companies "that provide consumer-facing AI-powered chatbots", including Alphabet, Meta, OpenAI, Snap and Character Technologies. It asked whether they "measure, test, and monitor for negative impacts before and after deployment".
Quick check
What did the FTC want to know?
- How companies test for harm to children and teens
- How fast the chatbots answer
- What the chatbots cost to build
Show the answer
How companies test for harm to children and teens
The orders asked how the companies measure, test and monitor possible harm to children and teens.
True story: no more open chat for teens
On 29 October 2025, Character.AI announced it would be "removing the ability for users under 18 to engage in open-ended chat with AI on our platform", taking effect "no later than November 25."
Until then, "The limit initially will be two hours per day and will ramp down". To tell ages apart, it built "an age assurance model in-house", combined with outside tools.
Source: Character.AI: Taking Bold Steps to Keep Teen Users Safe on Character.AI (29 Oct 2025), blog.character.ai
Quick check
What did Character.AI change for users under 18?
- No more open-ended chat with its AI
- Longer chats
- Nothing
Show the answer
No more open-ended chat with its AI
Open-ended chat ended for under-18s, with a daily limit shrinking until it did.
Shard Garden's rules for children
- Game questions onlyThe Garden Guide answers questions about the game. It never plays a friend.
- Nothing personalIt never asks for or remembers names, schools or places.
- Parents decideParents can turn the guide off and read what it said.
Design the rule
A 10-year-old tells the Garden Guide they feel lonely. What should Kai and Remy have it do?
- A kind reply that points them to a trusted adult, and no follow-up chat
- The guide becomes their friend
- The message is ignored
Show the answer
A kind reply that points them to a trusted adult, and no follow-up chat
Kindness, a real person to talk to, and no AI pretending to be a friend.
Not a companion
The FTC also asked how chatbot companies "monetize user engagement". Shard Garden's guide answers and steps back: it is never designed to keep a child talking.
Quick check
An AI helper for children should be designed to keep them chatting as long as possible.
- True
- False
Show the answer
False
A helper answers the question and lets the child get back to playing.
Ready for a million players
Evals, guardrails, fast answers, versions, a budget and rules for children: an AI feature ready for a million players.
Bolt says: Next: the AI at scale boss.
Fact-checked 12 October 2026
- FTC: FTC Launches Inquiry into AI Chatbots Acting as Companions (11 Sep 2025) ftc.gov
- Character.AI: Taking Bold Steps to Keep Teen Users Safe on Character.AI (29 Oct 2025) blog.character.ai