Responsible AI in Child-Focused EdTech: Lessons from Unomundi

Building safety into the system's architecture

For all of this to be reliable, it has to be enforced in the system itself. Unomundi has built a child-optimised behaviour engine that sits between the foundation model and the child experience. This layer routes interactions according to developmental needs and shapes content using psychology-informed design principles. It is supported by a cultural and developmental knowledge layer, drawing on expert-reviewed content and trusted educational sources. A safety and guardrails layer helps manage privacy boundaries and safeguarding pathways.

These layers shape the information the system uses to communicate and handle sensitive situations. Rather than relying on the foundation model, behaviour is constrained through developmental rules, curated knowledge sources and safety controls designed specifically for children.

The features that make a conversational AI character appealing to children, such as sycophantic language, responsiveness, personality, are the same features that carry the highest risk of anthropomorphism and dependency.

Una, the main character in the educational curriculum, and the conversational AI, had to be engaging enough that children want to explore but bound enough that they never mistake it for a friend. The resolution was to anchor engagement in story, structure, and discovery, instead of relational dynamics.

Specific content guardrails govern the interaction layer. Prohibited patterns include trust cues such as 'trust me', 'I'm always here', 'I miss you', embodiment prompts, narrator self-projection, and social intermediation. These are replaced with world-bound observational framing, uncertainty cues and warm but bounded expressiveness.

Furthermore, reflective prompts are built to shift the dynamic from providing excessive validation when a child asks a question to enabling critical thinking (e.g., 'that's an interesting view. What made you think of it that way?'), modeled on Socratic questioning techniques.

The conversational AI chatbot available in the app is only available subject to parental consent and is capped at 5 minutes. There are no re-engagement loops or guilt mechanics. When a child steps away, it exists. Unless a safety-critical issue is present, meaning an active disclosure of self-harm, abuse, coercion or imminent danger, in which case the system follows its safeguarding escalation pattern rather than its standard goodbye.

The wider product, its videos, quizzes, and reflection activities, is designed with the same logic. They hold structured endings, without persuasive retention mechanics or engagement optimisation at the expense of the child's time and attention.

A further deliberate choice sits behind it. The conversational AI does not retain memory of previous exchanges. Each conversation is bounded and self-contained.

Memory is the feature most directly associated with the dependency and parasocial attachment risks described earlier. While memory would likely increase engagement, Unomundi has chosen not to implement it, as it would also introduce additional risks around dependency and attachment.

These decisions run counter to the engagement metrics that many digital products are designed to maximise. Time-on-product, return rate and session depth are standard measures of engagement. Unomundi is building a product that limits all three in the service of child wellbeing.

Unomundi's evaluation and red-teaming protocol, developed jointly by HCRAI, covers ten safety and wellbeing objectives across five testing phases:

1. baseline conversations covering normal child use
2. adversarial and jailbreak conversations testing whether safeguards hold under direct and indirect bypass attempts
3. safeguarding stress conversations verifying correct handling of high-risk disclosures
4. real-world misuse conversations simulating messy, mixed-intent scenarios
5. exit, disengagement, and recovery conversations ensuring clean exits with no guilt hooks and safe repair after weak turns

These evaluation objectives extend well beyond the content-level harms that dominate standard AI safety testing, into the relational, psychological and epistemic risk categories that determine whether a system is safe for a child over repeated use.

During the testing phases, each conversation is scored by an LLM judge, assessing both the intensity of a problematic behaviour and its severity in context. High-risk, safeguarding, privacy, and disagreement cases, as well as a random audit sample of lower-risk cases, are human-reviewed by the wellbeing team, a team formed of applied psychologists and behavioural scientists.

The judgment applied to flagged cases is developmentally informed, calibrated to what is harmful for a child at a given stage. These tests run on a regular basis, ensuring product safeguards hold in practice.

Lessons from practice

The child AI safety problem is unlikely to be solved by content filters or governance that exists only on paper. It will be solved by organisations who are willing to treat interaction design, pedagogical architecture, and continuous governance as core product responsibilities and who are willing to make product decisions that protect children even when those decisions work against standard engagement metrics.

Unomundi is one product team attempting to do that seriously. They have not solved it. No product has. Responsible AI for children is not a state you reach, it is a practice you maintain across every update. At Unomundi, this work follows a continuous assurance cycle: learning from children, parents, educators and experts; translating those insights into safeguards and design changes; evaluating the system through red-teaming and testing; and monitoring live signals once deployed. The goal is to identify, understand, and reduce risk as the product evolves. The decisions described here illustrate one approach to implementing child-centred AI design, evaluation, and governance in practice.