The Gemini Robotics ER 2 model card prohibits safety-critical use

Under acceptable use, the model card asks users to exercise discretion before using the robotics models in production, commercial or public environments, and not to use them for safety-critical applications or work. It names healthcare and transport and, more broadly, settings where safety protocols matter and a malfunction could reasonably foreseeably lead to death, personal injury or property damage.

The API documentation is written in a milder register. It warns that generative models can make mistakes, that physical robots can cause damage, that hallucinations are possible and that responsibility sits with the user. It does not state a prohibition.

Keep those two apart when you quote them, because vendors and consultants frequently blur them. The model card carries the prohibition. The API documentation carries the warning. An operator citing the milder wording to justify a corridor pilot is quoting the wrong document.

Google reports that ER 2 outperforms its predecessor on Safety Instruction Following and Human Proximity benchmarks, and that it halts a humanoid when a person is nearby, resuming only once the area is clear. That is a real improvement and it is worth having.

It is not a safety function. The distinction is not pedantic, it is the entire basis of machinery law. A safety function has to behave the same way every time, has to fail in a defined direction, and has to be demonstrable to a third party. A probabilistic model that is right more often than its predecessor satisfies none of those requirements, and the number that would matter, the residual failure rate under adversarial conditions, is not the number that gets published.

In practice this means the model may propose that a machine stops. It may not be the thing that stops it. The stopping is done by a certified protective device that would have stopped the machine anyway.

For driverless industrial trucks and transport robots, EN ISO 3691-4 has been the harmonised type C standard since May 2024, and it requires person detection at performance level d per ISO 13849. That is the requirement a machine has to meet on a corridor where people walk.

A frequent market error is to cite ISO 13482 instead. ISO 13482 covers robots in physical contact with people, which is a different category. Vendors quote it regularly for transport machines, and it is worth checking which standard a supplier claims, because the answer tells you how carefully they have read their own documentation.

ISO 10218, revised in 2025, governs industrial robots and is a third category again. None of these standards has anything to say about a cloud planner, which is precisely why the planner cannot be part of the safety case.

The Machinery Regulation (EU) 2023/1230 applies from 20 January 2027 and replaces Directive 2006/42/EC. Machines placed on the market before that date keep their existing status. The provision that matters for anyone connecting an AI planner to existing hardware is the treatment of substantial modification.

A modification counts as substantial when a machine is changed physically or digitally after being placed on the market, in a way the manufacturer did not foresee, and when that change creates a new hazard or increases an existing risk such that new protective measures become necessary. Whoever performs a substantial modification is legally treated as the manufacturer and takes on the corresponding obligations, including conformity assessment and CE marking for the affected aspects.

Connecting an external planner that changes what a machine does, where it goes or under what conditions it operates is exactly the kind of change that invites this question. It does not automatically trigger it. But it is a question that has to be answered deliberately, in writing, before a pilot rather than after an incident.

None of this argues against piloting. It argues for piloting with the boundaries drawn explicitly, because the parties involved have genuinely different exposures.

• Model provider. Supplies a planner and, through the model card, explicitly declines responsibility for safety-critical use.
• Robot manufacturer. Holds the CE conformity for the machine as delivered, including its protective devices.
• Integrator. Connects planner, fleet layer and building systems, and has to make sure the connection does not alter the safety case.
• Operator. Runs the machine, carries the workplace safety obligation, and is the party a works council will ask.

In German-speaking operations the works council question arrives early, and it is usually about video. A planner that reasons over live camera feeds is processing images of employees at work, which raises codetermination under section 87 paragraph 1 number 6 of the Works Constitution Act on technical monitoring equipment. Bringing that up yourself, before it is discovered, tends to go considerably better than the alternative.

werob works as an independent systems integrator across Germany, Austria and Switzerland, which includes being direct about what a technology cannot yet carry. Our piece on mixed fleet orchestration covers the architectural side of the same release.