Understanding the openpilot Safety Model

We’ve been seeing questions about if modifications to openpilot violate our safety model or not. The safety model has three main principles.

1. The driver must always be paying attention.
2. The driver must always be capable of immediately retaking manual control of the vehicle.
3. The vehicle must not alter its trajectory too quickly for the driver to safely react.

Complying with 1 obviously depends on the driver, but we’ve followed industry standard practices and built both a camera based driver monitoring system like GM Super Cruise and a “hands on wheel” detector like Tesla Autopilot to help the driver stay focused. This code lives in driver_monitor.py, and as long as you don’t disable it or nerf it by lowering the strictness, you are in compliance with 1. While stock openpilot uses both, we consider one or the other acceptable as long as Tesla does. Just not neither.

2 is enforced by the safety code in the panda, our real time STMF4 bridge to the car. The panda has a state variable “controls_allowed” which determines if control messages are allowed to be sent on the CAN bus. You enter controls allowed state by turning on cruise control, and you exit by cancelling cruise control. The brake pedal must always immediately cancel the controls allowed state. In stock openpilot, the gas pedal will always cancel too, though there is an unsafe flag to the panda to allow gas while engaged, since both Super Cruise and Autopilot allow this.

3 is the most subtle. Cars can overpower humans, and we need to make sure the human is always the one in control. By using the CAN messages as designed for ADAS, we get a lot of protection here from the car’s built in safety model. Do not use messages not designed for ADAS or outside of the stock ADAS spec. In addition, after doing injection testing, we’ve written an extra layer of safety in the panda limiting how these messages can be used.

Remember that in a level 2 system, doing nothing is always a safe option. You must never rely on your car to take or maintain an action, you can only rely on it to not do things like jerk the wheel or keep acting after you’ve stepped on the brake.

The beauty of this safety model is that none of openpilot’s functional safety depends on the neural network, or even anything running on the EON. So feel free to mess with models, UI, tuning, controls, device hardware, or sensors. Leave the panda code and the driver monitoring alone, and while safety overall is a holistic thing, the functional safety will remain intact with many different modifications.