openpilot 0.9.8

As an openpilot user, this release brings you Gas Gating, a 10ms e2e latency reduction, a new Toyota longitudinal tune, and openpilot longitudinal control for Fords. You’ll also notice reduced driver monitoring false positives and your device running cooler thanks to an optimized image processing pipeline.

Under the hood, we’ve rewritten large parts of the stack to prepare for the next set of releases, where we’ll ship our biggest driving models yet. openpilot’s lateral stack is very mature with an end-to-end policy and auto-tuned controls. openpilot’s longitudinal control is less mature and will be the focus of the next set of releases, starting with Gas Gating to bring a bit of Experimental mode to Chill mode.

Gas Gating

The new Gas Gating feature makes driving in Chill mode smoother and more natural. Using the driving model’s predictions of when a human driver would press the gas or brake pedals (which are learned end-to-end as always), openpilot intelligently refrains from applying acceleration in situations where it wouldn’t be appropriate, like approaching a traffic light or taking an exit. This helps reduce unnecessary control inputs and results in a driving experience that’s both more comfortable and closely aligned with human intuition.

When it triggers, the path fades to grey. Instead of accelerating after this lead changes lanes, the car comfortably coasts until the user takes over for the red light.

Driver monitoring improvements

New driver monitoring model on GPU

Like the driving model, the DM model now fully runs on the GPU using tinygrad (#33397). This allows us to use a unified model runner in openpilot and more importantly, not worry about the quirks and limitations (for example, unsupported operations and loss of precision due to quantization) of using SNPE running at UINT8 on the DSP. Since the image transformation is also done on the GPU, one fewer copy of the input buffer is required.

With the new model we also retuned the end-to-end phone policy to reduce false positives. More DM chillness is expected.

Always-On Driver Monitoring toggle

It is fully optional, but for those who may find DM useful even with openpilot not engaged, it can now be toggled to always-on mode (#32205). There’s some filtering for low speeds (#33644) to allow alert-free maneuvers such as parking and pulling out of driveways.

Improved model runtime

Image processing pipeline moved to the ISP

Until now, openpilot used the GPU to process raw Bayer images from the cameras. While this gives a lot of flexibility, the GPU is a limited resource that should be used for running models!

The new ISP pipeline maintains all the same processing steps and produces indistinguishable images, all with 500mW less power draw and no GPU time. The ISP also processes both road-facing cameras in 0.1ms, which saves 10ms on openpilot’s end-to-end latency.

This is likely the first non-Android Snapdragon ISP driver, and hopefully the Qualcomm mainline kernel group is able to use some of this work to upstream more ISP support. Our driver supports both the IFE (#33720) and ICP/BPS (#33763) found in newer Qualcomm ISPs.

tinygrad runner

With the limited compute we’re able to stick to a windshield, optimizing the model runner has always been important. First, we used Qualcomm’s SNPE, then we wrote thneed to accelerate SNPE, and for the last couple years, tinygrad has been eating away at that stack.

Thanks to tinygrad’s new QCOM driver, we were able to fully switch the whole stack to tinygrad for both comma 3X and PC. Currently this simplifies the model runtime in terms of lines and dependencies, but in the future, this allows us to use any future tinygrad optimizations to ship bigger driving models. Soon, tinygrad will also support plugging a GPU directly into the comma 3X’s auxiliary USB-C port for really big models.

Improved testing

Previously, the model runners were tested with a simple reference test that asserted nothing accidentally changed during refactors. While that test sounds nice, it rarely failed, gave no coverage around actual driving behavior, and required manual updating when shipping a new model.

Shipping new models should be easy and fun, so we replaced that test with two new ones. The first removes the assert in the reference test and simply posts comparison plots in the PR description for the author to sanity check.

The second is an end-to-end driving test that spins up the whole openpilot stack inside the MetaDrive simulator and ensures openpilot can drive a pre-defined track. While our internal training stack has tons of driving behavior tests, this is the first in openpilot’s CI, and it covers everything from the model weights to openpilot’s IPC to the CAN messages sent out.

GPS-free Localizer

The localizer in openpilot estimates the current motion and orientation of the vehicle with a Kalman filter. The outputs are then used to calculate vehicle parameters such as steering angle offset and vehicle roll, which directly influence vehicle control.

In rare cases, erratic GPS measurements could degrade the localizer’s performance. In this release, GPS was removed as an input to this localizer (#33029), ensuring that faulty GPS measurements can never negatively affect vehicle control. This refactor also makes the localizer simpler and easier to maintain.

Longitudinal Maneuver Runner

This tool captures the full range of car-specific longitudinal behaviors in a quick, repeatable, and comparable report. Instead of basing proposed changes off of subjective driving feel, we can compare reports from before and after for a complete picture of the difference. This tool was used while making openpilot longitudinal the default for Ford Q3 (opendbc#1411) and improving the Toyota TSS2 longitudinal tune (opendbc#1513).

We have a repository of reports from users who have benchmarked their cars here. Explore the reports and try it out on your car. Instructions can be found here.

New Toyota TSS2 longitudinal tune

A long-time complaint was harsh braking and slow acceleration. We looked at how the Toyota powertrain control module (PCM) responds to acceleration requests and compensated for its consistent inaccuracies. For example, we observed that the PCM would often actuate more braking than was intended leading to an uncomfortable experience in stop and go.

To combat this we now apply a simple derivative layer in the Toyota car controller that winds down the acceleration request as the ego approaches the setpoint to prevent overshoot (opendbc#1559). Additionally, we found a bit in the ACC_CONTROL message that improves the laggy gas response (opendbc#1301). Its name is literally PMTBRKG (permit braking).

As seen in the plots, this approach still isn’t perfect, but it eliminates openpilot’s integral wind up that would occur due to the slow response by the car, and enables future improvements as we understand more about the PCM.

Firehose Mode

In preparation for training SOTA-scale diffusion and driving models, we’re scaling up our data ingestion pipeline 100x.

With Firehose Mode, your comma 3X connects directly to our backend and uploads as much data as it can. To switch on the “firehose,” simply connect your device to an unmetered internet connection (usually Wi-Fi) and a good power source.

Forks in the training set

Like Linux and Android, many comma 3X users run forks rather than the mainline distribution.

But their data is still valuable, so we’ve set up guidelines for fork maintainers who want to get into the training set. sunnypilot is the first fork ready for the training set, and hundreds of their users have already contributed data.

1m CI

To combat the phenomenon in codebases where CI gets slower and flakier over time, we introduced two new constraints: the rerun button is disabled and all tests have a one minute timeout.

Why one minute? It’s an arbitrary limit, but it also happens to be the minimum timeout in GitHub Actions. Now you can merge a PR in the time it takes to grab a drink from the fridge, all with the same coverage and improved reliability.

AGNOS 11

AGNOS 11 updates the base operating system on the comma 3X from Ubuntu 20.04 to 24.04 (agnos-builder#262), thanks to @andiradulescu.

QDL Mode

This AGNOS also removes Fastboot for three seconds of boot time savings. Flashing over QDL also makes your comma 3X unbrickable, since it’s implemented in the boot ROM, and it removes the comma Fastboot dance.

We wrote QDL.js, a JavaScript library for low level flashing of Qualcomm devices, to enable this switch for flash.comma.ai.

Cars

opendbc is the new home for cars

Up until now, porting a new car required working with three repositories: panda for safety code, opendbc for CAN message definitions, and openpilot for car interfacing code. Car safety (opendbc#1713) and interfacing (opendbc#1049) have been moved to opendbc for a streamlined car porting experience. Along with opendbc’s new 1-minute CI, this unification massively speeds up development time.

opendbc is now a fully self-contained project for interfacing with over 300 cars. Get started with:

pip install opendbc

Safety mutation testing

We’re excited to introduce safety mutation testing (panda#2040) to the test suite. In the same vein as the safety coverage report added in 0.9.5, this will allow us to catch safety code paths that are either not fully tested or written in an unmaintainable way.

Rivian

Rivian R1T and R1S 2022-24 (opendbc#1613) are now supported thanks to lukasloetkolben!

Ford Q4

Ford F-150, F-150 Hybrid, Ranger, and Mustang Mach-E are now supported!

Ford Q4 platform vehicles have more torque available than their Q3 counterparts. We implemented a new safety mechanism to limit the max curvature based on an estimated lateral acceleration dependent on speed (opendbc#1852).

Ford Q3

This release brings two enhancements to the Ford Q3 platform. The low speed steering rate limit was increased (panda#2065) and openpilot longitudinal is now default, allowing you to try out Experimental mode.

This marks the first new brand to gain openpilot longitudinal in release since GM was ported in 2018. This was enabled by the rigorous safety tests and refactors we’ve implemented in the last few releases. As usual for release, stock Automatic Emergency Braking is maintained while openpilot is in control.

Join the community

openpilot is built by comma together with the openpilot community. Get involved in the community to push the driving experience forward.

• Join the community Discord
• Post your driving feedback in #driving-feedback on Discord
• Upload your driving data with Firehose Mode
• Get started with openpilot development

comma is also hiring if you want to work on openpilot full-time.