openpilot 0.9.7

5 minute read

New driving model

This release includes improvements in the driving model. In 0.9.6, we made lateral planning completely end-to-end, with the model directly predicting a desired curvature. In this release, we made the model aware of its previous predicted curvature by feeding it back as an input (#31616). This made the model produce smoother lateral trajectories and more precise predictions in and around curves.

We also simplified the driving model architecture by removing the scaling layers that were used as last layers (#32309). These layers were highly sensitive to other training hyperparameters such as the batch size and the learning rate, making them unreliable in some training settings.

Bug Fixes

  • Apply augmentation to the desire input (lane changes, turns, etc.) during the policy model training (#31883)
  • Exclude some layers from weight decay regularization (#32141)

In 0.9.4, we introduced Navigate on openpilot in Experimental mode for prime & lite users. This was done using additional inputs to the driving model (picture of the map, navigation instructions). However, the driving model’s ability to execute the maneuvers required to follow navigation was limited, which made the experience unreliable.

In this release, we temporarily remove Navigate on openpilot from Experimental mode in order to focus on improving the driving model’s capabilities. Removing this feature allows us to reduce the complexity of the training stack, which makes us iterate faster on solving core problems. We plan on bringing it back in a better state as soon as these improvements are achieved.

Note: the openpilot turn-by-turn navigation feature is still available for prime & lite users and is not affected by this change.

New driver monitoring model

On rare occasions, namely when a passenger is present and appears distracted, the previous model would sometimes misclassify the driver as distracted instead. This was caused by training data bugs where the generated ground truths for the end-to-end phone bit were noisy and in some cases correlated more with the passenger instead of the driver.

To “fix” the training data, we spun up a local multimodal LLM server and asked it to specifically pay attention to the driver and pick out the aforementioned false positives in our training set. The resulting model (#32431) is thus much improved with these false positives and phone detection accuracy in general.

New route format

Instead of the current date and time, routes now have a monotonic counter and unique ID. This entirely removes openpilot’s dependence on time, which saves 500 lines of code (#31829, panda#1897) and eventually parts and steps in comma 3X production.

Before:
d9b97c1d3b8c39b2/2024-03-05--15-04-33

Now:
d9b97c1d3b8c39b2/0000002b--669b2ffc96

Discord Refactor

#current-projects is a new Discord forum to host discussions for longer term projects on the order of weeks to months. Currently, we’re discussing the new API for longitudinal control, abstracting out the car interface code from openpilot, getting rerun.io set up, and more. Join the discussion at discord.comma.ai!

There’s also a new #dev-connect-web channel for discussing comma connect and other comma web products. Along with that, we set up live previews for all connect pull requests for easy sharing and testing.

Cars

Follow distance button

Although openpilot doesn’t have an adjustable follow distance, there’s a “driving personality” setting that is analogous to follow distance. openpilot now uses the follow distance button on your steering wheel to toggle between the driving personalities, and it will be reflected on your car’s dash (#31792).

Fingerprinting without OBD-II CAN

When we started querying the car’s ECUs for their firmware versions, it made sense to start with the OBD-II port since all ECUs are accessible there for diagnostics. Now that we have all the ECUs logged for all ~300 supported cars, we know exactly which ECUs we need to uniquely identify a supported car, and we’re able to query only the relevant ECUs, which also happen to be accessible from the car harness. This is the first step towards making the comma power optional.

Fuzzy fingerprinting for Ford and Volkswagen

This release improves vehicle identification for Ford and Volkswagen. Unlike in the past, where in-house analysis of users’ fingerprints was needed, these two projects were driven by the community! Special thanks to community members jyoung8607 (Volkswagen) and incognitojam (Ford).

See the Ford pull request below to check out how external analysis on real user routes was performed using our open dataset, commaCarSegments.

  • Ford: improved fuzzy fingerprinting by using the platform code and model year hint (#31124)
  • Volkswagen: improved fuzzy fingerprinting by using the chassis code from the VIN (#32148)

A table of recent devices that didn’t immediately fingerprint on their first drive. Green highlights cases for the brands we improved in this release.

card

card is a new daemon split out from controlsd. It takes in the raw stream of CAN bytes from the car, and it builds the CAN messages to control the car. All car-specific code paths now live in card (#32380). Soon, we’ll pull all the car interfacing code out of openpilot and into opendbc, where openpilot will be a consumer of a generic library for talking to cars.

Simpler car ports

Previously, most car attributes were split across a few files. In #31542 we introduced a new data structure that holds all the information needed to port a new car, simplifying the process.

Here’s a car port for the Lexus LC from before the refactor and what it looks like now on master. Just 4 lines!

LEXUS_LC_TSS2 = ToyotaTSS2PlatformConfig(
  [ToyotaCarDocs("Lexus LC 2024")],
  CarSpecs(mass=2041., wheelbase=2.87, steerRatio=13.0, tireStiffnessFactor=0.444),
)

Bug Fixes

  • Ford: handle metric cruise set speed (#31463)
  • Hyundai CAN FD: fix inconsistent FW query without comma power (#32627)

Enhancements

  • Honda: fingerprint without the OBD-II port #31926
  • Hyundai CAN: fingerprint without the OBD-II port (#31969)
  • Hyundai: speed up FW query (#32022)
  • Kia: allow fuzzy fingerprinting for two new models (#31882)
  • Toyota: speed up FW query (#32235)

Car Ports

  • Support for hybrid variants of supported Ford models (#31568)
    • Ford Escape Hybrid 2020-22
    • Ford Escape Plug-in Hybrid 2020-22
    • Ford Explorer Hybrid 2020-23
    • Ford Focus Hybrid 2018
    • Ford Kuga Hybrid 2020-22
    • Ford Kuga Plug-in Hybrid 2020-22
    • Ford Maverick Hybrid 2022-24

Updated:

You may also enjoy

Autonomy

6 minute read

The autonomy team’s goal is to build robotic agents. Agents that are open-source, useful, and that you can fully own. Our main product and current focus is b...