Getting started with openpilot

“How do I get started with openpilot?” is a question we get asked a lot. So in this blog post we’ll answer that. We want to make it as easy as possible for people to experiment with, and potentially contribute to the openpilot ecosystem.

openpilot in a car

The main purpose of openpilot is to provide great quality ADAS functionality on a range of supported cars. If you have a supported car, driving with a comma device is a mostly plug-and-play experience. If you want to develop on openpilot, being a user is a good place to start to understand how it all works.

Forking

Forking is one of the great perks of open-source projects. The comma 3X is a great development environment that is designed to make tinkering with openpilot easy. It runs a customized Ubuntu 20.04 LTS that you can SSH into, with most of the stuff you are used to on a normal Linux system. We also architected openpilot such that as long as you do not mess with driver monitoring or panda safety you are free to do almost anything without having to be concerned about violating the safety model. You can change the color schemes, the UI, the tuning for your car, the longitudinal policy, and much more!

Porting a new car

A lot of cars in openpilot are initially supported and maintained due to community effort. If you’re interested in porting your own car, or just looking to understand the process, check out community contributor Jason Young’s talk during COMMA_CON 2023 which gives some insight about every step of the process.

Generally, if the car has Adaptive Cruise Control (ACC) and a Lane Keeping Assist System (LKAS), it can be ported with some exceptions noted here. Difficulty of the port depends on a few factors, such as similarity to other existing supported cars, if it is a facelift or new vehicle generation, or a new vehicle brand altogether.

While developing a port with a comma device, some tools that the openpilot team uses daily are listed below:

• cabana is a tool for reverse engineering CAN messages.
• test_car_model.py is a script that lets you leverage the car test suite that is run on every openpilot commit. It ensures validity of the defined CAN signals, checks for panda state and safety issues, and more.

openpilot on a PC

All parts of openpilot can run on a PC. You can read about how to set it up on your PC here.

Running a replay

The best place to start when openpilot is installed on your PC is to replay an openpilot drive. You can replay any drive you have access to (a demo drive is available to all). This replays all the messages that were sent by openpilot services at runtime. Run ui to see the openpilot UI as you would when it was running in the car. With replay running, you can run and experiment with other services, experiment with modifying the UI, etc…

Playing the CTF

To get a slightly deeper understanding of openpilot, you can play our CTF. It takes you through a few different parts of openpilot and is loads of fun.

Running openpilot in simulation

Since openpilot can run on PC, it can also be simulated in-the-loop. In openpilot master there is an interface to the CARLA simulator than can be used for this. Several people have ported openpilot to other simulators as a fun project. Here is a port of openpilot to GTA 5 and American Truck Simulator.

openpilot on a robot

Billy is a comma device attached to a comma body, which makes it a wheeled robot. The comma 3X is a great development kit for robotics, it has cameras, USB ports, microphones, speakers, a screen, and significant compute for running neural networks. The robot can be remote controlled with tools in openpilot. If you are interested in a comma body, keep in mind it is a dev-kit and not plug-and-play.

Contributing to mainline openpilot

Contributing to mainline openpilot when done right is satisfying and helpful. The best places to contribute are environment setup (supporting macOS, Ubuntu 23.04, etc…), documentation, tools, simulation, fixes for your car, and car ports. Particularly if you experience any issues or confusion when trying anything mentioned in this blog post, whether that’s installing openpilot on your machine or running a fork, think about how you can make it easier for the next person. Generally speaking, PRs that affect the driving or UI experience of all users are unlikely to get merged. It is impractical for external contributors to do the testing that is required. One extremely successful example of an external contributor is deanlee. Deanlee has been consistently fixing bugs and improving small things for years and has a whopping 12% of all openpilot commits.

openpilot-adjacent projects

A few other things worth exploring are the other repos we have open-sourced specifically to give people something experiment with:

• Our hiring challenge
• Our machine learning simulator dataset and challenge
• Our driving dataset with 2000minutes of openpilot logs
• Our car controls dataset with ~12500 hours of car steering logs
• Our dataset of 10k images of perfectly labeled driving images for semantic segmentation