Open Sourcing openpilot Development Tools

Why?

At this moment, comma.ai is a 12-person company. What we’ve already achieved would not have been possible without the help from a community of developers who shares our objective: make driving better and safer.

openpilot is open source software that provides enhanced driver assistance features for some of the most popular cars on the market. As of today, openpilot supports more than 50 combinations of car models/years, totaling to about 5 million cars in the US alone. We have also collected 7 million miles of driving data around the world.

We have therefore decided to take a step further and release some of the development tools that we use daily at comma. We can’t imagine developing without those tools, so why should we expect others to?

What’s being released

We added the tools to the openpilot GitHub repo under tools. It contains development tools compatible with any computer running Ubuntu 16.04.

What follows is a brief overview of some of the use cases where those tools can be utilized to review driving data, debug, reverse engineer cars and add new features.
The repo’s README should provide all the info you need to get started, as well as detailed information on the specific command lines. If you plan on executing some of the tutorials below, please complete the openpilot-tools setup first. We’ll assume that your current directory is the/tools.

Example 1: replay driving data

Hardware needed: none.

This is the most common scenario that you, as a developer, can encounter: when you were testing chffrplus or openpilot, some feature did not behave as expected. The comma explorer and cabana allow you to have some replay functionalities already: you can visualize what the model output was and what the CAN messages were, but a lot more information was logged and you can now have access to all of it.

The following example describes how you can replay the driving events (almost) exactly as they occurred at test time. Why “almost”? Our logging process cannot guarantee that events are recorded exactly in the same order as they occurred. ROS, an alternative and widely used framework for robotics, describes this well in its wiki.

The first step to replay the driving data is to download the segment(s) that you are interested in from the comma explorer. In case you have no personal driving data but you still want to test the tools, you can download a few driving segments here.

You can replay the data with the command:

python replay/unlogger.py

Example:

python replay/unlogger.py ‘99c94dc769b5d96e|2018-11-14–13-31-42’ /home/batman/unlogger_data
Within /home/batman/unlogger_data:
99c94dc769b5d96e|2018-11-14–13-31-42–0–fcamera.hevc
99c94dc769b5d96e|2018-11-14–13-31-42–0–rlog.bz2

Note that the publicly available driving segments use a different naming convention, which also works.
In another terminal, you can optionally run a debug visualizer:

Set env variable HORIZONTAL if the ui layout is too tall

python replay/ui.py

Together with the replay tools, you can also locally run most of the processes that normally run on your EON, such as controlsd.py and radard.py. Just start them before unlogger.py to prevent it from publishing conflicting outputs.

Example 2: debug car controls

Hardware needed: panda, giraffe, USB joystick.

It’s sometimes convenient to use a joystick when reverse engineering car controls or while testing edge cases.

Use the panda’s OBD-II port to connect with your car (via giraffe) and a USB cable to connect the panda to your PC. Also, connect a USB joystick to your PC.

joystickd.py runs a daemon that reads inputs from a joystick and publishes them over zmq. boardd.py sends the CAN messages from your PC to the panda; debug_controls.py is a mocked version of controlsd.py and uses input from a joystick to send controls to your car.

python carcontrols/joystickd.py# In another terminal:

Make sure the safety setting is hard-coded to ALL_OUTPUT

python replay/boardd.py

In another terminal:

python carcontrols/debug_controls.py

Example 3: stream replayed CAN messages to EON

Hardware needed: 2x panda, debug board, EON Dashcam DevKit.

It is possible to replay CAN messages as recorded during a drive and forward them to an EON sitting on your desk for debug.
Connect 2 pandas to the debug board. A panda connects to the PC, the other panda connects to the EON.

With MOCK=1 boardd will read logged can messages from a replay and send them to the panda.

MOCK=1 openpilot_tools/replay/boardd.py# In another terminal:
python replay/unlogger.py

Example 4: stream EON video data to a PC

Hardware needed: EON Dashcam DevKit, comma Smays.

You can connect your EON to your PC using the Ethernet cable provided with the comma Smays and you’ll be able to stream data from your EON, in real time, with low latency. A useful application is streaming the raw video frames at 20 fps, as captured by the EON’s camera.

ssh into the eon and run loggerd with the flag “–stream”. In ..selfdrive/manager.py you can change:
“loggerd”: (“selfdrive/loggerd”, [”./loggerd”]),

with:

“loggerd”: (“selfdrive/loggerd”, [”./loggerd”, “–stream”]),# On the PC, run the following to receive frames from the EON and re-publish them.

Set PYGAME env variable if you want to display the video stream

python streamer/streamerd.py

Welcomed contributions

• Documentation: code comments, better tutorials, etc…
• Support for other platforms other than Ubuntu 16.04.
• Performance improvements: the tools have been developed on high-performance workstations (12+ logical cores with 32+ GB of RAM), so they are not optimized for running efficiently. For example, ui.py might not be able to run real-time on most PCs.
• More tools: anything that you think might be helpful to you, comma and the whole community. Mentioned products: EON Dashcam DevKit, panda, giraffe, debug board, comma Smays.

Useful resources: openpilot, openpilot-tools, opendbc, panda firmware, openpilot port guide for Toyota, generic openpilot port guide.