-
Notifications
You must be signed in to change notification settings - Fork 9.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge remote-tracking branch 'upstream/master' into boardd-no-fault
- Loading branch information
Showing
77 changed files
with
5,632 additions
and
85 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Submodule cereal
deleted from
d0ceac
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| # What is cereal? [](https://github.com/commaai/cereal/actions) [](https://codecov.io/gh/commaai/cereal) | ||
|
|
||
| cereal is both a messaging spec for robotics systems as well as generic high performance IPC pub sub messaging with a single publisher and multiple subscribers. | ||
|
|
||
| Imagine this use case: | ||
| * A sensor process reads gyro measurements directly from an IMU and publishes a `sensorEvents` packet | ||
| * A calibration process subscribes to the `sensorEvents` packet to use the IMU | ||
| * A localization process subscribes to the `sensorEvents` packet to use the IMU also | ||
|
|
||
|
|
||
| ## Messaging Spec | ||
|
|
||
| You'll find the message types in [log.capnp](log.capnp). It uses [Cap'n proto](https://capnproto.org/capnp-tool.html) and defines one struct called `Event`. | ||
|
|
||
| All `Events` have a `logMonoTime` and a `valid`. Then a big union defines the packet type. | ||
|
|
||
| ### Best Practices | ||
|
|
||
| - **All fields must describe quantities in SI units**, unless otherwise specified in the field name. | ||
| - In the context of the message they are in, field names should be completely unambiguous. | ||
| - All values should be easy to plot and be human-readable with minimal parsing. | ||
|
|
||
| ### Maintaining backwards-compatibility | ||
|
|
||
| When making changes to the messaging spec you want to maintain backwards-compatibility, such that old logs can | ||
| be parsed with a new version of cereal. Adding structs and adding members to structs is generally safe, most other | ||
| things are not. Read more details [here](https://capnproto.org/language.html). | ||
|
|
||
| ### Custom forks | ||
|
|
||
| Forks of [openpilot](https://github.com/commaai/openpilot) might want to add things to the messaging | ||
| spec, however this could conflict with future changes made in mainline cereal/openpilot. Rebasing against mainline openpilot | ||
| then means breaking backwards-compatibility with all old logs of your fork. So we added reserved events in | ||
| [custom.capnp](custom.capnp) that we will leave empty in mainline cereal/openpilot. **If you only modify those, you can ensure your | ||
| fork will remain backwards-compatible with all versions of mainline cereal/openpilot and your fork.** | ||
|
|
||
| ## Pub Sub Backends | ||
|
|
||
| cereal supports two backends, one based on [zmq](https://zeromq.org/) and another called [msgq](messaging/msgq.cc), a custom pub sub based on shared memory that doesn't require the bytes to pass through the kernel. | ||
|
|
||
| Example | ||
| --- | ||
| ```python | ||
| import cereal.messaging as messaging | ||
|
|
||
| # in subscriber | ||
| sm = messaging.SubMaster(['sensorEvents']) | ||
| while 1: | ||
| sm.update() | ||
| print(sm['sensorEvents']) | ||
|
|
||
| ``` | ||
|
|
||
| ```python | ||
| # in publisher | ||
| pm = messaging.PubMaster(['sensorEvents']) | ||
| dat = messaging.new_message('sensorEvents', size=1) | ||
| dat.sensorEvents[0] = {"gyro": {"v": [0.1, -0.1, 0.1]}} | ||
| pm.send('sensorEvents', dat) | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| Import('env', 'envCython', 'arch', 'common', 'messaging') | ||
|
|
||
| import shutil | ||
|
|
||
| cereal_dir = Dir('.') | ||
| gen_dir = Dir('gen') | ||
| other_dir = Dir('#msgq/messaging') | ||
|
|
||
| # Build cereal | ||
| schema_files = ['log.capnp', 'car.capnp', 'legacy.capnp', 'custom.capnp'] | ||
| env.Command(["gen/c/include/c++.capnp.h"], [], "mkdir -p " + gen_dir.path + "/c/include && touch $TARGETS") | ||
| env.Command([f'gen/cpp/{s}.c++' for s in schema_files] + [f'gen/cpp/{s}.h' for s in schema_files], | ||
| schema_files, | ||
| f"capnpc --src-prefix={cereal_dir.path} $SOURCES -o c++:{gen_dir.path}/cpp/") | ||
|
|
||
| # TODO: remove non shared cereal and messaging | ||
| cereal_objects = env.SharedObject([f'gen/cpp/{s}.c++' for s in schema_files]) | ||
|
|
||
| cereal = env.Library('cereal', cereal_objects) | ||
| env.SharedLibrary('cereal_shared', cereal_objects) | ||
|
|
||
| # Build messaging | ||
|
|
||
| services_h = env.Command(['services.h'], ['services.py'], 'python3 ' + cereal_dir.path + '/services.py > $TARGET') | ||
| env.Program('messaging/bridge', ['messaging/bridge.cc'], LIBS=[messaging, 'zmq', common]) | ||
|
|
||
|
|
||
| socketmaster = env.SharedObject(['messaging/socketmaster.cc']) | ||
| socketmaster = env.Library('socketmaster', socketmaster) | ||
|
|
||
| Export('cereal', 'socketmaster') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| import os | ||
| import capnp | ||
|
|
||
| CEREAL_PATH = os.path.dirname(os.path.abspath(__file__)) | ||
| capnp.remove_import_hook() | ||
|
|
||
| log = capnp.load(os.path.join(CEREAL_PATH, "log.capnp")) | ||
| car = capnp.load(os.path.join(CEREAL_PATH, "car.capnp")) | ||
| custom = capnp.load(os.path.join(CEREAL_PATH, "custom.capnp")) |
Oops, something went wrong.