Skip to content

Commit

Permalink
feat(nebula_ros): add json schema (#153)
Browse files Browse the repository at this point in the history
* fix(hesai_hw_interface): add error handling

* check PTC command error codes, throw exception if necessary
* perform size checks before parsing responses
* emit errors on too-large payload size

* fix(hesai_decoder): print config instead of config address

* refactor(nebula_ros): combine Hesai wrapper nodes into one

This is step one of the single node refactoring of Nebula.
In this step, only the three Hesai wrapper nodes were combined into one, and no optimizations have been done that are made possible by this refactoring.

The next step is to do those optimizations (e.g. get rid of unnecessary pointcloud copying).

* refactor: remove pandarScan pub/sub, decode one packet at a time

* reword later

* temporary progress

* fix(hesai_ros_wrapper): remove changes wrongly merged during rebase

* fix(hesai_ros_wrapper): increase packet buffering to stop internal packet loss

* feat(nebula_common): add instrumentation tools

* feat(hesai_ros_wrapper): instrument code on critical path

* disable instrumentation

* feat(hesai): move received buffer into ros message instead of copy

* feat(hesai_ros_wrapper: add thread-safe queue between udp receiver and decoder, decode in separate thread

* feat(nebula_common): more instrumentation tools

* fix: update link to transport_drivers fork

* fix(hesai_hw_monitor_ros_wrapper): fixed wrong range given for S/N copy

* update GitHub PR view

* fix(hesai): print uint8, uint16 as numbers

* refactor(mt_queue): make variables more readable

* perf(hesai_ros_wrapper): update queue capacity to alleviate packet drops on ECU

* chore(hesai_ros_wrapper): remove unused pub/sub

* refactor(hesai_ros_wrapper): clean up control flow, member variables

* feat(hesai_ros_wrapper): publish/subscribe to legacy pandar_packets on demand

* change launch file back to single-threaded container

* attempt to make queue contention less bad

* chore(hesai_ros_wrapper): reduce logging output

* feat(hesai_ros_wrapper): print warning when connected to HW and receiving /pandar_packets

* feat(nebula_launch.py): throw error when trying to launch Hesai sensor and refer to hesai_launch_all_hw.xml

* chore: update cspell ignore

* refactor(nebula_ros): split single wrapper file into 3 sub-wrappers

* chore(velodyne_calibration_decoder): fix spelling

* chore(nebula_common): remove debug code

* chore(velodyne_calibration_decoder): fix spelling

* fix(hesai_decoder):  initialize last_phase to prevent empty pointcloud published on startup

* fix(nebula_tests): make Hesai tests compile again

* fix(nebula_examples): make Hesai examples compile again

* chore(velodyne_calibration_decoder): fix spelling once and for all

* chore(velodyne_calibration_decoder): fix spelling once and for all

* fix(nebula_examples): fix test failure (remove ament_lint_common)

* fix(hesai_hw_interface): add missing check for PTC error, make error type more readable

* refactor(hesai): re-introduce parameter update mechanism

* feat(hesai_ros): add watchdog timer for pointcloud output

* fix(hesai): change to possibly more accurate high_resolution_clock

* fix(hesai): fix crash on QT128

* refactor(hesai_hw_interface): refactor repeated error handling code

* fix(hesai_launch_all_hw.xml): set valid RPM when AT128 is selected

* refactor(hesai_decoder): remove redundant arguments for correction/calibration

* refactor(nebula_ros): move mt_queue to common

* refactor(velodyne)!: unify wrapper nodes. Currently WIP but compiling

* fix(velodyne_decoders): after  the last commit, point clouds weren't reset correctly. fixed  in this  commit

* fix(velodyne_tests): make tests compile with the new decoder API

* fix(velodyne_examples): make examples compile with new decoder API

* feat(velodyne_hw_interface): synchronous, null- and thread-safe HTTP requests

* fix(velodyne_launch_all_hw.xml): refactor to single-node

* fix(velodyne): make hw interface wrapper work with new hw  interface API

* fix(velodyne): implement correct locking behavior in hw monitor wrapper

* fix(nebula_tests): re-cut scans in existing reference data correctly (360deg). has been ~372deg before (#150)

* chore(velodyne_scan_decoder.hpp): explicitly initialize some fields for clarity

* feat(nebula_ros): add velodyne json schema

Signed-off-by: amadeuszsz <[email protected]>

* chore(nebula_ros): update config

Signed-off-by: amadeuszsz <[email protected]>

* chore(nebula_ros): update schema

Signed-off-by: amadeuszsz <[email protected]>

* chore(nebula): add json schema workflow

Signed-off-by: amadeuszsz <[email protected]>

* ci(nebula): use of autoware actions repo

Signed-off-by: amadeuszsz <[email protected]>

* chore(nebula_ros): add schema composition

Signed-off-by: amadeuszsz <[email protected]>

* chore(pre-commit): autoupdate hooks

* fix(hesai_decoder): print config instead of config address

* refactor(nebula_ros): combine Hesai wrapper nodes into one

This is step one of the single node refactoring of Nebula.
In this step, only the three Hesai wrapper nodes were combined into one, and no optimizations have been done that are made possible by this refactoring.

The next step is to do those optimizations (e.g. get rid of unnecessary pointcloud copying).

* refactor: remove pandarScan pub/sub, decode one packet at a time

* reword later

* temporary progress

* fix(hesai_ros_wrapper): remove changes wrongly merged during rebase

* fix(hesai_ros_wrapper): increase packet buffering to stop internal packet loss

* feat(nebula_common): add instrumentation tools

* feat(hesai_ros_wrapper): instrument code on critical path

* disable instrumentation

* feat(hesai): move received buffer into ros message instead of copy

* feat(hesai_ros_wrapper: add thread-safe queue between udp receiver and decoder, decode in separate thread

* feat(nebula_common): more instrumentation tools

* fix: update link to transport_drivers fork

* update GitHub PR view

* refactor(mt_queue): make variables more readable

* perf(hesai_ros_wrapper): update queue capacity to alleviate packet drops on ECU

* chore(hesai_ros_wrapper): remove unused pub/sub

* refactor(hesai_ros_wrapper): clean up control flow, member variables

* fix(hesai_hw_interface): add error handling

* check PTC command error codes, throw exception if necessary
* perform size checks before parsing responses
* emit errors on too-large payload size

* fix(hesai_hw_monitor_ros_wrapper): fixed wrong range given for S/N copy

* fix(hesai): print uint8, uint16 as numbers

* feat(hesai_ros_wrapper): publish/subscribe to legacy pandar_packets on demand

* change launch file back to single-threaded container

* attempt to make queue contention less bad

* chore(hesai_ros_wrapper): reduce logging output

* feat(hesai_ros_wrapper): print warning when connected to HW and receiving /pandar_packets

* feat(nebula_launch.py): throw error when trying to launch Hesai sensor and refer to hesai_launch_all_hw.xml

* refactor(nebula_ros): split single wrapper file into 3 sub-wrappers

* chore: update cspell ignore

* chore(velodyne_calibration_decoder): fix spelling

* chore(nebula_common): remove debug code

* chore(velodyne_calibration_decoder): fix spelling

* fix(hesai_decoder):  initialize last_phase to prevent empty pointcloud published on startup

* fix(nebula_tests): make Hesai tests compile again

* fix(nebula_examples): make Hesai examples compile again

* chore(velodyne_calibration_decoder): fix spelling once and for all

* fix(nebula_examples): fix test failure (remove ament_lint_common)

* fix(hesai_hw_interface): add missing check for PTC error, make error type more readable

* refactor(hesai): re-introduce parameter update mechanism

* feat(hesai_ros): add watchdog timer for pointcloud output

* fix(hesai): change to possibly more accurate high_resolution_clock

* fix(hesai): fix crash on QT128

* refactor(hesai_hw_interface): refactor repeated error handling code

* fix(hesai_launch_all_hw.xml): set valid RPM when AT128 is selected

* refactor(hesai_decoder): remove redundant arguments for correction/calibration

* refactor(nebula_ros): move mt_queue to common

* chore(nebula_examples): change output rosbag format to NebulaPackets, clean up code

* chore(hesai/decoder_wrapper): clarify watchdog behavior in decoder

* chore(hesai/hw_monitor_wrapper): fix typo in diagnostics name

* chore(hesai_ros_decoder_test): fix unclear naming in console output

* chore(nebula_ros): add schema composition for robosense

Signed-off-by: amadeuszsz <[email protected]>

* chore: run pre-commit and implement suggested fixes (not including copyright yet)

* fix(expected.hpp): revert explicit constructors

* chore: remove erroneously added node_modules folder

* chore(gitconfig): exclude node_modules from git

* fix(nebula_tests): missing serialization format

Signed-off-by: amadeuszsz <[email protected]>

---------

Signed-off-by: amadeuszsz <[email protected]>
Signed-off-by: amadeuszsz <[email protected]>
Co-authored-by: Max SCHMELLER <[email protected]>
Co-authored-by: Abraham Monrroy Cano <[email protected]>
Co-authored-by: Maximilian Schmeller <[email protected]>
Co-authored-by: Max Schmeller <[email protected]>
  • Loading branch information
5 people authored May 22, 2024
1 parent 2ec268f commit ad67d06
Show file tree
Hide file tree
Showing 113 changed files with 4,624 additions and 6,019 deletions.
36 changes: 19 additions & 17 deletions .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,47 +6,49 @@
"adctp",
"Adctp",
"AT",
"autosar",
"block_id",
"Bpearl",
"calib",
"DHAVE",
"Difop",
"extrinsics",
"fout",
"gprmc",
"gptp",
"Helios",
"Hesai",
"Idat",
"ipaddr",
"manc",
"memcpy",
"mkdoxy",
"Msop",
"nohup",
"nproc",
"ntoa",
"pandar",
"PANDAR",
"PANDARAT",
"PANDARQT",
"PANDARXT",
"Pdelay",
"Piyush",
"piyushk",
"QT",
"rclcpp",
"schedutil",
"srvs",
"STD_COUT",
"stds",
"struct",
"structs",
"UDP_SEQ",
"vccint",
"Vccint",
"Vdat",
"XT",
"XTM",
"DHAVE",
"Bpearl",
"Helios",
"Msop",
"Difop",
"gptp",
"Idat",
"Vdat",
"autosar",
"srvs",
"manc",
"ipaddr",
"ntoa",
"piyushk",
"Piyush",
"fout"
"yukkysaito"
]
}
}
40 changes: 40 additions & 0 deletions .github/workflows/json-schema-check.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: json-schema-check

on:
pull_request:
workflow_dispatch:

jobs:
check-if-relevant-files-changed:
runs-on: ubuntu-latest
outputs:
run-check: ${{ steps.paths_filter.outputs.json_or_yaml }}
steps:
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: paths_filter
with:
filters: |
json_or_yaml:
- '**/schema/*.schema.json'
- '**/schema/sub/.json'
- '**/config/**/*.param.yaml'
json-schema-check:
needs: check-if-relevant-files-changed
if: needs.check-if-relevant-files-changed.outputs.run-check == 'true'
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Run json-schema-check
uses: autowarefoundation/autoware-github-actions/json-schema-check@v1

no-relevant-changes:
needs: check-if-relevant-files-changed
if: needs.check-if-relevant-files-changed.outputs.run-check == 'false'
runs-on: ubuntu-latest
steps:
- name: Dummy step
run: echo "No relevant changes, passing check"
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,6 @@ site/

# qcreator stuff
CMakeLists.txt.user

# pre-commit
node_modules/
24 changes: 12 additions & 12 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ci:

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0
rev: v4.6.0
hooks:
- id: check-json
- id: check-merge-conflict
Expand All @@ -18,51 +18,51 @@ repos:
args: [--markdown-linebreak-ext=md]

- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.30.0
rev: v0.40.0
hooks:
- id: markdownlint
args: [-c, .markdownlint.yaml, --fix]

- repo: https://github.com/pre-commit/mirrors-prettier
rev: v2.5.1
rev: v4.0.0-alpha.8
hooks:
- id: prettier

- repo: https://github.com/adrienverge/yamllint
rev: v1.26.3
rev: v1.35.1
hooks:
- id: yamllint

- repo: https://github.com/tier4/pre-commit-hooks-ros
rev: v0.4.0
rev: v0.8.0
hooks:
- id: prettier-package-xml
- id: sort-package-xml

- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.8.0.3
rev: v0.10.0.1
hooks:
- id: shellcheck

- repo: https://github.com/scop/pre-commit-shfmt
rev: v3.4.2-1
rev: v3.8.0-1
hooks:
- id: shfmt
args: [-w, -s, -i=4]

- repo: https://github.com/pycqa/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort

- repo: https://github.com/psf/black
rev: 22.1.0
rev: 24.4.2
hooks:
- id: black
args: [--line-length=100]

- repo: https://github.com/PyCQA/flake8
rev: 4.0.1
rev: 7.0.0
hooks:
- id: flake8
additional_dependencies:
Expand All @@ -78,12 +78,12 @@ repos:
]

- repo: https://github.com/pre-commit/mirrors-clang-format
rev: v13.0.0
rev: v18.1.5
hooks:
- id: clang-format

- repo: https://github.com/cpplint/cpplint
rev: 1.5.5
rev: 1.6.1
hooks:
- id: cpplint
args: [--quiet]
Expand Down
4 changes: 2 additions & 2 deletions build_depends.repos
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ repositories:
# TCP version of transport drivers
transport_drivers:
type: git
url: https://github.com/MapIV/transport_drivers
version: boost
url: https://github.com/mojomex/transport_drivers
version: mutable-buffer-in-udp-callback
2 changes: 2 additions & 0 deletions nebula_common/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ ament_auto_add_library(nebula_common SHARED
src/velodyne/velodyne_calibration_decoder.cpp
)

target_link_libraries(nebula_common yaml-cpp)

ament_auto_package()

# Set ROS_DISTRO macros
Expand Down
Loading

0 comments on commit ad67d06

Please sign in to comment.