Skip to content

Releases: dora-rs/dora

Dora v0.3.7

18 Nov 05:08
@haixuanTao haixuanTao
v0.3.7
efe6cc9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Dora v0.3.7 Latest
Latest

Making Dora simpler

  • dora run command now removes the need for dora up and removes the need for background process: #703
dora run examples/camera/dataflow.yml
  • You can now build, run and start URL dataflow. #682
dora build/start https://raw.githubusercontent.com/dora-rs/dora/refs/heads/main/examples/camera/dataflow.yml
  • You can now pip install the dora cli for Linux, as well as all rust written nodes. #706
pip install dora-rs-cli
pip install dora-rerun

Releasing additional nodes

  • Vision Language Model (VLM): Qwenvl2 with data recording and finetuning with llama factory
dora build examples/vlm/dataflow.yml
dora run examples/vlm/dataflow.yml
Screencast.from.2024-11-18.04-23-10.webm
  • Audio: Whisper Turbo, Sidero Voice Activity Detection(VAD), Opus translation

speech-to-text example:

dora build examples/speech-to-text/dataflow.yml
dora run examples/speech-to-text/dataflow.yml
Screencast.from.2024-11-18.04-28-33.webm

translation example:

dora build examples/translation/dataflow_en_fr.yml
dora run examples/translation/dataflow_en_fr.yml
  • visualisation: Dora rerun with 3D visualisation. Time series and now in pip.
dora build examples/piper/play_dummy_inference.yml
dora run examples/piper/play_dummy_inference.yml
Screencast.from.2024-11-18.05-30-19.webm

Please checkout piper examples.

  • Hardware: Agilex Piper sdk, Astra camera with PyOrbbeckSdk and tracer mobile base

Known Issue still present in 0.3.7

  • Python dataflow does not stop gracefully #625
  • Queue_size is not properly respected #652

What's Changed

New Contributors

Contributors

phil-opp, renovate, and 6 other contributors
Assets 10
Loading

v0.3.7rc2

15 Nov 06:54
@haixuanTao haixuanTao
v0.3.7rc2
95f09e0
Compare
Choose a tag to compare
v0.3.7rc2 Pre-release
Pre-release 
Fix piper version
Assets 10
Loading

v0.3.7rc1

11 Nov 10:18
@haixuanTao haixuanTao
v0.3.7rc1
47fa3ab
Compare
Choose a tag to compare
v0.3.7rc1 Pre-release
Pre-release 
Fix CI/CD for pip release
Assets 10
Loading

v0.3.7rc0

13 Oct 13:18
@haixuanTao haixuanTao
v0.3.7rc0
4e51cd8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.7rc0 Pre-release
Pre-release

What's Changed

New Contributors

Full Changelog: v0.3.6...v0.3.7rc0

Contributors

phil-opp, renovate, and 4 other contributors
Assets 21
Loading

v0.3.6

28 Aug 05:01
@haixuanTao haixuanTao
v0.3.6
c6abec2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.6

Dora v0.3.6 Release

New Website Frontpage!

We've been doing a lot of demo at dora, and it was about time to put them in front so that people get a better understanding of what dora-rs stand for

https://dora-rs.ai/

Preview

Screenshot from 2024-08-28 11-20-26

Quicker installer!

As we're adding support for more platform such as linux arm musl, it was time to make installation simpler. So we copied other project installation script so that people can install dora in one line:

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/dora-rs/dora/main/install.sh | bash

Shipping the Node-Hub!

We have released a node-hub, that should make it a lot easier to get started with dora.

The idea is that, we will be able to directly pull pre-packaged node from pip and cargo and use them in our own project.

For example to get started with a video stream, instead of writing code or copying code from the dora project you can just write:

nodes:
  - id: camera
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/20
    outputs:
      - image
    env:
      CAPTURE_PATH: 0
      IMAGE_WIDTH: 640
      IMAGE_HEIGHT: 480

  - id: plot
    build: pip install opencv-plot
    path: opencv-plot
    inputs:
      image:
        source: camera/image
        queue_size: 1

Then, use:

dora build dataflow.yml
dora up
dora start dataflow.yml

This should reduce the necessity to copy paste code from one place to another as well as reduce the risk of path issues.

You can copy our CI/CD github worklfow : https://github.com/dora-rs/dora/blob/main/.github/workflows/node-hub-ci-cd.yml so that your nodes within your node-hub folder are tested and published on cargo and pip.

Dora Node should now be compatible between version from 0.3.6 forward!

Thanks to @phil-opp, we know have an independant dora-message crate that will be responsible for the communication of messages, and should be able to handle dora nodes from multiple version as long as dora-message version are compatible.

C++ Documentation :)

Thanks to @starlitxiling, we now have a first experimental C++ documentation to make it easier for people to get started in C++

https://dora-rs.ai/cpp/cpp-api.html

Preview

Screenshot from 2024-08-28 11-22-09

Fix dynamic node initialisation issues

We have noticed that dora sometimes had issues with initializing dynamic node and this release should fix this issue.

What's Changed

New Contributors

Full Changelog: v0.3.5...v0.3.6

Contributors

phil-opp, renovate, and 6 other contributors
Assets 21
Loading

v0.3.6-rc0

17 Aug 04:29
@haixuanTao haixuanTao
v0.3.6-rc0
1b7e544
Compare
Choose a tag to compare
v0.3.6-rc0 Pre-release
Pre-release

What's Changed

New Contributors

Full Changelog: v0.3.5-fix7...v0.3.5.6-rc0

Contributors

phil-opp, renovate, and 5 other contributors
Assets 21
Loading

v0.3.5-fix7

15 Jul 03:19
@haixuanTao haixuanTao
v0.3.5-fix7
39d7d49
Compare
Choose a tag to compare
v0.3.5-fix7 Pre-release
Pre-release 
Add installl shell
Assets 21
Loading

v0.3.5-fix6

06 Jul 11:25
@haixuanTao haixuanTao
v0.3.5-fix6
76223ee
Compare
Choose a tag to compare
v0.3.5-fix6 Pre-release
Pre-release 
Zip only binary
Assets 10
Loading

v0.3.5

03 Jul 16:46
@haixuanTao haixuanTao
v0.3.5
110ce99
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.5

dora 0.3.5 Release

Better Error Logging and Graceful Stopping

Before

018ffe79-1c0c-7b68-8886-d8cff079c232
  2024-06-09T19:28:22.877741Z ERROR dora_daemon::spawn: 018ffe79-1c0c-7b68-8886-d8cff079c232/webcam: 
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 11, in <module>
    node = Node()
RuntimeError: failed to init event stream

Caused by:
    subscribe failed: Some nodes exited before subscribing to dora: {NodeId("plot")}

    This is typically happens when an initialization error occurs
                    in the node or operator. To check the output of the failed
                    nodes, run `dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 plot`.

Location:
    apis/rust/node/src/event_stream/mod.rs:90:17

    at binaries/daemon/src/spawn.rs:371


  2024-06-09T19:28:23.263843Z ERROR dora_daemon: 
    018ffe79-1c0c-7b68-8886-d8cff079c232/object_detection failed with exit code 1.

    Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 object_detection
                            
    at binaries/daemon/src/lib.rs:1077

  2024-06-09T19:28:23.264003Z ERROR dora_coordinator: some nodes failed:
                            
  - webcam: 
    018ffe79-1c0c-7b68-8886-d8cff079c232/webcam failed with exit code 1.

    Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 webcam
                            

Location:
    /home/peter/Documents/work/dora/binaries/coordinator/src/listener.rs:90:56
    at binaries/coordinator/src/lib.rs:279

dataflow failed: errors occurred in dataflow 018ffe79-1c0c-7b68-8886-d8cff079c232:
- machine ``:
    some nodes failed:
      - webcam: 
        018ffe79-1c0c-7b68-8886-d8cff079c232/webcam failed with exit code 1.
    
        Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 webcam
                                
    
    Location:
        /home/peter/Documents/work/dora/binaries/coordinator/src/listener.rs:90:56

After

01907807-9461-738c-a503-c92b82e0d4c2
attaching to dataflow (use `--detach` to run in background)
ERROR webcam: exited with code 1 with stderr output:
---------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 12, in <module>
    assert False, "could not start dataflow"
AssertionError: could not start dataflow
---------------------------------------------------------------------------------

INFO  plot: node finished successfully
INFO  object_detection: node finished successfully


[ERROR]
Dataflow 01907807-9461-738c-a503-c92b82e0d4c2 failed:

Node `webcam` failed: exited with code 1 with stderr output:
---------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 12, in <module>
    assert False, "could not start dataflow"
AssertionError: could not start dataflow
---------------------------------------------------------------------------------

Dynamic Node

We're introducing dynamic node, to run node in a native way, whether it is by calling python, jupyter or cargo. All you have to do is specify that the node is dynamic in the dataflow and specify the node id within the node code and you will be able to run the node natively without using dora start.

Warning

We have noticed some bugs on MacOS/Windows with dynamic node: ##575

Getting Started

You can try minimal example:

cd examples/python-dataflow
dora start dataflow_dynamic.yml 
# In another terminal
python plot_dynamic.py
  • Then starting the dataflow
dora start dataflow.yaml
  • Then using one of the following method to start the node:

Using interactive python:

$ python
>>> from dora import Node
>>> node = Node("node_0")
>>> event = node.next()
>>> event
{'id': 'tick', 'type': 'INPUT', 'value': <pyarrow.lib.NullArray object at 0x7bda86924460>
0 nulls, 'kind': 'dora', 'metadata': {'open_telemetry_context': ''}}

Using Python

$ python my_script.py # <-- node = Node("node_0")
# Log will appear in your terminal

Using Rust

$ cargo run my_node # <-- DoraNode::init_from_node_id(node_id);
# Log will appear in your terminal

Using a jupyter notebook

python

Made all dora python object representable in python

All dora-rs now implements: __str__, __repr__, __dir__, __dict__

Meaning that from Python, you will always be able to represent dora object as a Python Object.

>>> node
Node()
>>> event = node.next()
>>> event
{'id': 'tick', 'type': 'INPUT', 'value': <pyarrow.lib.NullArray object at 0x7bda86924460>
0 nulls, 'kind': 'dora', 'metadata': {'open_telemetry_context': ''}}
>>>
>>> from dora import Ros2NodeOptions, Ros2QosPolicies
>>>
>>> policies = Ros2QosPolicies()
>>> policies
Ros2QosPolicies(durability=Ros2Durability.Volatile, liveliness=Ros2Liveliness.Automatic, lease_duration=inf, reliable=false, max_blocking_time=0.0, keep_all=false, keep_last=1)
>>> policies.__dict__
{'keep_all': False, 'keep_last': 1, 'liveliness': Ros2Liveliness.Automatic, 'durability': Ros2Durability.Volatile, 'reliable': False, 'max_blocking_time': 0.0, 'lease_duration': inf}

Thanks a million to @EricLBuehler!

Simplified the dataflow

We removed one level of hierarchy by removing custom that was referencing custom node.

We also use the keyword path instead of source that might be a bit too confusing.

This change is backward compatible and you can keep the state of your previous dataflow.

Before:

  - id: idefics2
    custom:
      source: ../nodes/idefics2_node_demo.py
      inputs:
        text: whisper/text_llm
      outputs:
        - speak
        - control

After:

  - id: idefics2
    path: ../nodes/idefics2_node_demo.py
    inputs:
      text: whisper/text_llm
    outputs:
      - speak
      - control

Enabled dora on multiple machines

We can now experiment dora on different machine!

You can getting started with our multi-daemon example

A very minimalist setup would look like this

# On machine A
dora coordinator
dora daemon --machine-id A

# On machine B
dora daemon --machine-id B --coordinator-addr <COORDINATOR_IP>:53290

# On machine A or B
dora start ./examples/multiple-daemons/dataflow.yml
nodes:
  - id: rust-node
    _unstable_deploy:
      machine: A
    path: abs/path/to/target/debug/rust-node
    inputs:
      tick: dora/timer/millis/10
    outputs:
      - random

  - id: rust-sink
    _unstable_deploy:
      machine: B
    path: abs/path/to/target/debug/rust-sink
    inputs:
      message: rust-node/random

Thanks a million to @Gege-Wang and @XxChang!

I would also thanks @Michael-J-Ward and @LyonRust for their help on this release as well!!!

Full Changelog

Read more

Contributors

phil-opp, LyonRust, and 6 other contributors
Assets 17
Loading

v0.3.5-rc1

03 Jul 16:10
@haixuanTao haixuanTao
v0.3.5-rc1
4f9cf05
Compare
Choose a tag to compare
v0.3.5-rc1 Pre-release
Pre-release 
bumping to v0.3.5-rc1
Assets 17
Loading