Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert AudioData stream to arrays of floats, and convert arrays of floats to AudioData #84

Open
wants to merge 31 commits into
base: master
Choose a base branch
from

Commits on May 19, 2023

  1. Configuration menu
    Copy the full SHA
    46e1c7f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a740590 View commit details
    Browse the repository at this point in the history
  3. py example doesn't work

    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    dba1563 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e830583 View commit details
    Browse the repository at this point in the history
  5. It looks like only one buffer is received, but I'm not handling it co…

    …rrectly so the callback gets triggered repeatedly with the same buffer. It seems like I need something else in the pipeline to receive the app_buffer.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    2f2f32b View commit details
    Browse the repository at this point in the history
  6. This may be the raw data- but what about 8-bit vs. 16-bit samples? Do…

    … I have to know that and cast the data appropriately, or can I force conversion to expected format?
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    16c637d View commit details
    Browse the repository at this point in the history
  7. I thought I was getting good decoded data from the microphone, but no…

    …w I only see five good samples and everything else is zero.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    ad36501 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    e65a238 View commit details
    Browse the repository at this point in the history
  9. Passing data as -1.0 to 1.0 float now, so receiving node doesn't have…

    … to understand it (should be 0 to 1.0?)
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    3c51d1e View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    c87d14f View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    9095bed View commit details
    Browse the repository at this point in the history
  12. Trying to convert ros messages of floating point arrays to gstreamer …

    …format, currently getting
    
    [ INFO] [/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/float_to_audio/src/float_to_audio.cpp]:[208] [emitted push]
    [ERROR] [/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/float_to_audio/src/float_to_audio.cpp]:[194] [gstreamer: Internal data flow error.]
    terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
      what():  boost: mutex
    
    Not sure if I'm specifying the float format correctly, x-raw-float is 0.10 and not 1.0.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    094a920 View commit details
    Browse the repository at this point in the history
  13. Should be able to trace problem with debug on:

    0:00:00.016236728 17257      0x19a4720 WARN           basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(int)28, channels=(int)1, width=(int)32, depth=(int)32, endianness=(int)1234, rate=(int)16000 in anything we support
    [ INFO] [/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/float_to_audio/src/float_to_audio.cpp]:[208] [emitted push 8]
    0:00:00.258644946 17257      0x19a4720 WARN           basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(int)28, channels=(int)1, width=(int)32, depth=(int)32, endianness=(int)1234, rate=(int)16000 in anything we support
    0:00:00.258680231 17257      0x19a4720 WARN           basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(int)28, channels=(int)1, width=(int)32, depth=(int)32, endianness=(int)1234, rate=(int)16000 in anything we support
    0:00:00.258704660 17257      0x19a4720 WARN                 basesrc gstbasesrc.c:2948:gst_base_src_loop:<source> error: Internal data flow error.
    0:00:00.258712146 17257      0x19a4720 WARN                 basesrc gstbasesrc.c:2948:gst_base_src_loop:<source> error: streaming task paused, reason not-negotiated (-4)
    0:00:00.258770927 17257      0x19a4720 WARN           basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(int)28, channels=(int)1, width=(int)32, depth=(int)32, endianness=(int)1234, rate=(int)16000 in anything we support
    0:00:00.258792146 17257      0x19a4720 WARN           basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(int)28, channels=(int)1, width=(int)32, depth=(int)32, endianness=(int)1234, rate=(int)16000 in anything we support
    [ERROR] [/audio_capture] [/home/lucasw/catkin_ws/src/audio_common/float_to_audio/src/float_to_audio.cpp]:[194] [gstreamer: Internal data flow error.]
    terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::lock_error> >'
      what():  boost: mutex lock failed in pthread_mutex_lock: Invalid argument
    Aborted (core dumped)
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    88f69cc View commit details
    Browse the repository at this point in the history
  14. Trying out paused, but the next problem seems to be with pads? gstpad…

    ….c gst_pad_push_data error pushing events, return not-negotiated (Seen on debug level 6)
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    4677fbe View commit details
    Browse the repository at this point in the history
  15. Trying out S16LE as the input type, but now back to 0:00:00.276624022…

    … 16823 0x224a000 WARN basetransform gstbasetransform.c:1414:gst_base_transform_setcaps:<filter> transform could not transform audio/x-raw, format=(string)S16LE, channels=(int)1, layout=(int)0, width=(int)16, depth=(int)16, rate=(int)16000, signed=(boolean)true in anything we support
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    9e7d9a1 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    6a0592b View commit details
    Browse the repository at this point in the history
  17. Finall got it! The whole problem was not specifying the layout as a s…

    …tring. The input and output rates do look different though
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    c8c1653 View commit details
    Browse the repository at this point in the history
  18. Making messages debug level, also adjusting visualization. Next step …

    …is to approximate nes sound chip with A-D voices, have four ros nodes in control of each, and then they can be played individually.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    4fe1db6 View commit details
    Browse the repository at this point in the history
  19. Configuration menu
    Copy the full SHA
    bba66bd View commit details
    Browse the repository at this point in the history
  20. Wave encoding doesn't work, trying to fix that- if I put too many cap…

    …s in it results in internal data error. Also trying to make noise sound more like online examples- I think when sample rate exceeds output sample rate there needs to be averaging. Also it is possible the mp3 compression is making the sound much different (could increase bitrate?), which is why I want wave to work.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    c642cbd View commit details
    Browse the repository at this point in the history
  21. flac seems to work, though I got a crash once (maybe a different node…

    … though), and can't get rqt to load the config file
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    18b4d29 View commit details
    Browse the repository at this point in the history
  22. disabling wave. Flac works but the waveform is really different, it s…

    …eems like there ought to be an auditory difference (or maybe I need the conversion to float to understand the decoded format better)
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    a4c7f70 View commit details
    Browse the repository at this point in the history
  23. Rename projects for consistency, but maybe float_to_audio should be a…

    …udio_from_float to have the same prefix as the rest of the projects? Also deleted a lot of commented out code. Inability to kill float_to_audio easily remains, also other major issue is that audio_to_float assumes format to convert to float, I think I understand gstreamer enough to make it do the floating point conversion.
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    52fe48c View commit details
    Browse the repository at this point in the history
  24. The launch file doesn't work when I launch it- something is wrong wit…

    …h float_to_audio, but when I run it manually it does work:
    
    rosrun float_to_audio float_to_audio audio:=audio2 samples:=decoded __name:=float_to_audio __ns:=/audio
    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    2fc59f5 View commit details
    Browse the repository at this point in the history
  25. Forgot to add this headers

    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    5b7fb73 View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    4584078 View commit details
    Browse the repository at this point in the history
  27. Configuration menu
    Copy the full SHA
    8bae90d View commit details
    Browse the repository at this point in the history
  28. Configuration menu
    Copy the full SHA
    f6a80dc View commit details
    Browse the repository at this point in the history
  29. flake8 compliance

    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    497bba7 View commit details
    Browse the repository at this point in the history
  30. flake8 compliance for view.py

    lucasw committed May 19, 2023
    Configuration menu
    Copy the full SHA
    d61ee03 View commit details
    Browse the repository at this point in the history
  31. Configuration menu
    Copy the full SHA
    e0a3be8 View commit details
    Browse the repository at this point in the history