Migrate to the picamera2 library for camera control & image acquisition

[W7CH]

Updated PR description by @ethanjli

This PR adds hardware support for Raspberry Pi cameras using the libcamera-based picamera2 library, as a replacement for the previous raspimjpeg-based image-acquisition routine. The work in this PR, which is done in two new subpackages - camera (for hardware support using the picamera2 library, and for management of camera settings) and imagernew (for the image-acquisition routine which should be clearer/easier to understand) - completes the majority of PlanktoScope/PlanktoScope#79 - after this PR is merged, the only remaining work required for completing PlanktoScope/PlanktoScope#79 will be to delete the raspimjpeg-based code and rename the new imagernew subpackage to imager. That remaining work should be done in a future PR after we've had a chance to test the changes from this PR.

The imagernew subpackage also makes progress on PlanktoScope/PlanktoScope#217 by changing the filename format from {hours}_{minutes}_{seconds}_{microseconds}.jpg to {year}-{month}-{day}_{hours}-{minutes}-{seconds}-{microseconds}.jpg (where index is the number of the frame in the sequence, starting with 0).

The camera subpackage also fixes the buggy backend behavior reported in PlanktoScope/PlanktoScope#166 by adding input validation and error handling for invalid camera settings (both in MQTT and in the hardware.json config file): instead of crashing the hardware controller, now those invalid values are ignored in favor of default settings.

This PR also introduces GitHub Actions CI checks for autoformatting, type-checking, and linting into all new modules, starting with the modules added by this PR. The new readme file in the control/ directory includes usage instructions on local developer environments.

QA/TODOs:

• Update the changelog
• Resolve any remaining linter complaints
• Check for any incorrect interactions between the auto-white-balance toggle and the manual white balance gains
• Manually compare new code with old imager code to see if anything has been accidentally omitted
• Check to ensure that all metadata in the metadata.json files from the old imager is still saved by the new imager
• Confirm that datasets from the new imager are successfully segmented.
• Confirm that the new filename scheme passes through metadata and can be uploaded/added to EcoTaxa successfully
• Ensure that erroneous inputs don't crash anything
• Ensure that a missing camera is handled gracefully
• Confirm that all "statistics" fields on the "Fluidic Acquisition" page of the Node-RED dashboard are still displayed correctly
• Make a PR for https://github.com/PlanktoScope/PlanktoScope/tree/feature/backend-controller-picamera2
• Update https://github.com/PlanktoScope/PlanktoScope/tree/feature/backend-controller-picamera2, and confirm that everything works as expected on a freshly-set-up SD card image
• Build adafruithat SD card image for testing
• Build pscopehat SD card image for testing
• Mark this PR as "ready to review"
• Get help from someone to do basic testing on a pscopehat-based PlanktoScope with the HQ camera

Work left for future PRs:

• Write unit tests
• Remove the old imager subpackage, and rename the new imagernew subpackage to imager
• Ensure compatibility with the 64-bit version of Raspberry Pi OS 12 (bookworm) and the Raspberry Pi 5 (as part of backend: Resolve incompatibilities between the Python backend and the Raspberry Pi 5 PlanktoScope#323)
• There are also a bunch of TODOs left in code comments, mainly related to refactoring of other parts of the hardware controller and reorganization of the MQTT API

Original PR description by @W7CH

This PR makes progress on PlanktoScope/PlanktoScope#79 by defining a new picamera2-based image-acquisition module (temporarily named imagernew, but which will be renamed in a future PR to imager in order to replace the raspimjpeg-based imager). This PR doesn't work yet:

• This branch contains all the code replicated from raspimjpeg to picamera2. The system still doesn't work with these modifications, even when the picamera class was used as a thread within the imager process (similar to the fact that raspimjpeg was used as a subprocess).

• What kind of issue does this branch try to resolve?
  Raspimjpeg is subject to certain limitations:
  -It is not an official Raspberry library. It was mainly present in "RPi Cam Web Interface" to control the RPi Cam via a browser.
  -It only supports cameras connected via the ribbon cable.
  -It appears to bypass or increase the exposure parameters defined in the camera's camera configuration, resulting in overexposed images.

• What are the commands used to test the code?
  The new python files of this branch are drafted on my personal laptop and then being updated on the planktoscope to test them. So, I first update them and then reboot the device to test the latest changes.

• What is the expected behavior?
  The expected behavior is to have the Pi camera working correctly and smoothly as if it was controlled by raspimjpeg. The camera preview needs to be displayed on the Node-RED dashboard, the camera parameters need to be updated directly from the dashboard and an image acquisition using the camera need to be launched.

• What is the current behavior?
  The camera preview isn't being displayed on the dashboard, neither on the optic configuration page, nor on fluidic acquisition page even though both streaming sources "/ps/hal/camera/streams/preview.mjpg" & "ip-address:8000/stream.mjpg" were tested. The light and stepper processes were isolated and the imager process was tested solely, but it didn't work (unlike raspimjpeg's case). The camera class was also used as a thread within the imager process, but it still doesn't work. In general, there is no sign that the camera is working!

[ethanjli]

@W7CH I figured out the source of the problem you were blocked on: basically, you had one process which started the camera (which writes to the StreamingOutput instance), and then you had a separate process which was trying to read data (the condition variable, and frames) from the StreamingOutput instance. However, StreamingOutput only shares state across threads, not across processes; the second process saw a different copy of the StreamingOutput instance in a different memory location (because of how forking child processes works) - so even though the camera called the notify_all method on a condition variable within its own process, that condition variable was not synchronized to the condition variable which the StreamingHandler's GET handler was waiting on. So I fixed your problem by moving the code for starting the camera into the same method where the StreamingServer is started, so that everything is happening in the same process. The fact that we could be calling code from different processes is something we'll need to continue to be careful about.

I noticed that the code are many placeholders/TODO comments for additional methods to implement. Now that I've unblocked progress on this pull request, do you plan to implement the remaining parts of this pull request? Or do we need to confirm with @tpollina (maybe at this week's software meeting) about who (e.g. me) will do the remaining work?

(update from the 2024-03-14 software meeting: yes, I will be fully taking over this PR)

[ethanjli]

I got a PlanktoScope HAT-based PlanktoScope (v2.6, I think? The label says v2.5 marine but the serial number says 2.6bU059) and identified some problems which needed to be fixed (see the two commits above). After that, now the hardware controller runs without crashing at startup and without obviously visible errors from the Node-RED dashboard. So I will merge this PR now, and we can start doing broader testing!