Gphoto2 basic implementation #117

jeffwitz · 2024-03-13T17:27:30Z

Hello,

You can find here a basic example of a continuous acquisition of gphoto2 compatible cameras.
This example should work on all the OS as it only uses python libraries.
There are 2 dependencies : gphoto2 and Pillow

There is an infinity of options that could be added in order to improve it, but is is a start.

The main idea, is that you set everything on the device and you just record with crappy.

Hope it can find its place in crappy.

import gphoto2 as gp
import numpy as np
from crappy.camera.meta_camera import Camera
import crappy
from datetime import datetime
from PIL import Image
from PIL.ExifTags import TAGS
from io import BytesIO
import time

class CameraGphoto2(Camera):
  def __init__(self) -> None:
      Camera.__init__(self)
      self.camera = None
      self.context = gp.Context()


  def open(self):
      self.camera = gp.Camera()
      self.camera.init(self.context)


  def get_image(self):
      file_path = self.camera.capture(gp.GP_CAPTURE_IMAGE, self.context)
      camera_file = self.camera.file_get(
          file_path.folder, file_path.name, gp.GP_FILE_TYPE_NORMAL)
      file_data = camera_file.get_data_and_size()
      image_stream = BytesIO(file_data)
      img = Image.open(image_stream)
      exif_data = img._getexif()
      date_time_str = time.time()
      img = np.array(img)
      print(date_time_str)
      return date_time_str,img[:,:,::-1]

  def close(self):
      if self.camera is not None:
          self.camera.exit(self.context)

if __name__ == '__main__':
  cam = crappy.blocks.Camera(
      'CameraGphoto2',  # Using the FakeCamera camera so that no hardware is
      # required
      config=True,  # Before the test starts, displays a configuration window
      # for configuring the camera
      display_images=True,  # During the test, the acquired images are
      # displayed in a dedicated window
      save_images=False,  # Here, we don't want the images to be recorded
      # Sticking to default for the other arguments
      )
  stop = crappy.blocks.StopButton(
      # No specific argument to give for this Block
  )
  crappy.start()

WeisLeDocto · 2024-03-13T21:15:11Z

Hi !

This new Camera object might ultimately make it to the main branch, but I think that even before including it on the development branch, some aspects need to be addressed:

There should be a way to select the device to read images from
It seems that EXIF metadata can be easily retrieved, it should be returned by the get_image method (probably only a relevant subset of all the returned tags)
There should be a way to manage the number of channels (or cast color images to grey levels)
A minimal documentation should be added, specifying the compatible platforms (Windows ?), the requirements if any (libgphoto2 ?), and the typical use cases for this new Camera.

As this object requires specific hardware for being tested, I think it will be included/moved in crappy.collection from version 2.1.0 onwards.

jeffwitz · 2024-03-18T13:10:38Z

I made all the improvements you ask for, please take a look at this class :

import gphoto2 as gp
import numpy as np
from crappy.camera.meta_camera import Camera
import os
from datetime import datetime
from PIL import Image
from PIL.ExifTags import TAGS
from io import BytesIO
import time
from typing import Optional,Tuple, List
import cv2
import sys

def interpret_exif_value(value: any) -> any:
    """Readable generic EXIF interpreter"""
    if isinstance(value, bytes):
        try:
            return value.decode('utf-8')
        except UnicodeDecodeError:
            return value.hex()
    elif isinstance(value, tuple) and all(isinstance(x, int) for x in value):
        return "/".join(map(str, value))
    elif isinstance(value, (list, tuple)):
        return [interpret_exif_value(x) for x in value]
    return value

class CameraGphoto2(Camera):
    """Class for reading images from agphoto2 compatible Camera.

    The CameraGphoto2 block is meant for reading images from a
    Gphoto2 Camera. It uses the :mod:`ghoto2` library for capturing images,
    and :mod:`cv2` for converting BGR images to black and white.

    It can read images from the all the gphoto2 compatible cameras  indifferently.

    Warning:
    Not tested in Windows, but there is no use of Linux API, only python libraries.
    available in pip
    .. versionadded:: ?
    """
    def __init__(self) -> None:
        """Instantiates the available settings."""

        Camera.__init__(self)
        self.camera = None
        self.context = gp.Context()
        self.model: Optional[str] = None
        self.port: Optional[str] = None
        self.add_choice_setting(name="channels",
                        choices=('1', '3'),
                        default='1')

    def open(self, model: Optional[str] = None,
             port: Optional[str] = None,
             **kwargs: any) -> None:
        """Open the camera `model` and `could be specified`"""
        self.model = model
        self.port = port
        self.set_all(**kwargs)

        self.cameras = gp.Camera.autodetect(self.context)
        self._port_info_list = gp.PortInfoList()
        self._port_info_list.load()

        camera_found = False
        for name, port in self.cameras:
            if (self.model is None or name == self.model) and (self.port is None or port == self.port):
                idx = self._port_info_list.lookup_path(port)
                if idx >= 0:
                    self.camera = gp.Camera()
                    self.camera.set_port_info(self._port_info_list[idx])
                    self.camera.init(self.context)
                    camera_found = True
                    break

        if not camera_found:
            print(f"Camera '{self.model}' on port '{self.port}' not found.")


    def get_image(self) -> Tuple[float, np.ndarray, dict]:
        """Simply acquire an image using gphoto2 library.
        The captured image is in GBR format, and converted into black and white if
        needed.
        Returns:
        The timeframe and the image.
        """
        file_path = self.camera.capture(gp.GP_CAPTURE_IMAGE, self.context)
        camera_file = self.camera.file_get(
            file_path.folder, file_path.name, gp.GP_FILE_TYPE_NORMAL)
        file_data = camera_file.get_data_and_size()
        image_stream = BytesIO(file_data)
        img = Image.open(image_stream)
        # Extract EXIF data
        t = time.time()
        # Extract and interpret EXIF data
        metadata = {}
        if hasattr(img, '_getexif'):
            exif_info = img._getexif()
            if exif_info is not None:
                for tag, value in exif_info.items():
                    decoded = TAGS.get(tag, tag)
                    if decoded is not 'MakerNote':
                        readable_value = interpret_exif_value(value)
                        metadata[decoded] = readable_value
                        print(f'{decoded} : {readable_value}')
        metadata = {'t(s)': t, **metadata}

        img=np.array(img)
        if self.channels == '1':
            return t, cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        else:
            return metadata,img[:,:,::-1]

    def close(self) -> None:
        """Close the camera in gphoto2 library"""
        if self.camera is not None:
            self.camera.exit(self.context)

you can try with:

import libgphoto2
import crappy



if __name__ == '__main__':

  # The Block in charge of acquiring the images and displaying them
  # It also displays a configuration windows before the test starts, in which
  # the user can tune a few parameters of the Camera
  # Here, a fake camera is used so that no hardware is required
  cam = crappy.blocks.Camera(
      'CameraGphoto2',  # Using the FakeCamera camera so that no hardware is
      # required
      model = 'Nikon Z6_2',
      port = 'usb:002,012',
      config = True,  # Before the test starts, displays a configuration window
      # for configuring the camera
      display_images = True,  # During the test, the acquired images are
      # displayed in a dedicated window
      save_images = True,  # Here, we don't want the images to be recorded
      # Sticking to default for the other arguments
      )

  # This Block allows the user to properly exit the script
  stop = crappy.blocks.StopButton(
      # No specific argument to give for this Block
  )
  # Mandatory line for starting the test, this call is blocking
  crappy.start()

I don't understand why metadada dict doesn't work. I copy the structure of ximea_xapi, so it should work

WeisLeDocto · 2024-03-18T23:05:51Z

Here's a refactored version, closer to the code style of Crappy

# coding: utf-8

import numpy as np
from io import BytesIO
from time import time
from typing import Optional, Tuple, Any, Union, List, Dict

from crappy.camera.meta_camera import Camera
from crappy import OptionalModule

try:
    from PIL import Image
    from PIL.ExifTags import TAGS
except (ImportError, ModuleNotFoundError):
    Image = TAGS = OptionalModule('Pillow')

try:
    import cv2
except (ImportError, ModuleNotFoundError):
    cv2 = OptionalModule('opencv-python')

try:
    import gphoto2 as gp
except (ImportError, ModuleNotFoundError):
    gp = OptionalModule('gphoto2')


# Is it really needed ? What is returned by PIL ?
def interpret_exif_value(value: Any) -> Union[str, List[str]]:
    """Readable generic EXIF interpreter"""

    if isinstance(value, bytes):
        try:
            return value.decode('utf-8')
        except UnicodeDecodeError:
            return value.hex()
    elif isinstance(value, tuple) and all(isinstance(x, int) for x in value):
        return "/".join(map(str, value))
    elif isinstance(value, (list, tuple)):
        return [interpret_exif_value(x) for x in value]
    return value


class CameraGphoto2(Camera):
    """Class for reading images from agphoto2 compatible Camera.

    The CameraGphoto2 block is meant for reading images from a
    Gphoto2 Camera. It uses the :mod:`ghoto2` library for capturing images,
    and :mod:`cv2` for converting BGR images to black and white.

    It can read images from the all the gphoto2 compatible cameras
    indifferently.

    Warning:
        Not tested in Windows, but there is no use of Linux API, only python
        libraries available in pip

    .. versionadded:: ?
    """

    def __init__(self) -> None:
        """Instantiates the available settings."""

        super().__init__()

        self.camera = None
        self.model: Optional[str] = None
        self.port: Optional[str] = None

        self.context = gp.Context()

    def open(self,
             model: Optional[str] = None,
             port: Optional[str] = None,
             **kwargs: Any) -> None:
        """Open the camera `model` and `could be specified`"""

        # Not actually needed, since it's only being used in open
        self.model = model
        self.port = port

        self.add_choice_setting(name="channels",
                                choices=('1', '3'),
                                default='1')

        # Maybe use gp.CameraList() ?
        # No need for instance attributes since it's only used in open
        # Except if a reference needs to be stored somewhere ?
        self.cameras = gp.Camera.autodetect(self.context)
        self._port_info_list = gp.PortInfoList()
        self._port_info_list.load()

        camera_found = False
        for name, port in self.cameras:
            if ((self.model is None or name == self.model) and
                (self.port is None or port == self.port)):
                idx = self._port_info_list.lookup_path(port)
                if idx >= 0:
                    self.camera = gp.Camera()
                    self.camera.set_port_info(self._port_info_list[idx])
                    self.camera.init(self.context)
                    camera_found = True
                    break

        if not camera_found:
            print(f"Camera '{self.model}' on port '{self.port}' not found.")
            # Raise exception
            # Message not generic enough, what about the case when no model
            # and/or port is specified ?

        self.set_all(**kwargs)

    def get_image(self) -> Tuple[Dict[str, Any], np.ndarray]:
        """Simply acquire an image using gphoto2 library.

        The captured image is in GBR format, and converted into black and white
        if needed.

        Returns:
            The timeframe and the image.
        """

        file_path = self.camera.capture(gp.GP_CAPTURE_IMAGE, self.context)
        camera_file = self.camera.file_get(
            file_path.folder, file_path.name, gp.GP_FILE_TYPE_NORMAL)
        img = Image.open(BytesIO(camera_file.get_data_and_size()))

        # Is it actually a good thing to retrieve all the exif tags ?
        metadata = dict()
        if hasattr(img, 'getexif'):
            exif_info = img.getexif()
            if exif_info is not None:
                for tag, value in exif_info.items():
                    decoded = TAGS.get(tag, tag)
                    if decoded is not 'MakerNote':
                        readable_value = interpret_exif_value(value)
                        metadata[decoded] = readable_value
                        print(f'{decoded} : {readable_value}')

        # Need the 'ImageUniqueID' key
        metadata = {'t(s)': time(), **metadata}

        img = np.array(img)
        if self.channels == '1':
            return metadata, cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        else:
            return metadata, img[:, :, ::-1]

    def close(self) -> None:
        """Close the camera in gphoto2 library"""

        if self.camera is not None:
            self.camera.exit(self.context)

Still several things that are bugging me, I commented them out in the code:

Is the interpret_exif_value function really necessary ? Doesn't PIL already perform some nice formatting when returning the tags ? (IDK myself)
Is it really desirable to save all the available EXIF tags in the metadata ? Would it still be user-friendly if the camera generates dozens of different tags ?
Several class attributes that could be simple variables since they're only being used in one method
The case when no matching camera is found should raise an exception, not just print text
My linter tells me that gp.Camera.autodetect() should take as arguments a gp.CameraList and a gp.Context. Seems to be confirmed by the source code. In this example they do not provide any argument though. Just bringing it up, in case you want to check on that.
As outlined in the documentation, the tutorials, and the examples of Crappy, the 'ImageUniqueID' is one of the two required keys in the metadata dictionary, along with 't(s)'. I assume that not providing it is the reason why metadata is not working in your case.

jeffwitz · 2024-03-19T09:43:04Z

here the last version that works with most of your advice:

import numpy as np
from crappy.camera.meta_camera import Camera
import os
from datetime import datetime
from io import BytesIO
import time
from typing import Optional,Tuple, List, Dict, Any
# import sys

try:
    from PIL import Image, ExifTags
except (ModuleNotFoundError, ImportError):
  pillow = OptionalModule("Pillow", "To use DSLR or compact cameras, please install the "
                         "official ghoto2 Python module : python -m pip instal Pillow")

try:
  import gphoto2 as gp
except (ModuleNotFoundError, ImportError):
  gphoto2 = OptionalModule("gphoto2", "To use DSLR or compact cameras, please install the "
                         "official ghoto2 Python module : python -m pip instal gphoto2")

try:
  import cv2
except (ModuleNotFoundError, ImportError):
  gphoto2 = OptionalModule("cv2", "Crappy needs OpenCV for video "
                         "official cv2 Python module : python -m pip instal opencv-python")


class CameraGphoto2(Camera):
    """Class for reading images from agphoto2 compatible Camera.

    The CameraGphoto2 block is meant for reading images from a
    Gphoto2 Camera. It uses the :mod:`ghoto2` library for capturing images,
    and :mod:`cv2` for converting BGR images to black and white.

    It can read images from the all the gphoto2 compatible cameras  indifferently.

    Warning:
    Not tested in Windows, but there is no use of Linux API, only python libraries.
    available in pip
    .. versionadded:: ?
    """
    def __init__(self) -> None:
        """Instantiates the available settings."""

        Camera.__init__(self)
        self.camera = None
        self.context = gp.Context()
        self.model: Optional[str] = None
        self.port: Optional[str] = None
        self.add_choice_setting(name="channels",
                        choices=('1', '3'),
                        default='1')
        self.num_image = 0

    def open(self, model: Optional[str] = None,
             port: Optional[str] = None,
             **kwargs: any) -> None:
        """Open the camera `model` and `could be specified`"""
        self.model = model
        self.port = port
        self.set_all(**kwargs)

        cameras = gp.Camera.autodetect(self.context)
        _port_info_list = gp.PortInfoList()
        _port_info_list.load()

        camera_found = False
        for name, port in cameras:
            if (self.model is None or name == self.model) and (self.port is None or port == self.port):
                idx = _port_info_list.lookup_path(port)
                if idx >= 0:
                    self.camera = gp.Camera()
                    self.camera.set_port_info(_port_info_list[idx])
                    self.camera.init(self.context)
                    camera_found = True
                    break

        if not camera_found:
            if self.model is not None and self.port is not None:
                raise IOError(f"Camera '{self.model}' on port '{self.port}' not found.")
            elif self.model is not None and self.port is None:
                raise IOError(f"Camera '{self.model}' not found.")
            elif self.model is None and self.port is None:
                raise IOError(f"No camera found found.")

    def get_image(self) -> Tuple[Dict[str, Any], np.ndarray]:
        """Simply acquire an image using gphoto2 library.
        The captured image is in GBR format, and converted into black and white if
        needed.
        Returns:
        The timeframe and the image.
        """
        file_path = self.camera.capture(gp.GP_CAPTURE_IMAGE, self.context)
        camera_file = self.camera.file_get(
            file_path.folder, file_path.name, gp.GP_FILE_TYPE_NORMAL)
        file_data = camera_file.get_data_and_size()
        image_stream = BytesIO(file_data)
        img = Image.open(image_stream)
        # Extract EXIF data
        t = time.time()
        # Extract and interpret EXIF data
        metadata = {}
        if hasattr(img, '_getexif'):
            exif_info = img._getexif()
            if exif_info is not None:
                for tag, value in exif_info.items():
                    decoded = ExifTags.TAGS.get(tag, tag)
                    if decoded in ["Model", "DateTime", "ExposureTime","ShutterSpeedValue", "FNumber","ApertureValue","FocalLength", "ISOSpeedRatings"]:
                        metadata[decoded] = value
        metadata = {'ImageUniqueID': self.num_image, **metadata}
        metadata = {'t(s)': t, **metadata}
        self.num_image+=1
        img=np.array(img)
        if self.channels == '1':
            metadata['channels'] = 'gray'
            return t, cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
        else:
            metadata['channels'] = 'color'
            return metadata,img[:,:,::-1]

    def close(self) -> None:
        """Close the camera in gphoto2 library"""
        if self.camera is not None:
            self.camera.exit(self.context)

WeisLeDocto · 2024-03-19T21:31:31Z

Was the code thoroughly tested on at least 2 different models of camera ?
If so, I'll perform a bit of refactoring and include it in the next release (probably not before a few weeks at least).

If you want credit for this addition, the easiest way is to open a pull request on the develop branch.
Otherwise, it is also possible to add you as an author of my own commit, but I might forget to do it.

jeffwitz · 2024-03-19T23:38:34Z

Was the code thoroughly tested on at least 2 different models of camera ? If so, I'll perform a bit of refactoring and include it in the next release (probably not before a few weeks at least).

Yes on a Nikon Z6 2 and a Canon EOS 450D simultaneously.

If you want credit for this addition, the easiest way is to open a pull request on the develop branch. Otherwise, it is also possible to add you as an author of my own commit, but I might forget to do it.

Could be interesting to learn how to do it. I will try.

It is important to note that this code lack the event management that will allow to acquire an image when the shot button is pressed or when the remote controller is used.

It would allow to sync to differents cameras, as for now each camera run without knowing the other and so without any sync.
It will be an improvement for the next version of the class, but I think this version covers a lot of cases already.

Thanks for your help

jeffwitz · 2024-04-27T05:08:56Z

pending pull request #118 so I close this issue

WeisLeDocto · 2024-04-27T14:17:37Z

It's just a detail, but closing the issue might actually not be the best move for several reasons:

The requested implementation hasn't been merged so far (still waiting for your input)
This issue and the pull requests are linked together, so they automatically share a common fate (see screenshot)
Closing the issue signals the associated task as done in the projects section, which is not the case (see second screenshot)

WeisLeDocto self-assigned this Mar 13, 2024

WeisLeDocto added the feature request User requests a new feature to be implemented label Mar 13, 2024

WeisLeDocto added this to @WeisLeDocto's contribution to Crappy Mar 19, 2024

github-project-automation bot moved this to Todo in @WeisLeDocto's contribution to Crappy Mar 19, 2024

WeisLeDocto linked a pull request Apr 14, 2024 that will close this issue

Add CameraGphoto2 to existing crappy available cameras #118

Open

jeffwitz closed this as completed Apr 27, 2024

github-project-automation bot moved this from Todo to Done in @WeisLeDocto's contribution to Crappy Apr 27, 2024

WeisLeDocto moved this from Done to In Progress in @WeisLeDocto's contribution to Crappy Apr 27, 2024

WeisLeDocto moved this from In Progress to Done in @WeisLeDocto's contribution to Crappy Apr 27, 2024

WeisLeDocto moved this from Done to In Progress in @WeisLeDocto's contribution to Crappy Apr 27, 2024

WeisLeDocto reopened this Nov 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gphoto2 basic implementation #117

Gphoto2 basic implementation #117

jeffwitz commented Mar 13, 2024

WeisLeDocto commented Mar 13, 2024

jeffwitz commented Mar 18, 2024 •

edited

Loading

WeisLeDocto commented Mar 18, 2024 •

edited

Loading

jeffwitz commented Mar 19, 2024

WeisLeDocto commented Mar 19, 2024

jeffwitz commented Mar 19, 2024 •

edited

Loading

jeffwitz commented Apr 27, 2024

WeisLeDocto commented Apr 27, 2024

Gphoto2 basic implementation #117

Gphoto2 basic implementation #117

Comments

jeffwitz commented Mar 13, 2024

WeisLeDocto commented Mar 13, 2024

jeffwitz commented Mar 18, 2024 • edited Loading

WeisLeDocto commented Mar 18, 2024 • edited Loading

jeffwitz commented Mar 19, 2024

WeisLeDocto commented Mar 19, 2024

jeffwitz commented Mar 19, 2024 • edited Loading

jeffwitz commented Apr 27, 2024

WeisLeDocto commented Apr 27, 2024

jeffwitz commented Mar 18, 2024 •

edited

Loading

WeisLeDocto commented Mar 18, 2024 •

edited

Loading

jeffwitz commented Mar 19, 2024 •

edited

Loading