ML4Science 2023: Phase masks implementations

README.rst

@@ -1,70 +1,20 @@
 =============
 LenslessPiCam
 =============
+Ahmed Elalamy (324610), Seif Hamed (312081), Ghita Tagemouati (330383)
 
-.. image:: https://readthedocs.org/projects/lensless/badge/?version=latest
-    :target: http://lensless.readthedocs.io/en/latest/
-    :alt: Documentation Status
-
-
-.. image:: https://joss.theoj.org/papers/10.21105/joss.04747/status.svg
-      :target: https://doi.org/10.21105/joss.04747
-      :alt: DOI
-
-.. image:: https://static.pepy.tech/badge/lensless
-      :target: https://www.pepy.tech/projects/lensless
-      :alt: Downloads
-
-
-*A Hardware and Software Toolkit for Lensless Computational Imaging with a Raspberry Pi*
------------------------------------------------------------------------------------------
-
-.. image:: https://github.com/LCAV/LenslessPiCam/raw/main/scripts/recon/example.png
-    :alt: Lensless imaging example
-    :align: center
-
-
-This toolkit has everything you need to perform imaging with a lensless
-camera. We make use of a low-cost implementation of DiffuserCam [1]_, 
-where we use a piece of tape instead of the lens and the
-`Raspberry Pi HQ camera sensor <https://www.raspberrypi.com/products/raspberry-pi-high-quality-camera>`__
-(the `V2 sensor <https://www.raspberrypi.com/products/camera-module-v2/>`__
-is also supported). Similar principles and methods can be used for a
-different lensless encoder and a different sensor. 
-
-*If you are interested in exploring reconstruction algorithms without building the camera, that is entirely possible!*
-The provided reconstruction algorithms can be used with the provided data or simulated data.
-
-We've also written a few Medium articles to guide users through the process
-of building the camera, measuring data with it, and reconstruction.
-They are all laid out in `this post <https://medium.com/@bezzam/a-complete-lensless-imaging-tutorial-hardware-software-and-algorithms-8873fa81a660>`__.
+Our work is mainly done in the trainable_mask.py and mask.py files.
 
 Setup 
 -----
 
-If you are just interested in using the reconstruction algorithms and 
-plotting / evaluation tools you can install the package via ``pip``:
+First, install the lensless package
 
 .. code:: bash
 
    pip install lensless
 
 
-For plotting, you may also need to install
-`Tk <https://stackoverflow.com/questions/5459444/tkinter-python-may-not-be-configured-for-tk>`__.
-
-
-For performing measurements, the expected workflow is to have a local 
-computer which interfaces remotely with a Raspberry Pi equipped with 
-the HQ camera sensor (or V2 sensor). Instructions on building the camera
-can be found `here <https://lensless.readthedocs.io/en/latest/building.html>`__.
-
-The software from this repository has to be installed on **both** your
-local machine and the Raspberry Pi. Note that we highly recommend using
-Python 3.9, as some Python library versions may not be available with 
-earlier versions of Python. Moreover, its `end-of-life <https://endoflife.date/python>`__ 
-is Oct 2025.
-
 *Local machine setup*
 =====================
 
@@ -92,101 +42,8 @@ install the library locally.
 
    # extra dependencies for local machine for plotting/reconstruction
    pip install -r recon_requirements.txt
+   pip install -r mask_requirements.txt
 
-   # (optional) try reconstruction on local machine
-   python scripts/recon/admm.py
-
-   # (optional) try reconstruction on local machine with GPU
-   python scripts/recon/admm.py -cn pytorch
-
-
-Note (25-04-2023): for using the :py:class:`~lensless.recon.apgd.APGD` reconstruction method based on Pycsou
-(now `Pyxu <https://github.com/matthieumeo/pyxu>`__), a specific commit has 
-to be installed (as there was no release at the time of implementation):
-
-.. code:: bash
-
-   pip install git+https://github.com/matthieumeo/pycsou.git@38e9929c29509d350a7ff12c514e2880fdc99d6e
-
-If PyTorch is installed, you will need to be sure to have PyTorch 2.0 or higher, 
-as Pycsou is not compatible with earlier versions of PyTorch. Moreover, 
-Pycsou requires Python within 
-`[3.9, 3.11) <https://github.com/matthieumeo/pycsou/blob/v2-dev/setup.cfg#L28>`__.
-
-Moreover, ``numba`` (requirement for Pycsou V2) may require an older version of NumPy:
-
-.. code:: bash
-
-   pip install numpy==1.23.5
-
-*Raspberry Pi setup*
-====================
-
-After `flashing your Raspberry Pi with SSH enabled <https://medium.com/@bezzam/setting-up-a-raspberry-pi-without-a-monitor-headless-9a3c2337f329>`__, 
-you need to set it up for `passwordless access <https://medium.com/@bezzam/headless-and-passwordless-interfacing-with-a-raspberry-pi-ssh-453dd75154c3>`__. 
-Do not set a password for your SSH key pair, as this will not work with the
-provided scripts.
-
-On the Raspberry Pi, you can then run the following commands (from the ``home`` 
-directory):
-
-.. code:: bash
-
-   # dependencies
-   sudo apt-get install -y libimage-exiftool-perl libatlas-base-dev \
-   python3-numpy python3-scipy python3-opencv
-   sudo pip3 install -U virtualenv
-
-   # download from GitHub
-   git clone [email protected]:LCAV/LenslessPiCam.git
-
-   # install in virtual environment
-   cd LenslessPiCam
-   virtualenv --system-site-packages -p python3 lensless_env
-   source lensless_env/bin/activate
-   pip install --no-deps -e .
-   pip install -r rpi_requirements.txt
-
-
-Acknowledgements
-----------------
-
-The idea of building a lensless camera from a Raspberry Pi and a piece of 
-tape comes from Prof. Laura Waller's group at UC Berkeley. So a huge kudos 
-to them for the idea and making tools/code/data available! Below is some of 
-the work that has inspired this toolkit:
-
-* `Build your own DiffuserCam tutorial <https://waller-lab.github.io/DiffuserCam/tutorial>`__.
-* `DiffuserCam Lensless MIR Flickr dataset <https://waller-lab.github.io/LenslessLearning/dataset.html>`__ [2]_. 
-
-A few students at EPFL have also contributed to this project:
-
-* Julien Sahli: support and extension of algorithms for 3D.
-* Yohann Perron: unrolled algorithms for reconstruction.
-
-Citing this work
-----------------
-
-If you use these tools in your own research, please cite the following:
-
-::
-
-   @article{Bezzam2023,
-      doi = {10.21105/joss.04747},
-      url = {https://doi.org/10.21105/joss.04747},
-      year = {2023},
-      publisher = {The Open Journal},
-      volume = {8},
-      number = {86},
-      pages = {4747},
-      author = {Eric Bezzam and Sepand Kashani and Martin Vetterli and Matthieu Simeoni},
-      title = {LenslessPiCam: A Hardware and Software Platform for Lensless Computational Imaging with a Raspberry Pi},
-      journal = {Journal of Open Source Software}
-   }
-
-References
-----------
-
-.. [1] Antipa, N., Kuo, G., Heckel, R., Mildenhall, B., Bostan, E., Ng, R., & Waller, L. (2018). DiffuserCam: lensless single-exposure 3D imaging. Optica, 5(1), 1-9.
+   # training with the height varying mask
+   python scripts/recon/train_unrolled.py -cn train_heightvarying
 
-.. [2] Monakhova, K., Yurtsever, J., Kuo, G., Antipa, N., Yanny, K., & Waller, L. (2019). Learned reconstructions for practical mask-based lensless imaging. Optics express, 27(20), 28075-28090.

configs/train_coded_aperture.yaml

@@ -0,0 +1,55 @@
+# python scripts/recon/train_unrolled.py -cn train_coded_aperture
+defaults:
+  - train_unrolledADMM
+  - _self_
+
+# Train Dataset
+files:
+  dataset: mnist   # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+  celeba_root: /scratch/bezzam
+  downsample: 16     # TODO use downsample simulation instead? 
+  n_files: 100
+  crop:
+    vertical: [810, 2240]
+    horizontal: [1310, 2750]
+
+torch_device: "cuda" 
+
+optimizer:
+  # type: Adam   # Adam, SGD...
+  # lr: 1e-4
+  type: SGD
+  lr: 0.01
+
+#Trainable Mask
+trainable_mask:
+  mask_type: TrainableCodedAperture
+  # optimizer: Adam
+  # mask_lr: 1e-3
+  optimizer: SGD
+  mask_lr: 0.01
+  L1_strength: False
+  binary: False
+  initial_value:
+    psf_wavelength: [550e-9]
+    method: MLS
+    n_bits: 8  # (2**n_bits-1, 2**n_bits-1)
+    # method: MURA
+    # n_bits: 25  # (4*nbits*1, 4*nbits*1)
+    # # -- applicable for phase masks
+    # design_wv: 550e-9   
+
+simulation:
+  grayscale: True
+  flip: False
+  scene2mask: 40e-2
+  mask2sensor: 2e-3
+  sensor: "rpi_hq"
+  object_height: 0.30
+
+training:
+  crop_preloss: True # crop region for computing loss
+  batch_size: 4
+  epoch: 25
+  eval_batch_size: 16
+  save_every: 1

configs/train_heightvarying.yaml

@@ -0,0 +1,43 @@
+# python scripts/recon/train_unrolled.py -cn train_multilens_array
+defaults:
+  - train_unrolledADMM
+  - _self_
+
+# Train Dataset
+files:
+  dataset: fashion_mnist   # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+  celeba_root: /scratch/bezzam
+  downsample: 16     # TODO use simulation instead?   
+  n_files: 100
+  crop:
+    vertical: [810, 2240]
+    horizontal: [1310, 2750]
+
+torch_device: "cuda:0" 
+
+#Trainable Mask
+trainable_mask:
+  mask_type: TrainableHeightVarying
+  optimizer: Adam
+  mask_lr: 1e-3
+  L1_strength: False
+  binary: False
+  initial_value: 
+    psf_wavelength: [550e-9]
+    design_wv: 550e-9
+
+simulation:
+  grayscale: True
+  flip: False
+  scene2mask: 40e-2
+  mask2sensor: 2e-3
+  sensor: "rpi_hq"
+  downsample: 16
+  object_height: 0.30
+
+training:
+  crop_preloss: True # crop region for computing loss
+  batch_size: 2
+  epoch: 25
+  eval_batch_size: 16
+  save_every: 1

configs/train_multilens_array.yaml

@@ -0,0 +1,44 @@
+# python scripts/recon/train_unrolled.py -cn train_multilens_array
+defaults:
+  - train_unrolledADMM
+  - _self_
+
+# Train Dataset
+files:
+  dataset: fashion_mnist   # Simulated : "mnist", "fashion_mnist", "cifar10", "CelebA". Measure :"DiffuserCam"
+  celeba_root: /scratch/bezzam
+  downsample: 16     # TODO use simulation instead?   
+  n_files: 100
+  crop:
+    vertical: [810, 2240]
+    horizontal: [1310, 2750]
+
+torch_device: "cpu" 
+
+#Trainable Mask
+trainable_mask:
+  mask_type: TrainableMultiLensArray
+  optimizer: Adam
+  mask_lr: 1e-3
+  L1_strength: False
+  binary: False
+  initial_value:
+    N : 10 #TODO: check this value ? 
+    psf_wavelength: [550e-9]
+    design_wv: 550e-9
+
+simulation:
+  grayscale: True
+  flip: False
+  scene2mask: 40e-2
+  mask2sensor: 2e-3
+  sensor: "rpi_hq"
+  downsample: 16
+  object_height: 0.30
+
+training:
+  crop_preloss: True # crop region for computing loss
+  batch_size: 2
+  epoch: 25
+  eval_batch_size: 16
+  save_every: 1

configs/train_unrolledADMM.yaml

@@ -84,6 +84,7 @@ trainable_mask:
   initial_value: psf
   grayscale: False
   mask_lr: 1e-3
+  optimizer: Adam  # Adam, SGD... (Pytorch class)
   L1_strength: 1.0  #False or float
 
 target: "object_plane"    # "original" or "object_plane" or "label"
@@ -129,7 +130,7 @@ training:
   crop_preloss: True # crop region for computing loss
 
 optimizer:
-  type: Adam
+  type: Adam  # Adam, SGD... (Pytorch class)
   lr: 1e-4
   slow_start: False  #float how much to reduce lr for first epoch
   # Decay LR in step fashion: https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.StepLR.html

lensless/eval/benchmark.py

@@ -124,8 +124,8 @@ def benchmark(
 
             for i, idx in enumerate(batch_idx):
                 if idx in save_idx:
-                    prediction_np = prediction.cpu().numpy()[i].squeeze()
-                    # switch to [H, W, C]
+                    prediction_np = prediction.cpu().numpy()[i]
+                    # switch to [H, W, C] for saving
                     prediction_np = np.moveaxis(prediction_np, 0, -1)
                     save_image(prediction_np, fp=os.path.join(output_dir, f"{idx}.png"))