New cyto tool: create cell locations file

.github/workflows/codecov.yml

@@ -2,33 +2,32 @@ name: Code coverage
 
 on:
   push:
-    branches: [ master ]
+    branches: [master]
   pull_request:
-    branches: [ master ]
+    branches: [master]
 
 jobs:
   run:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@master
-    - name: Setup Python
-      uses: actions/setup-python@master
-      with:
-        python-version: 3.7
-    - name: Generate coverage report
-      run: |
-        pip install pytest
-        pip install pytest-cov
-        pip install -r requirements.txt
-        pip install .[collate]
-        pytest --cov=./ --cov-report=xml
-    - name: Upload coverage to Codecov
-      uses: codecov/codecov-action@v1
-      with:
-        file: ./coverage.xml
-        files: ./coverage1.xml,./coverage2.xml
-        directory: ./coverage/reports/
-        flags: unittests
-        name: codecov-umbrella
-        fail_ci_if_error: false
-        path_to_write_report: ./coverage/codecov_report.gz
+      - uses: actions/checkout@master
+      - name: Setup Python
+        uses: actions/setup-python@master
+        with:
+          python-version: 3.9
+      - name: Generate coverage report
+        run: |
+          pip install pytest
+          pip install pytest-cov
+          pip install .[collate,cell_locations]
+          pytest --cov=./ --cov-report=xml
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v1
+        with:
+          file: ./coverage.xml
+          files: ./coverage1.xml,./coverage2.xml
+          directory: ./coverage/reports/
+          flags: unittests
+          name: codecov-umbrella
+          fail_ci_if_error: false
+          path_to_write_report: ./coverage/codecov_report.gz

.github/workflows/python-app.yml

@@ -5,33 +5,35 @@ name: Python build
 
 on:
   push:
-    branches: [ master ]
+    branches: [master]
   pull_request:
-    branches: [ master ]
+    branches: [master]
 
 jobs:
   build:
-
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
         python-version: [3.7, 3.8, 3.9]
         os: [ubuntu-latest, macos-latest]
     env:
-        OS: ${{ matrix.os }}
+      OS: ${{ matrix.os }}
+      # This is needed to avoid a warning from SQLAlchemy
+      # https://sqlalche.me/e/b8d9
+      # We can remove this once we upgrade to SQLAlchemy >= 2.0
+      SQLALCHEMY_SILENCE_UBER_WARNING: "1"
     steps:
-    - uses: actions/checkout@v2
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
+      - uses: actions/checkout@v2
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
 
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        pip install pytest
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-        pip install .[collate]
-    - name: Test with pytest
-      run: |
-        pytest
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install pytest
+          pip install .[collate,cell_locations]
+      - name: Test with pytest
+        run: |
+          pytest

.gitignore

@@ -9,4 +9,5 @@ build
 *.sqlite
 pycytominer/tests/test_data/collate/backend/**/*.csv
 !pycytominer/tests/test_data/collate/backend/**/*master.csv
+!pycytominer/tests/test_data/cell_locations_example_data/*.sqlite
 

README.md

@@ -45,11 +45,12 @@ Since the project is actively being developed, with new features added regularly
 # Example:
 pip install git+git://github.com/cytomining/pycytominer@2aa8638d7e505ab510f1d5282098dd59bb2cb470
 ```
+
 ### CSV collation
 
 If running your images on a cluster, unless you have a MySQL or similar large database set up then you will likely end up with lots of different folders from the different cluster runs (often one per well or one per site), each one containing an `Image.csv`, `Nuclei.csv`, etc.
 In order to look at full plates, therefore, we first need to collate all of these CSVs into a single file (currently SQLite) per plate.
-We currently do this with a library called [cytominer-database](https://github.com/cytomining/cytominer-database). 
+We currently do this with a library called [cytominer-database](https://github.com/cytomining/cytominer-database).
 
 If you want to perform this data collation inside pycytominer using the `cyto_utils` function `collate` (and/or you want to be able to run the tests and have them all pass!), you will need `cytominer-database==0.3.4`; this will change your installation commands slightly:
 
@@ -62,6 +63,43 @@ pip install "pycytominer[collate] @ git+git://github.com/cytomining/pycytominer@
 
 If using `pycytominer` in a conda environment, in order to run `collate.py`, you will also want to make sure to add `cytominer-database=0.3.4` to your list of dependencies.
 
+## Creating a cell locations lookup table
+
+The `CellLocation` class offers a convenient way to augment a [LoadData](https://cellprofiler-manual.s3.amazonaws.com/CPmanual/LoadData.html) file with X,Y locations of cells in each image.
+The locations information is obtained from a single cell SQLite file.
+
+To use this functionality, you will need to modify your installation command, similar to above:
+
+```bash
+# Example for general case commit:
+pip install "pycytominer[cell_locations] @ git+git://github.com/cytomining/pycytominer"
+```
+
+Example using this functionality:
+
+```bash
+metadata_input="s3://cellpainting-gallery/test-cpg0016-jump/source_4/workspace/load_data_csv/2021_08_23_Batch12/BR00126114/test_BR00126114_load_data_with_illum.parquet"
+single_single_cell_input="s3://cellpainting-gallery/test-cpg0016-jump/source_4/workspace/backend/2021_08_23_Batch12/BR00126114/test_BR00126114.sqlite"
+augmented_metadata_output="~/Desktop/load_data_with_illum_and_cell_location_subset.parquet"
+
+python \
+    -m pycytominer.cyto_utils.cell_locations_cmd \
+    --metadata_input ${metadata_input} \
+    --single_cell_input ${single_single_cell_input}   \
+    --augmented_metadata_output ${augmented_metadata_output} \
+    add_cell_location
+
+# Check the output
+
+python -c "import pandas as pd; print(pd.read_parquet('${augmented_metadata_output}').head())"
+
+# It should look something like this (depends on the width of your terminal):
+
+#   Metadata_Plate Metadata_Well Metadata_Site  ...                                   PathName_OrigRNA ImageNumber                                        CellCenters
+# 0     BR00126114           A01             1  ...  s3://cellpainting-gallery/cpg0016-jump/source_...           1  [{'Nuclei_Location_Center_X': 943.512129380054...
+# 1     BR00126114           A01             2  ...  s3://cellpainting-gallery/cpg0016-jump/source_...           2  [{'Nuclei_Location_Center_X': 29.9516027655562...
+```
+
 ## Usage
 
 Using pycytominer is simple and fun.