CHANGELOG

v0.4.0-beta.2 (2024-12-28)

Bug Fixes

fix: pass build argument for env container version (6364a77)
fix: ensure build script runs relative to itself (458af39)
fix: changelog output filename (33ba8bb)

Features

feat: bump versions for scvi-tools, scvelo, cupy-core (3d59228)
feat: generate conda-lock files in parallel (80fb3cf)
feat: commit the metadata and image version on build.sh (7a6b95d)
feat: add changelog templates (ec36a60)

Refactoring

refactor: full image naming (d536ff5)

v0.4.0-beta.1 (2024-12-28)

Bug Fixes

fix: default repository to localhost (1b1580f)
fix: revert back to docker.io for cache registry (31d4a08)
fix: Change permissions on /opt and /tmp to give read-write user permissions #34 (34092ef)
fix: permissions on /tmp and /opt to install

Fixes #34 (81f1fd7)

fix: quarto should download amd64, not arm64 (5435219)
fix(docker): permissions on /tmp and /opt in base dir (#34) (197b233)
fix: WIP temporarily pausing automatic github actions with selfhosted runner offline (4dc2b24)
fix: build should always occur on the main branch (#17) (7a50800)
fix: adjust GH Runner labels (#16) (2075c2d)

Build System

build: WIP docker compose build process (ac512e2)
build: create a main image (base env) conda lock file (e4ce16f)
build: remove the separate cache build. simplify export of layers (a0beb4b)
build: update conda-lock file for v0.4.0 (82aebef)
build: full image uses main image as base and copies python from full-env image. Improves build time with simultaneous multistage builds (238e6fb)
build: move copy steps to the end to improve caching (31958ab)
build: use the buildkit for base image (708ff6e)

Chores

chore: save the current state (cfc682a)
chore: temp disable workflows (431ae53)
chore: rename (accef9e)
chore: temp before pull (35eb2d2)
chore: Merge branch 'main' into develop (c6d433b)

Continuous Integration

ci: 0.4.0-beta.1

Automatically generated by python-semantic-release (9d74ab2)

Features

feat: pass in discrete conda file names as arguments from parameters (045c588)
feat: add tmux, argcomplete, and bump quarto to v1.6 (#36)

Resolves #35, #31, #26 (3095cb7)

feat: pass in rclone version parameter. simplify base image parameters (a875752)
feat: create a unified build script which sources ./parameters

will replace directory specific build scripts. WIP docker-compose build support (396e094)

feat: use a pre-build conda-lock file for the main image too (eb017c3)
feat: bump micromamba to 2.0.5; bump cuda to 12.6.3 (85bcd10)
feat: updated conda-lock (93f8ef7)
feat: add spatialdata (24160fc)
feat: pushing to a local registry (ff071a6)
feat: update for cuda 12.6 and new directory structure (01a6579)
feat: add jq to main

Fixes: #27 (4379d56)

feat: add github action to update the develop branch when a new commit to main is made (#30)

Also pausing all other github actions until our self-hosted runner is working again (01d4b8c)

feat: using CUDA 12.6.1 as base; add nvidia-container-images repo as subdataset (#29) (5ad67bf)
feat: bump quarto v1.5 --> 1.6 (64d75c8)
feat: add argcomplete

Fixes #31 (803af02)

feat: add tmux to base

Fixes #26 (d0022a5)

feat: add github action to update the develop branch when a new commit to main is made (cd2fa23)
feat: discrete python build stages; semantic versioning; packages: gitpython, yadm, pptx, bump quarto to 1.5.56; cleanup old images (#15)

Docker build

The main and full images now have a separate state just for building the python environment. This is done using the standard docker builder, not the docker buildx build due to an issue with hardlinks. The buildx builder has issues when moving conda packages across the hardlink from their downloaded directory to the environment directory. Could not resolve this using different conda commands or conda-lock. As a benefit, now this separate image improves caching, which improves build times.

Dependencies

yadm: efficient pulldown of configuration templates - helpful in ephemeral environments like docker
gitpython: automate git, specifically branch creation, in conjunction with datalad. datalad does not natively support actions on branches
semantic versioning with python-semantic-release
python-pptx: generate PowerPoint presentations with python - useful when we need to quickly combine figures and send to colleagues in PowerPoint format for review.
Upgrade quarto version to 1.5.56 (7a4cdae)

Performance Improvements

perf: move labels to the end for better caching (1ded2cc)

Refactoring

refactor: rename files for the new build system (64ad553)
refactor: rename base image dockerfile name (a32283c)
refactor: move labels to the end of the dockerfile to improve caching (8184549)
refactor: Rename python environment directory (#28) (c02d757)

Unknown

Merge branch 'FacsimiLab:main' into develop (eea7796)
Merge branch 'FacsimiLab:main' into develop (a07d956)

v0.2.5 (2024-08-27)

Bug Fixes

fix: build should always occur on the main branch (faeec95)
fix: adding additional gh runner tags (1e04e4c)
fix: runs on label having error picking up (29cb883)
fix: workflow permissions (fddfb21)
fix(code): change the changelog output filename (454d50b)
fix: remove git commit from action due to failure (5e2f5f3)
fix: path to image_version (4a54073)
fix: corrected spelling of environment and set push-all repository (8f18f06)
fix: gpu --> gpus (7998d2f)
fix: cat to echo (782b6cf)
fix: appending directly to the github step summary || removed building on every commit (4e2ce0d)
fix: cleanup python package check (fb65d99)
fix: build arguments for base (054c9f7)
fix: add condition for action (6f73e25)
fix: add condition for action (c4072e5)
fix: change permissions to 755 (7b4df80)
fix: adding a git (56e7eb7)
fix: cleanup (0bc4248)
fix: change back to selfhosted due to docker container size limitation (cdec609)
fix: append to overwrite and add gitconfig (25e9862)
fix: append to overwrite and add gitconfig (9abbf06)
fix: missing -e flag for environment (391e888)
fix: GH action (054592b)
fix: was not passing in image_version.txt (d2a781b)
fix: build all was not building all (f4e8334)
fix: runs on statement (b704129)
fix: branch specification (09ee265)
fix: branch specification (b458f8f)
fix: pushing the main-env in two stages also (13dd422)
fix: push full images (9bccc7b)
fix: buildall was not building all (78a94bd)
fix: add docker pull (4cedb0a)
fix: multiple jobs (08f5f48)
fix: matrix server (42ff5f1)
fix: use current facsimilab label to test image (3f052b3)
fix: pass in conda env name (29f53c2)
fix: order of action (c59e099)
fix: adjust name of action (6f151a7)
fix: pull images by tags (67b054b)

Build System

build: update conda lock file (b58aad5)
build: v0.2.5-delta on gitea (161dd67)
build(dependencies): update / pin depedencies based on CVE reports (153d604)

Chores

chore: chmod after backup restoration (09de4ef)
chore: removing extraneous workflows (ce36ac8)
chore: rename main-env and full-env to main and full

In a previous commit, we deleted the single stage image builds. now we are renaming the multi-stage ones to the original directory names. This will clean up the repository. (c0baff2)

chore: cleanup extraneous dev_test_full and related workflows (58c5d52)

Continuous Integration

ci: 0.3.0-beta.1

Automatically generated by python-semantic-release (e6d3715)

ci: set git safe directory to allow write permissions (06c56bb)
ci(code): apply docker labels and push (c403d44)
ci: starting build on selfhosted ubuntu22 runners not the custom images (758478c)
ci: starting build on selfhosted ubuntu22 runners (a89fc61)
ci(build): testing containers version wise (a501a09)

Features

feat: build containers using cuda 12.6.1 and ubuntu 22.04 as the base image (ad3ef42)
feat: add nvidia-cuda image repo as a subdataset (478cfdf)
feat: add semantic-versioning configuration (a017b56)
feat(dependencies): bump quarto to version 1.5.56 (dc28e8e)
feat(dependencies): add gitpython (f724d64)
feat: mount local user's git config to the devcontainer (bedfbee)
feat(dependencies): yadm dotfile apt package (3368173)
feat: add bash prompt coloring (0cd3e57)
feat(dependencies): add powerpoint python package to full image (20f17c5)
feat: commit test results to git (cd6df7f)
feat: add matrix notification (d5cec47)
feat: have the python environments checked after the build is complete (3b194ba)
feat: Installing docker directly into the runner rather than in facsimilab (ceae5d1)
feat: add in apt caches (bd72fb0)
feat: explicitly pull the base image for each build (48b4041)
feat: pin exact sha of cuda image; cleanup (866d1a4)
feat: pin exact sha of cuda image; cleanup (9707833)
feat: switch the python package check to github and expand space (206529b)
feat: add build conditions (cff9f85)
feat: simplified build and push workflow (48ebe3e)
feat: simplify bleeding edge (f34951f)
feat: simplify bleeding edge (de15ecc)
feat: simplify bleeding edge (62160ce)
feat: add exact commit SHA for the cuda base (e3b6a0d)
feat: add more labels and cleanup (a9be95c)
feat: add more OCI labels (1e54867)
feat: script to clear cache (fbe4830)
feat: cache export and image export separate for main (15b96bb)
feat: add docker login, push main image directly (635a89c)
feat: new action for bleeding edge (each commit) (f7200cc)
feat: new action for bleeding edge (each commit) (24f6e8b)
feat: add additional notifications (09f4ad2)
feat: change to matrix action (95540b2)
feat: add webhook notification additional (5ac4e21)
feat: add webhook notification (4ce1606)
feat: add readme and better build completion statements (440e0cb)
feat: add gitea actions (619ee26)
feat: building from full-env and main-env (0937c50)
feat: build main image in 2 separate containers, pushing each stage (f64e11e)
feat: set default to copy environment, not link (423811e)
feat: building image with some CVE addressed, lightning issue WIP

created github issue on lightning dependency which is different from PIP vs. conda conda cannot solve the environment with the updated lightning. will update in the next one. (7f29646)

feat: add build-and-push to main for workflow-dispatch (988bdd6)

Refactoring

refactor(docker): changed directory name for clarity (1d389db)
refactor(docker): removing deprecated code with single stage main and full image builds

!BREAKING CHANGE: the older build-all script pointed to the /base, /main, /full directories, which are no longer present (2d432e1)

Testing

test: contribution to develop branch to see if actions run (#18) (c5c6b3a)
test(dependencies): refactor / minor changes to gh actions (35887fc)
test(code): change path of image version (ef637a4)

Unknown

v0.2.5 - Update packages, multi-stage build with standard and buildx, ci / tests (e9f1989)
Merge branch 'main' into version/0.2.5 (38905c2)
[DATALAD] Added subdataset (c1400d2)
Merge branch 'main' into develop (a801138)
add action (694cf6f)
add testing results (949b91c)
add actions (282237a)
trying to pass in gpu (dac127f)
debugging the job summary build (9413ed9)
Merge branch 'gitea' of github.com:FacsimiLab/dev-platform into gitea (102dc55)
change version (48c3a8f)
fix concurrancy (e37eade)
Merge branch 'gitea' of github.com:pranavmishra90/facsimilab-platform into gitea (bad7a5a)
Update build-and-push.yml (0d35691)
remove conditions for the test compose action (4c75f9e)
Merge branch 'gitea' of github.com:pranavmishra90/facsimilab-platform into gitea (a78a95f)
adding more commands including copy (b1bcbba)
adding more commands including copy (ae10e7b)
add quotes (5837efc)
add quotes (6dfcf19)
add quotes (987a0b0)
try new (e615601)
try new (2e8e71e)
change to selfhosted (aea2c7c)
change full to main image (99bdf73)
running a test using a docker container run (7221ad2)
test container workflow (1eceda5)
get image info (8514f23)
get image info (523fd12)
get image info (65e9ddf)
get image info (b733ff3)
get image info (c954638)
using workspace var (dfa7263)
changing to run (89fde89)
changing to run (4da9c6a)
fix missing tree (1060779)
testing (a3f0903)
using actoins used in protocol timer (4b78590)
fix concurrancy and -e flag (e0c4c92)
tmp remove concurrancy (4f13427)
modify actions (4164ab9)
change name to start action (f6e4c2f)
improved workflows (fa8d739)
Fixing full image WIP (479c8ca)
ix: runs on (2855187)
v0.2.5-beta8 - CI working on main image, now building all (0f4fa2d)
removed cuda submodule [WIP] (303b6e7)
Merge branch 'main' into develop (520140f)

v0.2.4 (2024-08-09)

Bug Fixes

fix: ci cleanup was erroring out; push was difficult to track (cf4ceec)
fix: error in spelling filename (c418d64)
fix: move condarc into the final build stage (b09130a)
fix: remove extraneous files, get standard jupyter notebook working for a dockerized setup (bf5ebbf)
fix(test): Mount gitconfig in github runners (254ec62)
fix: cat to echo (03c50ec)
fix(ci): main and full image prints (c922e5c)
fix(ci): header typo (efd13ba)
fix(ci): Job summary sources (9ab96bc)
fix(test): correct python path for main container (b7f2c55)
fix(test): correct python path for main container (aa9cead)
fix(test): Add tests for main and full image to the build and push (b74fb57)
fix: change docker label to latest (394eaaf)
fix(wip): removing tty terminal (7036fb8)
fix: runs on (084085d)
fix: Add credentials to ghcr.io (63e2ae0)
fix: container label (685df5f)
fix: docker tags and repository to ghcr (1ad2e19)
fix: Fix problem with seeing and logging the push to docker repos (9358b5d)
fix: Removed ~/ from being mounted which interferes with docker configuration (2522376)
fix: build and push scripts can be run from any directory (5d5b91e)

Build System

build(dependencies): bump python package versions (1867952)
build(docker): New script to build only the full container, tag as dev, and push to Docker (808418c)

Code Style

style: cleanup (d4cf036)

Continuous Integration

ci: trying checkout v4 (39b707d)
ci: trying checkout v3 (975f168)
ci: do not perform clean checkout due to error (c1df9ee)
ci: Add condition for run (6b0e573)
ci(test): check datalad for main (3889537)
ci: ci: Add filters to the pull request (0de54c8)
ci(docker): Push only to docker, not gitea (4484e75)
ci(dependencies): Check python packages in main and full images (cae239b)
ci(dependencies): Test only on a push to main (ed41b24)
ci(docker): WIP build and push (2d1fb0a)
ci(docker): Add condition to check if a new build_version is detected (63ca558)
ci(docker): Build and push a tagged image version (bb7706e)
ci: create a job summary (3894897)
ci: add selfhosted runner (4a2c5a6)
ci(test): Results to an upload artifact (121b48f)
ci(test): Running versions inside of the docker container (9c6e5b6)
ci(wip): using dockerhub (b5b8996)
ci: Init a dev facsimilab container test (75c0f12)

Documentation

docs: (#5) Update readme, licenses for the GPU capable multi image facsimilab platform

Merge pull request #5 from pranavmishra90/develop (ba96eb3)

docs: Update readme for the GPU capable multi image facsimilab platform (e6f47ce)

Features

feat: modified readme for pranavmishra90's images (0d28f8a)
feat: making small change for ci (3acfe74)
feat: full image build and buildx engines working. testing buildx ci (619aa30)
feat: add the nvidia/cuda base image as part of build-all

We want to ensure that our ubuntu base image is up to date (d063567)

feat: rename build_version.txt to image_version.txt for ease of autocomplete (7b2778d)
feat: improved caching (cf0c0b4)
feat: improved workflow, simplified copy statements (9be923f)
feat: start pushing docker base images in background during build (e5f1f14)
feat(dependencies): Removing zenodo_get due to reducancy with pooch (48bfafc)
feat: Add dockerfile syntax (64bb65b)
feat: add openpyxl for reading excel files (dbfacd5)
feat: WIP v0.2.5 with added and updated packages, improved ci, tested on tutorials (ec202a4)
feat: Add bbknn (94458e4)
feat: Add primer3-py (a2ab3a4)
feat: Add faiss-gpu (66f3551)
feat: Add dask (8986b9e)
feat: Add current state (50222a8)
feat(ci): when adding python packages, build only the full image (956601a)
feat: set default environment (81cfe9d)
feat: Add additional scverse packages, rmm (6fc4650)
feat: add new packages and notebooks for testing (0747686)
feat: results for versions (ef714ac)
feat: Pass through ports for jupyter server in docker container (0a32d33)
feat(dependencies): Add fa2 v0.3.5 (3ab6c59)
feat(dependencies): Add ForceAtlas2 algorithm for network visualization

Add accompanying citation: 10.1371/journal.pone.0098679

Starting new version --> v0.2.5 (ea153df)

feat: Add references in biblatex format (eb577c0)
feat: Dynamic build and push scripts; conda-lock

version number is picked up from a text file inside of the /docker dir this is then fed into the build-all.sh and push-all.sh files

we are now using a conda-lock file to ensure proper images. This is programmatically generated from the environment.yml file inside of the full container's directory. (c926fab)

feat(dependency): Add pipx, preferring native docker builder, cleanup (c5b0f2c)
feat(build): WIP building v0.2.4-lock (93ddd6e)
feat: add local bare metal conda lock (37e14c8)
feat(dependency): WIP testing cuda 12.4 container

Building platform:

feat(dependency): bump datalad 1.1 (79f78c3)
feat: move quick deploy (b529e0e)
feat(devcontainer): switch back to full container (a72e4e4)

Performance Improvements

perf(dependencies): Do not rerun a conda-lock generation (1aa62ae)

Testing

test(dependencies): Add addtional packages to be tested

Taken from the Prrx1 notebook for hardware and software testing (c2956fe)

test(dependencies): Check current package versions and see if updates are possible (91c7139)
test: checking versions of python packages in the container (5f2041c)

Unknown

Using a conda lock file to fix GPU-enable pytorch + dependencies not being added correctly (e7f380e)
[DATALAD] Added subdataset (705e263)
[DATALAD] Recorded changes (f914234)
[DATALAD] new dataset (2eca83d)
[AUTO]Docker images built and pushed

Files changed: M docker/metadata/01-base_metadata.json M docker/metadata/02-main_metadata.json M docker/metadata/03-full_metadata.json M testing/results/full-image.txt M testing/results/main-image.txt (e6d5f8f)

Merge branch 'main' into develop (9dfd507)
having issue with jaxlib installation not detecting cuda. testing on older image (8ff5f28)
having issue with jaxlib installation not detecting cuda. testing on older image (dffe3fd)
moving out doublet detection (50a11b4)
Removed code-workspace from git (a66a961)
doc(licenses): Add Nvidia CUDA docker image license (fbb336f)
Merge branch 'main' into develop v0.2.1 (578ba06)

v0.2.1 (2024-07-04)

Bug Fixes

fix(docker): .profile --> .bash_aliases. removed /bin/conda causing init error. installed main base into base env (07b1c00)
fix(docker): remove failing autoactivate of facsimilab env (9596784)

Build System

build(docker): build scripts working for all images (11879f8)

Documentation

docs(licenses): Add rclone license (3596cf3)

Features

feat(docker): Add rclone for datalad remote siblings. .bash_aliases. Base and facsimilab env (#4) v0.2.1
Add / Fix: [WIP] Working docker image with CUDA12.1

Why: cuda was not working on the previous method of config

How:

base image has changed to nvidia's image. no longer using vscode since that is image bloat

Pytorch is being installed with pip with a cuda requirement. This is uninstalling the version from conda and reinstalling

Tags: nvidia, cuda, pytorch, pip, conda

Fix: torch version printout
Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage

Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage

feat(docker): Multistage builds

faster testing of docker builds. Improved debugging with python packages

feat: Add dynamic load/run of scvi model if it does / does not exist

Determine if a trained scvi model exists. If so, load it. If not, train a new model.

feat(docker): Updated method for passing version number; push script which adds latest tag
feat(docker): Add dockerignore
feat(docker): updated quick build and docker-compose files for testing
feat(docker): preparing v0.2.1 which adds back rclone
feat(docker): Add sound alert on script completion
fix(docker): remove failing autoactivate of facsimilab env
fix(docker): .profile --> .bash_aliases. removed /bin/conda causing init error. installed main base into base env
build(docker): build scripts working for all images
docs(licenses): Add rclone license (6115a0d)
feat(docker): Add rclone, use.bash_aliases, use discrete base environment for facsimilab-main

WIP v0.2.1 Add rclone, use .bash_aliases, use discrete base environment for facsimilab-main (e987f9f)

feat(docker): Add sound alert on script completion (f94ca6a)
feat(docker): preparing v0.2.1 which adds back rclone (96c9e41)

Unknown

Merge branch 'main' into develop (73e9a2f)

v0.2.0 (2024-07-04)

Features

feat(docker): Creation of multiple docker images (#2) v0.2.0
Add / Fix: [WIP] Working docker image with CUDA12.1

Why: cuda was not working on the previous method of config

How:

base image has changed to nvidia's image. no longer using vscode since that is image bloat

Pytorch is being installed with pip with a cuda requirement. This is uninstalling the version from conda and reinstalling

Tags: nvidia, cuda, pytorch, pip, conda

Fix: torch version printout
Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage

Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage

feat(docker): Multistage builds

faster testing of docker builds. Improved debugging with python packages

feat: Add dynamic load/run of scvi model if it does / does not exist

Determine if a trained scvi model exists. If so, load it. If not, train a new model.

feat(docker): Updated method for passing version number; push script which adds latest tag
feat(docker): Add dockerignore
feat(docker): updated quick build and docker-compose files for testing (92fe9cd)
feat(docker): updated quick build and docker-compose files for testing (a4f43a9)
feat(docker): Add dockerignore (f49490b)
feat(docker): Updated method for passing version number; push script which adds latest tag (fb4d039)
feat: Add dynamic load/run of scvi model if it does / does not exist

Determine if a trained scvi model exists. If so, load it. If not, train a new model. (5438df0)

feat(docker): Multistage builds

faster testing of docker builds. Improved debugging with python packages (0821ac8)

Unknown

Merge pull request #1 from pranavmishra90/feat/multiple-images

v0.2.0 - Creation of multiple docker images (2a5d5c3)

Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage (35f9d5a)

Add: 3 docker containers with increasingly larger sizes and number of packages

Why: Want to provide minimal docker containers while improving build efficiency

How: multi-stage builds

Tags: docker, multi-stage (57781c4)

Fix: torch version printout (445f226)
Add / Fix: [WIP] Working docker image with CUDA12.1

Why: cuda was not working on the previous method of config

How:

base image has changed to nvidia's image. no longer using vscode since that is image bloat

Pytorch is being installed with pip with a cuda requirement. This is uninstalling the version from conda and reinstalling

Tags: nvidia, cuda, pytorch, pip, conda (23013e1)

v0.1.0 (2024-05-07)

Unknown

Fix / Rephrase: Migration to Facsimilab

Replacing references to singulab. (d63a9ea)

Migration to Facsimilab

We have renamed the SinguLab project to FacsimiLab!

Our new name is a portmonteau of facsimile and laboratory. Not only is it "more unique", it is also more reflective of the project's objective: to create a reproducible research environment with identical packages and platforms. Using environment files and OCI compatible build instructions, FacsimiLab will be the path forward for replacing "it works on my computer" with "I've looked at your data and reproduced your results".

We hope you follow our project's ongoing development at github.com/FascimiLab and pranavmishra90/FacsimiLab-platform.

This GitHub repository at pranavmishra90/SinguLab is now archived. (0fc5557)

Bump scvi-tools to 1.1 (ac2b127)
WIP (aad2ffe)
Name change (37f1c44)
Adding major licenses and references (88da37a)
Bump: Quarto to 1.4.550

Why: Increase from Quarto 1.3 to 1.4 has many additional features found at: https://quarto.org/docs/blog/posts/2024-01-24-1.4-release/

How: https://github.com/quarto-dev/quarto-cli/releases/download/ v1.4.550/quarto-1.4.550-linux-amd64.deb

Tags: quarto (6404173)

Pin 4.21.2-39 for coder (1f54160)
Replacing singulab with facsimilab (6585827)
Name change to FacsimiLab and archive repository

Name change to FacsimiLab and archive repository (b4295c6)

Modify readme to explain name change and archive status (b843ca1)
Modify readme to explain name change and archive status (4cd654b)
Merge pull request #7 from pranavmishra90/debug

New CPU based image. Removed deprecated jupyter server. GPU container is WIP (b72efc8)

chmod (b9b6627)
Fix Docker image printout at the end of build (dec7c52)
Fix: docker image names, documentation (54501a0)
Removed the original jupyter notebook base image which is no longer being used (706ed63)
WIP: Testing Nvidia CUDA (0abdc06)
Add (WIP): Nvidia CUDA based container for GPU accelerated analysis

Why: performing next-gen -omics experiments in the CPU takes many multiples longer than using a GPU. We will leverage Nvidia's CUDA for GPU based analyses.

How:

Using nvidia/cuda:12.3.1-base-ubuntu22.04 as a base container. Tags: GPU, Nvidia, CUDA (937bc74)

Now testing scRNAseq env copied from the experiment (7499aed)
begin debugging (b77cfbb)
Rename directory (389494f)
Update python environment (4ccac7f)
WIP VSCoder with mamba setup (68a0fab)
Starting a vscode version (d0d284b)
Add: apt cache for builds; Bump: python 3.10; Fix: mamba over conda;

Why: We want to utilize the docker buildx build for caching the apt install steps.

Python 3.10 is stable on our test bench computer

Mamba is faster than conda

Tags: docker build, mamba, conda, python (e9ac76a)

Make: using the newer docker image; Add: Traefik labels

Why: old name is no longer being used / updated. Adding functionality for Traefik on VPS

Tags: docker, traefik (e662b04)

Merge branch 'main' into develop

Signed-off-by: Pranav Kumar Mishra 62562712+pranavmishra90@users.noreply.github.com (b1be74d)

Upgrade pre-commit-hooks and add code workspace file (5b09cbb)
Merge branch 'develop' of github.com:pranavmishra90/SinguLab into develop (ceffeb7)
Add: 'feature/singularity' into develop

Apptainer (Singularity) has a number of advantages over standard docker setups. Among them is the ability to run rootless containers in HPC configurations. Adding the ability to create an Apptainer image from the docker image.

Changes to be committed: new file: singularity/Singularity (7b42785)

Creating a Singularity (Apptainer) image (4f6de56)
Merge pull request #1 from pranavmishra90/feature/docker

Change workdir to ~/work (f83fad4)

Change workdir to ~/work (0825d98)
Merge branch 'main' into develop (496c225)
Merge branch 'develop': Fixing LF endings (a68451e)
Using submodules for the coder templates (7a2e181)
Removed coder directory (00a0590)
VSCode is working! had to remove dotfiles though (c036e87)
Merge branch 'feature/coder' of github.com:pranavmishra90/SinguLab into feature/coder (8c2fa72)
Fix: VSCode, documentation for terraform (269488a)
Fixing vscode plugin, but newer change is available on server (e04d028)
testing a new terraform file (ef6e178)
Fix: replaced jovyan/lab with coder/work for username and dir (ce4894f)
changed ports (1febd25)
removed yadm and .jupyter directory (13ae7ed)
added vscode directory (0180b4c)
changing all files to LF ending (25dc1fd)
Fix: basic improvements with docker-compose readme and changelog (c066e02)
Merge branch 'main' into develop (5cda0e6)
Add: coder .tf file; Fix: user permissions and naming of dockerfile

Why: furthering self-contained environment

How: Using coder/coder

Tags: datascience (9447f7c)

pre-v0.0.8 (af617b4)
this is not building at the fix-permissions file, so changing branches (80481fd)
added coder/code-server, adjusted image version to 0.0.8 (a046d69)
added vscode extension for jupyter lab (210eb57)
Add vscode server from Coder to the image (bda6a60)
Fix permissions to 777 so that the permissions work inside the container (ad91114)
moved cachebust (452b1d1)
adding primer3-py via bioconda (25bc890)
copying the rclone install file into the container, then running (ecedde2)
copying the rclone install file into the container, then running (1a32f2f)
fix minor issues in the README (a9f31ce)
fix python version in README (c20bee7)
Merge branch 'main' into develop (9442b17)
New features: rclone for git-annex, datalad via conda, cachebust

Squashed commit of the following:

commit 96e083fee1e0af75f0d0061468011aa8f2798b7d Author: Pranav Kumar Mishra 62562712+pranavmishra90@users.noreply.github.com Date: Tue Sep 6 01:12:04 2022 -0500

Fix: Now using conda for datalad, Add: cachebust

Why: apt-get datalad was pulling v0.15 vs newest 0.17. Now using conda

mamba via conda-forge somehow was not pulling the latest either

How: removed apt datalad, added conda datalad package

Tags: Datalad, conda, apt, dockerfile

commit b20706463c6df9fcca97ba2876de58554fffc06e Author: Pranav Kumar Mishra 62562712+pranavmishra90@users.noreply.github.com Date: Mon Sep 5 18:22:56 2022 -0500

Add: Git-Annex-Remote-rclone
Why: can use rclone as a datalad sibling
How: added repo + rclone config ([`a6abb0b`](https://github.com/pranavmishra90/facsimilab-platform/commit/a6abb0bf12c688819b325d5e864b03090612cb16))

Fix: Now using conda for datalad, Add: cachebust

Why: apt-get datalad was pulling v0.15 vs newest 0.17. Now using conda

mamba via conda-forge somehow was not pulling the latest either

How: removed apt datalad, added conda datalad package

Tags: Datalad, conda, apt, dockerfile (96e083f)

Add: Git-Annex-Remote-rclone Why: can use rclone as a datalad sibling How: added repo + rclone config (b207064)
fixing errors from merge (3488d09)
Merge branch 'develop' (7042b6a)
change gnupg2 to gnupg, fix permissions (a2f4b96)
Fix: Dockerfile, docker-compose to follow nightly builds if wanted

Why: Using vscode to build containers was not picking up all changes

How: created new tag :nightly, which will be pushed to gitea and github

Tags: Docker (f1fa0a8)

Created nightly tag and streamlined process for vscode containers

commit f1fa0a8de8dffd7fac8bc2e16765c8405ff99400 Author: Pranav Kumar Mishra 62562712+pranavmishra90@users.noreply.github.com Date: Wed Aug 24 21:35:02 2022 -0500

Fix: Dockerfile, docker-compose to follow nightly builds if wanted

Why: Using vscode to build containers was not picking up all changes

How: created new tag :nightly, which will be pushed to gitea and github

Tags: Docker ([`64d8d98`](https://github.com/pranavmishra90/facsimilab-platform/commit/64d8d986e5b600f48426b2c56ca15cb1982cccc5))

Optimize: Moved custom apt-packages to the bottom || Add: exa (for ls)

Why: Docker image taking a long time to rebuild because the additional packages which are being added modify the earlier layers

Now building a 'fixed' base with my changes at the end Note a proximal change is fixing the python version to 3.9

How: Move RUN command for my addons to the end Add 'exa' to this run command

Tags: Dockerfile (1873813)

Add: Missing files from jupyter/base-notebook for image creation

Why: required to create the image

How: taken from jupyter/base-notebook

Tags: (a59d639)

(Created seperate git repository specifically for this container) . Why: Difficult to maintain image creation inside a specific branch of the first experiment's git repository

This is following the YODA principles outlined in the README

How: Created seperate git repo, pushed

Tags: Docker, Dockerfile, docker-compose (475aba2)

Initial commit (e888b16)

Files

changelog.md

Latest commit

History

changelog.md

File metadata and controls

CHANGELOG

v0.4.0-beta.2 (2024-12-28)

Bug Fixes

Features

Refactoring

v0.4.0-beta.1 (2024-12-28)

Bug Fixes

Build System

Chores

Continuous Integration

Features

Docker build

Dependencies

Performance Improvements

Refactoring

Unknown

v0.2.5 (2024-08-27)

Bug Fixes

Build System

Chores

Continuous Integration

Features

Refactoring

Testing

Unknown

v0.2.4 (2024-08-09)

Bug Fixes

Build System

Code Style

Continuous Integration

Documentation

Features

Performance Improvements

Testing

Unknown

v0.2.1 (2024-07-04)

Bug Fixes

Build System

Documentation

Features

Unknown

v0.2.0 (2024-07-04)

Features

Unknown

v0.1.0 (2024-05-07)

Unknown