Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat] initialize from an existing environment #162

Merged
merged 15 commits into from
Apr 30, 2024
Merged

Conversation

AlbertDeFusco
Copy link
Contributor

@AlbertDeFusco AlbertDeFusco commented Apr 24, 2024

In this PR

  • implement conda-project init --from-environment <name or prefix>
  • update to conda-lock 2.5.6
    • 2.3.0 removes the need for a temporary lockfile
    • 2.5.6 fixes input hash updates on re-lock
  • timestamp now added to lockfile
  • on conda-project install only re-lock if the current platform needs to be re-locked
    • previously the lockfile was re-solved if any platform needed it

Using the --from-environment <name or prefix path> flag the environment.yml and conda-lock.<env-name>yml files will be generated from an existing conda environment.

The conda-lock.<env-name>.yml for the current platform is constructed by performing a full conda env export so that the environment is captured as-is into the lockfile. Only packages for the current platform are written in to the lock. When someone attempts to install on another platform conda-project will automatically lock the missing platforms, but not modify the original platform, unless the environment.yml file was changed.

The environment.yml captures the requested conda and pip packages as follows. You'll see that conda packages are prefixed with its channel and the full version is captured in the spec even if a specific version was not originally requested. This helps keep the environment close to the original if the original platform is re-locked and when other platforms are locked.

You'll see that all four major platforms are listed in the environment.yml, but a full cross-platform solve is not attempted by default. The user could do conda project init --from-environment <env-or-path> --lock, which will capture the current platform according to the above procedure and then lock the remaining platforms.

name: data-science-work
channels:
  - defaults
  - anaconda-cloud
  - conda-forge
dependencies:
  - defaults::hvplot=0.9.2
  - defaults::pip=23.3.1
  - defaults::scikit-learn=1.3.0
  - defaults::pandas=2.1.4
  - conda-forge::wheel-filename=1.4.1
  - defaults::panel=1.3.8
  - defaults::python=3.11.5
  - defaults::python-libarchive-c=2.9
  - defaults::ipykernel=6.25.0
  - defaults::intake=0.6.8
  - conda-forge::ratelimit=2.2.1
  - pip:
    - anaconda-catalogs==0.2.0
    - torch==2.2.2
variables:
prefix: /Users/adefusco/Applications/miniconda3/envs/data-science-work
platforms:
  - osx-arm64
  - linux-64
  - osx-64
  - win-64

@AlbertDeFusco AlbertDeFusco requested a review from mattkram April 24, 2024 19:06
Copy link

codecov bot commented Apr 24, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.23%. Comparing base (8aab898) to head (1dabb92).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #162      +/-   ##
==========================================
+ Coverage   97.89%   98.23%   +0.33%     
==========================================
  Files           9        9              
  Lines         996     1074      +78     
==========================================
+ Hits          975     1055      +80     
+ Misses         21       19       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* no need for temporary lockfile
* add timestamp to lockfile output
* only re-lock platforms when the hash changes (or forced)
@AlbertDeFusco AlbertDeFusco requested a review from jjhelmus April 25, 2024 15:54
@jjhelmus
Copy link

LGTM

@AlbertDeFusco AlbertDeFusco removed the request for review from mattkram April 30, 2024 12:25
@AlbertDeFusco AlbertDeFusco merged commit 635ce24 into main Apr 30, 2024
88 checks passed
@AlbertDeFusco AlbertDeFusco deleted the feat/init-from-env branch April 30, 2024 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants