Skip to content

v0.0.33

Compare
Choose a tag to compare
@jlewitt1 jlewitt1 released this 04 Sep 17:42
· 308 commits to main since this release
bbbd641

Highlights

Docker dev & prod workflows

We released a new guide showing how you can use the same docker image on a Runhouse cluster for both local development and production.

Local Development: We now support passing in requirements to the Runhouse env object instead of preemptively sending the env to the function, ensuring that local changes are synced over.

dev_env = rh.env(name="dev_env", env_vars={"HF_TOKEN": "****"}, reqs=["diffusers", "transformers"])

# we pass in the env object to ensure any requirements specified in the env will be synced onto the cluster properly
dev_fn = rh.function(is_transformers_available).to(cluster, env=dev_env)

Production: The docker image holds all the packages and versions we want, ensuring reproducibility and reduce ingress costs of installing dependencies on each production cluster.

Improvements

  • If no package version is specified, install the same version that is installed locally (#1058)
  • Convert folder to a Module (#995)
  • Support constructing an empty env with only a name (#1095)
  • Support loading an on-demand cluster from name locally in addition to Den (#1093)
  • Add option to skip loading resources from Den (#1096)
  • Update client connection flow to reduce check server calls (#1098, #1141)
  • Updating Folder APIs when interacting with a cluster (#1147, #1116, #1108)
  • Add launched properties to ondemand cluster config (#1139)
  • Stream logs in SSH setup commands (#1211)
  • Logging updates and improvements (#1177, #1178, #1204)
  • Update command running for Kubernetes, and support Docker images with Kubernetes clusters (#1173, #1174)

Clean up

  • Remove sshfs dependency (#1129)
  • Convert cluster rsync method to public (#1082)

Bug Fixes

  • Remove callable param from Folder APIs (#1101)
  • Fix folder construction in the Package (#1083)
  • importlib support for Python 3.7 (#1073)
  • Fix conda installation on cluster (#1176)
  • Have modules created in an env be put within that env (#1194)

Deprecations

Deprecate no longer maintained resources and functionality

Examples

  • Docker workflow guide (#1094)
  • FastAPI RAG app with custom vector embeddings (#1118)
  • Torch image classification example (#1086)
  • Airflow Torch training with Runhouse (#1086)
  • LoRA fine-tuning from a notebook example (#1086)