Skip to content

v0.0.28
Compare
Choose a tag to compare
@jlewitt1 jlewitt1 released this 30 May 18:47
· 601 commits to main since this release
v0.0.28
61da772
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Highlights

runhouse status: Improving visibility into cluster utilization and memory consumption

Improved Cluster Status

Runhouse now provides a more comprehensive view of the cluster's utilization and memory consumption, providing more coverage over the true utilization numbers across each worker and head node of a the cluster.

Information surfaced includes: PID, CPU utilization, memory consumption, and GPU utilization (where relevant).
This data can be viewed as part of the runhouse status CLI command:

GPU Cluster

>> runhouse status
/sashab/rh-basic-gpu
😈 Runhouse Daemon is running 🏃
Runhouse v0.0.28
server pid: 29486
• server port: 32300
• den auth: True
• server connection type: ssh
• backend config:
  • resource subtype: OnDemandCluster
  • use local telemetry: False
  • domain: None
  • server host: 0.0.0.0
  • ips: ['35.171.157.49']
  • resource subtype: OnDemandCluster
  • autostop mins: autostop disabled
Serving 🍦 :
• _cluster_default_env (runhouse.Env)
  This environment has only python packages installed, if such provided. No resources were found.
• np_pd_env (runhouse.Env) | pid: 29672 | node: head (35.171.157.49)
  CPU: 0.0% | Memory: 0.13 / 16 Gb (0.85%)
  • /sashab/summer (runhouse.Function)
  • mult (runhouse.Function)
• sd_env (runhouse.Env) | pid: 29812 | node: head (35.171.157.49)
  CPU: 1.0% | Memory: 4.47 / 16 Gb (28.95%)
  GPU: 0.0% | Memory: 6.89 / 23 Gb (29.96%)
  • sd_generate (runhouse.Function)

CPU cluster

>> runhouse status

/sashab/rh-basic-cpu
😈 Runhouse Daemon is running 🏃
Runhouse v0.0.28
server pid: 29395
• server port: 32300
• den auth: True
• server connection type: ssh
• backend config:
  • resource subtype: OnDemandCluster
  • use local telemetry: False
  • domain: None
  • server host: 0.0.0.0
  • ips: ['52.207.212.159']
  • resource subtype: OnDemandCluster
  • autostop mins: autostop disabled
Serving 🍦 :
• _cluster_default_env (runhouse.Env)
  This environment has only python packages installed, if such provided. No resources were found.
• sd_env (runhouse.Env) | pid: 29716 | node: head (52.207.212.159)
  CPU: 0.0% | Memory: 0.13 / 8 Gb (1.65%)
  This environment has only python packages installed, if such provided. No resources were found.
• np_pd_env (runhouse.Env) | pid: 29578 | node: head (52.207.212.159)
  CPU: 0.0% | Memory: 0.13 / 8 Gb (1.71%)
  • /sashab/summer (runhouse.Function)
  • mult (runhouse.Function)

Improvements

  • Cluster status displays additional information. (#653)
  • Polling den with cluster status data (#806)
  • Prevent exposing user Runhouse API tokens on the cluster by saving a modified hashed API token (#797)
  • Use env vars in default env creation (#798)
  • Login flow improvements (#796)

Bug Fixes

  • Fix undefined path when pip installing a folder (#826)
  • Don't pass basic auth to password cluster HTTP calls (#823)
  • Fix env installations that contain a provider secret (#822)
  • Refresh sys.path upon loading a new module (#818)

Docs & Examples

  • Update domain docs (#812)
  • Add default env section to envs tutorial (#810)
  • Minor improvements to parallel embedding example (#795)
Assets 2
Loading