Skip to content
This repository has been archived by the owner on Jul 16, 2020. It is now read-only.

Returning FULL when diskspace is low on a node no longer makes sense. #1541

Open
markdryan opened this issue Oct 24, 2017 · 0 comments
Open

Comments

@markdryan
Copy link
Contributor

The notion of FULL for disk spaces is totally broken at the moment. We broke it when we moved VM instances to ceph only and never fixed it.

The problem is that launcher reports FULL when there is less than 80GBs on the drive on which /var/lib/ciao/instances is located. But this doesn’t make sense any more.

  1. Storage for VM instances is taken from the ceph cluster so the amount of free space in /var/lib/ciao/instances is mostly irrelevant, unless you’re down to your last 10kb or so.
  2. Container instances do use local storage but they do not store their state in /var/lib/ciao/instances.
  3. Even if we were correctly computing the free space on a node for container instances, and we determine that there is no space left for any more containers, should we return FULL? Returning FULL, would prevent us from hosting any more VM instances on this node, even though these instances are not disk space bound and should be able to run happily.

To really fix this we’d need to have two different FULL statuses and we’d need to find a way to accurately determine how much disk space is available to containers on a given node. Perhaps there’s some way we can ask docker for this info.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant