Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Request idling containers exit when the container limit is reached #1120

Merged
merged 1 commit into from
Nov 14, 2024

Conversation

shepmaster
Copy link
Member

We let WebSocket connections keep {stable,beta,nightly} containers
running after the execution finishes. These idle containers still
count against the enforced container limits even though they aren't
doing useful work.

This change allows the owner of those idle containers to know that
someone else wants their slot and then they can make the containers
exit earlier than they otherwise would have.

This can be best shown by setting the container limit very
low (e.g. 2) and then opening that many plus one more browser
tabs (e.g. 3). Before this change, the last tab would need to wait for
the idle timeout (defaults to 60 seconds) before they could have a
chance to run their code.

= Known issues

There's a race that can cause containers to spuriously exit, but that
should only occur when we are near the cap of containers anyway. More
details are in comments.

= Future thoughts / limitations

One WebSocket connection could have both an active container and an
idle one (e.g. stable and nightly). This case is not handled, but it
may also be comparatively rare.

The non-WebSocket endpoints can request other containers to exit, but
they won't ever decrement the exit request counter themselves. This
means that if you hit the server many times at these endpoints, then
future WebSocket connections will exit earlier than they need
to. Hopefully this is also comparatively rare.

We let WebSocket connections keep {stable,beta,nightly} containers
running after the execution finishes. These idle containers still
count against the enforced container limits even though they aren't
doing useful work.

This change allows the owner of those idle containers to know that
someone else wants their slot and then they can make the containers
exit earlier than they otherwise would have.

This can be best shown by setting the container limit very
low (e.g. 2) and then opening that many plus one more browser
tabs (e.g. 3). Before this change, the last tab would need to wait for
the idle timeout (defaults to 60 seconds) before they could have a
chance to run their code.

= Known issues

There's a race that can cause containers to spuriously exit, but that
should only occur when we are near the cap of containers anyway. More
details are in comments.

= Future thoughts / limitations

One WebSocket connection could have *both* an active container and an
idle one (e.g. stable and nightly). This case is not handled, but it
may also be comparatively rare.

The non-WebSocket endpoints can request other containers to exit, but
they won't ever decrement the exit request counter themselves. This
means that if you hit the server many times at these endpoints, then
future WebSocket connections will exit earlier than they need
to. Hopefully this is also comparatively rare.
@shepmaster shepmaster added the maintenance Keeping the wheels turning label Nov 13, 2024
@shepmaster shepmaster merged commit 0f564e6 into main Nov 14, 2024
12 checks passed
@shepmaster shepmaster deleted the sharing-is-caring branch November 14, 2024 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Keeping the wheels turning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant