Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dobby replicate images to regions #228

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Dobby replicate images to regions #228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
14953d6
WIP Dobby: method to transfer image to @regions
Davis-A Mar 11, 2024
8495249
WIP InABox: replicate fminabox to all configured regions
Davis-A Mar 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions lib/Dobby/Client.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,60 @@ async sub delete_url ($self, $path) {
return;
}

async sub transfer_image_to_regions ($self, $image, $regions) {

my $image_id = $image->{id};

for my $region ($regions->@*) {
next if grep { $_ eq $region } $image->{regions}->@*;

my $res = await $self->http->do_request(
method => 'POST',
uri => $self->api_base . "/images/$image_id/actions",
headers => {
'Authorization' => "Bearer " . $self->bearer_token,
},

content_type => 'application/json',
content => encode_json({
type => 'transfer',
region => $region,
}),
);

next if $res->is_success;

# # # notes
# # # if an image has already been transferred we get a 422 code and this reply
# # # {
# # # "id": "unprocessable_entity",
# # # "message": "This image has already been transferred to this region."
# # # }
# # # If an image transfer is in progress we get a 422 and this reply
# # # {
# # # "id": "unprocessable_entity",
# # # "message": "The image is already being transferred to that region."
# # # }
# # # We shouldn't encounter the former (well small race between the last time we checked and the upcoming post) because we check in the for loop
# # # But we will definitely encounter cases where someone might try to spin up a box after the transfer has been requested but it isn't there yet
# # # we need to handle that case, ideally we should just handle both

# Handle errors we expect
# 1. We've already requested the transfer and it is still in progress
# 2. We've raced and the transfer has completed since we got $image
if (! $res->is_success && $res->code == 422) {
my $json = $res->decoded_content(charset => undef);
my $data = decode_json($json);
# transfer already in progress
next if $data->{message} eq "The image is already being transferred to that region.";
# race!
next if $data->{message} eq "This image has already been transferred to this region.";
}

die "error replication image $image_id to $region: " . $res->as_string;
}
}

async sub create_droplet ($self, $arg) {
state @required_keys = qw( name region size tags image ssh_keys );

Expand Down
2 changes: 2 additions & 0 deletions lib/Synergy/Reactor/InABox.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,8 @@ async sub handle_create ($self, $event, $switches) {
join ', ',
grep { $snapshot_regions{$_} } $self->box_datacentres->@*;

await $self->dobby->transfer_image_to_regions($snapshot, $self->box_datacentres);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems okay... but won't immediately fix the problem since these transfers can take awhile. What might be better/nice would be if you tried to create it in a region that didn't exist, synergy said: "Hey, that region isn't currently available. I'll start a transfer, and let you know when it finishes. In the meantime, you could create it in a different region with /datacentre ..."

So then she'd need to handle the case where a transfer is already underway and track that instead of starting a new one... etc etc.. (And if more people requested creating boxes, she'd want to add them to the list of people to notify when the region transfer has succeeded...) And then she'd also have to account for noticing a newer inabox image has been created that is in all the regions, so the current waiting things don't matter and she should just notify people everything is good....

This is all... a bit much more work than what you have here, and maybe not worth it (though I like the idea).

So maybe what we do is use this, but have synergy say something like this:

        "I'm unable to create snapshot in region '$region'.  Available compatible regions are $compatible_regions." I've kicked off a transfer of fminabox to that region, and it should be available in the next hour or less."


if ($compatible_regions) {
return await $event->reply(
"I'm unable to create snapshot in region '$region'. Available compatible regions are $compatible_regions."
Expand Down