Minion::Backend::Pg: Postpone migrations to first connection #1886
Replies: 2 comments 14 replies
-
|
This is incorrect, |
Beta Was this translation helpful? Give feedback.
-
|
I also think that the connection already happens in |
Beta Was this translation helpful? Give feedback.
-
|
... or line 131. |
Beta Was this translation helpful? Give feedback.
-
|
No, the migration is not loaded there. And |
Beta Was this translation helpful? Give feedback.
-
|
OK. Maybe I'm wrong and the migration is not loaded there. |
Beta Was this translation helpful? Give feedback.
-
|
Just checked: I was wrong - the migration is not executed by the web app, but when the Minion scheduler is started. |
Beta Was this translation helpful? Give feedback.
-
|
Then perhaps they shouldn't initialize Mojo::Pg?
Just as an example, initialize if app->mode eq production, but if mode eq
cli, don't.
Of course mode is not the best condition to base this on, but it
illustrates the point. Environment variable? Config value? I'd think
there's some way for a command to influence the startup of the application,
but I can't think of the most ideal at the moment.
…On Fri, Nov 19, 2021, 12:26 PM Bernhard Graf ***@***.***> wrote:
OK. Maybe I'm wrong and the migration is not loaded there.
But the constructor connects to the database server to determine the
server version. This is still a problem for the commands that do not
have/need database access.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#1886 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAD6K6Q3D5ZFARLRNJH2BMLUM2JFRANCNFSM5IMUNCRQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
Beta Was this translation helpful? Give feedback.
-
|
Hmm, yes, that's quite a pickle! Thanks for the clear report about how that all worked out. Ok, what if you move the call to load the Minion plugin into a "Useful for reconfiguring application servers dynamically" ... That's exactly what you're looking for right? With that proposed configuration, your app by default doesn't load Minion, but instead reconfigures the application server dynamically by loading Minion However, how will commands that need Minion (such as the minion commands) be able to function because the Minion plugin won't get loaded? Well, go back to the I'd hate to have to suggest a bunch of if statements or a regular expression. Perhaps the cleanest way would be to add an attribute to commands that shouldn't load Minion. By moving Minion plugin load to both hooks, it sounds like it will get loaded only once, depending on application invocation. In other words, invocation is either a command or a server, but not both. It could become a problem, I suppose, if you have a command that needs Minion and also loads a server. It's not really a problem, I don't think, but it does register the plugin both times in that rare situation. A final scenario is an innovation where you neither run a command nor start a server. I think that's technically possible, and what happens is you get an application usage response. But, what if you intended to run your app in some other way, similar to morbo. You wouldn't need to call a command, and you wouldn't need to start a server. In this case, the Minion plugin wouldn't load. Instead, you'd have to load it directly in order to use it. Hopefully you can figure out something that works, and relatively elegantly. Hoping to be able to see what you come with, or what the next pickle is! |
Beta Was this translation helpful? Give feedback.
-
|
Hi Stefan, so I changed MyApp to this (basically): package MyApp;
use Mojo::Base 'Mojolicious', -signatures;
sub startup ($self) {
my $config = $self->plugin('NotYAMLConfig');
$self->hook(before_command => sub ($command, $) {
$self->log->debug("Command " . ref($command));
$self->plugin(Minion => {Pg => $config->{database}}) if ref($command) =~ /^Minion::Command/;
});
$self->hook(before_server_start => sub ($server, $) {
$self->log->debug("Server " . ref($server));
$self->plugin(Minion => {Pg => $config->{database}}
}
...
}
This looks promising. But... Yes, Minion is not there (even when I remove the condition It seems loading the So thanks buddy, but again no luck. :-/ Your last point about the final scenario is too abstract for me, but I have the feeling that this is currently not my biggest concern. |
Beta Was this translation helpful? Give feedback.
-
|
Perhaps call L8 in your own app's startup, not a hook? Will that solve all the problems? If so, could you create your own package MyApp;
use Mojo::Base 'Mojolicious', -signatures;
sub startup ($self) {
my $config = $self->plugin('NotYAMLConfig');
$self->plugin('MyMinion' => {Pg => $config->{database}});
}
package Mojolicious::Plugin::MyMinion;
use Mojo::Base 'Mojolicious::Plugin', -signatures;
sub register {
my ($self, $app, $conf) = @_;
push @{$app->commands->namespaces}, 'Minion::Command';
$self->hook(before_command => sub ($command, $) {
$self->log->debug("Command " . ref($command));
$self->_plugin_minion($app, $conf) if ref($command) =~ /^Minion::Command/;
});
$self->hook(before_server_start => sub ($server, $) {
$self->log->debug("Server " . ref($server));
$self->_plugin_minion($app, $conf);
});
}
sub _plugin_minion ($self, $app, $conf) {
my $minion = Minion->new(%$conf)->app($app);
$app->helper(minion => sub {$minion});
} |
Beta Was this translation helpful? Give feedback.
-
|
Here's an interesting technique, but I reckon it'll still depend on conditional activation, likely a configuration. Use a Dummy backend for Minion: package Minion::Backend::Dummy;
use Mojo::Base 'Minion::Backend';
sub auto_retry_job { 0 }
sub broadcast { 0 }
sub dequeue { {} }
sub enqueue { undef }
sub fail_job { 0 }
sub finish_job { 0 }
sub history { {} }
sub list_jobs { {} }
sub list_locks { {} }
sub list_workers { {} }
sub lock { 0 }
sub new { shift->SUPER::new }
sub note { 0 }
sub receive { [] }
sub register_worker { undef }
sub remove_job { 0 }
sub repair { }
sub reset { }
sub retry_job { 0 }
sub stats { {} }
sub unlock { 0 }
sub unregister_worker { }Use it like so: |
Beta Was this translation helpful? Give feedback.
-
|
Well, I want so solve a problem, but without rewriting large parts of the used CPAN module(s). But since I still think that the version check in that module is not a good idea (or not implemented in the best way) I opened this issue: mojolicious/minion#121 |
Beta Was this translation helpful? Give feedback.
-
After switching from
Minion::Backend::mysqltoMinion::Backend::PgI have issues with application commands:The constructor of
Minion::Backend::Pgtries to run its migrations as soon it is instantiated - this seems to be the case if you havein your application class.
As a consequence migrations are tried even when a simple
Mojolicous::Commandis executed by an application, i.e.This command fails when the database - which is not required for the command - is not accessible. OTOH the migration is always tried if the database can be reached, which is not always desired, and slows down the commands in any case.
In a project based on Mojolicious we use a couple of dev commands in the
Authornamespace (i.e. for CI/CD) that don't have and don't need access to the database. This fails now.Currently I'm not aware of a way to find out in the application code by which
Mojolicous::Commandit was started in order to conditionally skip the Minion-plugin loading. Even then it would be a clumsy work-around.As indicated I did not have these issues with
Minion::Backend::mysqlthat postpones migration to the first database connection event. I would really appreciate IfMinion::Backend::Pgwould do that in the same way.What do you think?
Beta Was this translation helpful? Give feedback.
All reactions