Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry sooner than every 24h in case of unavailable base schedule #111

Open
2 tasks
thbar opened this issue Jan 18, 2021 · 0 comments
Open
2 tasks

Retry sooner than every 24h in case of unavailable base schedule #111

thbar opened this issue Jan 18, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@thbar
Copy link
Collaborator

thbar commented Jan 18, 2021

The work done in #107/#109 already improved the boot process by ensuring that a dataset is not "unregistered" (disappears from the routes) completely until next deploy if the static part is not available at boot time.

After #109 is merged, though, in case of unavailability at boot, the dataset download will only be retried 24h later at the moment, via this code:

impl actix::Actor for BaseScheduleReloader {
type Context = actix::Context<Self>;
fn started(&mut self, ctx: &mut Self::Context) {
info!(self.log, "Starting the base schedule updater actor");
ctx.run_interval(std::time::Duration::from_secs(60 * 60 * 24), |act, ctx| {
info!(act.log, "reloading baseschedule data");
act.update_data(ctx);
});
}
}

Detecting the unavailability (at boot and also later) and rescheduling with shorter timeframes (possibly with a form of backoff) would be a real improvement.

Todos

  • Figure out how to move forward in time, to implement proper tests on this workflow. It could be done via introspection of scheduled run_interval in Actix, if available, or via another form of tweak.
  • Make sure to avoid cascading downloads (e.g. all of a sudden, we end up downloading multiple times) - see cascading failures in distributed systems for the overall idea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant