When Should I Use Chronos?
How can you schedule periodic tasks into your Mesos cluster? For example, if you have a large ETL job that needs to run every night, how do you guarantee consistent runs across the cluster while making sure that the work is scheduled fairly and with a minimum of duplicated effort? This is where Chronos shines.
Chronos (formerly an AirBnB project, but now
maintained by Mesosphere) occupies the role that
cron does on a single
machine. It can execute shell scripts or Docker containers, and has a nice web
UI to see the state of your jobs (pictured below, screenshot courtesy of the
Chronos documentation site.)
Chronos actually improves on
cron's scheduling techniques by using
ISO8601 repeating intervals
instead of the
cron syntax you may be used to. This can cause some confusion
at first, but ultimately gives you much more flexibility in deciding when to
schedule your tasks. As a simple example, let’s say you want to schedule a task
to run at midnight at the beginning of every month. In
cron syntax, that would
0 0 1 * * /path/to/command where in ISO8601 syntax, that’s
R/2015-01-01T00:00:00/P1M (and Chronos’ web UI can help you when constructing
Using a system like Chronos also gives your scheduled jobs a measure of fault
tolerance. In a situation where you’re just using
cron, if the server goes
down then your tasks will definitely not be run. With Chronos, multiple servers
can still fail and the jobs will still be scheduled.
You can also specify a dependency graph of tasks. For example if you have an ETL job, each stage depends on the previous stage being finished. In this case, you would schedule the extract stage on a schedule. The transform stage would then depend on the extract stage, and the load stage on the transform stage. Chronos will keep all of this straight for you, and only schedule work when dependencies are met.
When should I not use Chronos?
- When your task is a long-running task. Chronos assumes that tasks will be run and finish with an exit code.
- Chronos was recently the subject of Kyle Kingsbury’s Call Me Maybe series and exhibited some strange behavior when subjected to network partitions. In short: if your jobs absolutely must be run exactly on time, consider another option. See the recommendations near the bottom of his post for more info.
So is Chronos a good fit for the work you want to do with your Mesos cluster? Now you know! Here’s a cheat sheet based off of the information in this post to make things easier: