@fghaas and I have been thinking about the backup and restore functionality in Tutor. Although the documentation offers one way of doing this, we prefer a more automated way that is also suitable for a Kubernetes environment. It is probably better to include this in Tutor itself, rather than a plugin. So far, we have landed on a design like this:
For the local environment: tutor local (backup|restore) <service>
tutor local backup mongodb runs docker exec to kick off a mongodb dump, to stdout. It has an optional -o/--output-file option to dump to a named file instead.
tutor local restore mongodb runs docker exec to kick off a mongodb restore, from stdin. It has an optional -i/--input-file option to restore from named file instead.
Same for tutor local (backup|restore) (mysql|caddy) with mysqldump and tar -gz.
For Kubernetes: tutor k8s (backup|restore) <service>
Creates a Kubernetes Job to do the backup/restore, to/from an S3 bucket. It would be expected that that bucket has object versioning enabled, so the restore command should support a --version option that allows one to restore a particular version of a backup.
Both tutor k8s backup and tutor k8s restore should also support a --schedule and --unschedule option which, based on config variables containing a cron expression, create or remove a recurring CronJob instead of a one-time Job. (The reason why you’d want recurring backups is self-evident; recurring restores are helpful for the “run a warm standby site for Disaster Recovery purposes” scenario).
What are your thoughts about this design?
In a reply to an old thread about the same topic, you said:
Have you started the work on this already? If not, I can take this on.
That could be an option, but it feels like an overkill to introduce a dependency for something that the user can easily set up with cron on their Docker host. And that would only be useful if the user is running tutor local in a production environment.
My assumption is that most tutor users are running tutor local and if the functionality is there for k8s deployment to schedule the backups, people will expect it to be there for local deployment backup.
Hey @foadlind,
Thanks for your thoughts on this matter. If you are going to add this to Tutor core, then you will have to address a variety of scenarios:
Some people run mysql or mongodb outside of Tutor (RUN_MYSQL=false, RUN_MONGODB=false). Thus, these people will not be able to run docker exec. Instead, they should run docker run.
Most people who run Tutor on Kubernetes do not have access to an S3 account. Thus, we need to make it possible for them to store backups elsewhere (where exactly I do not know).
A comprehensive backup system would have to also backup media assets, and that for every application, such as ecommerce/discovery.
Automated, periodical database dumps are not acceptable for everyone, because they lock the databases for a long while (unless I’m mistaken).
I understand your need for backups, and I agree that it would be better if we provided everyone with a simple solution to perform backups. But then we would have to take into account all these fairly complex scenarios. For that reason, I think that I would prefer to just give basic instructions on how to perform backups, and then leave the responsibility of actually performing the backups to a separate piece of software – i.e: a Tutor plugin. But I’m open to alternatives.
Yeah you are right. There will be a lot of scenarios that need to be covered if this is implemented as a native Tutor backup/restore functionality. We are working on a plugin that is roughly based on the design I described. We’ll share that with the community once it’s ready.
