Introducing the backup/restore plugin

Hi everyone!

I would like to announce the release of our backup and restore plugin, called tutor-contrib-backup. It provides backup and restore functionality for MySQL, MongoDB, and Caddy services in both local and Kubernetes Tutor deployments.

In a local Tutor deployment, you can run the backup from the command line with the tutor local backup command. This will create a dump of MySQL and MongoDB, and create a copy of the Caddy data directory. The backups are stored as a single tar file in $(tutor config printroot)/env/backup/. You can then copy the Tutor config root folder to a new host and restore your Open edX environment with the restore command: tutor local restore.

In Kubernetes, the plugin runs the backup job as a CronJob by default. You can also run the backup job from the command line. In both cases the backup tar file is stored in an S3 bucket. Then you can use the restore command to restore your Open edX environment. You can even schedule the restore as a CronJob to periodically download the latest backup and restore your environment. This can, for example, be useful if you want to maintain a standby site for disaster recovery purposes.

The plugin is new and experimental, but we are working on it to make it production-ready. Please go ahead and give it a try and let us know what you think :slight_smile:

10 Likes

Since the release we have added some more features to the plugin:

  • You can now individually exclude services from restore on both Local and Kubernetes deployments,
  • The backup tar file integrity is checked after uploading to, and downloading from S3,
  • There is better logging with timestamps for each operation during backup and restore.
2 Likes

I followed the instructions mentioned in this repo. I got the following error in building the docker image.

$ tutor images build backup

Building image docker.io/backup:0.0.6

docker build -t docker.io/backup:0.0.6 /home/ubuntu/.local/share/tutor/env/plugins/backup/build/backup

unable to prepare context: path "/home/ubuntu/.local/share/tutor/env/plugins/backup/build/backup" not found

Error: Command failed with status 1: docker build -t docker.io/backup:0.0.6 /home/ubuntu/.local/share/tutor/env/plugins/backup/build/backup

Hi @Harryp ,

Before using command tutor images build backup
Did you run command tutor config save?

Thanks @thinnguyen for pointing this out. I noticed that the tutor config save step was missing from the docs. It’s added now.
@Harryp let us know if running tutor config save before building the image solved the problem.

1 Like

@foadlind thanks for fixing tutor images build backup. But I got another problem when I restart the server:

lms-worker_1                 |     storage=get_video_image_storage(),
lms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/edxval/utils.py", line 158, in get_video_image_storage
lms-worker_1                 |     return get_storage_class(
lms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto.py", line 253, in __init__
lms-worker_1                 |     check_location(self)
lms-worker_1                 |   File "/openedx/venv/lib/python3.8/site-packages/storages/utils.py", line 84, in check_location
lms-worker_1                 |     raise ImproperlyConfigured(
lms-worker_1                 | django.core.exceptions.ImproperlyConfigured: S3BotoStorage.location cannot begin with a leading slash. Found '/openedx/media/'. **Use 'openedx/media/' instead.**

Where I can fix this back slash problem. Thank you !

That is unrelated to the backup plugin. See error in lms job execution · Issue #342 · overhangio/tutor · GitHub for a possible fix.

Just taking the liberty to add that this specific comment from that issue

I recommend the first option, since minio is required to run Open edX on K8s.

… is no longer current, because you now have the option of running without MinIO and using tutor-contrib-s3 instead.

1 Like

Thank you @fghaas and @foadlind now fixed issues without loading S3 plugin. The problem came from default values aws kyes as @regis mentioned here. I removed keys from OPENEDX_AWS_SECRET_ACCESS_KEY and added them to BACKUP_S3_SECRET_ACCESS_KEY.

One more question, tutor local backup
creates tar file but does not push it into the bucket. Should I install S3 plugin as you mentioned

Currently, the plugin does not upload the backup tar file to S3 when running tutor local. It will only do that when Tutor is running in Kubernetes. In a tutor local deployment, the the tar file is stored in $(tutor config printroot)/env/backup/.

1 Like

@foadlind thanks for the update. I use AWS-CLI to sync data from instances to s3, and vice-versa. Just required a two-step process. Temporary resolved the issue