@regis, I’ve been thinking about the issue of private Open edX images and how to manage and deploy them with Tutor, and I have an idea here.
Come Maple, pretty much everyone will need to build custom images for almost every Open edX instance they run — because they effectively won’t be able to run without MFE, and MFE needs an image rebuild for every theme change. Also many users will be running Comprehensive Themes that change per platform they manage, which also requires that they run custom openedx images for each platform. (Not to mention that XBlock requirements may change per platform, or even local Open edX patches.)
So, we should have a way to easily manage private images, both for the local build use case and for deployment to Kubernetes. And for the Kubernetes use case, it should be equally applicable no matter if people self-host their Kubernetes, or run fully managed on AWS EKS, or Digital Ocean, or OVH, or whatever.
We also already assume that every Open edX environment has access to a storage platform that exposes the S3 API: either to AWS S3 itself, or via Minio.
So, how about this idea:
- Tutor would optionally deploy its own registry when running in
local mode, meaning it spins up a local registry service.
- Tutor would also optionally deploy the registry in
k8s mode, by adding a Deployment containing of a single Pod that deploys a container from the same image, and a Service exposing that registry.
-
In either of these cases, Tutor would configure the registry to use the S3 storage backend. That means that the registry service is completely stateless, and all of its persistent data lives in S3.
- In the
tutor k8s use case, the registry service endpoint could get plugged into Caddy as an additional backend, meaning it gets HTTPS termination and Let’s Encrypt certificates for free, and doesn’t consume an additional external IP. However, if we wanted to support TLS in a completely standalone fashion (meaning, the registry is able to run even if it’s the only Pod or Service left in the cluster), we could also use the registry service’s built-in Let’s Encrypt support.
This way, depending on how organizations organize their development workflow,
- if they don’t use automated/reproducible image builds out of their CI, developers can build images locally and push them directly to the registry, as long as they have the S3 credentials for the appropriate bucket.
- if the image build workflow is CI driven, the organization sets S3 credentials as a Secret in their CI infrastructure, and then pushes them to a local registry running in their CI environment, which also populates the S3 bucket (as an alternative to using a container image registry that may be specific to a CI).
Either way, the Tutor-managed containers, whether they’re running locally or in Kubernetes, then have the ability to pull from this private registry, which exists once per Tutor platform.
This way the private image management approach is unified, and it doesn’t matter
(a) where Tutor users run their Kubernetes, or if they run Kubernetes at all (the same approach might be useful for building images that are then used to deploy Open edX with, say, Nomad),
(b) what CI Tutor users run, or if they run with a CI at all.
Also, this whole functionality should be straightforward to make entirely optional, in other words, would suit itself to being a Tutor plugin. And (even when that plugin is enabled) for any upstream public images that the Tutor user does want to use unmodified, those would of course bypass the private registry altogether, as they do now.
What do you think of this? Do consider that this may be an exceptionally silly idea, because I may be overlooking something completely obvious that’s a deal-breaker for this kind of approach. So please poke holes into these thoughts. Thanks!
Cheers,
Florian
As a Tutor maintainer, I could certainly add dozens of