AWS S3 configuration for K8S

Tutor
1

Hi @regis,

Can we configure aws s3 for storing all the objects to s3?
I am able to see only “OPENEDX_AWS_ACCESS_KEY”, “OPENEDX_AWS_SECRET_ACCESS_KEY” in config.yml file.

How to add bucket name and region for configuration.

2

Hi @ramshankar! Out of the box, Tutor only comes with MinIO support, which is an emulator for S3: https://pypi.org/project/tutor-minio/
However, Adolfo developed an S3 plugin: https://github.com/hastexo/tutor-s3 I don’t know if it’s stable or maintained, but you could give it a try.

3

Thanks @regis for the clarity.

Can I run minio plugin as a separate service and just map the minio URL and bucket details in the config.yml file like way we connect s3?

4

Yes, absolutely. Just set the right configuration values, as described in the plugin docs: tutor-minio · PyPI

1 Like
5

Hi @regis,

When I enable the minio plugin and give the tutor k8s quickstart the minio pod is creating and it is working fine.

But, when I want to run the minio as a separate service in a different k8s cluster, is there any way to stop/not to deploy the minio pod in the parent cluster? Some thing like ACTIVATE_MYSQL=false in config.yml?

6

This is a good point. There are different ways to approach your problem:

  1. If you want to connect directly to S3, then the MinIO plugin is overkill, as it comes with a MinIO server which you don’t really need. In that case, you can either a) fork the plugin to modify it and point it to S3, or b) use the tutor-s3 plugin mentioned earlier.
  2. If you are ok to connect to S3, or to the equivalent blob storage in Azure, through MinIO, then you want to run your MinIO server in gateway mode. I mention this solution because @pcliu opened a promising pull request that leverages MinIO’s gateway mode, and I think this is a very elegant solution. In this solution, the client connects to S3/Azure through MinIO, thus abstracting away most of the connection intricacies.
  3. If you do want to use MinIO, but run it in a separate cluster, then it does not make much sense for you to run the MinIO server that comes with the tutor-minio plugin. In that case, it would very much make sense to create a ACTIVATE_MINIO_SERVER configuration variable.

If your use case is #3, can you please open an issue on the tutor-minio repo? You should describe your use case as precisely as possible and link to this conversation.

7

@regis Raised an issue in git repo. https://github.com/overhangio/tutor-minio/issues/3#issue-640630083

1 Like