Only setting files getting values from environment variables

Tutor
1

We know Open edX is monolithic and we have some workarounds to run it in a container. When we build a container in a CI/CD pipeline, the best practices (The Twelve Factors) say to avoid store config as constants in the code.

An app’s config is everything that is likely to vary between deploys (staging, production, developer environments, etc). This includes:

  • Resource handles to the database, Memcached, and other backing services
  • Credentials to external services such as Amazon S3 or Twitter
  • Per-deploy values such as the canonical hostname for the deploy

Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.

Tutor saves all configuration to the files:

  • config/cms-env.json
  • config/lms-env.json
  • settings/cms/production.py
  • settings/lms/production.py

With this, we need to create secrets every build to store the files that have sensitive information and end up with many secret versions (cms-env-{COMMIT_HASH_1}, cms-env-v{COMMIT_HASH_2} …).

I think if we have an option to generate a setting file getting the values that can be changed between environments from the system environment variables would be a good feature.

We can have after tutor config save command just production.py file with something like this:

# json env files settings
SITE_NAME = os.getenv('SITE_NAME')
...
DATABASES = {
    "default": {
      "ENGINE": "django.db.backends.mysql",
      "HOST": os.getenv('MYSQL_HOST'),
      "PORT": os.getenv('MYSQL_PORT'),
      "NAME": os.getenv('MYSQL_NAME'),
      "USER": os.getenv('MYSQL_USER'),
      "PASSWORD": os.getenv('MYSQL_PASSWORD'),
      "ATOMIC_REQUESTS": true,
      "OPTIONS": {
        "init_command": "SET sql_mode='STRICT_TRANS_TABLES'"
      }
    }
  }
...
# python settings file settings
mongodb_parameters = {
    "host": os.getenv('MONGO_USER'),
    "port": os.getenv('MONGO_USER'),
    "user": os.getenv('MONGO_USER'),
    "password": os.getenv('MONGO_USER'),
    "db": os.getenv('MONGO_USER'),
}
...

What do you think about it?

2

You can use environment variables in the form of TUTOR_PARAM1=VALUE1 for the variables in the config.yml the environment variables will be used before the variables set in the config.yml file.

For example, if you set export TUTOR_ENABLE_HTTPS=false, while your config.yml is ENABLE_HTTP: true, Tutor will render the settings as if ENABLE_HTTPS: false is set.

For your example, you can set the system environment variables TUTOR_MYSQL_* and TUTOR_MONGO_*.

You can refer to the Tutor documentation for this behavior: Configuration and customisation — Tutor documentation

Or from the system environment:
export TUTOR_PARAM1=VALUE1

1 Like
3

Yes, yes, tutor supports environment variables.

However I’m talking about the Open edX (django setting file) loads the values from the environment variables and not from a YML file.

Also not store sensitive information in files (yml and python).

4

What problem would this solve for you? Can you solve it with a plugin? Do you think other Tutor users should benefit from this?

5

I needed to do a workaround to create a CI/CD pipeline to use the same container image in all environments (dev, staging, and prod) using the default Open edX (Django) config loading (from files).

I’m thinking about a plugin for that, if it’s worth it … the integration with others plugins, and if it is possible to be generic to resolve my problem and of other users.

But the ability to deploy the Open edX in a container Orchestrator (k8s, swarm) using environment variables/secrets (the default used in a container) to store the setting would be great.
We could even use cloud-managed service like Cloud Run, ECS, Fargate.

What do you think, negative and positive points?

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.