MongoDB Failover / Replica Set Support

Hey there!
Happy Wednesday night!
Hope all has been well!

I was curious about connecting to high availability replicasets with tutor & mongodb if configuration for that was possible?
I’m thinking it would probably be some custom editing to files?

I was looking at this:

and the ‘replicaSet’ item in the dictionary peeked my curiosity.
OpenEdx, (I think) uses pymongo 2.9.1, there docs here covering highly available mongo clusters and what not:
>>> MongoClient("mongodb://morton.local:27017,morton.local:27018,morton.local:27019")
MongoClient(['morton.local:27019', 'morton.local:27017', 'morton.local:27018'])

made me wonder, if it’s just that easy for the library to handle potentially different mongodb nodes like:
MongoClient(“172.XX.YY.BBB:27017”, “172.XX.YY.CCC:27017”, “172.XX.YY.HHH:27017”) …and if that happened to work successfully (of which I’ll probably try tomorrow … or latter tonight when I can’t sleep) …but if that wasn’t too difficult to do from the PyMongo library perspective … how might that translate to configuration and or tutor edits?

Thanks for any input, hazings, thoughts, feedback, and such! :slight_smile:

So I was curious, inside the “auth.json” file that has the templating, would it work to say, anywhere the “DOC_STORE_CONFIG” is listed modify the “host” to be changed to “replicaSet”, and ensuring that the MONGODB_HOST is actually like: AAA.BBB.CCC.GGG, AAA.BBB.CCC.EEE, AAA.BBB.CCC.DDD - given that we use standard port of 27017?
I’m starting to test this to try to see what impact the edits have.

 10     "ENGINE": "xmodule.contentstore.mongo.MongoContentStore",
 11     "DOC_STORE_CONFIG": {
 12       "host": "{{ MONGODB_HOST }}",
 13       "port": {{ MONGODB_PORT }},
 15       "user": "{{ MONGODB_USERNAME }}",
 16       "password": "{{ MONGODB_PASSWORD }}",
 17       {% endif %}
 18       "db": "{{ MONGODB_DATABASE }}"
 19     }
 20   },
 22     "replicaSet": "{{ MONGODB_HOST }}",
 24     "user": "{{ MONGODB_USERNAME }}",
 25     "password": "{{ MONGODB_PASSWORD }}",
 26     {% else %}
 27     "user": null,
 28     "password": null,
 29     {% endif %}
 30     "db": "{{ MONGODB_DATABASE }}"

However, I have noted that there is:

Initialising forum...

docker-compose -f /home/ec2-user/.local/share/tutor/env/local/docker-compose.yml -f /home/ec2-user/.local/share/tutor/env/local/ --project-name tutor_local -f /home/ec2-user/.local/share/tutor/env/local/ run --rm forum-job sh -e -c export MONGOHQ_URL="mongodb://$MONGODB_AUTH$MONGODB_HOST:$MONGODB_PORT/cs_comments_service"

bundle exec rake search:initialize

bundle exec rake search:rebuild_index

thinking that the: “mongodb://$MONGODB_AUTH$MONGODB_HOST:$MONGODB_PORT/cs_comments_service” would need custom edits, since the mogodb connection url will be different for using a replica set

Does any of y’all think this may be the right path to head down?

Are you guys working together? This PR addressing replica sets was opened just this morning.

Hahaha wow! That’s so uncanny! Super coincidental! :smiley::laughing:

But no, haha, we’re not working together hahah

@irishgordo, hi!

With latest tutor(from source code) you are able to set up MongoDB connection you want. I created simple plugin like this:

name: mongodb_replicaset
version: 0.1.0
  openedx-common-settings: |
    mongodb_parameters = {
        "read_preference": "PRIMARY",
        "replicaSet": "{{ MONGODB_REPLICASET }}",
        "host": [{% for host in MONGODB_HOST %}"{{ host }}"{{ "," if not loop.last }}{% endfor %}],
        "port": {{ MONGODB_PORT }},
        "user": "{{ MONGODB_USERNAME }}",
        "password": "{{ MONGODB_PASSWORD }}",
        "db": "{{ MONGODB_DATABASE }}"
    DOC_STORE_CONFIG = mongodb_parameters
        "ENGINE": "xmodule.contentstore.mongo.MongoContentStore",

    # Load module store settings from config files
    update_module_store_settings(MODULESTORE, doc_store_settings=DOC_STORE_CONFIG)
1 Like

Hey @odemakov that pull request was great! and perfect timing too!
I manually implemented those changes to our auth.json file and config.yml file and it’s working like a dream on the server!
And thanks to the work you did I am gaining a deeper understanding of how the templating with
is kinda taking place

I haven’t read up on plugins with tutor and should :slight_smile: