In tutor v10.2.2, forum is built with
ENV MONGOID_AUTH_MECH “”
Does that mean no authentication is needed for forum to connect to mongodb?
In my environment, forum cannot connect to mongodb as my mongodb needs authentication.
Is there a way to set MONGOID_AUTH_MECH through configration?
I haven’t looked into this, but you might try:
Check the value:
tutor config printvalue MONGOID_AUTH_MECH ?
Configure the value:
tutor config save --set MONGOID_AUTH_MECH="yourvalue"
I’m not quite familiar with mongodb authentication mechanisms, but this value was set there to avoid a crash when it is undefined. This variable describes just the authentication mechanism, not the credentials. The latter are defined by the MONGODB_USERNAME and MONGODB_PASSWORD Tutor settings.
sorry to refer that I am running on k8s.
Seems that MONGOID_AUTH_MECH is not passed as env in deployments.yaml.
@regis yes, username and password can be passed in.
I am running on k8s and using a mongdb service, and in previsous tutor version, forum can connect to mongodb. With v10.2.2 built image, mongodb cannot be connected util I set
MONGOID_AUTH_MECH env to “:scram” in deployments.yml.
Interesting. I suggest you pass the MONGOID_AUTH_MECH environment variable to the forum container. You can do this by defining a custom docker-compose.override.yml file:
$ cat "$(tutor config printroot)/env/local/docker-compose.override.yml"
version: "3.7"
services:
forum:
environment:
- "MONGOID_AUTH_MECH=:scram"
Hi,
Can you help me with this issue:
I deploy the mongodb rs with authentication for mongodb as example: root/rootpwd on cs_comments_service database
Update the deployment for forum path as following
- name: MONGODB_AUTH
value: "root:rootpwd@"
- name: MONGOID_AUTH_MECH
value: ":mongodb_cr"
- name: MONGODB_HOST
value: "mongodb-0.mongodb-headless.default.svc.cluster.local"
- name: MONGODB_PORT
value: "27017"
I got this error when starting the forum service:
W, [2021-01-11T14:28:12.604779 #13] WARN – : Overwriting existing field _id in class User.
W, [2021-01-11T14:28:12.711481 #13] WARN – : MONGODB | Unsupported client option ‘max_retries’. It will be ignored.
W, [2021-01-11T14:28:12.711931 #13] WARN – : MONGODB | Unsupported client option ‘retry_interval’. It will be ignored.
W, [2021-01-11T14:28:12.712137 #13] WARN – : MONGODB | Unsupported client option ‘timeout’. It will be ignored.
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/cr/conversation.rb:130:in `validate!’: User root is not authorized to access cs_comments_service. (Mongo::Auth::Unauthorized)
Do you have idea on how to fix this?
I used the docker image: overhangio/openedx-forum:10.0.11
Thank you for your help.
Hey @vthily I was facing the same issue, and I just happened to follow @regis’s post #6 suggestion. One should override two docker-compose files that is docker-compose.yml & docker-compose.jobs.yml files, by following @regis’s recommended approach to override the default docker-compose files. Also make sure that MongoDB server’s user is using the same authentication mechanism that which you are passing in the docker-compose files.
Hi @vaunramesh26 ,
This solution works for local deployment . How we can implement this for k8s deployment?
@vthily Do this solution works for you? How ?
Yes, the solution works for me too.
In the k8s, we have to create a job to rebuild the index,
apiVersion: batch/v1
kind: Job
metadata:
labels: <yourlabels>
name: forum-job
spec:
template:
spec:
containers:
- args:
- sh
- -e
- -c
- 'bundle exec rake search:initialize
bundle exec rake search:rebuild_index'
env:
- name: SEARCH_SERVER
value: "your search server endpoint"
- name: MONGODB_AUTH
value: "user:password@"
- name: MONGOID_AUTH_MECH
value: ":scram"
- name: MONGODB_HOST
value: "mongodb_host"
- name: MONGODB_PORT
value: "27017"
image: docker.io/overhangio/openedx-forum:11.2.0
name: forum
restartPolicy: Never
Thanks a lot @vthily ,
Do you add this job using tutor custom plugin, or just apply this with kubectl ?
We apply it one time with kubectl.
Hi @vthily, @imantaba, all,
Were you able to fix this? If I try with an empty MONGODB_AUTH it works well, but when I set a username and password I get the same problem.
I tried to run the job to build the indexes without success:
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: job
app.kubernetes.io/name: forum-job-patch
name: forum-job-patch
spec:
template:
spec:
containers:
- args:
- sh
- -e
- -c
- 'bundle exec rake search:initialize
bundle exec rake search:rebuild_index'
env:
- name: SEARCH_SERVER
value: http://elasticsearch:9200
- name: MONGODB_AUTH
value: mongouser:mongopwd@
- name: MONGODB_HOST
value: mongodb
- name: MONGODB_PORT
value: '27017'
- name: MONGOID_AUTH_MECH
value: ":scram"
image: docker.io/overhangio/openedx-forum:12.0.4
name: forum
restartPolicy: Never
Waiting for mongodb/elasticsearch...
2021/09/14 13:12:12 Waiting for: tcp://mongodb:27017
2021/09/14 13:12:12 Waiting for: http://elasticsearch:9200
2021/09/14 13:12:12 Connected to tcp://mongodb:27017
2021/09/14 13:12:12 Received 200 from http://elasticsearch:9200
/openedx/cs_comments_service/lib/tasks/flags.rake:6: warning: already initialized constant ROOT
/openedx/cs_comments_service/lib/tasks/deep_search.rake:7: warning: previous definition of ROOT was here
/openedx/cs_comments_service/lib/tasks/kpis.rake:7: warning: already initialized constant ROOT
/openedx/cs_comments_service/lib/tasks/flags.rake:6: warning: previous definition of ROOT was here
/openedx/cs_comments_service/models/constants.rb:2: warning: already initialized constant COURSE_ID
/openedx/cs_comments_service/lib/tasks/db.rake:28: warning: previous definition of COURSE_ID was here
rake aborted!
Mongo::Auth::Unauthorized: User mongouser is not authorized to access cs_comments_service.
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:489:in `validate!'
...
I tried adding manually the user in mongodb with the same result:
> use admin
> db.createUser({user:"mongouser", pwd:"mongopwd", roles:['userAdminAnyDatabase']})
Successfully added user: { "user" : "mongouser", "roles" : [ "userAdminAnyDatabase" ] }
I have to set username and password because I am trying to connect to an external MongoDB, but it requires authentication to connect.
Any ideas?
I think you can start with ssh to that mongodb instance and test with the connection string to make sure it works in cli. You can inspect the list of users, database.
Other thing: user was added to the admin database, but the connection string doesn’t seems to include that as the authDatabase. You might need to check that too.
I think I made it.
I am not expert in MongoDB (or in any other thing), but this is what I’ve learned.
admin DB to store users. There you can grant permissions to all or other specific DBs. Usually clients should authenticate users using the admin DB, and then access whatever DB that holds the data. However, it’s possible to store users in a specific DB.admin DB is used for authentication by default (see the url sepec)admin, cs_comments_service and openedx databases to have everything working properly.I have created PR #493 to add MONGOID_AUTH_MECH as parameter and to the jobs and deployments.
Hi all!
@regis has found that the problem was that setting MONGOID_AUTH_MECH="" in the forum dockerfile actually prevented Ruby to use the default “:scram” value. This is because an empty env var would not be nil, while an unset var yes.
Now the PR with the fix was merged to master. Please check if it solves the issues reported before.
Looks like it broke something, as reported in github.
Now local quickstart is failing the forum-job:
Creating tutor_local_forum-job_run ... done
Waiting for mongodb/elasticsearch...
2021/09/17 14:12:34 Waiting for: tcp://mongodb:27017
2021/09/17 14:12:34 Waiting for: http://elasticsearch:9200
2021/09/17 14:12:34 Received 200 from http://elasticsearch:9200
2021/09/17 14:12:34 Connected to tcp://mongodb:27017
/openedx/cs_comments_service/lib/tasks/db.rake:28: warning: already initialized constant COURSE_ID
/openedx/cs_comments_service/models/constants.rb:2: warning: previous definition of COURSE_ID was here
/openedx/cs_comments_service/lib/tasks/kpis.rake:7: warning: already initialized constant ROOT
/openedx/cs_comments_service/lib/tasks/deep_search.rake:7: warning: previous definition of ROOT was here
/openedx/cs_comments_service/lib/tasks/flags.rake:6: warning: already initialized constant ROOT
/openedx/cs_comments_service/lib/tasks/kpis.rake:7: warning: previous definition of ROOT was here
W, [2021-09-17T14:12:40.285471 #13] WARN -- : Overwriting existing field _id in class User.
W, [2021-09-17T14:12:40.395846 #13] WARN -- : MONGODB | Unsupported client option 'max_retries'. It will be ignored.
W, [2021-09-17T14:12:40.395983 #13] WARN -- : MONGODB | Unsupported client option 'retry_interval'. It will be ignored.
W, [2021-09-17T14:12:40.396101 #13] WARN -- : MONGODB | Unsupported client option 'timeout'. It will be ignored.
rake aborted!
NoMethodError: undefined method `encode' for nil:NilClass
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/user.rb:87:in `encoded_name'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:320:in `first_bare'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:263:in `client_first_message'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:181:in `start'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram.rb:58:in `login'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:249:in `block in authenticate!'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server.rb:267:in `handle_auth_failure!'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:248:in `authenticate!'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:100:in `connect!'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connectable.rb:84:in `ensure_connected'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:270:in `write'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:229:in `deliver'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:144:in `block in dispatch'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/monitoring/publishable.rb:47:in `publish_command'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:143:in `dispatch'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/operation/shared/executable.rb:33:in `block in dispatch_message'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection_pool.rb:110:in `with_connection'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server.rb:251:in `with_connection'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/operation/shared/executable.rb:32:in `dispatch_message'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/operation/find/op_msg.rb:46:in `execute'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/operation/find.rb:43:in `execute'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/collection/view/iterable.rb:82:in `send_initial_query'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/collection/view/iterable.rb:42:in `block in each'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/retryable.rb:44:in `read_with_retry'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/collection/view/iterable.rb:40:in `each'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongoid-7.0.5/lib/mongoid/query_cache.rb:227:in `each'
/openedx/cs_comments_service/app.rb:173:in `map'
/openedx/cs_comments_service/app.rb:173:in `<top (required)>'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/activesupport-6.0.2.1/lib/active_support/dependencies.rb:325:in `require'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/activesupport-6.0.2.1/lib/active_support/dependencies.rb:325:in `block in require'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/activesupport-6.0.2.1/lib/active_support/dependencies.rb:291:in `load_dependency'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/activesupport-6.0.2.1/lib/active_support/dependencies.rb:325:in `require'
/openedx/cs_comments_service/Rakefile:24:in `block in <top (required)>'
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/openedx/ruby/bin/bundle:23:in `load'
/openedx/ruby/bin/bundle:23:in `<main>'
Tasks: TOP => search:initialize => environment
(See full trace by running task with --trace)
ERROR: 1
Error: Command failed with status 1: docker-compose -f /Users/andres/Workspace/tutor-test/env/local/docker-compose.yml -f /Users/andres/Workspace/tutor-test/env/local/docker-compose.prod.yml --project-name tutor_local -f /Users/andres/Workspace/tutor-test/env/local/docker-compose.jobs.yml run --rm forum-job sh -e -c bundle exec rake search:initialize
bundle exec rake search:rebuild_indices
@regis @andres Still i had authentication problem with external MongoDB for k8s deployment
tutor, version 12.1.6
Waiting for mongodb/elasticsearch...
2021/11/08 06:51:38 Waiting for: tcp://edx-mongo-mongodb.edx-mongo.svc.cluster.local:27017
2021/11/08 06:51:38 Waiting for: http://elasticsearch:9200
2021/11/08 06:51:38 Received 200 from http://elasticsearch:9200
2021/11/08 06:51:38 Connected to tcp://edx-mongo-mongodb.edx-mongo.svc.cluster.local:27017
W, [2021-11-08T06:51:40.630423 #13] WARN -- : Overwriting existing field _id in class User.
W, [2021-11-08T06:51:40.673717 #13] WARN -- : MONGODB | Unsupported client option 'max_retries'. It will be ignored.
W, [2021-11-08T06:51:40.673795 #13] WARN -- : MONGODB | Unsupported client option 'retry_interval'. It will be ignored.
W, [2021-11-08T06:51:40.673820 #13] WARN -- : MONGODB | Unsupported client option 'timeout'. It will be ignored.
/openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:489:in `validate!': User admin is not authorized to access org2-cs_comments_service. (Mongo::Auth::Unauthorized)
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:484:in `validate_first_message!'
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram/conversation.rb:113:in `continue'
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/auth/scram.rb:60:in `login'
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:249:in `block in authenticate!'
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server.rb:267:in `handle_auth_failure!'
from /openedx/cs_comments_service/vendor/bundle/ruby/2.5.0/gems/mongo-2.5.3/lib/mongo/server/connection.rb:248:in `authenticate!'
Thanks for the report @imantaba. It’s really difficult for me to help you troubleshoot this, as I don’t have access to a remote mongodb cluster. Do you think you could investigate the issue?