Ecommerce unable to communicate to LMS

Hi!

We’re in the middle of setting up a fresh install of Tutor for production use, so that means we have public URL’s created and resolvable in public DNS for all components and plugins that we intend to use.

We’ve installed the Discovery and Ecommerce plugins and can get to the admin pages just fine, and the course admin pages in Ecommerce.
The installation was done by following the steps on PyPi (excluding the parts that specify if we’ve built from source, as we’ve used the binary)

The only thing we’ve done differently is modify the public URL on ECOMMERCE_HOST to a different public domain than the default. In this case, it is something similar to ecommerce.topleveldomain.x. The LMS is on something like lms.topleveldomain.x. (Note, I’ve had to replace dot com with x, my user account on the forum won’t let me post more than 2 links)

I’ve tried to create a new course in the course admin page (…/courses/new/) but I’m getting the following error when running tutor local logs -f --tail=10 and clicking the save button on the admin page:

ecommerce_1 | 2020-04-23 03:37:41,568 ERROR 11 [ecommerce.courses.publishers] /openedx/ecommerce/ecommerce/courses/publishers.py:112 - Failed to publish commerce data for [course-v1:test+C001+2020_01] to LMS.
ecommerce_1 | Traceback (most recent call last):
ecommerce_1 | File “/openedx/ecommerce/ecommerce/courses/publishers.py”, line 100, in publish
ecommerce_1 | commerce_api_client = site.siteconfiguration.commerce_api_client
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/django/utils/functional.py”, line 35, in get
ecommerce_1 | res = instance.dict[self.name] = self.func(instance)
ecommerce_1 | File “/openedx/ecommerce/ecommerce/core/models.py”, line 457, in commerce_api_client
ecommerce_1 | return EdxRestApiClient(self.build_lms_url(’/api/commerce/v1/’), jwt=self.access_token)
ecommerce_1 | File “/openedx/ecommerce/ecommerce/core/models.py”, line 384, in access_token
ecommerce_1 | token_type=‘jwt’
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/edx_rest_api_client/client.py”, line 142, in get_oauth_access_token
ecommerce_1 | return get_oauth_access_token(url, client_id, client_secret, token_type=token_type)
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/edx_rest_api_client/client.py”, line 72, in get_oauth_access_token
ecommerce_1 | ‘User-Agent’: USER_AGENT,
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/requests/api.py”, line 116, in post
ecommerce_1 | return request(‘post’, url, data=data, json=json, **kwargs)
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/requests/api.py”, line 60, in request
ecommerce_1 | return session.request(method=method, url=url, **kwargs)
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/requests/sessions.py”, line 533, in request
ecommerce_1 | resp = self.send(prep, **send_kwargs)
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/requests/sessions.py”, line 646, in send
ecommerce_1 | r = adapter.send(request, **kwargs)
ecommerce_1 | File “/openedx/venv/local/lib/python2.7/site-packages/requests/adapters.py”, line 516, in send
ecommerce_1 | raise ConnectionError(e, request=request)
ecommerce_1 | ConnectionError: HTTPSConnectionPool(host=‘lms.topleveldomain.com’, port=443): Max retries exceeded with url: /oauth2/access_token (Caused by NewConnectionError(’<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1045c010d0>: Failed to establish a new connection: [Errno 111] Connection refused’,))
ecommerce_1 | 2020-04-23 03:37:41,574 ERROR 11 [ecommerce.extensions.api.serializers] /openedx/ecommerce/ecommerce/extensions/api/serializers.py:585 - Failed to save and publish [course-v1:test+C001+2020_01]: [Failed to publish commerce data for course-v1:test+C001+2020_01 to LMS.]

The error we receive on the course creation page is the ‘Failed to publish to LMS’ one that shows in the logs:

It appears that the ecommerce container cannot reach the LMS using the LMS_HOST public URL for whatever reason, or nginx is blocking access.

Things I have tried so far:

  • Re running quickstart, running reboots, etc. (All components are running on one host, with Docker)
  • Disabling HTTPS
  • Attempting to resolve/ping the LMS_HOST from the ecommerce container, but the container doesn’t have the ping binary
  • Looking through the config files for nginx etc to see if there were host whitelists anywhere

One thing I tried fixed it, but I feel like it’s a hack and isn’t meant to be working that way:

  1. Log in to the ecommerce Django admin panel
  2. Browse to Core -> Site Configurations
  3. Open the Site Configuration for ecommerce.topleveldomain.com
  4. Modify the Partner from ‘Open edX’ to ‘Open edX - Development’
  5. Save
  6. Re-try course creation, then everything works just fine!

I also get the exact same error if I browse to the ecommerce site, click the Login button, Login, then I get redirected back to Ecommerce but I get a ‘Server Error’ message. The output of the tutor logs is the exact same Connection Refused Error 111 as before. I have a screenshot of this but I can’t put it in my post because my user account is only new.

I’m not sure if this is a bug or configuration error on my part, thanks in advance!

I have actually discovered this was due to a NAT hair-pinning issue, where the firewall behind the Tutor server was blocking connections to the public URL. After modifying the firewall to accommodate NAT hair-pinning, this has completely fixed up that issue.

2 Likes