Cache Program Configuration

Looking for detailed guidance on how the LMS and Discovery systems in a tutor deployment should be configured so that the command

tutor local run lms ./manage.py lms cache_programs
will work.

I have the discovery plugin installed with tutor 3.8.

I have created a program type and program in the discovery admin view. I am trying to get that program visible on the lms site when I go to explore available courses.

When I refresh the metadata and reindex I see that it mentions the 1 program in the results after the refresh.

Originally I was getting the error ‘skipping, no site configuration’ until I added a site configuration to the LMS.

I originally put this in the JSON box for site configuration:

{
“COURSE_CATALOG_API_URL”:“http://discovery:8000
}

I also tried it with the URL: ‘https://discovery:8000/api/v1’. Both gave the same type of error which is below.

2020-01-01 23:22:28,079 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:116 - Received 0 UUIDs for site modifiedSiteName.com
2020-01-01 23:22:28,080 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:143 - Requesting pathways for modifiedSiteName.com.
2020-01-01 23:22:28,085 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:153 - Failed to retrieve pathways for site: modifiedSiteName.com.
Traceback (most recent call last):
File “/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py”, line 147, in get_pathways
new_pathways = client.pathways.get(exclude_utm=1, page=next_page)
File “/openedx/venv/local/lib/python2.7/site-packages/slumber/init.py”, line 155, in get
resp = self._request(“GET”, params=kwargs)
File “/openedx/venv/local/lib/python2.7/site-packages/slumber/init.py”, line 101, in _request
raise exception_class(“Client Error %s: %s” % (resp.status_code, url), response=resp, content=resp.content)
HttpClientError: Client Error 400: http://discovery:8000/api/v1/pathways/
2020-01-01 23:22:28,085 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:159 - Received 0 pathways for site modifiedSiteName.com
2020-01-01 23:22:28,086 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:72 - Caching UUIDs for 0 programs for site modifiedSiteName.com.
2020-01-01 23:22:28,086 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:79 - Caching ids for 0 pathways for site modifiedSiteName.com.
2020-01-01 23:22:28,087 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:85 - Caching details for 0 programs.
2020-01-01 23:22:28,087 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:90 - Caching details for 0 pathways.

I noticed its reporting an HTTP error with 400 which makes me think it is just a permission issue hitting that url. Therefore, I tried changing the URL config to ‘https’. When I make this change I get the following error when it tries to make the call:

2020-01-01 23:43:29,749 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:153 - Failed to retrieve pathways for site: modifiedSiteName.com.
Traceback (most recent call last):
File “/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py”, line 147, in get_pathways
new_pathways = client.pathways.get(exclude_utm=1, page=next_page)
File “/openedx/venv/local/lib/python2.7/site-packages/slumber/init.py”, line 155, in get
resp = self._request(“GET”, params=kwargs)
File “/openedx/venv/local/lib/python2.7/site-packages/slumber/init.py”, line 97, in _request
resp = self._store[“session”].request(method, url, data=data, params=params, files=files, headers=headers)
File “/openedx/venv/local/lib/python2.7/site-packages/requests/sessions.py”, line 533, in request
resp = self.send(prep, **send_kwargs)
File “/openedx/venv/local/lib/python2.7/site-packages/requests/sessions.py”, line 646, in send
r = adapter.send(request, **kwargs)
File “/openedx/venv/local/lib/python2.7/site-packages/requests/adapters.py”, line 514, in send
raise SSLError(e, request=request)
SSLError: HTTPSConnectionPool(host=‘discovery’, port=8000): Max retries exceeded with url: /pathways/?exclude_utm=1&page=1 (Caused by SSLError(SSLEOFError(8, u’EOF occurred in violation of protocol (_ssl.c:590)'),))

I saw this post and used some of it to make changes in attempt to get the catalog update working:

Hoping @regis or @cacciaresi can help shed some light on getting the discovery and lms communication working for programs.

1 Like

Hi @sbrewer, the error that you see in the logs is related with pathways which is not related (I think) with programs.

The expected output that you should see is this for existing programs being cached:

2019-12-12 13:51:46,619 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:72 - Caching UUIDs for 1 programs for site courses.sitename.com.
2019-12-12 13:51:46,622 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:79 - Caching ids for 0 pathways for site courses.sitename.com.
2019-12-12 13:51:46,624 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] cache_programs.py:85 - Caching details for 1 programs.

So in your logs no programs were detected, some questions that I may ask while troubleshooting are:

  • is the program active?
  • does the program include at least 1 course?
  • is the partner related with the program correctly configured in discovery?
  • Can you get the program info when you hit discovery service endpoint: /api/v1/programs/?

Maybe answering these question can help you to solve the issue.

Regards,

@cacciaresi Do you have links to the API documentation? I have searched all over Open EDX and can’t find the documentation for the APIs. I am used to have an API doc page or Swagger for the APIs so I can piece together how to consume them. The only API documentation I have found to date is for an Enterprise product associated with Open EDX.

As far as I know, there is no documentation for these APIs. But maybe I’m wrong…

There is a lot going on right now regarding API docs. See for example this new repo: https://github.com/edx/api-doc-tools/pull/13#event-2944698130
So even though there is no public documentation right now, it’s going to change in the near future :crossed_fingers:

1 Like

Hi there. I have the same problem as @sbrewer but, instead of getting an HTTP error with 400, mine return HTTP error with 403 as show below :

2020-08-17 08:25:22,176 INFO 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] cache_programs.py:177 - Requesting pathways for openedx_sitename
2020-08-17 08:25:22,193 ERROR 1 [openedx.core.djangoapps.catalog.management.commands.cache_programs] [user None] cache_programs.py:187 - Failed to retrieve pathways for site: openedx_sitename
Traceback (most recent call last):
File “/openedx/edx-platform/openedx/core/djangoapps/catalog/management/commands/cache_programs.py”, line 181, in get_pathways
new_pathways = client.pathways.get(exclude_utm=1, page=next_page)
File “/openedx/venv/lib/python3.5/site-packages/slumber/init.py”, line 155, in get
resp = self._request(“GET”, params=kwargs)
File “/openedx/venv/lib/python3.5/site-packages/slumber/init.py”, line 101, in _request
raise exception_class(“Client Error %s: %s” % (resp.status_code, url), response=resp, content=resp.content)
slumber.exceptions.HttpClientError: Client Error 403: http://discovery:8000/api/v1/pathways/

As mention by @cacciaresi, I’ve made sure that :

  • The programs are active
  • All programs include at least 1 course
  • The partner is correctly configured in discovery
  • I can get the programs list in the swagger of discovery API

Honestly, I don’t know why I get this error.
Please, help !