Discovery plugin error on a clean fresh openedx installation

Tutor
1

Hi every one.
I installed openedx tutor based with only a demo course and an admin user. I than enabled discovery plugin : tutor plugins enable discovery save the environment tutor config save and then I run tutor local quickstart as instructed here https://pypi.org/project/tutor-discovery/

I got this error which persisted and repeated for eternity.

discovery-plugin
discovery-plugin955×625 35.2 KB

I aborted the installation but noticed that that all the containers are up and running without issues.

I then deleated all the env (sudo rm -r .local) and started a new fresh installation with updated images but without course even demo, nor users. I got the same persistent messages.

When aborting Crtl c
I receive :

discovery-error
discovery-error958×63 2.92 KB

docker ps shows all the containers up including the discovery container.

And tutor local logs --follow discovery shows a repeating :

discovery_1         | 2021-01-03 10:27:19,996 **ERROR** 6 [django.request] /openedx/venv/lib/python3.8/site-packages/django/utils/log.py:222 -                               **Internal Server Error**: /
discovery_1         | Traceback (most recent call last):
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
discovery_1         |     return self.cursor.execute(sql, params)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 71, in execute
discovery_1         |     return self.cursor.execute(query, args)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
discovery_1         |     res = self._query(query)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
discovery_1         |     db.query(q)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
discovery_1         |     _mysql.connection.query(self, query)
discovery_1         | MySQLdb._exceptions.**ProgrammingError**: (1146, "Table 'discovery.django_site' doesn't exist")
discovery_1         |
discovery_1         | The above exception was the direct cause of the following exception:
discovery_1         |
discovery_1         | Traceback (most recent call last):
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
discovery_1         |     response = get_response(request)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/utils/deprecation.py", line 93, in __call__
discovery_1         |     response = self.process_request(request)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/contrib/sites/middleware.py", line 12, in process_request
discovery_1         |     request.site = get_current_site(request)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/contrib/sites/shortcuts.py", line 13, in get_current_site
discovery_1         |     return Site.objects.get_current(request)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django_sites_extensions/models.py", line 43, in patched_get_curren                              t
discovery_1         |     return self._get_site_by_request(request)  # pylint: disable=protected-access
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django_sites_extensions/models.py", line 95, in patched_get_site_b                              y_request
discovery_1         |     site = self.get(domain__iexact=host)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/manager.py", line 82, in manager_method
discovery_1         |     return getattr(self.get_queryset(), name)(*args, **kwargs)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 402, in get
discovery_1         |     num = len(clone)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 256, in __len__
discovery_1         |     self._fetch_all()
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 1242, in _fetch_all
discovery_1         |     self._result_cache = list(self._iterable_class(self))
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 55, in __iter__
discovery_1         |     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1142, in execute_sql
discovery_1         |     cursor.execute(sql, params)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 67, in execute
discovery_1         |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
discovery_1         |     return executor(sql, params, many, context)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
discovery_1         |     return self.cursor.execute(sql, params)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/utils.py", line 89, in __exit__
discovery_1         |     raise dj_exc_value.with_traceback(traceback) from exc_value
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
discovery_1         |     return self.cursor.execute(sql, params)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 71, in execute
discovery_1         |     return self.cursor.execute(query, args)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
discovery_1         |     res = self._query(query)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
discovery_1         |     db.query(q)
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 259, in query
discovery_1         |     _mysql.connection.query(self, query)
discovery_1         | django.db.utils.**ProgrammingError**: (1146, "Table 'discovery.django_site' doesn't exist")
discovery_1         | 2021-01-03 10:27:19,996 **ERROR** 6 [django.request] /openedx/venv/lib/python3.8/site-packages/django/utils/log.py:222 - **Internal Server Error**: /
discovery_1         | Traceback (most recent call last):
discovery_1         |   File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
discovery_1         |     return self.cursor.execute(sql, params)

This is a new fresh installation with no demo course nor super/admin user.
n.b : discovery subdomain is set and accessible.

Is this normal? or do I need to set partners/categories/programs prior to install the plugin?

Any suggestions from fellows whose succeeded to install descovery/ecommerce plugins?

Thanks

2

Since I have installed discovery plugin, commands like tutor local quickstart or tutor local init keep showing :

{
- Requesting course run page 1...
2021-01-05 21:49:30,554 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off _make_request(...) for 58.6s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&us ername=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6e7a90>: Failed to establish a new connection: [Errno 110]Connection timed out')))
}

The quickstart or initialisation goes to end when I uninstall the plugin !!??

I then installed course demo and a stuff/admin and reinstalled the discovery plugin and get the same behaviour (Errn110).
The discovery admin is accessible.
I have to Ctrl+c to stop the loop.

{
Running migrations:
  No migrations to apply.
python manage.py install_es_indexes
2021-01-05 21:47:08,672 INFO 10 [course_discovery.apps.core.management.commands.                                                         install_es_indexes] /openedx/discovery/course_discovery/apps/core/management/com                                                         mands/install_es_indexes.py:19 - Attempting to establish initial connection to E                                                         lasticsearch host [http://elasticsearch:9200]...
2021-01-05 21:47:08,674 INFO 10 [course_discovery.apps.core.management.commands.                                                         install_es_indexes] /openedx/discovery/course_discovery/apps/core/management/com                                                         mands/install_es_indexes.py:21 - ...success!
2021-01-05 21:47:08,674 INFO 10 [course_discovery.apps.core.utils] /openedx/disc                                                         overy/course_discovery/apps/core/utils.py:18 - Making sure alias [catalog] exist                                                         s...
2021-01-05 21:47:08,698 INFO 10 [course_discovery.apps.core.utils] /openedx/disc                                                         overy/course_discovery/apps/core/utils.py:22 - ...alias exists.
2021-01-05 21:47:11,724 INFO 13 [course_discovery.apps.core.management.commands.                                                         create_or_update_partner] /openedx/discovery/course_discovery/apps/core/manageme                                                         nt/commands/create_or_update_partner.py:127 - Partner updated with code dev
2021-01-05 21:47:14,709 INFO 16 [course_discovery.apps.core.management.commands.                                                         create_or_update_partner] /openedx/discovery/course_discovery/apps/core/manageme                                                         nt/commands/create_or_update_partner.py:127 - Partner updated with code openedx
2021-01-05 21:47:17,763 INFO 19 [course_discovery.apps.course_metadata.managemen                                                         t.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/cou                                                         rse_metadata/management/commands/refresh_course_metadata.py:126 - Command is not                                                          using threads to write data.
2021-01-05 21:47:17,771 INFO 19 [course_discovery.apps.course_metadata.managemen                                                         t.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/cou                                                         rse_metadata/management/commands/refresh_course_metadata.py:164 - Executing Load                                                         er [https://afaq-online.com/api/courses/v1/]
2021-01-05 21:47:19,664 INFO 19 [course_discovery.apps.course_metadata.data_load                                                         ers.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/a                                                         pi.py:47 - Refreshing Courses and CourseRuns from https://afaq-online.com/api/co                                                         urses/v1/...
2021-01-05 21:47:19,664 INFO 19 [course_discovery.apps.course_metadata.data_load                                                         ers.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/a                                                         pi.py:93 - Requesting course run page 1...
2021-01-05 21:49:30,554 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packa                                                         ges/backoff/_common.py:86 - Backing off _make_request(...) for 58.6s (requests.e                                                         xceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443)                                                         : Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&us                                                         ername=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnect                                                         ion object at 0x7f316c6e7a90>: Failed to establish a new connection: [Errno 110]                                                          Connection timed out')))
2021-01-05 21:50:29,195 INFO 19 [course_discovery.apps.course_metadata.data_load                                                         ers.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/a                                                         pi.py:93 - Requesting course run page 1...
2021-01-05 21:52:38,988 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.                                     py:86 - Backing off _make_request(...) for 94.2s (requests.exceptions.ConnectionError: HTTPSConnecti                                     onPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?pa                                     ge=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnect                                     ion object at 0x7f316c6e7ca0>: Failed to establish a new connection: [Errno 110] Connection timed ou                                     t')))
2021-01-05 21:54:13,264 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/di                                     scovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1.                                     ..
2021-01-05 21:56:24,268 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.                                     py:86 - Backing off _make_request(...) for 145.2s (requests.exceptions.ConnectionError: HTTPSConnect                                     ionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?p                                     age=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnec                                     tion object at 0x7f316c6d8fa0>: Failed to establish a new connection: [Errno 110] Connection timed o                                     ut')))
2021-01-05 21:58:49,601 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/di                                     scovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1.                                     ..
2021-01-05 22:01:00,753 ERROR 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:101 - Giving up _make_request(...) after 4 tries (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a71c0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:01:00,766 INFO 19 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off run_loader(...) for 0.7s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a71c0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:01:01,527 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:47 - Refreshing Courses and CourseRuns from https://afaq-online.com/api/courses/v1/...
2021-01-05 22:01:01,529 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1...
2021-01-05 22:03:11,789 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off _make_request(...) for 27.8s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a76a0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:03:39,567 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1...
2021-01-05 22:05:49,482 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off _make_request(...) for 93.7s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a7880>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:07:23,219 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1...
2021-01-05 22:09:32,714 INFO 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off _make_request(...) for 215.9s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a7af0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:13:08,743 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1...
2021-01-05 22:15:18,841 ERROR 19 [backoff] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:101 - Giving up _make_request(...) after 4 tries (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a7dc0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:15:18,847 INFO 19 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/venv/lib/python3.8/site-packages/backoff/_common.py:86 - Backing off run_loader(...) for 17.3s (requests.exceptions.ConnectionError: HTTPSConnectionPool(host='afaq-online.com', port=443): Max retries exceeded with url: /api/courses/v1/courses/?page=1&page_size=50&username=discovery (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f316c6a7dc0>: Failed to establish a new connection: [Errno 110] Connection timed out')))
2021-01-05 22:15:36,237 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:47 - Refreshing Courses and CourseRuns from https://afaq-online.com/api/courses/v1/...
2021-01-05 22:15:36,238 INFO 19 [course_discovery.apps.course_metadata.data_loaders.api] /openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py:93 - Requesting course run page 1...
^CTraceback (most recent call last):
  File "./manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 165, in handle
    success = execute_loader(
  File "/openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 36, in execute_loader
    run_loader()
  File "/openedx/venv/lib/python3.8/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 33, in run_loader
    return loader_class(*loader_args).ingest()
  File "/openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py", line 50, in ingest
    response = self._make_request(initial_page)
  File "/openedx/venv/lib/python3.8/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/openedx/discovery/course_discovery/apps/course_metadata/data_loaders/api.py", line 95, in _make_request
    response = self.api_client.get(self.api_url + '/courses/', params=params)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 543, in get
    return self.request('GET', url, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/edx_rest_api_client/client.py", line 283, in request
    return super(OAuthAPIClient, self).request(method, url, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 309, in connect
    conn = self._new_conn()
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/openedx/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
KeyboardInterrupt
Error: Command failed with status 130: docker-compose -f /home/muss/.local/share/tutor/env/local/docker-compose.yml -f /home/muss/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/muss/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm discovery-job sh -e -c make migrate

# Development partners
./manage.py create_or_update_partner  \
  --site-id 1 \
  --site-domain discovery.afaq-online.com:8381 \
  --code dev --name "Open edX - development" \
  --lms-url="http://lms:8000" \
  --courses-api-url "http://afaq-online.com:8000/api/courses/v1/"

# Production partner
./manage.py create_or_update_partner  \
  --site-id 2 \
  --site-domain discovery.afaq-online.com \
  --code openedx --name "Open edX" \
  --lms-url="http://lms:8000" \
  --courses-api-url "https://afaq-online.com/api/courses/v1/"

./manage.py refresh_course_metadata --partner_code=openedx
./manage.py update_index --disable-change-limit

}

I am trying this install/uinstall of discovery plugin for tree days with the same result !!?? Eventhought, Iam on a clean fresh tutor installation. Is this a bug related the Koa ?
Your help is really appreciated.