Error during install discovery & ecommerce HTTPConnectionPool(host=‘lms’, port=8000): Read timed out

Hi
Got an error when executed “tutor local quickstart”

Applying waffle.0003_update_strings_for_i18n... OK
    python manage.py install_es_indexes
    2020-10-20 10:47:22,037 INFO 12 [course_discovery.apps.core.management.commands.install_es_indexes] /openedx/discovery/course_discovery/apps/core/management/commands/install_es_indexes.py:19 - Attempting to establish initial connection to Elasticsearch host [http://elasticsearch:9200]...
    2020-10-20 10:47:22,046 INFO 12 [course_discovery.apps.core.management.commands.install_es_indexes] /openedx/discovery/course_discovery/apps/core/management/commands/install_es_indexes.py:21 - ...success!
    2020-10-20 10:47:22,047 INFO 12 [course_discovery.apps.core.utils] /openedx/discovery/course_discovery/apps/core/utils.py:18 - Making sure alias [catalog] exists...
    2020-10-20 10:47:22,054 WARNING 12 [elasticsearch] /openedx/venv/lib/python3.5/site-packages/elasticsearch/connection/base.py:82 - HEAD /_alias/catalog [status:404 request:0.006s]
    2020-10-20 10:47:22,054 WARNING 12 [elasticsearch] /openedx/venv/lib/python3.5/site-packages/elasticsearch/connection/base.py:82 - HEAD /_alias/catalog [status:404 request:0.006s]
    2020-10-20 10:47:23,294 INFO 12 [course_discovery.apps.core.utils] /openedx/discovery/course_discovery/apps/core/utils.py:53 - ...index [catalog_20201020_104722] created.
    2020-10-20 10:47:23,317 INFO 12 [course_discovery.apps.core.utils] /openedx/discovery/course_discovery/apps/core/utils.py:33 - ...alias updated.
    2020-10-20 10:47:28,126 INFO 17 [course_discovery.apps.core.management.commands.create_or_update_partner] /openedx/discovery/course_discovery/apps/core/management/commands/create_or_update_partner.py:127 - Partner created with code dev
    2020-10-20 10:47:32,760 INFO 22 [course_discovery.apps.core.management.commands.create_or_update_partner] /openedx/discovery/course_discovery/apps/core/management/commands/create_or_update_partner.py:127 - Partner created with code openedx
    2020-10-20 10:47:37,677 INFO 27 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:116 - Command is not using threads to write data.
    2020-10-20 10:47:37,681 INFO 27 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:153 - Executing Loader [https://my-edx-domain.com/api/courses/v1/]
    2020-10-20 10:47:42,693 ERROR 27 [course_discovery.apps.course_metadata.management.commands.refresh_course_metadata] /openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py:28 - CoursesApiDataLoader failed!
    Traceback (most recent call last):
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 426, in _make_request
        six.raise_from(e, None)
      File "<string>", line 3, in raise_from
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 421, in _make_request
        httplib_response = conn.getresponse()
      File "/usr/lib/python3.5/http/client.py", line 1225, in getresponse
        response.begin()
      File "/usr/lib/python3.5/http/client.py", line 307, in begin
        version, status, reason = self._read_status()
      File "/usr/lib/python3.5/http/client.py", line 268, in _read_status
        line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
      File "/usr/lib/python3.5/socket.py", line 575, in readinto
        return self._sock.recv_into(b)
    socket.timeout: timed out

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/openedx/venv/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
        timeout=timeout
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 725, in urlopen
        method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/util/retry.py", line 403, in increment
        raise six.reraise(type(error), error, _stacktrace)
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/packages/six.py", line 735, in reraise
        raise value
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 677, in urlopen
        chunked=chunked,
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 428, in _make_request
        self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
      File "/openedx/venv/lib/python3.5/site-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
        self, url, "Read timed out. (read timeout=%s)" % timeout_value
    urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='lms', port=8000): Read timed out. (read timeout=5)

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/openedx/discovery/course_discovery/apps/course_metadata/management/commands/refresh_course_metadata.py", line 25, in execute_loader
        loader_class(*loader_args).ingest()
      File "/openedx/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 31, in __init__
        self.username = self.get_username_from_client(self.api_client)
      File "/openedx/discovery/course_discovery/apps/course_metadata/data_loaders/__init__.py", line 41, in get_username_from_client
        token = client.get_jwt_access_token()
      File "/openedx/venv/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 270, in get_jwt_access_token
        self._ensure_authentication()
      File "/openedx/venv/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 254, in _ensure_authentication
        timeout=self._timeout,
      File "/openedx/venv/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 170, in get_and_cache_oauth_access_token
        timeout=timeout,
      File "/openedx/venv/lib/python3.5/site-packages/edx_rest_api_client/client.py", line 112, in get_oauth_access_token
        timeout=timeout
      File "/openedx/venv/lib/python3.5/site-packages/requests/api.py", line 119, in post
        return request('post', url, data=data, json=json, **kwargs)
      File "/openedx/venv/lib/python3.5/site-packages/requests/api.py", line 61, in request
        return session.request(method=method, url=url, **kwargs)
      File "/openedx/venv/lib/python3.5/site-packages/requests/sessions.py", line 530, in request
        resp = self.send(prep, **send_kwargs)
      File "/openedx/venv/lib/python3.5/site-packages/requests/sessions.py", line 643, in send
        r = adapter.send(request, **kwargs)
      File "/openedx/venv/lib/python3.5/site-packages/requests/adapters.py", line 529, in send
        raise ReadTimeout(e, request=request)
    requests.exceptions.ReadTimeout: HTTPConnectionPool(host='lms', port=8000): Read timed out. (read timeout=5)
    CommandError: One or more of the data loaders above failed.
    Error: Command failed with status 1: docker-compose -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.yml -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/ubuntu/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm discovery-job sh -e -c make migrate

    # Development partners
    ./manage.py create_or_update_partner  \
      --site-id 1 \
      --site-domain discovery.my-edx-domain.com:8381 \
      --code dev --name "Open edX - development" \
      --lms-url="http://lms:8000" \
      --courses-api-url "http://my-edx-domain.com:8000/api/courses/v1/"
     
    # Production partner
    ./manage.py create_or_update_partner  \
      --site-id 2 \
      --site-domain discovery.my-edx-domain.com \
      --code openedx --name "Open edX" \
      --lms-url="http://lms:8000" \
      --courses-api-url "https://my-edx-domain.com/api/courses/v1/"

    ./manage.py refresh_course_metadata --partner_code=openedx
    ./manage.py update_index --disable-change-limit

versions

tutor, version 10.3.1

discovery==10.1.7
ecommerce==10.1.6
lts==10.0.3 (disabled)
minio==10.1.3 (disabled)
notes==10.1.4 (disabled)
xqueue==10.1.2 (disabled)

instance amazon ec2 t3a.large (2cpu, 8gb)

thanks in advance

Hi @alex,
can you try

$ tutor local init --limit discovery

for initializing and migrations ? than

$ tutor local quickstart

Hi there I’m getting the same error even after trying the suggested fix. I’ve logged in and changed the effective url in discoveryadmin but it still comes back to faling on not contacting lms:8000

Sometimes if your local machine runs really slow (for example, the memory is not sufficient), the api call between discovery and lms may timeout.

I upped the resources available in azure and it worked this time thanks good call