Tutor in development, error 1 import imp

Leonn · November 22, 2021, 3:50pm

Hi tutor community!

Right now we are trying to setup a development environment for our developer to work on customizing openedx. Unfortunately we are running is now errors here and there.

Lets start with setting up the dev env:

We did it following the docs
We get the server running by running ‘tutor dev runserver lms’
Now we want to mount our own local fork by overriding the docker-compose volumes using these docs
We get the following error when running tutor dev runserver lms:

 The lms service will be available at http://local.overhang.io:8000
docker-compose -f /home/leon/.local/share/tutor/env/local/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.override.yml --project-name tutor_dev run --rm --service-ports lms
Creating tutor_dev_lms_run ... done
Setting file permissions for user openedx...
File permissions set.
2021-11-22 15:45:41,785 WARNING 1 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/newrelic/core/application.py:26: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

Traceback (most recent call last):
  File "/openedx/edx-platform/openedx/core/djangoapps/coursegraph/tasks.py", line 19, in <module>
    from py2neo.matching import NodeMatcher
ModuleNotFoundError: No module named 'py2neo.matching'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./manage.py", line 103, in <module>
    startup.run()
  File "/openedx/edx-platform/lms/startup.py", line 20, in run
    django.setup()
  File "/openedx/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/openedx/venv/lib/python3.8/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
  File "/openedx/venv/lib/python3.8/site-packages/django/apps/config.py", line 116, in create
    mod = import_module(mod_path)
  File "/opt/pyenv/versions/3.8.6/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/openedx/core/djangoapps/coursegraph/apps.py", line 11, in <module>
    class CoursegraphConfig(AppConfig):
  File "/openedx/edx-platform/openedx/core/djangoapps/coursegraph/apps.py", line 17, in CoursegraphConfig
    from openedx.core.djangoapps.coursegraph import tasks
  File "/openedx/edx-platform/openedx/core/djangoapps/coursegraph/tasks.py", line 21, in <module>
    from py2neo import NodeMatcher
ImportError: cannot import name 'NodeMatcher' from 'py2neo' (/openedx/venv/src/py2neo/py2neo/__init__.py)
ERROR: 1
Error: Command failed with status 1: docker-compose -f /home/leon/.local/share/tutor/env/local/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.override.yml --project-name tutor_dev run --rm --service-ports lms

We have been searching for /openedx/venv/lib/python3.8/site-packes/nerelic/core/application.py to change the change import imp. Unfortunately we can’t find it, probably due to it being in an venv that is not remotely accessible(?).

The override yml:

version: "3.7"
services:
  lms:
    volumes:
      - /home/leon/edxsource/edx-platform/:/openedx/edx-platform
  cms:
    volumes:
      - /home/leon/edxsource/edx-platform/:/openedx/edx-platform
  lms-worker:
    volumes:
      - /home/leon/edxsource/edx-platform/:/openedx/edx-platform
  cms-worker:
    volumes:
      - /home/leon/edxsource/edx-platform/:/openedx/edx-platform

Hopefully you guys can give on some insights on this situation. Haven’t been able to find this error by searching the docs and this forum.

We did found this Tutor Analysis openedx.atlassian.net/wiki/spaces/AC/pages/2451276919/Tutor+Analysis. Here they are talking about copying container’s venv to the host using the bindmount command. In our view this couldn’t be possible since the container doesn’t exist yet(?). I’m sure we are missing some conceptual vision in this matter. Hopefully you guys can help us a bit.

Thanks in advance!

regis · November 25, 2021, 8:04am

Did you follow the instructions on how to prepare the edx-platform repo for mounting? Open edX development — Tutor documentation

In particular: Configuration and customisation — Tutor documentation

Given that you have from py2neo import NodeMatcher on line 21 of “edx-platform/openedx/core/djangoapps/coursegraph/tasks.py”, my guess is that you did not create your fork on top of the lilac.2 tag, but on top of master:

github.com

edx/edx-platform/blob/open-release/lilac.master/openedx/core/djangoapps/coursegraph/tasks.py

"""
This file contains a management command for exporting the modulestore to
neo4j, a graph database.
"""


import logging

from celery import shared_task
from django.utils import timezone
from edx_django_utils.cache import RequestCache
from edx_django_utils.monitoring import set_code_owner_attribute
from opaque_keys.edx.keys import CourseKey
from py2neo import Graph, Node, Relationship, authenticate, NodeSelector
from py2neo.compat import integer, string


log = logging.getLogger(__name__)
celery_log = logging.getLogger('edx.celery.task')

This file has been truncated. show original

Leonn · November 26, 2021, 6:23pm

Thanks a lot for your answer @regis , and my beginner question! Indeed it was about forking the Lilac tag. Slowly but surely I’m getting a hang of Tutor and how it can be implemented. We get the customized theme’s going, custom fork and ready for our pipeline.

Now we want to bind our fork, which is cloned to our development ubuntu server, to the dev containers. Following this docs i’ve made the following override yaml:

version: "3.7"
services:
  lms:
    volumes:
      - /home/leon/openedx/:/openedx/edx-platform
  cms:
    volumes:
      - /home/leon/openedx/:/openedx/edx-platform
  lms-worker:
    volumes:
      - /home/leon/openedx/:/openedx/edx-platform
  cms-worker:
    volumes:
      - /home/leon/openedx/:/openedx/edx-platform

I tried to remove the slash at the end of openedx, didn’t work. I’m receiving the following error when running tutor dev runserver lms.


leon@docker:~/openedx$ tutor dev runserver lms
The lms service will be available at http://local.overhang.io:8000
docker-compose -f /home/leon/.local/share/tutor/env/local/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.override.yml --project-name tutor_dev run --rm --service-ports lms
Creating tutor_dev_lms_run ... done
Setting file permissions for user openedx...
File permissions set.
Traceback (most recent call last):
  File "./manage.py", line 120, in <module>
    startup.run()
  File "/openedx/edx-platform/lms/startup.py", line 20, in run
    django.setup()
  File "/openedx/venv/lib/python3.8/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/openedx/venv/lib/python3.8/site-packages/django/apps/registry.py", line 114, in populate
    app_config.import_models()
  File "/openedx/venv/lib/python3.8/site-packages/django/apps/config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "/opt/pyenv/versions/3.8.6/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/openedx/edx-platform/lms/djangoapps/bulk_email/models.py", line 19, in <module>
    from openedx.core.djangoapps.course_groups.cohorts import get_cohort_by_name
  File "/openedx/edx-platform/openedx/core/djangoapps/course_groups/cohorts.py", line 21, in <module>
    from lms.djangoapps.courseware import courses
  File "/openedx/edx-platform/lms/djangoapps/courseware/courses.py", line 45, in <module>
    from lms.djangoapps.courseware.module_render import get_module
  File "/openedx/edx-platform/lms/djangoapps/courseware/module_render.py", line 62, in <module>
    from lms.djangoapps.lms_xblock.runtime import LmsModuleSystem
  File "/openedx/edx-platform/lms/djangoapps/lms_xblock/runtime.py", line 20, in <module>
    from xmodule.library_tools import LibraryToolsService
  File "/openedx/edx-platform/common/lib/xmodule/xmodule/library_tools.py", line 13, in <module>
    from openedx.core.djangoapps.content_libraries import api as library_api
  File "/openedx/edx-platform/openedx/core/djangoapps/content_libraries/api.py", line 57, in <module>
    from openedx.core.djangoapps.content_libraries import permissions
  File "/openedx/edx-platform/openedx/core/djangoapps/content_libraries/permissions.py", line 7, in <module>
    from openedx.core.djangoapps.content_libraries.models import ContentLibraryPermission
  File "/openedx/edx-platform/openedx/core/djangoapps/content_libraries/models.py", line 31, in <module>
    class ContentLibrary(models.Model):
  File "/openedx/venv/lib/python3.8/site-packages/django/db/models/base.py", line 108, in __new__
    raise RuntimeError(
RuntimeError: Model class openedx.core.djangoapps.content_libraries.models.ContentLibrary doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
ERROR: 1
Error: Command failed with status 1: docker-compose -f /home/leon/.local/share/tutor/env/local/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.yml -f /home/leon/.local/share/tutor/env/dev/docker-compose.override.yml --project-name tutor_dev run --rm --service-ports lms

Besides this error, I am able to build an image from the same repo. I guess that shows me that the repo is forked correctly, from Lilac 2. Hopefully this information is sufficient. Thanks in advance.

regis · November 29, 2021, 9:14am

Again, did you follow these instructions? Open edX development — Tutor documentation

In particular, did you run pip install --requirement requirements/edx/development.txt? It will be easier for us to troubleshoot your issue if you give us more details about which steps you ran.

system · February 27, 2022, 9:14am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.