Upgrading Tutor to lilac.master -- a braindump

I’m in the process of upgrading Tutor to open-release/lilac.master. As usual, I discover a lot of tiny things that need to be modified. I’m using this topic as a brain dump for others to follow my progress. Also, these notes might be useful for others attempting a similar process.

If you are interested in testing tutor with Lilac, you may take a look at the lilac branch: WIP feat: upgrade to open-release/lilac.master by regisb · Pull Request #421 · overhangio/tutor · GitHub Beware! There be dragons! :dragon_face:

3 Likes

Upgrading edx-platform

  • got rid of the pre_common_all.py setting template because we don’t have to ignore deprecation warnings anymore – unless we do?
  • turns out there are a few deprecation warnings related to etree, but they are not too bad:
2021-04-13 19:23:13,065 WARNING 7 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/lib/python3.8/site-packages/newrelic/console.py:84: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
  prototype = wrapper.__name__[3:] + ' ' + inspect.formatargspec(                  

2021-04-13 19:23:13,203 WARNING 7 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/edx-platform/lms/djangoapps/course_wiki/plugins/markdownedx/wiki_plugin.py:5: DeprecationWarning: 'etree' is depre
cated. Use 'xml.etree.ElementTree' instead.
  from lms.djangoapps.course_wiki.plugins.markdownedx import mdx_mathjax, mdx_video

2021-04-13 19:23:15,447 WARNING 7 [py.warnings] [user None] [ip None] warnings.py:109 - /openedx/venv/src/django-wiki/wiki/plugins/links/wiki_plugin.py:9: DeprecationWarning: 'etree' is deprecated. Use 'xml.etre
e.ElementTree' instead.
  from wiki.plugins.links.mdx.djangowikilinks import WikiPathExtension

That’s all we have to do to upgrade edx-platform? That’s surprisingly little.

Aaaah no. There is an Elasticsearch connection error while running tutor local import democourse. That’s because the ELASTIC_SEARCH_CONFIG env token was renamed to ELASTIC_SEARCH_CONFIG_ES7. The funny thing is that the line is prefixed by a comment:

# TODO: Once we have successfully upgraded to ES7, switch this back to ELASTIC_SEARCH_CONFIG.

More issues:

Upgrading Elasticsearch

  • We need to define the TAKE_FILE_OWNERSHIP=1 env var to make sure that mounted volumes are writable: Install Elasticsearch with Docker | Elasticsearch Guide [7.12] | Elastic
  • We need to increase vm.max_map_count on the host. Is this new or is it because I’m running on a new computer? Should all users do this? sudo sysctl -w vm.max_map_count=262144
  • to run as a single node, we need to pass discovery.type=single-node otherwise we get “the default discovery settings are unsuitable for production use” errors.
  • the heap size settings must be passed as environment variables.

Upgrading the forums

  • The SEARCH_SERVER environment variabled was replaced by SEARCH_SERVER_ES7.
  • The search:rebuild_index rake task was renamed to search:rebuild_indices

Quickstart works!

2 Likes

I don’t know about everybody else, but I certainly need to do this. From my own /etc/sysctl.conf:

# For elasticsearch on docker
vm.max_map_count=524288

studio works too \o/

wow so much change such beautiful

Hmmm the changes come from “Section Highlights”. By git-blaming some of the code I stumbled upon this PR: https://github.com/edx/edx-platform/pull/26744

wtf is Braze? It’s a “customer engagement platform”: https://www.braze.com/ Hmmmm I’m pretty sure I don’t want to use that… Luckily there aren’t too many hits for “braze” in the edx-platform source code. Most of them come from polish translation strings. :poland:

But does it mean that we need to configure s3 by default to store course highlights? No, because the EXPORT_COURSE_METADATA_FLAG waffle flag is disabled by default.

There is now a “reindex” button which seems to work – except on the demo course which is missing sjson transcripts, but that’s nothing new.

1 Like

I found out a major bug in edx-platform, put my hacker hat on and swiftly proposed a fix: fix: extra ")}" in 500 error page by regisb · Pull Request #27325 · edx/edx-platform · GitHub wow such talent

1 Like

Hey, there are now type annotations in edx-platform! refactor: centralize checks for canonical courseware experience & URL… · edx/edx-platform@9b37e7d · GitHub go mypy!

1 Like

Hi regis, I will look into the discovery plugin for upgrade. @regis

1 Like

Hi @regis I would like to contribute on tutor-notes.If you have some link or guide I’ll be glad to read it before start. Regards

1 Like

Hi regis, have you built images for lilac branch, or I need to built images by myself for testing?@regis

Hey @NeOneSoft @pcliu! You will want to take a look at this topic which I just wrote: How to upgrade a Tutor plugin

Please let me know there if you have any further question!

1 Like

A WIP branch for discovery plugin can be found here: tutor-discovery for lilac (WIP)

Good job! Would you like to open a WIP pull request, such that we can make comments?

Yes, PR has been created

1 Like

What’s the best way to contribute to the mfe plugin? A Fork and PR to the lilac branch?

Yes, that would be great! Note that I’m working on it right now, so I might push-force things.

Account Microfrontend

The Account MFE makes weird calls to http://localhost:18360/demographics/api/v1/demographics/ urls. It seems that this feature is gated by the ENABLE_DEMOGRAPHICS_COLLECTION env setting. I have no idea what this “demographics” backend is supposed to do.

Nope, setting ENABLE_DEMOGRAPHICS_COLLECTION=false does not disable the call to the demographics backend… For that we actually need to set DEMOGRAPHICS_BASE_URL=null.

I can also see that we make calls to http://LMS/api/coaching/v1/users/3/. These calls end in 404. This is because COACHING_ENABLED=true in development (now switching to false).

Well, neither of these work. Is that because these variables are evaluated as process.env.COACHING_ENABLED || false? Maybe we should set these variables to empty strings to disable these features? YES! That did the trick.

Ok, I was planning to work on one of the TODO items. I’m going to look at " re-build the mfe docker image on init/start".

Oh also, I’ve tried the mfe plugin with our customised gradebook and a custom mfe (dashboard) and both worked!