Missing js/css files missing from openedx Docker image (in studio)

gostdio · April 3, 2022, 2:53pm

After building this image, I find that the files & Uploads can not work.
http://studio.local.overhang.io/assets/course-v1:pu+cs101+2022/

In normal version, there should be in /openedx/staticfile/common/js/vendor
But in this version. these file are missed.

http://studio.local.overhang.io/static/studio/common/js/vendor/runtime.min.js
http://studio.local.overhang.io/static/studio/common/js/vendor/common.min.js
http://studio.local.overhang.io/static/studio/common/js/vendor/assets.min.js

In previously, I can build and work fine.
But right now,I don’t what is the different. each time I build it, these files are missing.

right now, only these files:
app@8193ee0ed41e:~/staticfiles/common/js/vendor$

backbone.222432686a33.js jquery.js require.e7199843dfd4.js
backbone.js jquery-migrate.351c89ef1b09.js require.js
backbone.paginator.e3292f22e57f.js jquery-migrate.js sinon.13eb38efdb17.js
backbone.paginator.js jquery.scrollTo.525edcc52fe8.js sinon.js
bootstrap.bundle.6866f102282d.js jquery.scrollTo.js Squire.6b82cd99daf8.js
bootstrap.bundle.js moment-timezone-with-data.60142e6c4416.js Squire.js
hls.5e0c7e1b3bfd.js moment-timezone-with-data.js underscore.f893e294cde6.js
hls.js moment-with-locales.e3cc27d412a1.js underscore.js
index.55c2f095862b.js moment-with-locales.js underscore.string.09290e54cc04.js
index.js picturefill.1d4a46c343cb.js underscore.string.js
jquery.888d4551b8db.js picturefill.js

normally should have :
accessibilityPolicy.min.6ed21ceeb8c5.js hls.5e0c7e1b3bfd.js
accessibilityPolicy.min.js hls.js
accessibilityPolicy.min.js.ec9076436b1d.map i18nMessages.min.328c46f83e32.js
accessibilityPolicy.min.js.map i18nMessages.min.js
ar.e694d008b122.json i18nMessages.min.js.ff71861378bc.map
ar.json i18nMessages.min.js.map
assets.min.b62a8df5a8dd.js index.55c2f095862b.js
assets.min.js index.js
assets.min.js.4e5bf55d622a.map jquery.888d4551b8db.js
assets.min.js.map jquery.js
backbone.222432686a33.js jquery-migrate.351c89ef1b09.js
backbone.js jquery-migrate.js
backbone.paginator.e3292f22e57f.js jquery.scrollTo.525edcc52fe8.js
backbone.paginator.js jquery.scrollTo.js
bootstrap.bundle.6866f102282d.js moment-timezone-with-data.60142e6c4416.js
bootstrap.bundle.js moment-timezone-with-data.js
common.min.1158603b80f5.js moment-with-locales.e3cc27d412a1.js
common.min.js moment-with-locales.js
common.min.js.c29db3e5b80d.map picturefill.1d4a46c343cb.js
common.min.js.map picturefill.js
courseHealthCheck.min.f3309b55bfe7.js require.e7199843dfd4.js
courseHealthCheck.min.js require.js
courseHealthCheck.min.js.bc6d881731b8.map runtime.min.0fc95669347a.js
courseHealthCheck.min.js.map runtime.min.js
courseOutlineHealthCheck.min.1b49356b6325.js runtime.min.js.86ed3dfa5d25.map
courseOutlineHealthCheck.min.js runtime.min.js.map
courseOutlineHealthCheck.min.js.259e8ceeb039.map sinon.13eb38efdb17.js
courseOutlineHealthCheck.min.js.map sinon.js
editImageModal.min.706669698cc1.js Squire.6b82cd99daf8.js
editImageModal.min.js Squire.js
editImageModal.min.js.ee35da03198d.map underscore.f893e294cde6.js
editImageModal.min.js.map underscore.js
es_419.5e7a47c23b2a.json underscore.string.09290e54cc04.js
es_419.json underscore.string.js
fr.b482b4a5c417.json zh_CN.159604f8b813.json
fr.json zh_CN.json

I run:
tutor images build openedx
How could I check the logs?

ToddLichty · April 5, 2022, 6:34pm

This exact issue just happened to me and I am still running Lilac. All the assets.min.js, runtime.min.js files were missing from the CMS.

uetuluk · April 6, 2022, 8:36am

I just noticed that I also have the same problem in my development instance running Lilac.

Joel · April 6, 2022, 8:57am

Hi @ToddLichty and @uetuluk
I had same issue on Maple as detailed here: Files & Uploads page is blank

By pulling the docker image I was able to resolve it. See my reply at the end and change your version number according to your config

uetuluk · April 6, 2022, 9:11am

I am using a fork of edx-platform, so I cannot use the prebuilt docker image unfortunately.

ToddLichty · April 6, 2022, 11:21am

I am using a fork as well. What I don’t understand is what would have changed from last week to this week?

Does anyone know how those files are generated and minified?

ToddLichty · April 6, 2022, 11:52am

So I continued to do some more investigation. Maybe this will help someone figure out what is going on.

I restored a snapshot I had from last week when the page loaded fine. Doing a view source on the page, I can see this piece of code:

    <div id="root" class="SFE"></div>
        <script type="text/javascript" src="/static/studio/common/js/vendor/runtime.min.0fc95669347a.js"></script>
        <script type="text/javascript" src="/static/studio/common/js/vendor/common.min.1158603b80f5.js"></script>
        <script type="text/javascript" src="/static/studio/common/js/vendor/assets.min.b62a8df5a8dd.js"></script>

    </div>

Everything loads fine. Looking at the same page on the latest build, I see this:

    <div id="root" class="SFE"></div>
        <script type="text/javascript" src="/static/studio/common/js/vendor/runtime.min.js"></script>
        <script type="text/javascript" src="/static/studio/common/js/vendor/common.min.js"></script>
        <script type="text/javascript" src="/static/studio/common/js/vendor/assets.min.js"></script>

    </div>

As was originally pointed out by @gostdio , those files are now missing. Also, the corresponding css (assets.min.cs, common.min.css) files are missing as well.

uetuluk · April 7, 2022, 2:33am

I believe the files are generated during the openedx-assets script section of the Dockerfile.

I have attached the Lilac version of the Dockerfile for reference.

github.com

overhangio/tutor/blob/e89ff49223ef4624058ab77ccb72c1394abb8be5/tutor/templates/build/openedx/Dockerfile#L187-L195

      
        
            RUN openedx-assets xmodule \
                && openedx-assets npm \
                && openedx-assets webpack --env=prod \
                && openedx-assets common
            COPY ./themes/ /openedx/themes/
            RUN openedx-assets themes \
                && openedx-assets collect --settings=tutor.assets \
                # De-duplicate static assets with symlinks
                && rdfind -makesymlinks true -followsymlinks true /openedx/staticfiles/

The openedx-assets script from Tutor uses the following Python file for managing the assets: pavelib/assets.py

https://github.com/openedx/edx-platform/blob/open-release/lilac.3/pavelib/assets.py

After further probing, I found that the @edx/studio-frontend npm module does not include a dist folder anymore. The dist folder includes most of the files mentioned by @gostdio as missing.

uetuluk · April 7, 2022, 5:39am

A quick solution that I found is to pin the @edx/studio-frontend to 1.17.0 which is the latest version that comes with the dist folder.

ToddLichty · April 7, 2022, 11:10am

@uetuluk Can I ask what file you modified to pin the @edx/studio-frontend to 1.17.0?

ToddLichty · April 7, 2022, 11:49am

For anyone else that comes across this, I modified the package.json in my edx-platform fork:

I changed this line:

"@edx/studio-frontend": "^1.16.22",

to this:

"@edx/studio-frontend": "1.17.0",

Great work @uetuluk for figuring this out!

jramnai · April 15, 2022, 12:56pm

We have wasted 2 days resolving this thing.

Thanks, @uetuluk @ToddLichty

erickhgm · April 15, 2022, 2:14pm

@ToddLichty @uetuluk @jramnai

Hello guys, how are you?

I’m using open-release/maple.2 with Tutor 13.1.8 and I’m having this problem.
This release already has @edx/studio-frontend 1.17.0.

These are the missing files that I noticed:

static/studio/common/css/vendor/common.min.css
static/studio/common/css/vendor/courseOutlineHealthCheck.min.css
static/studio/common/css/vendor/assets.min.css
static/studio/common/js/vendor/runtime.min.js
static/studio/common/js/vendor/common.min.js
static/studio/common/js/vendor/courseOutlineHealthCheck.min.js
static/studio/common/js/vendor/assets.min.js

Affected pages:

Files & Uploads
Content / Outline
Tools / Checklist

uetuluk · April 20, 2022, 3:39am

I can confirm this problem also exists with open-release/maple.3.

@erickhgm The @edx/studio-frontend should be pinned to 1.17.0. The default package.json file has the line “^1.17.0” which will use the most updated version of the package.

github.com

openedx/edx-platform/blob/b8e31d47ed7dd06f53d88167c4063afc433a06fe/package.json#L10

      
        
            {
              "name": "edx",
              "version": "0.1.0",
              "dependencies": {
                "@edx/brand-edx.org": "^1.3.3",
                "@edx/edx-bootstrap": "1.0.4",
                "@edx/edx-proctoring": "^1.5.0",
                "@edx/frontend-component-cookie-policy-banner": "1.0.0",
                "@edx/paragon": "2.6.4",
                "@edx/studio-frontend": "^1.17.0",
                "axios": "^0.21.1",
                "babel-core": "6.26.0",
                "babel-loader": "6.4.1",
                "babel-plugin-transform-class-properties": "6.24.1",
                "babel-plugin-transform-object-assign": "6.22.0",
                "babel-plugin-transform-object-rest-spread": "6.26.0",
                "babel-polyfill": "6.26.0",
                "babel-preset-env": "1.6.1",
                "babel-preset-react": "6.24.1",
                "backbone": "1.4.0",

uetuluk · April 20, 2022, 3:42am

@regis Sorry for bothering you.

This problem also exists with open-release/maple.3 using Tutor 13.1.11.

Do you know why the dist folder is not included in new versions of the @edx/studio-frontend package?

Thank you.

erickhgm · April 20, 2022, 4:28am

It’s true, I hadn’t noticed that detail. I will test …
Thank you @uetuluk

erickhgm · April 20, 2022, 2:47pm

@regis @uetuluk
Do we have an issue with this?
Is a problem of edx/studio-frontend or tutor build?

regis · April 24, 2022, 6:56am

Thanks for your work and your investigation everyone! Please let me know if I understand the problem correctly:

The @edx/studio-frontend package is a requirement of edx-platform, with version ^1.17.0, which basically mean “anything between 1.0.0 (included) and 2.0.0 (excluded)” (see: https://semver.npmjs.com/).
The @edx/studio-frontend package was recently (~ April 4th) updated to 1.18.2. Because of the unpinned required requirement in edx-platform, this causes new builds of Docker images to upgrade to the newest release.
The 1.18.2 release does not ship with some required files, such as node_modules/@edx/studio-frontend/dist/common.min.js, thus causing errors in the studio.
The files that are missing in 1.18.2 were present in 1.17.0, as we can see by running:

$ find node_modules/@edx/studio-frontend/ -name "common*.css"
node_modules/@edx/studio-frontend/dist/common.min.js

The first version in which the dist/ folder was missing is 1.18.0: https://registry.npmjs.org/@edx/studio-frontend/-/studio-frontend-1.18.0.tgz Compare with 1.17.0 where the dist/ foldr was present: https://registry.npmjs.org/@edx/studio-frontend/-/studio-frontend-1.17.0.tgz

It seems to me that there are two bugs here, both upstream:

The @edx/studio-frontend package should have a version pinned to 1.17.0 in edx-platform’s package.json.
The 1.18.* versions of package @edx/studio-frontend should include the dist/ folder.

I have no idea why edX.org is not affected by (2). My guess is that npm packages are either installed in place or stored in a CDN, which caches older versions for some reason.

I propose to fix the issue in Tutor by pinning the studio-frontend version in the open-release/maple.master branch of edx-platform. Would someone like to open a pull request on edx-platform to that effect? You should link to this comment in your PR description. Then, open a PR on Tutor to cherry-pick your commit.

Thanks again for your work!

uetuluk · April 25, 2022, 5:48am

I can open the pull request for the edx-platform.

Thank you @regis for organizing the findings in an easy to present format.

Edit: I have opened the following PR for this issue: Pin npm package @edx/studio-frontend to 1.17.0 to fix missing assets by uetuluk · Pull Request #30309 · openedx/edx-platform · GitHub

erickhgm · April 25, 2022, 12:54pm

Shouldn’t the new version of edx/studio-frontend be fixed?
Using the 1.17.0 in edx-platform’s package.json would be a temporary solution, right?

I started working with Open edX about 3 months ago and I see many opportunities for improvement, especially in the documentation.
It will be a pleasure to contribute …

Everything working fine, I can open the PR to git cherry-pick.