Issue with SCORM

We are experimenting with using SCORM and noticed that we had the XBlock installed as it comes as part of the Tutor install (Awesome! :slight_smile: ).

We are having some issues when uploading the Scorm Zip. The file does get uploaded but we get an error when saving with the error " Studio’s having trouble saving your work".

The error from the CMS pod is below and I am about to start having a proper look at why it isn’t working for us as I can’t see anything obvious straight away. Has anybody run into this issue before?

2020-12-09 16:40:40,884 INFO 472 [botocore.vendored.requests.packages.urllib3.connectionpool] [user None] connectionpool.py:735 - Starting new HTTPS connection (2): s3.eu-west-2.amazonaws.com
2020-12-09 16:40:42,333 ERROR 472 [root] [user None] signals.py:23 - Uncaught exception from None
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/pyenv/versions/3.5.9/lib/python3.5/contextlib.py", line 30, in inner
    return func(*args, **kwds)
  File "/openedx/venv/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "/openedx/edx-platform/cms/djangoapps/contentstore/views/component.py", line 478, in component_handler
    resp = handler_descriptor.handle(handler, req, suffix)
  File "/openedx/venv/lib/python3.5/site-packages/xblock/mixins.py", line 89, in handle
    return self.runtime.handle(self, handler_name, request, suffix)
  File "/openedx/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1453, in handle
    return super(MetricsMixin, self).handle(block, handler_name, request, suffix=suffix)
  File "/openedx/venv/lib/python3.5/site-packages/xblock/runtime.py", line 1063, in handle
    results = handler(request, suffix)
  File "/openedx/venv/lib/python3.5/site-packages/openedxscorm/scormxblock.py", line 217, in studio_submit
    self.extract_package(package_file)
  File "/openedx/venv/lib/python3.5/site-packages/openedxscorm/scormxblock.py", line 275, in extract_package
    ContentFile(scorm_zipfile.read(zipinfo.filename)),
  File "/openedx/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/openedx/venv/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 495, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/openedx/venv/lib/python3.5/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/openedx/venv/lib/python3.5/site-packages/boto3/s3/inject.py", line 513, in object_upload_fileobj
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/openedx/venv/lib/python3.5/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
    return future.result()
  File "/openedx/venv/lib/python3.5/site-packages/s3transfer/futures.py", line 73, in result
    return self._coordinator.result()
  File "/openedx/venv/lib/python3.5/site-packages/s3transfer/futures.py", line 233, in result
    raise self._exception
  File "/openedx/venv/lib/python3.5/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/openedx/venv/lib/python3.5/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/openedx/venv/lib/python3.5/site-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/openedx/venv/lib/python3.5/site-packages/botocore/client.py", line 317, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/openedx/venv/lib/python3.5/site-packages/botocore/client.py", line 589, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/openedx/venv/lib/python3.5/site-packages/botocore/client.py", line 644, in _convert_to_request_dict
    api_params, operation_model)
  File "/openedx/venv/lib/python3.5/site-packages/botocore/validate.py", line 291, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter ContentType, value: b'text/javascript', type: <class 'bytes'>, valid types: <class 'str'>

Which tutor version are you using (tutor --version)? And which plugins? (tutor plugins list)

Its tutor-openedx-10.5.2.

The only plugins we have installed are the ones we have been building around S3 and custom configuration (password management, course visibility, dashboard search, emails etc) and are all yaml files.

The run time for Tutor is built as part of our deployment (in Drone) so we don’t have a permanent Tutor install set up to run “tutor plugins list” but I can get a printout from a deployment - just need a colleague to confirm the pull request and will update the response.

I have seen this error somewhere but can’t seem to figure out where exactly… Would you be able to share the package zip file, as a private message if necessary?
S3 configuration with Ora2 is tricky – did you draw inspiration from the minio plugin? (you should)

Yeah - sure - that is no problem at all. It is only a test file - we have tried the 2004 and 1.2 format - as SCORM\iSpring (the tool we are using) is fairly new to us (read as we don’t really know what we are doing :rofl: ).

S3 was REALLY problematic as we needed it to work with Signed URLs (auto-generated and expire after a pre-defined period). We started looking at the S3 module recommended on the forum but that didn’t do what we wanted.

We did look at Minio and quite a few posts but in the end we actually ended up going through the git repo for OpenEDX and grep’ping for the setting we needed - as well as boto documentation. It took us a while to get the various parts working (profile images, media root, videos, course import/export, grading, all the data downloads from the web interface, ORA2 etc). We don’t really use the ORA assessments at the moment so we only did really lightweight testing on it - our main concern was media, profile images, course exports and grading (we had to have signed URLs for any data downloads from the Instructor web menu)).

It works really, really well now - we have courses running and users using the system and haven’t had issues with S3 (well apart from this one :slight_smile: ). We do have a few things still left to iron out but they are quite low on the backlog (profile images are hard-wired to a default region - so the bucket has to reside there) but the other settings are all in London region - its only profile pics. I think our next thing is to explore MongoDB tho - as its limiting us to one Availability Zone because of the EBS volumes I think.

Ill grab the plugin list tomorrow and can send a link for the SCORM file as well :slight_smile: Really appreciate the response :slight_smile:

I am encountering this same error with the minio plugin in k8s while trying to upload scorm package, any clues how to correct it?

tutor 11.0.4

Plugins
discovery==11.0.0 (disabled)
ecommerce==11.0.0 (disabled)
license==11.0.0 (disabled)
minio==11.0.0
notes==11.0.0 (disabled)
xqueue==11.0.0 (disabled)

fwiw, i also have an issue reading from the /media folder when the plugin is disabled, that I was hoping this plugin might fix. After uploading a scorm I get the 404 not found on the scorm’s index_lms.html file from LMS. i have seen an nginx config referenced in other tickets that fixes that but I don’t see it in my generated nginx. I have a pretty standard config, haven’t changed anything but themes.

This is a weird error. I have also encountered it, in the same circumstances (minio plugin enabled) but did not manage to consistently reproduce the issue. For instance, I could not reproduce the error in development mode. This looks very much like an upstream django-storage issue, but I did not manage to find out where the error came from.

Can you think of any workarounds I can try? is it fairly easy to try to use straight aws s3?

@regis So in my testing I’ve found that whether talking to minio or s3 directly, I get this error. It does sound like something in django-storages. In both cases, the file is getting uploaded fine, but then something happens after, perhaps in setting the policy? here is my stacktrace in case it helps

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/openedx/venv/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/pyenv/versions/3.8.6/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/openedx/venv/lib/python3.8/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
    return view_func(request, *args, **kwargs)
  File "./cms/djangoapps/contentstore/views/component.py", line 479, in component_handler
    resp = handler_descriptor.handle(handler, req, suffix)
  File "/openedx/venv/lib/python3.8/site-packages/xblock/mixins.py", line 85, in handle
    return self.runtime.handle(self, handler_name, request, suffix)
  File "/openedx/edx-platform/common/lib/xmodule/xmodule/x_module.py", line 1453, in handle
    return super(MetricsMixin, self).handle(block, handler_name, request, suffix=suffix)
  File "/openedx/venv/lib/python3.8/site-packages/xblock/runtime.py", line 1060, in handle
    results = handler(request, suffix)
  File "/openedx/venv/lib/python3.8/site-packages/openedxscorm/scormxblock.py", line 217, in studio_submit
    self.extract_package(package_file)
  File "/openedx/venv/lib/python3.8/site-packages/openedxscorm/scormxblock.py", line 273, in extract_package
    self.storage.save(
  File "/openedx/venv/lib/python3.8/site-packages/django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 495, in _save
    self._save_content(obj, content, parameters=parameters)
  File "/openedx/venv/lib/python3.8/site-packages/storages/backends/s3boto3.py", line 510, in _save_content
    obj.upload_fileobj(content, ExtraArgs=put_parameters)
  File "/openedx/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 511, in object_upload_fileobj
  File "/openedx/venv/lib/python3.8/site-packages/boto3/s3/inject.py", line 431, in upload_fileobj
    Callback=Callback, SourceClient=SourceClient, Config=Config)
  File "/openedx/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 73, in result
  File "/openedx/venv/lib/python3.8/site-packages/s3transfer/futures.py", line 233, in result
    self._result = result
  File "/openedx/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/openedx/venv/lib/python3.8/site-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 317, in _api_call
    return 's3' + suffix
  File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 588, in _make_api_call
    event_emitter, request_signer, service_model, loader,
  File "/openedx/venv/lib/python3.8/site-packages/botocore/client.py", line 643, in _convert_to_request_dict
    'client_region': self.meta.region_name,
  File "/openedx/venv/lib/python3.8/site-packages/botocore/validate.py", line 291, in serialize_to_request
    def serialize_to_request(self, parameters, operation_model):
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter ContentType, value: b'text/javascript', type: <class 'bytes'>, valid types: <class 'str'>

Edit:
I’ve tested this in both Juniper and Koa, with minio and aws s3, and i still see it. It can’t just be me at this point? What I notice is the botocore, boto3, and django-storage versions are the same between juniper and koa, and they all seem to be out of date. I figure it’s either something to do with that or something in my settings that isn’t right but there’s not a lot to mess up in s3 settings so I don’t know what could be wrong, especially since this config did work at one time.

OK I found the explanation. This is actually a bug in django-pipeline 1.7.0, which was later resolved in 2.0.3. I described the issue more throughly here: https://github.com/overhangio/openedx-scorm-xblock/issues/16
We’ll have to see whether we can push a django-pipeline upgrade upstream.

@regis can you give me any pointers on how to get a forked django-pipeline repo into the system? it doesn’t seem to work when I add a forked module to the private.txt in tutor, so I forked the edx-platform repo and attempted to update it in source instead. pip show shows me the updated module but I get errors when attempting to use it. I’m in desperate need of a fix, my system is unusable if I cannot upload scorm files.

It is unnecessary to install a patched django-pipeline, the fix is now part of v11.0.5 https://github.com/overhangio/tutor/releases/tag/v11.0.5

1 Like

This is awesome, thank you :slight_smile:

If we want to upgrade I can see from the docs we can run quickstart - would that delete our pod persistent volumes? Just trying to work out how we do an upgrade :slight_smile:

Yes, thank you very much - I am hitting this error now that I can upload scorms, I cannot read them.

Error grabbing 1.2 API-SecurityError:Blocked a frame with origin "https://minio-lms.mydomain.com" from accessing a cross-origin frame.

@regis Which setting should I change to make that work? I have not messed with any headers or cors settings yet.

No, running quickstart will never delete your data. “When in doubt, run quickstart”!

1 Like