Is SOFTWARE_SECURE part of the config.yml?

bryancr89 · April 8, 2020, 9:11pm

Hi there,

I ran tutor local quickstart and created an ecommerce course with the verification ID checked
When I try to upload the photos I get an error, so I’m wondering if I’m missing something in the config.yml

This is the error I’m getting:

lms_1               | 2020-04-08 20:44:44,050 ERROR 38 [root] signals.py:21 - Uncaught exception from None
lms_1               | Traceback (most recent call last):
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
lms_1               |     response = get_response(request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
lms_1               |     response = self._get_response(request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
lms_1               |     response = self.process_exception_by_middleware(e, request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
lms_1               |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
lms_1               |     return self.dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 863, in dispatch
lms_1               |     return super(SubmitPhotosView, self).dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
lms_1               |     return handler(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
lms_1               |     return view_func(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 185, in inner
lms_1               |     return func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 916, in post
lms_1               |     self._submit_attempt(request.user, face_image, photo_id_image, initial_verification)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 1045, in _submit_attempt
lms_1               |     attempt.upload_face_image(face_image)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 84, in with_status_check
lms_1               |     return func(obj, *args, **kwargs)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 585, in upload_face_image
lms_1               |     aes_key_str = settings.VERIFY_STUDENT["SOFTWARE_SECURE"]["FACE_IMAGE_AES_KEY"]
lms_1               | KeyError: 'SOFTWARE_SECURE'
lms_1               | 2020-04-08 20:44:44,051 ERROR 38 [django.request] exception.py:135 - Internal Server Error: /verify_student/submit-photos/
lms_1               | Traceback (most recent call last):
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
lms_1               |     response = get_response(request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
lms_1               |     response = self._get_response(request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
lms_1               |     response = self.process_exception_by_middleware(e, request)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
lms_1               |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
lms_1               |     return self.dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 863, in dispatch
lms_1               |     return super(SubmitPhotosView, self).dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
lms_1               |     return handler(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
lms_1               |     return view_func(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 67, in _wrapper
lms_1               |     return bound_func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 185, in inner
lms_1               |     return func(*args, **kwargs)
lms_1               |   File "/openedx/venv/local/lib/python2.7/site-packages/django/utils/decorators.py", line 63, in bound_func
lms_1               |     return func.__get__(self, type(self))(*args2, **kwargs2)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 916, in post
lms_1               |     self._submit_attempt(request.user, face_image, photo_id_image, initial_verification)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 1045, in _submit_attempt
lms_1               |     attempt.upload_face_image(face_image)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 84, in with_status_check
lms_1               |     return func(obj, *args, **kwargs)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 585, in upload_face_image
lms_1               |     aes_key_str = settings.VERIFY_STUDENT["SOFTWARE_SECURE"]["FACE_IMAGE_AES_KEY"]
lms_1               | KeyError: 'SOFTWARE_SECURE'

regis · April 13, 2020, 5:33pm

Hmmmm It looks like you need to customize the VERIFY_STUDENT setting from the LMS. I see that by default, its value is:

VERIFY_STUDENT = {
    "DAYS_GOOD_FOR": 365, 
    "EXPIRING_SOON_WINDOW": 28,
}

Apparently, you need to add the SOFTWARE_SECURE key to this dict. You would have to dig into the edx-platform source code to figure what would be the proper value. I honestly have no idea and would need to further investigate to understand that.

bryancr89 · May 26, 2020, 6:15pm

Is this verification performed outside of EDX?
And yes I added it to the env variables for LMS

ejklock · October 1, 2020, 7:51pm

hi @bryancr89 @regis!

I’m trying do activate these feature . The part of set SOFTWARE_SECURE key i did with a tutor plugin

github.com

ejklock/tutor-setstudentverification/blob/master/setstudentverificationkey.yml

name: setstudentverificationkey
version: 0.1.0
patches:
  lms-env: |
    "VERIFY_STUDENT":{
       "DAYS_GOOD_FOR": 365,
       "EXPIRING_SOON_WINDOW": 28,
       "SOFTWARE_SECURE":{
         "FACE_IMAGE_AES_KEY":"357638792F423F4528482B4D6251655468576D5A7134743777397A2443264629"
        }
    }

The image is successfully sent and encrypted. However, Edx tries to send this image to an S3 bucket. I activated the tutor-minio plugin to have this feature. But now Edx accuses him of not having permission to send the photo to the bucket

lms_1               | 2020-10-01 19:31:42,011 ERROR 286 [django.request] [user 4] log.py:228 - Internal Server Error: /verify_student/submit-photos/
lms_1               | Traceback (most recent call last):
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
lms_1               |     response = get_response(request)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 115, in _get_response
lms_1               |     response = self.process_exception_by_middleware(e, request)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 113, in _get_response
lms_1               |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 71, in view
lms_1               |     return self.dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/utils/decorators.py", line 45, in _wrapper
lms_1               |     return bound_method(*args, **kwargs)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 865, in dispatch
lms_1               |     return super(SubmitPhotosView, self).dispatch(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 97, in dispatch
lms_1               |     return handler(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/utils/decorators.py", line 45, in _wrapper
lms_1               |     return bound_method(*args, **kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
lms_1               |     return view_func(request, *args, **kwargs)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/utils/decorators.py", line 45, in _wrapper
lms_1               |     return bound_method(*args, **kwargs)
lms_1               |   File "/opt/pyenv/versions/3.5.9/lib/python3.5/contextlib.py", line 30, in inner
lms_1               |     return func(*args, **kwds)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 918, in post
lms_1               |     self._submit_attempt(request.user, face_image, photo_id_image, initial_verification)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/views.py", line 1047, in _submit_attempt
lms_1               |     attempt.upload_face_image(face_image)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 87, in with_status_check
lms_1               |     return func(obj, *args, **kwargs)
lms_1               |   File "/openedx/edx-platform/lms/djangoapps/verify_student/models.py", line 699, in upload_face_image
lms_1               |     self._storage.save(path, buff)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/django/core/files/storage.py", line 52, in save
lms_1               |     return self._save(name, content)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/storages/backends/s3boto.py", line 425, in _save
lms_1               |     key = self.bucket.get_key(encoded_name)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/boto/s3/bucket.py", line 193, in get_key
lms_1               |     key, resp = self._get_key_internal(key_name, headers, query_args_l)
lms_1               |   File "/openedx/venv/lib/python3.5/site-packages/boto/s3/bucket.py", line 231, in _get_key_internal
lms_1               |     response.status, response.reason, '')
lms_1               | boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden

I looked at the source code of the function that implements this and found this line

github.com

edx/edx-platform/blob/88ae23eb61b4b4fcbd640eec94f93c4ca09d164f/lms/djangoapps/verify_student/models.py

# -*- coding: utf-8 -*-
"""
Models for Student Identity Verification

This is where we put any models relating to establishing the real-life identity
of a student over a period of time. Right now, the only models are the abstract
`PhotoVerification`, and its one concrete implementation
`SoftwareSecurePhotoVerification`. The hope is to keep as much of the
photo verification process as generic as possible.
"""

import base64
import codecs
import functools
import json
import logging
import os.path
import uuid
from datetime import timedelta
from email.utils import formatdate

This file has been truncated. show original

@regis the S3_BUCKET config would it have to be set for him to try to write in the tutor-minio ?

regis · October 5, 2020, 7:22am

@ejklock According to this piece of code, I think it’s better if you actually don’t add any value to the settings.VERIFY_STUDENT["SOFTWARE_SECURE"] configuration. That way, all settings will (should?) be loaded from settings.AWS_* values.

Note however that this piece of code relies on the legacy S3BotoStorage backend, not on S3Boto3Storage. This might require different configuration: https://django-storages.readthedocs.io/en/1.8/backends/amazon-S3.html

I encourage you to debug the get_storage function to check what are the attributes of the storage backend, as well as the url where it tries to save the data.

vejeta · January 11, 2021, 12:01pm

Hello! Are there any updates regarding this?

To those that managed to make it work:

On behalf of tutor, is there need to install any plugin (self-made or tutor-minio)
Where are the place to set up the AWS S3 settings?

ejklock · January 13, 2021, 12:44pm

@regis

I managed to work Photo Verification ID on OpenEdx Tutor.

Default Storage to tutor-minio
I made work with tutor-minio changing de storage class from s3boto to s3boto3. The below line ilustrates de changing

image842×47 12.2 KB
Software Secure Keys (SOFTWARE_SECURE)
Based on test in the edx-platform source code, i just copy and paste from tests to devstack.py (on lms/envs) and all things work properly (only for test propouses). And all process of student verification works and all images submited has encrtypted and sended to tutor-minio storage.

@regis
Previosly i had a tutor plugin to add SOFTWARE_SECURE configs on tutor instance. But now i have some troubles to put RSA_PUBLIC_KEY on plugin. How is the properly way to add the RSA_PUBLIC_KEY string to .yml plugin?

My plugin:

github.com

ejklock/tutor-setstudentverification/blob/master/setstudentverificationkey.yml

name: setstudentverificationkey
version: 0.1.0
patches:
  lms-env: |
    "VERIFY_STUDENT":{
       "DAYS_GOOD_FOR": 365,
       "EXPIRING_SOON_WINDOW": 28,
       "SOFTWARE_SECURE":{
         "FACE_IMAGE_AES_KEY":"357638792F423F4528482B4D6251655468576D5A7134743777397A2443264629"
        }
    }

Best regards

scwall · January 13, 2021, 1:14pm

Hello I was just on the same question, we just migrated from ironwood to koa and I wanted to add this more cleanly than in the py file. If you’re interested I created a little lib quickly for work to get the users and the ID card (I just published it on github) from minio. it’s made with click because we’re using it on a laravel project. If it can inspire you. the libraries may not be just any more I had to readapt the decryption library from python 2 to python 3.

scwall/decrypt-verify-student (github.com)

scwall · January 13, 2021, 3:22pm

I added a small readme if needed at the bookstore to explain how to get minio’s pictures back.
I summarize here
an api must be created to retrieve the information sent from openedx to the platform
add the platform to the configuration file. This one returns a json here is an example

{
    "EdX-ID": "ad216c07-*-3a25b56b3787",
    "ExpectedName": "ad216c07-*-3a25b56b3787",
    "PhotoID": "https://*/photo_id/ad216c07*-3a25b56b3787?Signature=*4%3D&Expires=19*9&AWSAccessKeyId=openedx",
    "PhotoIDKey": "SHA1dmWUiwH4U0q/H+rIejx5Ez6ZPvMporQRj8U9y4wItHz1rVcF2a88XyOWuINmzVNix0xHgyZaGU0VjfviyKVEKCvJP1cQgdA0Ec7*XR4zk2ULNRf5+FVKhOeL1MLFGtRsYzJf/lFsfRsOCS+vxrTvBKQCwGVN6GU/wbFaZgi5WrO30erRMY1MTgDLdw9pdwl3k0EV+YZ5XF7GXIP5P+Wo2EVLqUB7F3ZlI/fkrG7rqZiOZQ==↵",
    "UserPhoto": "https://*/openedx/face/ad216c07-503a-460c-8b75-3a25b56b3787?Signature=DqF2uQy8hKkTwptbL*EJw%3D&Expires=1922*9&AWSAccessKeyId=openedx",
    "UserPhotoKey": "lcGPipE0DzYmLcnSPn+Est0SQJlvrl*aijfpkJKiV+0oDLKY0WBIrcKzyKeR+FUkjCjOPOXmtX9kNbVIzazZFeI4RszD0vwX7WPQrL4iegmMW7CeHrjnvyU5/czScA2f9A9UCTwPALmxFCowUH4dbNCaZQKm5Saj37FSM5UNSC3D1b+5+5ps75171IBEB3aQgMaG6LdxXy1XCinbbTyWaiabyMQojP4FduoYOscbE45ghKrLsYP4TkMXQ7LqAl4E1HrfzFjHfx5hNYqFNoRfeRlKwshZicGZeMkqTzjKw2j2v4NzPKQ==↵",
    "SendResponseTo": "https://******/verify_student/results_callback"
}

then you have to retrieve the information and transmit it to the order.

Then you can send back a request if it’s ok or refused on ironwood on the other hand the verification of the message by contribution to the encrypted message was not activated it is just necessary to send back to him on the new version I have not looked at yet.
the api key is the one you specified in the config file for identity

POST https://*/verify_student/results_callback
Content-Type: application/json
Authorization: _  API_KEY:_

{"Result": "PASS",
    "EdX-ID":"550fa42f-*-8bbb-702ee7407c94"
}

or

{"Result": "FAIL",
    "EdX-ID":"550fa42f-36-*-8bbb-702ee7407c94",
    "Reason": "not the same person".
}

scwall · January 21, 2021, 5:28pm

Hello,
Well here you go @ejklock I’ve started to modify the photo system for juniper and the python 3 version.

I’ll send the new changes to the repository to answer the original question you have to convert the key to byte and give it the line feed back manually \n
example :

RSA_PUBLIC_KEY":b"-----BEGIN PUBLIC KEY-----\nMIIBIj*****Upz\nc+eYc4Pyflb/WpfgYATggkoQdnsdplmvPtQr85+utgqKkxOh+PvYGW8QNUzjLIu4\n5/GlyvBa82i1jRMgEAxGI95bz7j9DtH+7mnj+06zR5xHwT49jK0zMs5MjMaz5WRq\nBUNkz7dxWzDrYJZQx230sPp6upy1Y5H5O8SnJVdghsh8sNciS4Bo4ZONQ3giBwxz\nh48svrspz1MIsOoShjbAdfG+4VX7sVwYlw2rnQeRsMH5/xpnNeqtScyOMoz0N9UDG\ndtRMNGa2MihAg7zh7/zckbUrtf+o5wQtlCJL1Kdj4EjshqYvCxzWnSM+MaYAjb3M\nEQIDAQAB\n-----END PUBLIC KEY-----",

Small info a problem I encountered for sending photos with the api it has become a periodic celery task it must be activated in the tasks under the name of lms.djangoapps.verify_student.tasks.send_request_to_ss_for_user

ejklock · January 21, 2021, 5:43pm

Hello @scwall, I got it in another way too. Creating a plugin with cookiecutter and, in the patches folder, include the settings in the files openx-lms-development-settings and openx-lms-production-settings

Thanks for update

imantaba · April 24, 2021, 8:15am

Hello @ejklock , Can you please explain your solution?
To be honest, I couldn’t get how verification process works or I need a third party account from Software Secure (PSIOnline).

I appreciate it if you can provide a step by step guide so everyone can learn from it.

Thanks