Emails fail with SSL: UNSUPPORTED_PROTOCOL after upgrade

dancasey · January 11, 2021, 1:02pm

After upgrading from 10.2.0 to 11.0.6, I am unable to send emails, with error
request.is_secure()): SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)')

Do I need to change settings for the latest version or am I missing something?

I have the SMTP settings set in the $TUTOR_ROOT/config.yml file as

SMTP_HOST: mail.blacknight.com
SMTP_PASSWORD: <password>
SMTP_PORT: 587
SMTP_USERNAME: <username>
SMTP_USE_TLS: true

Below is more of the log.

lms_1            | 2021-01-11 12:45:24,420 INFO 6 [audit] [user None] [ip 52.30.64.47] password_reset.py:599 - Password reset initiated for user dan.casey@ucc.ie.
lms_1            | 2021-01-11 12:45:24,552 ERROR 6 [openedx.core.djangoapps.user_api.helpers] [user None] [ip 52.30.64.47] helpers.py:86 - An unexpected error occurred when calling 'request_password_change' with arguments '('dan.casey@ucc.ie', True)' and keyword arguments '{}' from File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 610, in password_change_request_handler
lms_1            |     request_password_change(email, request.is_secure()): SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)')
lms_1            | Traceback (most recent call last):
lms_1            |   File "./openedx/core/djangoapps/user_api/helpers.py", line 54, in _wrapped
lms_1            |     return func(*args, **kwargs)
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 268, in request_password_change
lms_1            |     form.save(
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 223, in save
lms_1            |     send_password_reset_email_for_user(user, request)
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 170, in send_password_reset_email_for_user
lms_1            |     ace.send(msg)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/ace.py", line 56, in send
lms_1            |     delivery.deliver(channel, rendered_message, msg)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/delivery.py", line 52, in deliver
lms_1            |     channel.deliver(message, rendered_message)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/channel/django_email.py", line 93, in deliver
lms_1            |     mail.send()
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 306, in send
lms_1            |     return self.get_connection(fail_silently).send_messages([self])
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 103, in send_messages
lms_1            |     new_conn_created = self.open()
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 68, in open
lms_1            |     self.connection.starttls(keyfile=self.ssl_keyfile, certfile=self.ssl_certfile)
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/smtplib.py", line 774, in starttls
lms_1            |     self.sock = context.wrap_socket(self.sock,
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 500, in wrap_socket
lms_1            |     return self.sslsocket_class._create(
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 1040, in _create
lms_1            |     self.do_handshake()
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 1309, in do_handshake
lms_1            |     self._sslobj.do_handshake()
lms_1            | ssl.SSLError: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)
lms_1            | 2021-01-11 12:45:24,555 ERROR 6 [edx.student] [user None] [ip 52.30.64.47] password_reset.py:635 - Error occured during password change for user dan.casey@ucc.ie: An unexpected error occurred when calling 'request_password_change' with arguments '('dan.casey@ucc.ie', True)' and keyword arguments '{}' from File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 610, in password_change_request_handler
lms_1            |     request_password_change(email, request.is_secure()): SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)')
lms_1            | Traceback (most recent call last):
lms_1            |   File "./openedx/core/djangoapps/user_api/helpers.py", line 54, in _wrapped
lms_1            |     return func(*args, **kwargs)
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 268, in request_password_change
lms_1            |     form.save(
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 223, in save
lms_1            |     send_password_reset_email_for_user(user, request)
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 170, in send_password_reset_email_for_user
lms_1            |     ace.send(msg)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/ace.py", line 56, in send
lms_1            |     delivery.deliver(channel, rendered_message, msg)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/delivery.py", line 52, in deliver
lms_1            |     channel.deliver(message, rendered_message)
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/edx_ace/channel/django_email.py", line 93, in deliver
lms_1            |     mail.send()
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/message.py", line 306, in send
lms_1            |     return self.get_connection(fail_silently).send_messages([self])
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 103, in send_messages
lms_1            |     new_conn_created = self.open()
lms_1            |   File "/openedx/venv/lib/python3.8/site-packages/django/core/mail/backends/smtp.py", line 68, in open
lms_1            |     self.connection.starttls(keyfile=self.ssl_keyfile, certfile=self.ssl_certfile)
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/smtplib.py", line 774, in starttls
lms_1            |     self.sock = context.wrap_socket(self.sock,
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 500, in wrap_socket
lms_1            |     return self.sslsocket_class._create(
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 1040, in _create
lms_1            |     self.do_handshake()
lms_1            |   File "/opt/pyenv/versions/3.8.6/lib/python3.8/ssl.py", line 1309, in do_handshake
lms_1            |     self._sslobj.do_handshake()
lms_1            | ssl.SSLError: [SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)
lms_1            | 
lms_1            | During handling of the above exception, another exception occurred:
lms_1            | 
lms_1            | Traceback (most recent call last):
lms_1            |   File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 610, in password_change_request_handler
lms_1            |     request_password_change(email, request.is_secure())
lms_1            |   File "./openedx/core/djangoapps/user_api/helpers.py", line 87, in _wrapped
lms_1            |     raise api_error(msg)
lms_1            | openedx.core.djangoapps.user_api.errors.UserAPIInternalError: An unexpected error occurred when calling 'request_password_change' with arguments '('dan.casey@ucc.ie', True)' and keyword arguments '{}' from File "./openedx/core/djangoapps/user_authn/views/password_reset.py", line 610, in password_change_request_handler
lms_1            |     request_password_change(email, request.is_secure()): SSLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:1124)')
lms_1            | 2021-01-11 12:45:24,559 ERROR 6 [django.request] [user None] [ip None] log.py:222 - Internal Server Error: /account/password
nginx_1          | 172.18.0.3 - - [11/Jan/2021:12:45:24 +0000] http://learn.climateireland.ie "POST /account/password HTTP/1.1" 500 59 "https://learn.climateireland.ie/reset?next=%2F" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36" "52.30.64.47"
lms_1            | [pid: 6|app: 0|req: 2/3] 172.18.0.13 () {62 vars in 1667 bytes} [Mon Jan 11 12:45:24 2021] POST /account/password => generated 59 bytes in 253 msecs (HTTP/1.0 500) 6 headers in 228 bytes (1 switches on core 0)

dancasey · January 11, 2021, 11:13pm

I think the issue is that the mail server I am using only uses TLS1 and the updated docker images default minimum is TLS1_1+, but I am not certain of this.

Any advice would be greatly appreciated. I have reverted back to tutor v10 for now.

Update: I will try to change the MinProtocol and CipherString or the LMS openssl config as per this post, over the next few days. But anticipating this to be problematic in testing locally (non-production )and may require changing the Dockerfile to include line below, but my experience of Dockerfiles is limited to watching Tutor log outputs so will see how far I get. Link to post1, post

RUN sed -i 's/TLSv1.2/TLSv1.0/g' /etc/ssl/openssl.cnf

regis · January 13, 2021, 4:54pm

Hi @dancasey! Please comment here if you feel like the Docker image should include additional libraries to resolve your problem. However, I don’t expect we’ll be adding support for TLS1.0 which is deprecated (as far as I remember?). Are you relying on a well-known provider for email?

dancasey · January 13, 2021, 5:34pm

Our emails use Blacknight mail hosting which would be one most popular Domain/Hosting providers in Ireland. After contacting they have said that they use TLS1.0 and that they will not be upgrading for another year. Which is both surprising and slightly worrying.

I have tried to update the OpenSSL config in the running LMS container but it didn’t correct my issue (though I might have done it wrong) and since Caddy 2 can not be set up to use anything less than TLS1.2 trying to facilitate TLS1.0 in newer versions of Tutor is probably a bad idea.

For now I am happy to stay with Tutor V10, but I may look at this issue myself again in a few weeks to see if I can sort something for our specific needs. Working with Tutor has been a great experience in introducing me to containers and proxies, so firstly thank you and secondly…

Is the outgoing mail from the LMS routed through Caddy?

Because if that is the case I will not be trying to alter other containers for TLS1.0

thanks

regis · January 13, 2021, 8:46pm

No, it is not. I’m curious to learn if you discover a workaround for your issue by patching the openedx Docker image. If you do, other people could use the same fix with the help of a plugin (which I assume you would provide? )

dancasey · January 14, 2021, 9:43am

Of course . If I come up with a solution I will shout it from the roof tops and share it with the world.

dancasey · May 11, 2022, 11:23am

Over a year later…committed to the upgrade and wrote the plugin patch. Though it is probably discouraged to reduce the minimum supported TLS this solution might help others and demonstrate plugin patches to the Ubuntu level of the OpenEdx image.

To recap on the long message above.

Issue: After upgrading from tutor v10 emails were not sending, failing with error SLError(1, '[SSL: UNSUPPORTED_PROTOCOL] unsupported protocol
Explanation: Newer docker images use Ubuntu 20 which has TLSV1.2 set as default minimum
Solution: Developed plugin to allow TLS1.0 by editing /etc/ssl/openssl.cnf of the docker image

name: enable-tlsv1
version: 0.1.0
patches:
  openedx-dockerfile: |
    RUN sed  -i '1i openssl_conf = default_conf' /etc/ssl/openssl.cnf && \
    sed -i -e '$a[default_conf]\n\nssl_conf = ssl_sect\n\n[ssl_sect]\n\nsystem_default = system_default_sect\n\n[system_default_sect] \nMinProtocol = TLSv1\nCipherString = DEFAULT:@SECLEVEL=1' /etc/ssl/openssl.cnf