Increase File upload and Download size

[Mahendra]

Hi Everyone,

Recently I installed Maple using tutor. After the installation, all is working fine. But during course export, I’m facing the issue. The course export process working fine, but when I click on the “Download Exported course” button it’s download a partial file. For example, if my exported file size is 150MB, then it’ll download 25-30 MB and break the connection.

The same issue happened when the user upload a file in Staff Graded Xblock. If I try multiple times it’s working fine. I can’t say my users to upload file multiple times because it creates a bad user experience.

My server configuration:
RAM: 12 GB
CPU: 2

tutor version = 13.1.0

Hereby I’m sharing logs during the issue:
Caddy logs:
caddy_1 | {"level":"error","ts":1652784313.8148022,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"http2: stream closed"}

CMS logs:

cms_1            | Tue May 17 10:45:13 2022 - uwsgi_response_write_body_do(): Connection reset by peer [core/writer.c line 429] during GET /export_output/course-v1:demo+DemoX+2022 (172.18.0.4)
cms_1            | OSError: write error

For file upload size i updated MAX_ASSET_UPLOAD_FILE_SIZE_IN_MB = 250MB and also updated below configuration in .local/share/tutor/env/apps/caddy/Caddyfile

my.lms.domain {
    @favicon_matcher {
        path_regexp ^(.*)/favicon.ico$
    }
    rewrite @favicon_matcher /static/images/favicon.ico

    # Limit profile image upload size
    request_body /api/profile_images/*/*/upload {
        max_size 250MB
    }
    request_body {
        max_size 250MB
    }
    import proxy "lms:8000"
}

my.studio.domain {
    @favicon_matcher {
        path_regexp ^(.*)/favicon.ico$
    }
    rewrite @favicon_matcher /static/images/favicon.ico

    request_body {
        max_size 250MB
    }
    import proxy "cms:8000"
}

Do I need to update any configurations?

Can anyone please help?

[regis]

Let’s focus on the question of downloading course export files. The request body max_size statements should not have any impact because they affect the request body size, not the response.

According to this conversation the “aborting with incomplete response” message seems to indicate that the client was disconnected during download.

1. Are you quite sure that your internet connection is 100% functional?
2. Did you manage to reproduce the issue while running locally on local.overhang.io?
3. Maybe it’s a timeout issue. How long does it take before the download is interrupted?
4. Is there any proxy between your browser and the Caddy server?
5. How much free space is available on the server? (you can check with df -lh)

EDIT:

I just simulated a very slow download by rate-limiting curl to 1kb/s and I got the following error from caddy after 2:47:24:

caddy_1                      | {"level":"error","ts":1652794443.8251243,"logger":"http.handlers.reverse_proxy","msg":"reading from backend","error":"read tcp 172.21.0.5:46040->172.21.0.8:800
0: read: connection reset by peer"}                                                                                                                                                           
caddy_1                      | {"level":"error","ts":1652794443.8257854,"logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","error":"read tcp 172.21.0.5:46040->1
72.21.0.8:8000: read: connection reset by peer"}                                                                                                                                              
caddy_1                      | {"level":"info","ts":1652794443.8259127,"logger":"http.log.access.log1","msg":"handled request","request":{"remote_addr":"172.21.0.1:39408","proto":"HTTP/1.1",
"method":"GET","host":"studio.local.overhang.io","uri":"/export_output/course-v1:overhangio+test2+20210901"},"user_id":"","duration":1739.570407253,"size":13480944,"status":200}

Note that in my case there is no uwsgi log.

[Mahendra]

Hi @regis

Please see my below comments on your every question.

1. Are you quite sure that your internet connection is 100% functional?
   Yes, My internet connection is 100% functional and good.
2. Did you manage to reproduce the issue while running locally on local.overhang.io?
   Locally it’s working fine. Only getting issues in a production setup.
3. Maybe it’s a timeout issue. How long does it take before the download is interrupted?
   Download interrupted after 5-10 seconds. If I try multiple times to download then it works.
4. Is there any proxy between your browser and the Caddy server?
   I’m not sure, because I just did the tutor-based installation and there are no modifications. How can I check proxy between your browser and the Caddy server?
5. How much free space is available on the server? (you can check with df -lh)
   86GB of free space is available on the server.
   
   

Please let me know if you need any more details.

[regis]

I am unable to reproduce the issue in production, with https on. I do get the “aborting with incomplete response” error message when I manually interrupt the download from the client.

Are you running any storage-related plugin, like MinIO? (in other words: what is the output of tutor plugins list?)

[Mahendra]

Hi @regis

I’m running only two plugins 1) Forum and 2)MFE. The rest of other plugins are disabled.

Just want to confirm, is it an issue because of the wrong worker configuration? In the edx-platform/lms/docker_lms_gunicorn.py file I have below configuration.

preload_app = False
timeout = 300
bind = "127.0.0.1:8000"
pythonpath = "/edx/app/edxapp/edx-platform"
max_requests = 50
workers = 17

[regis]

@Mahendra

4. Is there any proxy between your browser and the Caddy server?
   I’m not sure, because I just did the tutor-based installation and there are no modifications.

A web proxy is a web server that would sit between your client and your Tutor installation. If you cannot reproduce the issue locally, then it’s almost certainly an issue with your server or your client configuration. For instance, there could a firewall preventing you from downloading large files.