MINIO gateway to azure storage

I know this is a newbie settings issue but I am trying to work through the gateway settings for azure

So I know I set the following MINIO_GATEWAY: true

I’m assuming I need to set the keys to the azure keys but beyond that I am getting a bit stumped in the layout of this

thanks for any help

1 Like

Not exactly :slight_smile: You should set this variable to be the same as the arguments that you would pass to the minio gateway command. For instance, for Azure, you would set this setting to azure. The MINIO_ACCESS_KEY and MINIO_SECRET_KEY should then refer to your azurestorageaccountname and azurestorageaccountkey.

Not very intuitive, I know… If it works for you please report your findings here and I’ll update the README.

its not working so far

I set the via config save
MINIO_ACCESS_KEY= storageaccountname
MINIO_SECRET_KEY= accesskeylong
MINIO_Gateway= azure

leaving the base openedx settings for the OPENEDX_AWS keys

when running save and run a local quick start
it fails at

`Plugin minio: running pre-init for service minio

docker-compose -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.yml -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm minio-job sh -e -c mc config host add minio http://minio:9000 openedx BuXLVVNGBO6a0ThtsQFHpAzS --api s3v4
mc mb --ignore-existing minio/openedx minio/openedxuploads minio/openedxvideos

Make common file upload bucket public (e.g: for forum image upload)
mc policy set public minio/openedx
Creating tutor_local_minio-job_run … done
mc: Configuration written to /root/.mc/config.json. Please update your access credentials.
mc: Successfully created /root/.mc/share.
mc: Initialized share uploads /root/.mc/share/uploads.json file.
mc: Initialized share downloads /root/.mc/share/downloads.json file.
Added minio successfully.
mc: Unable to make bucket minio/openedx. Put “http://minio:9000/openedx/”: dial tcp: lookup minio on 127.0.0.11:53: no such host

looking at the init file

mc config host add minio http://minio:9000 {{ OPENEDX_AWS_ACCESS_KEY }} {{ OPENEDX_AWS_SECRET_ACCESS_KEY }} --api s3v4

it seems to be at that stage it will not accept the gateway settings and not access the service

Can you get the logs from the minio container? Run: tutor local logs minio.

thanks for looking at this

in this case the bucket it should be looking for is not openedx.blob.core.windows.net but .MINIO_ACCESS_KEY.core.windows.net

I think this is because the value is linked to the original settings in the docker local config has

# MinIO
minio:
  image: {{ MINIO_DOCKER_IMAGE }}
  {% if not MINIO_GATEWAY %}
  volumes:
    - ../../data/minio:/data
  {% endif %}
  environment:
    MINIO_ACCESS_KEY: "{{ OPENEDX_AWS_ACCESS_KEY }}"
    MINIO_SECRET_KEY: "{{ OPENEDX_AWS_SECRET_ACCESS_KEY }}"
  {% if MINIO_GATEWAY %}
  command: gateway {{ MINIO_GATEWAY }} --address ":9000"
  {% else %}
  command: server --address ":9000" /data
  {% endif %}
  restart: unless-stopped

and always sets the access key to the aws access key which is openedx

Attaching to tutor_local_minio_1
minio_1            |
minio_1            |  You are running an older version of MinIO released 3 months ago
minio_1            |  Update: docker pull minio/minio:RELEASE.2020-11-25T22-36-25Z
minio_1            |
minio_1            |
minio_1            | Endpoint:  http://172.18.0.8:9000  http://127.0.0.1:9000
minio_1            |
minio_1            | Browser Access:
minio_1            |    http://172.18.0.8:9000  http://127.0.0.1:9000
minio_1            |
minio_1            | Object API (Amazon S3 compatible):
minio_1            |    Go:         https://docs.min.io/docs/golang-client-quickstart-guide
minio_1            |    Java:       https://docs.min.io/docs/java-client-quickstart-guide
minio_1            |    Python:     https://docs.min.io/docs/python-client-quickstart-guide
minio_1            |    JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
minio_1            |    .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
minio_1            |
minio_1            | API: PutBucket(bucket=openedx)
minio_1            | Time: 08:53:02 UTC 11/26/2020
minio_1            | DeploymentID: db920808-9b08-4e2f-97ff-3e92b01b1f19
minio_1            | RequestID: 164B02C4FB1F9A89
minio_1            | RemoteHost: 172.18.0.17
minio_1            | Host: minio:9000
minio_1            | UserAgent: MinIO (linux; amd64) minio-go/v7.0.6 mc/2020-10-03T02:54:56Z
minio_1            | Error: Put "https://openedx.blob.core.windows.net/openedx?restype=container&timeout=1501": dial tcp: lookup openedx.blob.core.windows.net on 127.0.0.11: 53: no such host
minio_1            |        4: cmd/api-errors.go:1980:cmd.toAPIErrorCode()
minio_1            |        3: cmd/api-errors.go:2005:cmd.toAPIError()
minio_1            |        2: cmd/bucket-handlers.go:616:cmd.objectAPIHandlers.PutBucketHandler()
minio_1            |        1: net/http/server.go:2041:http.HandlerFunc.ServeHTTP()

Can you please try to set the MINIO_BUCKET setting to the same value as MINIO_ACCESS_KEY? The following should work:

tutor config save --set MINIO_BUCKET=$(tutor config printvalue MINIO_ACCESS_KEY)

okay thats worked but now having issues with setting public access fails

reating tutor_local_minio-job_run ... done
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
Added `minio` successfully.
Bucket created successfully `minio/openedx`.
Bucket created successfully `minio/openedxuploads`.
Bucket created successfully `minio/openedxvideos`.
mc: <ERROR> Unable to set policy `public` for `minio/openedx`. A header you provided implies functionality that is not implemented.
Error: Command failed with status 1: docker-compose -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.yml -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.prod.yml --project-name tutor_local -f /home/AzureUser/.local/share/tutor/env/local/docker-compose.jobs.yml run --rm minio-job sh -e -c mc config host add minio http://minio:9000 tutorstore XXXXXXXXX   api s3v4
mc mb --ignore-existing minio/openedx minio/openedxuploads minio/openedxvideos

# Make common file upload bucket public (e.g: for forum image upload)
mc policy set public minio/openedx

At this point I don’t know enough about Azure object storage to propose a solution. You need to investigate existing bucket policies to make the bucket publicly readable.

Okay got that working by going into the bucket and setting the buckets to publicly readable

Thanks for the help ! your work is amazing

Now to see how I actually get this to work in the studio with scorm packages

Thanks! Please report your findings here!

Also, depending on your Scorm package provider, you might stumble upon issues where the Scorm package expects that it is served from the same domain as the LMS/CMS – and this will fail for your MinIO configuration. There are people who have found fixes, but they are a little “advanced”.