ECS Support for tutor openedx

dariothornhill · June 26, 2020, 5:40pm

Has there been any attempt to add ECS support to the tutor project. If I were to add this would I have interest and support from others on this item?

regis · June 28, 2020, 1:59pm

Hi @dariothornhill! What is ECS?

dariothornhill · June 29, 2020, 4:34pm

Hi@regis
Elastic Container Service, it is a managed container orchestration service offered by AWS. It abstracts the underlying compute resources from the containerized services similar to kubernetes. More information can be found here https://aws.amazon.com/ecs/.

In my particular use case my team has deployed OpenEdx using tutor on a single EC2 instance running docker and docker-compose. We want to move to ECS for scalability and to have the option to deploy a mix of tutor docker containers and AWS managed services.

We have tried the Kubernetes approach before with EKS (Elastic Kubernetes Service) but it is more complex than ECS and is more expensive.

We would really love to be able to do this without having to give up using tutor so we decided to ask if others would be interesting in supporting especially with best practices for including this deploy option.

regis · June 29, 2020, 5:44pm

@dariothornhill,
I’m pretty sure it’s possible to run Open edX on ECS with the Docker images created by Tutor. As far as I understand, ECS is yet another container orchestrator, like docker-compose, Kubernetes or any one of the many existing alternatives. Because it’s yet another alternative implementation that is not compliant with other cloud vendors, I’m personally not interested in developing and maintaining that gateway myself – or at least, not for free. (If you decide to hire me to implement this, I can either develop it from A to Z or act as support of your technical team.)

Unless I’m mistaken, the ECS CLI supports the definition of services via docker-compose files, just like tutor local. Have you tried simply running ECS with the docker-compose.yml file generated by tutor config save? This might just work with minimal changes.

dariothornhill · June 30, 2020, 11:19am

Hi @regis,
I totally understand, I’ll give your suggestion a shot.

cacciaresi · July 1, 2020, 10:02pm

Hi @dariothornhill, to answer your initial questions, I’m interested and also I’m able to support but the issue is that I don’t have previous ECS knowledge .

We are currently also using EC2 instance with tutor local to deploy openedx.

I’ve been reviewing ECS, I’ve try to deploy the docker-compose.yml ‘as-is’ and found some “issues”, like that there is a limit for 10 containers to run for a task (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-quotas.html), maybe running the entire docker-compose as a task is not the correct approach.

Also, by default 512mb of RAM is assigned to all containers, that for mysql or es won’t work.

So, with minimal changes I’ve managed to start 2 tasks running all containers, obviously the application was not running correctly so that is when I notice that was difficult to execute a command on a running container (I wanted to understand why the lms container was not running).

Based on what I’ve review, there is no “simple” way to run a command on a running container using ECS, this is important for a potential implementation in Tutor, maybe I’m wrong with my comment, maybe @regis can add his point of view here, or maybe you know how to do it but if we don’t have a proper way to interact easily with running containers there may be issues.

I notice that there is an issue regarding this topic: https://github.com/aws/containers-roadmap/issues/187 which references in some cases to: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ec2-run-command.html but I didnt’t check the details there.

To sum up, I’m interested and I’d be able to help maybe testing?

dariothornhill · July 3, 2020, 2:22pm

That’s awesome to hear! I’m working with an AWS consulting partner on this initiative i’m sure the team would welcome to collboration.

TimothyJAndrus · July 31, 2020, 2:12am

@dariothornhill I am interested in collaboration with any of you. Let me know! We are an online university with 135,000+ students and we are migrating to tutor.

cacciaresi · July 31, 2020, 2:49pm

@TimothyJAndrus @dariothornhill do you have something to start from? I’ll be also interested to help you.

joefoster-wgu · November 28, 2022, 4:14pm

Has there been any progress on this? We’re hoping to deploy tutor on Fargate containers inside ECS and could use any help/advice that anyone might have.