There are many different options available to run containers on AWS, so we would like to determine the best way to run Clowder in containers on AWS.
| Table of Contents |
|---|
Scenario A: Docker Compose via EC2
...
- Identical to what we do on OpenStack
Overview
The most familiar option would be to use AWS EC2 to spin up one or more raw virtual machines, precisely how we do on OpenStack today.
...
- Prices? What do we know about our expected workload?
- We could do some CPU/memory/IO profiling on existing systems
Scenario C: Kubernetes via EKS - Kubernetes
...
where AWS manages your master nodes?
Overview
AWS has a newer service called EKS (Elastic Kubernetes Service) that can be used to deploy several EC2 instances as a Kubernetes cluster.
...
- Prices? What do we know about our expected workload?
- We could do some CPU/memory/IO profiling on existing systems
Scenario D: ECS via FarGate - Containers, minus the Infrastructure?
Overview
AWS has an even newer service called FarGate that can be used to deploy containers directly to the cloud without worrying about the machines they will run on.
...
- Prices? What do we know about our expected workload?
- We could do some CPU/memory/IO profiling on existing systems
- LMA: do we need to provide monitoring/logging/alerts ourselves? Perhaps this is something that AWS FarGate provides for its containers?
- Persistent storage: are Docker volumes safely backed by EBS, EFS, or some other persistent storage options?
- What are the costs involved with this storage?
- Does this give us any additional S3 support?
- Shared storage: do we need shared storage (e.g. NFS/EFS)? Is such a thing provided by FarGate?
- What are the costs involved with this storage?
- Networking: How do the container networks work? Does it simply use the configured networks in the docker-compose.yaml as expected?
Scenario E...? ECS via EC2 - Containers, minus the familiar parts of the Infrastructure?
Overview
Before FarGate, the ECS option offered by ECS was center around EC2 and their custom taskDefinitions.
You first need to spin up an ECS cluster (using the UI or CLI). Your cluster size determines how many tasks you can run concurrently.
You can then define your tasks using a special AWS-flavored JSON/YAML syntax, then submit it to the ECS to run your tasks in containers.
This scenario appears to be the most work with less (or less obvious) methods for cutting costs.
Pros
- ?? (no obvious advantage yet over say FarGate)
Cons
- Task definitions appear to be highly customized and likely AWS-specific
Example
https://aws.amazon.com/getting-started/tutorials/deploy-docker-containers/
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_GetStarted_EC2.html
Pricing
Unclear, but I suspect that EC2 Instance Pricing may apply
Open Questions
- Why would we even consider this over FarGate, especially if our application already leverages Docker Compose? Is it somehow cheaper in the long-term?