AWS container services, and tracking LLM benchmarks

I have been using GCP & Azure, but wanted to explore container services available in AWS. Here are a few notes based on findings:

Amazon ECS — proprietary container service, not based on Kubernetes. Supports two launch modes:
- EC2 — you manage the EC2 instances; good if you already have reserved instances.
- Fargate — serverless.
Amazon EKS (Elastic Kubernetes Service) — typical, like GKE & AKS.
App Runner — very similar to Cloud Run, but still lacking. Higher-level than ECS in terms of abstraction.
- Can’t really scale to zero? The issue was open — but on further reading it has been implemented: it scales to zero after 5 minutes idle and resumes on HTTPS traffic with a cold start, like Cloud Run.
- For production, I guess we should stick to Fargate.
AWS Lambda — great and popular, if the use case runs under 15 minutes.

Model Analysis Tracker — great website to track model performance and comparisons.