We operate AWS for our own products and for a client platform with real users and real money flowing through it. Between March and September this year we made most of the mistakes a small team can make on AWS and wrote them up as a 29-article series. This is the condensed version: the setup we now start every project with, what each piece is for, and where the bodies are buried.
The short version: three accounts under one organisation, every resource in CDK, containers without a cluster, deploys through GitHub with no long-lived keys, observability that does not belong to one vendor, and a monthly cost review. None of it is clever. All of it is written down.
Accounts first: dev, staging, production, from day one
The first thing we create is not a server. It is an AWS Organization with three member accounts: dev, staging and production. Same CDK code, different parameters, hard IAM boundaries between them. A mistake in dev cannot touch production data because it cannot even see it.
We have inherited too many platforms where "staging" was a second database on the production server and dev was someone's laptop. Fixing that after launch is a migration; doing it before launch is an afternoon. We made the case in dev, staging, prod from day one and showed the code layout in three AWS accounts, one CDK codebase.
Deploys reach those accounts from GitHub Actions through OIDC federation: a role per account, trusted only for the right repository and branch, with no access keys stored anywhere. When an engineer leaves, nothing needs rotating. The setup is a hundred lines of CDK and we published it in GitHub OIDC deploy roles per AWS account.
Everything in CDK, and what that costs you
Every resource is TypeScript in a CDK app: accounts, networking, databases, services, DNS, alarms, IAM. The same language as the application, reviewed in the same pull requests, tested with the same runner. Nobody clicks in the console; if something was clicked, it is drift and it gets imported or deleted. We are relaxed about the tool and strict about the principle, as we explained in infrastructure as code does not mean Terraform.
CDK has sharp edges and we have cut ourselves on most of them. CloudFormation stacks have a limit of five hundred resources, and a platform with three applications, queues, functions and alarms reaches it faster than you expect; we hit 493 and had to split stacks without recreating anything, which is its own procedure (CloudFormation 500-resource limit: split a stack). A refactor that changed a logical ID once deleted and recreated two App Runner services in production; the post-mortem is in CloudFormation deleted our App Runner services, and the rule since then is that every diff touching a stateful or public-facing resource is read by two people before deploy.
Secrets never live in the template. Database passwords, API keys and webhook secrets sit in Secrets Manager or SSM and are injected at container start; the container role can read exactly those ARNs and nothing else. The trade-offs are in secrets in CDK: Secrets Manager, SSM or nothing in the template.
Containers without the cluster tax
You do not need Kubernetes. For products under a few hundred requests per second, which is most products, ECS on Fargate or App Runner deliver the same reliability with a fraction of the surface area. We explained the reasoning in you do not need Kubernetes and have not regretted it once.
App Runner is our default for web applications: push an image, get HTTPS and auto-scaling, manage it from CDK like any other resource. It has real limits, a 120-second gateway timeout and a one-minute cold instance among them, and we know exactly where they are because we ran into every one of them. The honest review is App Runner after six months in production. ECS on Fargate is the step up when a service needs a VPC-internal path, sidecars or fine-grained scaling.
Images are distroless and multi-stage, which halved image size and removed a whole category of CVE noise from the scanner (Docker images for Node in 2026). Health checks verify the database connection and the one downstream dependency that matters, not just that the port is open, because health checks that lie are worse than none.
Data: managed, backed up, and actually restored
PostgreSQL on RDS or Aurora, encrypted, in private subnets, with automated backups and a restore that has been rehearsed. That last part is the one most teams skip. We restore a production snapshot into the staging account every month, run the smoke tests against it and time the whole thing, because backups you never restored are not backups.
For small workloads with bursty traffic we like Aurora Serverless v2; we also know its half-ACU floor means it is never free, and we priced it honestly in Aurora Serverless v2 review. Schema changes are expand-and-contract, never a locked table during a deploy (zero downtime is a schema migration discipline), and the database password rotates on a schedule without anyone noticing (rotating the database password without downtime).
Queues are SQS and EventBridge, not Kafka, until there is a reason (queues without Kafka). Workers are containers or Lambda functions depending on duration, and every asynchronous job is idempotent, because it will run twice.
Observability that stays yours
Logs, traces and metrics go through OpenTelemetry to whichever backend the client already pays for, or to a self-hosted stack when they do not want to pay for one. The application does not know which; it emits standard signals and the exporter decides. When the client changed observability vendors this year, the application change was zero lines. The argument is in your logs should not know which cloud and your traces should not know which cloud, and the pricing reality in the real cost of managed observability at 100 GB a day.
Alerts are few and each one maps to a page in the runbook. The runbook is written for a team that does not have on-call, because most of our clients do not: what to check, what to click, whom to call, in that order (incident runbook for a team without on-call). Alerts nobody acts on get deleted; alerts that do not know which cloud explains the routing.
The bill, reviewed monthly
Cost is a design input, not a surprise at month end. The most expensive line on a small AWS bill is usually the one nobody remembers creating: a NAT gateway serving a private subnet that could have used a VPC endpoint (NAT gateway: the most expensive line you do not see). Then oversized databases, logs retained forever, and compute nobody committed to a savings plan.
Every month we go through the bill line by line with the client. For our own three-person company the exercise is public, in the AWS bill of a three-person startup, and the reasoning about commitments in reserved, savings plans, or nothing at small scale. Data protection policies in CloudWatch keep personal data out of the logs before it becomes a compliance problem (CloudWatch data protection policies for PII).
What an engagement looks like
An assessment week first: read-only access to the accounts, the pipelines and the last three bills, and a written report with priorities. Then the foundation: accounts, networking, CDK skeleton, OIDC deploys, the first service running the new way. Then migration, environment by environment, keeping the old path live until the new one is verified, never a big-bang weekend. Then either a clean handover with training, or we stay on for the monthly review and on-demand support.
If your infrastructure was built by hand, or your deploys are done by the one person who cannot go on holiday, tell us about it. The assessment week is fixed-scope and the report is yours to take anywhere.