Skip to content

Infrastructure

Every AWS resource CivicDog runs on is provisioned by Terraform, in a dedicated cd-infra repo, kept completely separate from application code.

cd-infra/terraform/
├── bootstrap/ # State bucket (S3 + KMS) + GitHub OIDC provider — one-time, local state
├── networking/ # VPC, subnets, security groups
├── rds/ # Single-AZ Postgres
├── airflow/ # EC2 host running cd-etl via Docker + Watchtower
└── cd-api/ # Lambda + API Gateway + RDS Proxy + GitHub OIDC deploy role

Each directory is an independent root module with its own state — there’s no single terraform apply for everything. Later modules read earlier ones’ outputs via terraform_remote_state. State lives in S3 with Terraform’s native use_lockfile locking (Terraform ≥ 1.10) — no DynamoDB lock table needed.

terraform apply is manual-only, by design — there’s no auto-apply workflow. Every PR runs terraform fmt -check, terraform validate, and a Trivy IaC security scan; a human runs apply.

Category Services
Compute Lambda (cd-api), EC2 (self-hosted Airflow)
API API Gateway REST API, with API keys + usage plan
Database RDS Postgres (single-AZ, KMS-encrypted), RDS Proxy
Networking VPC (2 AZs), public/private subnets, single NAT Gateway, security groups
IAM Scoped roles for EC2, Lambda, RDS Proxy, and a GitHub OIDC deploy role
Secrets Secrets Manager (Congress API key, DB credentials), 4 customer-managed KMS keys
Storage S3 (Terraform state — versioned, encrypted, public access blocked)

There’s deliberately no ECS/EKS, no CloudFront, no ALB, and no VPN — see the trade-offs below for why.

  • Zero public IPs on compute. RDS, the Airflow EC2 instance, and Lambda all live in private subnets.
  • SSM Session Manager, not a bastion or VPN. The only way to shell into the Airflow host or port-forward to RDS is through SSM — no open SSH port, no VPN infrastructure to maintain.
  • Four separate customer-managed KMS keys (state bucket, RDS, Airflow secrets/EBS, cd-api secrets/Lambda env vars), each with rotation enabled — a resource compromise in one domain doesn’t grant decrypt access to another.
  • No static DB credentials at runtime. The RDS master password is Secrets-Manager-managed by RDS itself (never touches Terraform state as plaintext). App-level roles (cd_etl_app, cd_api_app) are least-privilege, generated by Terraform, and bootstrapped into Postgres out-of-band.
  • IAM built empirically. The Terraform deployer’s own IAM policy was built up one real AccessDenied error at a time in the console, rather than guessed from docs upfront — a deliberate practice, not an oversight.

Real numbers, documented in the repo rather than assumed:

Decision Alternative considered Cost
Self-hosted Airflow on EC2 + Watchtower AWS MWAA ~$6–15/mo vs. ~$358/mo
SSM Session Manager AWS Client VPN Free vs. ~$72/mo per subnet association
Single shared NAT Gateway One NAT Gateway per AZ ~$32/mo vs. ~$64/mo

Documented honestly rather than hidden:

  • No CloudWatch alarms or dashboards yet. Lambda and API Gateway get default CloudWatch Logs, but there’s no alerting on top of them.
  • Single environment. Everything runs in one AWS account with no dev/staging split — API Gateway’s stage is literally named prod. This is a deliberate MVP trade-off (“flip it on in whichever PR first needs it, not preemptively”), not an oversight.
  • API key auth is a stopgap. cd-api is currently protected by a static API Gateway key rather than something more granular like IAM auth or OAuth scopes.