Energy Connect Platform
TypeScript/Node.js connectivity layer for EV chargers and vehicles — my largest system
Problem. An energy platform sits in front of OEM APIs that time out, rate-limit and fail in ways the product above them must never notice. One flaky vendor cannot be allowed to degrade the whole platform — and a retried request must never start a car charging twice.
Solution. A connectivity layer built to isolate failure. Commands are written to a durable PostgreSQL queue with an idempotency key, workers claim rows using FOR UPDATE SKIP LOCKED so two workers never pick up the same command, and every provider call is wrapped in retry plus circuit-breaker logic. Structured logs, Prometheus metrics and an SLO snapshot make failure visible; a React operations dashboard shows device state, queue depth and SLO health. Five backend tests run in GitHub Actions against a live PostgreSQL service container.
[React ops dashboard] [client / integrator]
\ /
v v
[Node.js API — TypeScript]
| enqueue command + idempotency-key
v
[PostgreSQL — devices + command_queue]
^ FOR UPDATE SKIP LOCKED
|
[Node.js worker pool]
| retry (exp. backoff) + circuit breaker
v
[Simulated OEM A] [Simulated OEM B]
observability: structured JSON logs · Prometheus metrics · SLO snapshot
infra: AWS CDK > CloudFormation > ECS/Fargate · RDS · CloudWatch alarms
// engineering decisions
- Why a PostgreSQL queue instead of SQS or Kafka? The command state and the queue state have to stay consistent with each other. Keeping both in one transactional database removes a whole class of "the message was sent but the row was never written" bugs, and at this scale a broker would be infrastructure I have to justify rather than use.
- Why FOR UPDATE SKIP LOCKED? It is the standard way to let several workers drain one table concurrently: each worker locks the rows it claims and skips whatever is already locked, so throughput scales with worker count instead of serialising behind a single lock.
- Why a circuit breaker on top of retries? Retrying against a provider that is already down turns one outage into a self-inflicted load test. The breaker trips after a failure threshold and fails fast until the provider recovers.
- Why idempotency keys at the API edge? Clients retry. Without a uniqueness constraint on the key, a network hiccup silently becomes two START_CHARGING commands — the exact bug that is invisible in tests and expensive in production.
- Why AWS CDK rather than raw CloudFormation? The stack is typed TypeScript, so the infrastructure is reviewed by the same compiler as the application. cdk synth still emits plain CloudFormation.
// lessons learned
- Reliability code is only real if it is tested against failure. The retry and circuit-breaker units are tested with deliberately failing providers — proving the breaker actually opens is the whole point.
- Splitting API and worker changed the design. Once the two processes cannot share memory, every piece of coordination has to become explicit state in the database — which is exactly what made the queue design necessary.
- Future work: real OEM adapters behind the existing provider interface, Grafana dashboards checked into the repo, and blue/green deploys on ECS.