When the Next Customer Has Nowhere to Go

Most autoscaling stories start with a graph. CPU spikes. Requests pile up. A new replica appears. The graph calms down. Everybody goes back to sleep.

That is not the story I lived.

My story started with a quieter question, the kind that shows up when you are building something real and nobody is coming to save you:

What happens when this server is full — and the next customer still tries to sign up?

The product that forced the question

I built FaaS — Failover as a Service. Hot standby for Supabase apps. If the primary database dies, the app keeps working. When it comes back, the writes made during the outage get replayed. That is the product people see.

Underneath it, every customer is not a request bouncing through a stateless pool. Every customer is a home. A dedicated standby database. Containers. Replication. A tunnel. Routing at the edge. All of it sitting on a specific EC2 host with a finite amount of disk and a finite number of neighbors.

I pack those homes onto servers the way you pack apartments onto a floor. Eight customers to a box, give or take, and I leave headroom so disk does not become a 2 a.m. surprise. That packing is the whole game. Get it wrong and you either waste money on empty machines or you strand a paying customer because there is no room left.

The obvious answer was the wrong answer

If you have spent any time in AWS, the reflex is automatic: Auto Scaling Group. CPU goes up, add instances. CPU goes down, take them away. Done.

That reflex would have failed me.

An Auto Scaling Group is built for identical copies of the same thing. Web servers. Workers. Anything you can kill and replace without asking where someone's data lives. FaaS is the opposite. Customer twelve cannot move because CPU got spicy. Customer twelve has a database on that box. Their failover path points there. Their tunnel points there. You do not "scale out a replica." You make a new floor in the building before the next tenant shows up with a suitcase and nowhere to unpack.

CPU was never the signal. The signal was occupancy. How many customers are on this host. How full is the disk. Is there already a new host being born. If there isn't — and the next signup has nowhere to go — that is not a metrics problem. That is a broken product.

So I built the thing I actually needed

I did what I always do. I saw the failure mode and I implemented the loop that handles it.

The system watches the fleet. If any active host hits about six customers — before the hard ceiling of eight — or disk crosses sixty percent, it does not page me. It starts a job. Golden image. Launch the instance. Bootstrap Postgres, Docker, the network, the tunnel. Prove the box can actually take a customer. Register it in the fleet. Then it is live.

That job is a state machine, not a hope. Creating the image. Launching. Initializing. Completed. Or failed, with an alert, because silent failure is how you wake up to a signup that died three hours ago.

Here is the part I am proudest of. The new customer does not have to wait on a nightly cron to notice we are full. Signup itself pushes the machine forward.If every host is packed, we do not return a dead error and shrug. We start the next server — or we keep advancing the one already spinning — and we tell the product: a new box is coming, retry in about eight minutes.

Capacity stopped being an ops secret. It became part of the path the customer is already on.

Watch the loop

Same idea as the FaaS failover diagram. Click a state, or let it auto-play. The interesting part is not a CPU spike. It is a customer who still has a floor to land on.

What that feels like in the real world

Picture it without the jargon.

A host is sitting at six out of eight customers. Disk is climbing. The next person hits subscribe. In a normal small system, that is where a human gets involved. SSH. Console. Launch an instance by hand. Forget a tunnel. Forget to register the host. The signup fails. You find out later.

In this system, the threshold already fired. Or the signup fires it. A new server starts becoming a home. The UI does not pretend the world is fine. It says the honest thing: we are setting up another server, it should be ready soon.Then the retry lands on a box that exists.

That is autoscaling to me. Not a prettier CloudWatch alarm. A customer who still has a place to go.

Why this is not “just AWS”

People hear autoscaling and they think the cloud already solved this. For stateless services, it did. For sticky customers — one database, one set of containers, one host — the cloud gives you bricks. It does not give you the building.

The building is the judgment:

• Scale on density and disk, not load average.
• Bootstrap a real host, not another identical container.
• Put the new box on the map — fleet registry, tunnel, routing — or it might as well not exist.
• Let the critical path drive the work, so you are not a slave to cron.
• Tell the human waiting on signup the truth, with an ETA, instead of a hard failure.

That loop is what I keep building, whether the label is failover, healing, or capacity. Detect the bad state. Do the thing. Verify it worked. Return to normal. No meeting required.

The thing I actually know how to do

I am not a salesperson. I am the person who sees a production problem and cannot leave it as a slide. Corporate life is full of platforms blamed for bad data, policies that replace ownership, and calls where nothing ships. I got tired of that. So I built systems where the answer is implementation.

FaaS made me invent this because I was the platform. I packed the tenants. I owned the 2 a.m. version of “the box is full.” Autoscaling, in that world, is not a feature checkbox. It is the difference between a product that can grow and a product that dies the first time it succeeds.

Autoscaling is not adding copies of a server. It is making sure the next customer still has a home.

That is the story. I saw the problem. I shipped the solution. The graph never asked me to.

FaaS is the product that forced this loop into existence. failoveraas.com — hot standby for Supabase, with a fleet that knows how to make room.

— CG, August 13, 2026. Written because the interesting part was never the graph.