Stop being afraid of your own production
We are not building Kubernetes. We are building the ability to stop being afraid.
Six products, twenty-three tenants, one small team.
"You don't need Kubernetes" and other expensive convictions
It is still fashionable, in engineering chats, to say that "not every company needs Kubernetes."
Maybe that is true. Maybe there are still businesses for which three virtual machines, or a single physical server, is the ceiling of ambition.
Maybe there are companies that do not need repeatability. That do not need disaster recovery. That do not need a standard API for asking the system "what is your current state?" That do not need a way to measure drift, or to prove its absence. That do not need to preview a change before applying it.
Such companies may exist. But let us be honest about what we are arguing over.
We are not really arguing about Kubernetes. Kubernetes is one available tool with a known API. It is not a panacea. We are arguing about what engineers living inside the system actually want. Less stress. Less fear. Repeatability. The ability to walk up to the platform and ask "what is your current state?" and "what changes if I deploy this?" and get an answer, not a hunch.
Complexity does not disappear. You cannot throw it out. You can only hide it.
And there are exactly two places to hide it. The first is the head of one engineer. The second is fragments of knowledge scattered across many heads, several chat channels, and a wiki nobody opens.
Six products, six companies, one snowflake fleet
Our holding owns six products. They used to be six companies, with six engineering teams. We inherited their servers the way you inherit an old house: the furniture is still inside, and nobody alive remembers where the keys are kept.
Their services ran on plain hosts. Sometimes without any virtualization at all.
In 2025, adding healthchecks to every running Docker container felt like a milestone. The work is not finished.
With a fleet that large, where many machines were already in production when we got the password, repeatable configuration is hard. Somewhere in the fleet there is a server nobody tries to upgrade. It runs Ubuntu 18. It still serves a critical part of the infrastructure. Everyone knows about it. Nobody touches it.
Teams changed. People came and people left. With the drift already there, putting things in order felt impossible. Most knowledge about how the system actually works lived in the heads of a few engineers. Some of those engineers had already left the company.
"You just need to write proper Ansible playbooks. The infrastructure is fine."
I still think about that sentence.
In the ten to fifteen years some services have existed, no team has managed to write those proper playbooks. Not from lack of trying. There are playbooks. There were several attempts to make every infrastructure change go through them. Every attempt failed.
Yes, some changes can be applied by running an Ansible playbook. You will never know exactly which ones.
And when you do run one, it reports drift. In a world of snowflake servers, drift is not information. It is fear.
The role moves to the team
After yet another CTO was dismissed for failing to deliver a high pace of change, the role of the CTO simply moved to the team.
We became SRE. We became DevOps. We became the people who know what to do to make production earn a couple of dollars.
We were the team that could not complete a single milestone, because we were either blocked or putting out a fire.
It is not a tools problem
We sat down and asked what we were missing. The fear of change had to go. The pace at which change must reach production today does not give you the luxury of being afraid of your own platform.
Problems of this kind are never solved by tools. You cannot shout buzzwords and expect them to work as spells.
Problems of this kind are administrative. They are about process, ownership, and where information is allowed to live.
When you understand the constraints of the system you are inside, you can build something that works inside those constraints.
Three principles
The first. Everything is data: every piece of knowledge, every config value, every parameter, every fact someone once stated in a chat. The data lives in one place only: in code. YAML, JSON, Python. The format does not matter. The commit is the source of truth.
The second. Once everything is data, you can build views on top of it. The system becomes legible. You can ask it questions. You can show a new engineer the whole map on one page.
The third. Every engineering team must speak the same language.
"12-factor is a clever trick to make us do more work than usual."
I am quoting. I am not exaggerating.
Why not just buy a platform
The reflex answer in 2026 is "use Backstage / Crossplane / Argo / Helm + Helmfile / GitOps." We have tried most of them. We still use some of them.
Each one solves a piece. None solves the problem a small team has when it stands behind six products, twenty-three tenants, two hypervisor families, and institutional memory that lives in chat threads from five years ago.
Backstage gives you a developer portal. It does not tell you whether your tenant registry and your cluster manifest agree on what the same database is named.
Crossplane gives you a control plane for cloud resources. It does not tell you that a service you deployed two years ago is still pinned to a base image you have since deprecated everywhere else.
Argo gives you GitOps. It does not tell you that the YAML it is faithfully applying was generated by a Helm chart whose values.yaml is the sixteenth layer of a merge nobody can read end to end.
They are all good tools. They are tools.
What we needed was a model. A list of every tenant in one file. A list of every service in another. A list of every environment in a third. Names a new engineer can grep on Monday morning and find on Tuesday afternoon. A registry the platform itself reads, the CLI itself reads, the AI agents read.
Don't buy a platform. Model your platform.
Two months in Singapore
In late March 2026, we still served Asia from Europe.
The servers that handled real-time requests from partners for users across Asia lived in a European datacenter. Every request from an Asian mobile carrier flew ten thousand kilometres, was decided, and flew back. The fixed cost of that round trip was 160 to 180 milliseconds. Partners gave us 200.
Most of the time we were too late.
We had planned the move to Singapore for years. The platform finally made it cheap. A few inv commands. Three Terraform stacks. A handful of services described in code. The pods came up in Singapore. Nginx answered. Every request returned 204 No Content.
Then we began to learn what the bare metal had been hiding for ten years.
A configuration table contained a single row for "Asia" that pointed to a server in Europe, because Asia had always been Europe and the table had no language for describing it otherwise.
A node-location parameter had not been changed in eleven years. It still resolved to a constant called Europe.
A model name in one configuration table did not match the model name the code was looking for. The mismatch had been silently producing a default value for years. On a warm system the default did not matter. On a cold cluster, every request returned 204.
The story I will remember longest is about the optimizer.
In Singapore on day one, a misconfiguration meant that no request produced a successful outcome. The optimizer dutifully counted. The optimizer dutifully denied. Within the first hour, every key in Singapore was permanently poisoned. Even after we fixed the misconfiguration, the poisoned keys remained denied by design.
We were debugging a system that was lying to us on purpose, because in Europe lying on purpose had always been the correct behaviour.
"Thank god this didn't go out on real traffic. We would have been wrecked."
The Singapore deploy had been broken from the day the pods came up. Nobody saw it because no live traffic was running through it yet. Every bug we dug out would have shipped as a production incident if we had switched traffic on day one.
The Singapore cluster now exists as a few hundred lines of Python in a repository. Every parameter is a dict entry. Every dependency between stacks is a list element. Every secret is a reference to a known store. Everything learned over those two months sits in the same code that generated the first deploy.
The next region is the United States. We will not run that investigation again. We will run a generator.
What we count today
Six product lines. Twenty-three tenants in the registry. Seventy-six Kubernetes environments. About a hundred dynamically generated developer namespaces. A hundred and thirty services in the catalog. Forty-nine Terraform stacks. Forty-three commands on the CLI. Three datacenters. Two hypervisor layers under one model.
Two numbers matter more than the rest.
The CLI layer has been touched, in the entire history of the repository, by five engineers. The configuration registry has been touched by thirty-two.
The behaviour of the platform is owned by a small group. The configuration is contributed by everyone who has a service to ship.
Infrastructure always changes. The tax compounds.
Infrastructure changes. It changes at every scale. A new region. A vendor migration. A compliance pocket. A product spin-off. A failed experiment that must be deleted cleanly. A Kubernetes minor that drops an API. A cheaper hypervisor. A new product feature that triples your storage footprint overnight.
Each such change is a tax. The tax accumulates.
Call it by its other name: technical debt. It hangs over the team like a sword of Damocles, and the team knows it.
Multi-tenancy from day zero is not over-engineering. It is the only hedge against this tax.
By "multi-tenancy" we do not mean the SaaS definition. We mean the engineering one: every concept in the platform is a parameter, not a constant. Anywhere you write "prod" you have taken on a debt. Anywhere you write tenants["prod"] you have paid it down.
Infrastructure Heaven is a publication by SimploLab Tech, a platform-engineering consultancy. More at simplolab.tech.
SimploLab Tech builds infrastructure platforms for teams that need production to become legible, repeatable, and less dependent on tribal knowledge.