Modern Infra and DevOps

Apr 5, 2020

Infra tools should be simple, stable, and well-understood. They should compose well together, so it's easy to swap out any individual part for a better one.

Kubernetes is not any these things:

Kubernetes is a bit like systemd, Container Edition. It takes an everything-and-the-kitchen-sink approach to solving what amounts to pretty simple, independent problems, and in doing so makes the problems seem overwhelmingly complex with hundreds of pages of documentation and (literally) millions of lines of code. It promises to abstract away all the complexity of managing infra, but its promises come up short. When things break—and they will break—you're left digging through millions of lines of code to identify the problem.

Kubernetes has incredible ideas at its core... but its implementation is so needlessly complex that it must be replaced.

Our field can do better.

Over the next several posts, I'll break down each major feature of Kubernetes and show how we can achieve it in 2k simple LOC or less with a series of small tools that work well together.

These are tools I've developed over the past four years of working on, managing, and deploying from 1 to 40 servers as a founder and CTO of a startup. We average 100% uptime and deploy 5-10 times per day with a team of just four engineers. Our team's experience ranges from "fresh out of college" to "sysadmin'd thousands of servers for a living." These tools work for all of us.

The tools laid out here should scale well from one to about about a hundred servers and with some minor tweaks could be adapted to scale much larger.

If you adopt this toolset (or way of thinking), you'll get:

  1. Concurrent, rolling, repeatable, and FAST deploys and rollbacks
  2. Self-healing infrastructure that automatically routes around problems
  3. Automatic TLS on all services with a simple service mesh
  4. Infrastructure stored in git and managed on the command line
  5. Binpacking to minimize costs
  6. Secret management without dedicated servers
  7. Log aggregation, broadcasting, and observability
  8. Safe and repeatable database migrations at scale

And all of this with tools that any one person can read, understand, and debug quickly and fully. If you don't like one of the tools, that's OK! None of these tools are perfect. Swap it out for something else.

That's the whole point.

Parts:

  1. Modern Infra and DevOps
  2. Concurrent, Rolling, and Repeatable Deploys
  3. Self-healing infrastructure and service mesh with TLS
  4. Infrastructure stored in git, managed on the command line
  5. Binpacking to minimize costs
  6. Better secret management
  7. Log aggregation, broadcasting, and observability
  8. Safe and repeatable database migrations at scale

This will be updated as more parts are released.