The Tax of Virtualization
Why running a full Guest OS for one small app is a resource tragedy — and why the industry was desperate for something leaner.
#The Bill Arrives
VMs solved the environment problem. They gave us isolation, portability, and reproducibility. But they came with a bill attached — one that the industry accepted for years because there was no alternative, and then grew increasingly unwilling to pay as the architecture of software itself changed.
The bill is this: every VM carries a full operating system it doesn't need.
#What's Actually Inside a VM
When you boot an Ubuntu 22.04 VM, before your application has started, before a single request has been processed, this is what's already consuming memory:
Linux kernel ~100 MB
systemd (init system) ~50 MB
system libraries (libc...) ~200 MB
background daemons:
cron ~5 MB
sshd ~5 MB
rsyslog ~5 MB
networkd, udevd... ~15 MB
─────────────────────────────────────
OS overhead total ~380 MB+And that's a minimal Ubuntu install. A standard cloud image with common utilities adds another 200–400 MB. By the time your application process starts, you've already consumed roughly 500 MB to 2 GB of RAM for infrastructure that serves the hypervisor, not your code.
Now look at what your application actually needs. A small Python web service handling user requests might use 150–300 MB at steady state. A Node.js API, perhaps 100–200 MB. Your application is the small thing. The operating system underneath it is the large thing.
#The Density Problem
Density is how many workloads you can fit on a given piece of hardware. It directly determines your infrastructure cost.
On a 32 GB server running VMs with 2 GB of OS overhead and 512 MB of app overhead per VM, you fit roughly 10 workloads. The math:
32 GB total RAM
- 2 GB hypervisor overhead
= 30 GB available
Per VM: 2 GB (OS) + 0.5 GB (app) = 2.5 GB
30 GB ÷ 2.5 GB = ~12 VMs maximumOf those 32 GB of RAM, roughly 24 GB — 75% — is consumed by operating systems. Not your code. Not your users' requests. Just twelve copies of Ubuntu running in parallel, each believing it's the only OS on the machine.
This was the deal you made with VMs. It was acceptable when applications were large monoliths that genuinely needed 4–8 GB of RAM themselves. The OS overhead was proportionally small. But software was changing.
#The Microservices Shift
Through the 2010s, the industry moved away from monoliths toward microservices — smaller, focused services that each do one thing. An e-commerce platform that was once a single application became 30 separate services: an auth service, a product catalogue service, an inventory service, a payments service, a notification service, a search service...
Each of these services was small. A focused microservice might need 128 MB of RAM to run. Put it in a VM and you've wrapped 128 MB of application in 2 GB of operating system. The ratio inverted — the overhead was now ten times the payload.
And microservices needed to scale independently. If the payments service was getting hammered, you'd want to spin up more instances of just that service — not more instances of the entire application. But spinning up a new VM took 30–60 seconds. By the time the new instance was ready, the traffic spike had passed.
Traffic spike detected at 14:32:07
VM provisioning begins at 14:32:07
VM boots at 14:33:15 (68 seconds later)
App process starts at 14:33:28
Ready to serve at 14:33:30
Spike subsides at 14:33:00The new instance was ready 30 seconds after the spike ended. You'd paid for a VM that served zero requests.
#The Boot Time Wall
The 30–60 second VM boot time wasn't an implementation detail that could be optimized away. It was structural.
Booting a Linux kernel means:
1. BIOS/UEFI initialization
2. Bootloader (GRUB) runs
3. Kernel decompresses and loads into memory
4. Kernel initializes hardware drivers
5. initramfs mounts and pivots to real root filesystem
6. systemd starts (PID 1)
7. systemd starts all configured services in dependency order
8. Networking comes up
9. Your application daemon startsEvery step is necessary. Every step takes time. A carefully tuned minimal Linux system can boot in 5–10 seconds. A standard cloud image takes 30–60 seconds. There is no shortcut — this is what it means to start an operating system.
For long-running production servers that you boot once and run for months, this was fine. But for ephemeral workloads — jobs that run for seconds, functions triggered by a single HTTP request, scaling events that need to respond in milliseconds — 45-second cold starts were a hard architectural wall.
#The Storage Cost
It's not just RAM. VM images are large.
A minimal Ubuntu cloud image is around 600 MB compressed, 2.5 GB uncompressed. Add your application, its dependencies, and a runtime, and a typical VM image is 4–10 GB. Storing, transferring, and versioning these images had real costs.
Deploying a new version of your application meant building a new 8 GB VM image, uploading it to your image registry, distributing it to every host, and replacing the running VMs. The iteration cycle was measured in tens of minutes.
Compare this to what you actually changed: perhaps a 2 MB application binary. You were shipping 8 GB to deliver 2 MB of change.
#The Hidden CPU Cost
Even when idle, VMs impose CPU overhead. The hypervisor must intercept and translate hardware instructions from the Guest OS — operations like memory management, I/O, and interrupt handling that the Guest OS thinks it's doing to real hardware but are actually going to the hypervisor for mediation.
Type 2 hypervisors (VirtualBox, VMware Workstation) add significant CPU overhead because every hardware call goes through two layers: the hypervisor and the host OS. Type 1 hypervisors (ESXi, KVM) are better, but the translation layer never disappears entirely.
For CPU-intensive workloads this overhead could be 5–15% of raw performance — burned on virtualization infrastructure rather than your application's actual work.
#The Question That Had to Be Asked
By the early 2010s, a generation of infrastructure engineers was sitting with the same uncomfortable calculation:
We need isolation. We need portability. But we're paying 2 GB of RAM and 45 seconds of boot time for every 128 MB microservice we run. There has to be a better way.
The better way existed. It had been sitting in the Linux kernel, largely unused outside of a few specialized contexts, for years. It didn't require a full Guest OS. It didn't require a hypervisor. It offered isolation at the process level using features the kernel already provided.
The features were called namespaces and cgroups. The technique was called OS-level virtualization. And a company called dotCloud was about to wrap them in a tool that anyone could use.
But before we get to Docker, we need to understand what dotCloud actually built on — the Linux kernel primitives that make containers possible. That's the next phase of this course.
Key Takeaway: Virtual machines carry a full Guest OS that consumes 2 GB of RAM and 30–60 seconds of boot time per workload, regardless of how small the application is. As the industry moved to microservices — small, independent services needing independent scaling — the VM overhead became proportionally absurd. A 128 MB service wrapped in 2 GB of operating system, taking 45 seconds to start, cannot be the unit of horizontal scaling. The industry needed process-level isolation without the OS tax. That's what containers deliver.