The VMware Exodus

With Broadcom's acquisition of VMware causing massive licensing fee hikes, enterprise companies are scrambling for alternatives. The default reaction is often a lift-and-shift to AWS or Azure. However, this often replaces licensing bloat with compute bloat.

In this tear-down, I dissect how I re-architected a high-traffic SaaS from a heavy AWS EC2/EKS environment to a bare-metal Proxmox LXC cluster, resulting in a 70% reduction in monthly cloud costs while increasing I/O performance.

The Problem: AWS Compute Bloat

The client was running a microservices architecture on AWS. They utilized EKS (Elastic Kubernetes Service) for orchestration, RDS for their PostgreSQL databases, and Elasticache for Redis. The monthly bill was nearing $12,000, primarily driven by compute instances (EC2), NAT Gateway data transfer, and managed service premiums.

The Solution: Proxmox Virtual Environment

We pivoted to a collocated bare-metal approach using Proxmox VE. Proxmox allows for both KVM-based virtual machines and lightweight LXC (Linux Containers).

1. Replacing EKS with LXC and Docker Swarm

Instead of heavy KVM virtual machines, we deployed stateful services directly onto Proxmox LXC containers. LXC shares the host kernel, providing bare-metal performance with zero virtualization overhead. For stateless microservices, we utilized a lightweight Docker Swarm cluster running across 3 VM nodes.

2. The Storage Layer: ZFS

AWS EBS volumes are expensive and charge for IOPS. We deployed NVMe drives in a ZFS RAID10 configuration on the bare-metal servers. ZFS provides native compression (lz4), snapshots, and incredible read/write speeds that dwarf standard EBS volumes, all without the per-IOPS cost.

3. Networking and Ingress

We replaced AWS ALB with a highly available HAProxy + Keepalived setup across two Proxmox nodes. This eliminated the hourly load balancer fees and provided deeper control over SSL termination and request routing.

The Outcome

By moving away from managed public cloud services and leveraging the raw power of bare-metal with Proxmox, the infrastructure cost dropped from $12,000/mo to approximately $3,500/mo (including colocation and bandwidth). Latency decreased due to the elimination of the AWS virtualization layer.

Looking to escape cloud vendor lock-in? I specialize in Proxmox architecture and Kubernetes migrations. Contact me to discuss your infrastructure.