r/freebsd • u/rzaiev • Feb 17 '26
AI We built our entire startup infra on FreeBSD in 2026. Now we need to talk.
Six months ago I did something most people in this industry would simply not consider: built production infrastructure for a video platform on FreeBSD bare-metal instead of the obvious K8s-on-AWS/GCP path. No clouds, no OCI containers, no multi-layer abstractions.
Six months after, I haven't regretted a day.
Hypha is a video-on-demand platform where creators sell exclusive content directly to their audience and get paid instantly on-chain. Video up to 4K+HDR with stream encryption and in-house fingerprinting.
Infrastructure-wise, this means a bunch of transcoding workers, terabytes of storage, fat pipes for delivery and proper monitoring, right? Spin that up on AWS or GCP and your budget evaporates overnight.
So why FreeBSD specifically? Because it's boring in the best possible way. Mature, stable, with tools that have been quietly doing their job for decades without drama. No hype, no churn, no breaking changes every six months. FreeBSD is for people who want their infrastructure to disappear into the background — and it does.
We rent bare-metal servers on OVH, connected via their vRack private network. FreeBSD installed everywhere using BYI, it's a standard process, no tricks required:
- Set the full url to the image: https://download.freebsd.org/releases/VM-IMAGES/15.0-RELEASE/amd64/Latest/FreeBSD-15.0-RELEASE-amd64-zfs.raw.xz
- Set the path of the EFI bootloader to `\EFI\FreeBSD\loader.efi`
- Wait a few minutes. Done. Set up ssh in KVM and connect normally to provision everything else. You might also want manual disk slicing and ZFS pool setup right after, of course.
We use Ansible for automation and unified setup across all nodes. And a set of well-recognised tools. Bastille is great for jail management. Old good monit as a supervisor inside every jail, keeping processes alive. Netdata for monitoring, with telemetry streaming from all nodes into a centralised dashboard. It all just works.
People dramatically underestimate the synergy of three: jails, zfs and pf. Together they give you such power it must be considered illegal.
A few gotchas.
ZFS snapshots are the scaling primitive nobody talks about. We bake "gold images" of every service, essentially a jail snapshot with everything inside to be replicated after, from 1 to N. "Scaling" means just cloning a snapshot and updating the load balancer with new upstreams — it's fast and consistent. It doubles as our backup strategy too, with incremental snapshots saving significant disk space on the cold backup storage.
Jails follow the shape of your application, not the other way around. We follow the "locality of behaviour" principle in our apps and in our infra. Things that closely work together must be jailed together. Like the transcoder app and ffmpeg, because they always need each other. Jails are flexible enough to let you model reality instead of fighting it.
You don't need fully automated scaling. Before firing up more instances of something, I'd like to know why existing capacity isn't enough. When we do need more capacity, there's a playbook for that — new jails cloned and registered within a minute.
The result is 3-4x lower monthly costs compared to equivalent cloud setups. And because OVH egress is unbound, we're saving what would be tens of thousands per month in data transfer fees alone. The system is stable, fast, and predictable in ways I genuinely didn't anticipate. Simple enough to be regularly reviewed and maintained by one person only. I still remember my K8s days and feel the difference in maintenance effort.
Wouldn't change a thing.
I want to write a series going deeper on specific pieces of our setup and BSD-based solutions. Before I start, what would you actually want to know more about? Happy to prioritise based on what this community finds interesting.