My network setup: MikroTik, pfSense, and VLANs
How my homelab network is currently built with MikroTik, pfSense and VLANs.
My network slowly grew from a simple home network into something I actually have to think about before moving a cable.
That sounds heavier than it is, but with multiple Proxmox nodes, self-hosted services, IoT devices, test environments and storage traffic, a flat network quickly becomes annoying. Not broken right away, just inconvenient. Especially when you need to find out why something does not work, or why a device can reach a place where it has no business being.
That is why I now use pfSense, a MikroTik CRS309 and multiple VLANs.
The switch in the middle
The MikroTik CRS309 is where most things come together. Connected to it are my Ubiquiti UniFi U7 Pro XG access point, my modem, six Proxmox nodes and a 1GbE uplink to a TP-Link TL-SG108.
The TP-Link is unmanaged and mainly handles devices that do not need a fast link. Simple IoT devices, for example, or my test node. Not everything needs to sit directly on 10GbE.
pfSense handles routing and firewall rules. The MikroTik handles the switching layer: which port gets which VLAN, what runs tagged over a trunk and what should be presented untagged to a device.
That last part is where mistakes happen quickly. One wrong VLAN on a port and you start troubleshooting in the wrong place.
Why VLANs?
I do not want Proxmox management on the same network as guests or IoT devices. I also do not want a test VM to accidentally reach things it has nothing to do with.
A separate VLAN does not solve everything by itself. Without firewall rules, it is mostly administration. But it gives a clear base: this is management, this is servers, this is guests, this is IoT, this is storage.
That makes rules easier to write and easier to understand later.
The segments
My management VLAN is for interfaces such as Proxmox, switches and pfSense. I want to keep that network as small as possible. Only devices I use for management need access.
For normal services I use a server and services VLAN. That is where applications, VMs and containers run when they need to be reachable inside my environment, but do not need to sit next to management interfaces.
For experiments I have a test VLAN. Temporary VMs, old installs and software I do not trust yet go there. If something breaks, I mainly want it to stay there.
Guests get their own network with internet access, but without access to internal services. That is not a special enterprise rule, just common sense for Wi-Fi you give to someone else.
I also keep IoT devices separate. Most of those devices have no reason to see my internal infrastructure. Usually they need internet, sometimes one specific service, and nothing more.
For Ceph I use a separate storage VLAN. That traffic should not be mixed with normal client traffic, management and applications. It is busy enough on its own.
pfSense and MikroTik together
pfSense decides what is allowed to talk between VLANs. By default I want to allow as little as possible and then open up only what is needed.
The MikroTik makes sure traffic arrives on the right port with the right tag. For Proxmox nodes and the access point, multiple VLANs run tagged over one link. For simpler devices, traffic needs to be presented untagged.
In theory that is straightforward. In practice it still needs attention, especially when an unmanaged switch is involved. Then you really need to know which VLAN is going there untagged, because after that you cannot tag anything anymore.
What it gives me
The biggest benefit is not speed. The 10GbE backbone is nice, but for me the real gain is clarity.
If something does not work, I can search more directly. Is it pfSense? VLAN tagging on the MikroTik? The port configuration? Or just the VM itself? That saves a lot of time.
It also keeps test work and daily services from blending together too much. I can try something new without feeling like the whole network is involved.
Next topic
After networking, backups naturally come up. Not as the most exciting part of the homelab, but as the part that decides how much risk I am willing to take.
So next I want to write down how I use Proxmox Backup Server, why I want backups separate from the primary cluster and where the setup still needs work.