Proxmox VE 8 Routed Network Configuration

In Proxmox VE 8 there are some changes to the way we configure a routed setup. The Proxmox VE 7 routed setup advises that you configure additional IPv4 directly on the interface. Whilst this is still the best configuration for PVE 7. Proxmox advises that In a PVE 8 routed network, IPs are configured within the /etc/network/interfaces.d/vm-routes file. This allows you to add further IPs to the server without disconnecting existing services. In PVE 7 when adding new IPs you had to either reboot the Dedicated Server or run ifdown and ifup. Both require a maintenance window.

This guide assumes you have already installed Proxmox VE 8 from ISO or by installing the required packages from the command line. Before proceeding check what your adapter is called. In our example, our network adapter is called enp5s0. Use ip a and check.

2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 9c:6b:00:26:48:51 brd ff:ff:ff:ff:ff:ff

Allow IP Forwarding

In all F2H OS distributions, we disable IP forwarding as standard. In a routed configuration our bridge will not be connected physically to the ethernet adapter. So we need to enable forwarding for IPv4 and IPv6. This is done in the /etc/sysctl.conf file. Or by issuing the following commands. The latter is only in force until the server is rebooted. Edit the file to make the changes permanent.

sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv6.conf.all.forwarding=1

Proxmox VE 8 Routed Network Setup

• Source Files
  • Interfaces file
  • vm-routes file

You can download the interfaces file from our CDN to your Proxmox 8 host using the below command.

wget -O /etc/network/interfaces https://f2h.ha-cdn.com/proxmox/pve8/pve8_interfaces.txt

Open up the interfaces files in a new tab. Let’s go through a PVE 8 routed network configuration.

Proxmox VE 8 /etc/network/interfaces file

source /etc/network/interfaces.d/*

auto lo
iface lo inet loopback

iface lo inet6 loopback

auto enp5s0
iface enp5s0 inet static
        address SERVER_IPv4/26
        gateway IP_GATEWAY
        up route add -net SERVER-IPv4 netmask 255.255.255.255 gw IP_GATEWAY dev enp5s0

# IPv6 Settings

iface enp5s0 inet6 static
        address IPV6IP::1
        netmask 128
        gateway fe80::1
        up sysctl -p

• On the physical interface enter your server’s IPv4 address followed by a /26 subnet. In the gateway enter the gateway IP.

• Use up route add to add a default route when the interface is brought up.
  • Only enter the main IPv4 of the server here.
  • Do not append a subnet and add the gateway.
  • Don’t forget to modify your adapter’s name if it is different to snp5s0.

• For IPv6, we do the same. Enter your IPv6 address and append 1 followed by a /64 subnet.
  • The gateway is always on fe80::1
    

Configure Bridge And Private Network

So now we move on to the virtual configuration. Our bridge and private network configuration.

auto vmbr0
iface vmbr0 inet static
        address SERVER_IPv4/32
        bridge-ports none
        bridge-stp off
        bridge-fd 0

iface vmbr0 inet6 static
        address IPv6::2
        up ip -6 route add 2a01:4f8::/64 dev vmbr0

auto vmbr1
iface vmbr1 inet static
        address 10.212.25.0
        netmask 255.255.255.0
        bridge-ports none
        bridge-stp off
        bridge-fd 0
        bridge-vlan-aware yes
        bridge-vids 2-4094
     post-up echo 1 > /proc/sys/net/ipv4/ip_forward
     post-up   iptables -t nat -A POSTROUTING -s '10.212.25.0/24' -o vmbr0 -j MASQUERADE
     post-down iptables -t nat -D POSTROUTING -s '10.212.25.0/24' -o vmbr0 -j MASQUERADE

• In the address space, we again append the server’s main IPv4 address but this time we use a /32 subnet.
• In the IPv6 address, we increment the IPv6 address by one.
  • The physical IPv6 interface is configured to run on ::1/64. So the virtual interface will work on ::2/128.

• Add a static route to route the entire IPv6 subnet via vmbr0. Replace with the first two octets of your IPv6 subnet.

Notice the subnets on the IPv6 addresses. Just like the physical interface, we change the subnet to /128.

• Optionally configure a Proxmox private network to work on vmbr1. And you can change the IPv4 used on the private network if required. In the interface add a route to send all connections from a private IP via vmbr0 so servers with a private IP have access to the outside world.

This is the entire networking file for the Proxmox VE 8 routed configuration. It’s a lot less cluttered as opposed to PVE 7 because we have not yet configured any additional IPv4. So what about IP subnets? Let’s do them next.

Configure Routed IPs on PVE 8

Remember at the top of this file we had source /etc/network/interfaces.d/*. That’s where we will add our additional IPs for use on virtual servers. Create the file /etc/network/interfaces.d/vm-routes or download our copy with the command below.

# Manually create file

nano  /etc/network/interfaces.d/vm-routes

# Download our vm-routes file

wget -O /etc/network/interfaces.d/vm-routes https://f2h.ha-cdn.com/proxmox/pve8/vm-routes.txt

In this file, we will configure our subnets. It is possible to use any type of subnet. From a single /32 IP to a /26 subnet.

iface vmbr0 inet static
        up ip route add 144.xx.1.2/29 dev vmbr0
        up ip route add 148.xx.53.6/29 dev vmbr0
        up ip route add 148.xx.21.1/29 dev vmbr0
        up ip route add 148.xx.93.176/29 dev vmbr0
        up ip route add 148.xx.99.64/29 dev vmbr0
        up ip route add 148.xx.241.104/29 dev vmbr0
        up ip route add 136.xx.253.160/29 dev vmbr0
        up ip route add 136.xx.232.32/29 dev vmbr0


iface vmbr0 inet6 static
        up ip -6 route add 2a01:xxx:xxx:2969::0/64 dev vmbr0

At the bottom ensure you add a default route for your IPv6 subnet. We have already used IPs ::1 and ::2 on our physical and virtual adapters so the IP needs a lower value. ::0 in this case.

To activate the configuration reboot the Dedicated Server and the process is complete.

Adding Further Subnets

From time to time, you may need to add further subnets and even remove some. To achieve this just add them to the vm-routes file and reload the network with ifreload –all.