<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>Larvitz Blog - Networking</title><link href="https://hofstede.it/" rel="alternate"/><link href="https://hofstede.it/feeds/networking.atom.xml" rel="self"/><id>https://hofstede.it/</id><updated>2026-08-29T00:00:00+02:00</updated><subtitle>FreeBSD, Linux, all things cleanly engineered</subtitle><entry><title>I Wanted an IPv4 Toast and Accidentally Built an IP Service</title><link href="https://hofstede.it/i-wanted-an-ipv4-toast-and-accidentally-built-an-ip-service/" rel="alternate"/><published>2026-08-29T00:00:00+02:00</published><updated>2026-08-29T00:00:00+02:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-08-29:/i-wanted-an-ipv4-toast-and-accidentally-built-an-ip-service/</id><summary type="html">&lt;p&gt;At 10:15 this morning I added a gently condescending toast for IPv4-only visitors to hofstede.it. It called ipify, which felt increasingly inappropriate for somebody operating an &lt;span class="caps"&gt;ASN&lt;/span&gt;. Three hours later I had written a Rust &lt;span class="caps"&gt;IP&lt;/span&gt;-information service, put it behind Caddy, given it three &lt;span class="caps"&gt;DNS&lt;/span&gt; names, and deployed it in its own dual-stack FreeBSD &lt;span class="caps"&gt;VNET&lt;/span&gt; jail. This is how a two-line dependency became&amp;nbsp;infrastructure.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="ip.hofstede.it showing an IPv6 address, provider, location, and AS201379" src="https://hofstede.it/images/2026-08-29-ipv4-toast-ip-service.webp" title="The finished service reporting an IPv6 connection from inside AS201379"&gt;&lt;/p&gt;
&lt;p&gt;At 10:15 this morning I added a small feature to &lt;a href="https://hofstede.it"&gt;hofstede.it&lt;/a&gt;, my static personal website. Visitors whose providers still cannot deliver IPv6 would get this toast in the lower-right&amp;nbsp;corner:&lt;/p&gt;
&lt;p&gt;&lt;img alt="The Legacy IP detected toast on hofstede.it, explaining that the visitor arrived over IPv4" src="https://hofstede.it/images/2026-08-29-ipv4-toast-hofstede-it.webp" title="The small toast on hofstede.it that started the entire escalation"&gt;&lt;/p&gt;
&lt;p&gt;The text is deliberately&amp;nbsp;direct:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Legacy &lt;span class="caps"&gt;IP&lt;/span&gt;&amp;nbsp;detected&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;You reached this website on a legacy version of the Internet Protocol (IPv4, 1981). It seems your provider does not support the current generation of Internet technology. Do ask them about it — IPv6 has only been a standard since&amp;nbsp;1998.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Gently condescending, technically defensible, and dismissible. The dismissal lives&amp;nbsp;in &lt;code&gt;localStorage&lt;/code&gt;, so nobody gets lectured twice. I considered this an admirably restrained piece of web&amp;nbsp;development.&lt;/p&gt;
&lt;p&gt;There was one problem. The website is static, so its JavaScript cannot ask the server which address connected to it. It had to ask somebody else. The first version used&amp;nbsp;ipify.&lt;/p&gt;
&lt;p&gt;For most websites that would have been the end of the matter. For somebody who &lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;operates an autonomous system&lt;/a&gt;, announces his own address space, runs his own authoritative &lt;span class="caps"&gt;DNS&lt;/span&gt;, and has written an entire article insisting that &lt;a href="https://hofstede.it/ipv6-foundations-the-internet-protocol-you-should-already-be-using/"&gt;IPv6 is the current Internet Protocol&lt;/a&gt;, outsourcing “which &lt;span class="caps"&gt;IP&lt;/span&gt; address made this request?” began to feel inappropriate almost&amp;nbsp;immediately.&lt;/p&gt;
&lt;p&gt;At 13:27 the website no longer depended on ipify. By then I had also written a Rust web service, added MaxMind lookups, built a server-rendered interface, configured three &lt;span class="caps"&gt;DNS&lt;/span&gt; names, hardened a reverse-proxy trust boundary, written an rc.d service, and deployed the whole thing in a dedicated FreeBSD &lt;span class="caps"&gt;VNET&lt;/span&gt;&amp;nbsp;jail.&lt;/p&gt;
&lt;p&gt;This article is about the intervening three&amp;nbsp;hours.&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/h2&gt;
&lt;div class="toc"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-harmless-original-idea"&gt;The Harmless Original&amp;nbsp;Idea&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#surely-i-can-replace-two-urls"&gt;Surely I Can Replace Two&amp;nbsp;URLs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#one-binary-because-restraint-had-to-appear-somewhere"&gt;One Binary, Because Restraint Had to Appear&amp;nbsp;Somewhere&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-reverse-proxy-knows-who-you-are"&gt;The Reverse Proxy Knows Who You&amp;nbsp;Are&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#naturally-it-needed-its-own-jail"&gt;Naturally, It Needed Its Own&amp;nbsp;Jail&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#three-names-for-two-protocols"&gt;Three Names for Two&amp;nbsp;Protocols&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-escalation-in-git-timestamps"&gt;The Escalation, in Git&amp;nbsp;Timestamps&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#was-this-overengineering"&gt;Was This&amp;nbsp;Overengineering?&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;h2 id="the-harmless-original-idea"&gt;The Harmless Original&amp;nbsp;Idea&lt;/h2&gt;
&lt;p&gt;The first implementation called two public&amp;nbsp;endpoints:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;IP_ECHO_URL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;https://api64.ipify.org&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;IPV6_PROBE_URL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;https://api6.ipify.org&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The first name is dual-stack, so it reports whichever protocol the browser happened to use. The second is IPv6-only. That distinction matters because one IPv4 connection does &lt;strong&gt;not&lt;/strong&gt; prove that the visitor lacks&amp;nbsp;IPv6.&lt;/p&gt;
&lt;p&gt;On a dual-stack network, the browser uses Happy Eyeballs: it attempts the available paths with carefully staggered timing and takes the one that wins. IPv4 can occasionally win that race even when IPv6 works perfectly. Toasting that user for having an obsolete provider would be satisfying but&amp;nbsp;wrong.&lt;/p&gt;
&lt;p&gt;The test is therefore deliberately&amp;nbsp;conservative:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kd"&gt;function&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;checkLegacyIp&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="kd"&gt;let&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;address&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="nx"&gt;address&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;fetchText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;IP_ECHO_URL&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;catch&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;address&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;includes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;:&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;fetchText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;IPV6_PROBE_URL&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;catch&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;        &lt;/span&gt;&lt;span class="nx"&gt;showLegacyIpNotice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;address&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;If the dual-stack request returns an IPv6 address, all is well. If it returns IPv4, the browser gets one more chance against an &lt;span class="caps"&gt;AAAA&lt;/span&gt;-only name. Only an IPv4 result &lt;strong&gt;and&lt;/strong&gt; a failed IPv6 probe produce the toast. A timeout, content blocker, captive portal, or other ambiguous failure keeps the site quiet rather than turning uncertainty into a&amp;nbsp;lecture.&lt;/p&gt;
&lt;p&gt;That part survived the afternoon unchanged. The only offensive lines were the two&amp;nbsp;constants.&lt;/p&gt;
&lt;h2 id="surely-i-can-replace-two-urls"&gt;Surely I Can Replace Two&amp;nbsp;URLs&lt;/h2&gt;
&lt;p&gt;An &lt;span class="caps"&gt;IP&lt;/span&gt; echo service can be one line in almost any server-side language. Accept a connection, return the peer address, go home. I did not do&amp;nbsp;that.&lt;/p&gt;
&lt;p&gt;Once the endpoint was going to exist under my own domain, it seemed rude to make it useful only to one function in one JavaScript&amp;nbsp;file. &lt;code&gt;curl&lt;/code&gt; should be able to use it. If it is useful&amp;nbsp;from &lt;code&gt;curl&lt;/code&gt;, it should return clean plain text. Machines might want &lt;span class="caps"&gt;JSON&lt;/span&gt;. MaxMind databases already exist in my infrastructure for GeoIP policy, so the service may as well expose the approximate location and &lt;span class="caps"&gt;ASN&lt;/span&gt;. If that data exists, a human visiting the root &lt;span class="caps"&gt;URL&lt;/span&gt; deserves something nicer than one undecorated&amp;nbsp;line.&lt;/p&gt;
&lt;p&gt;This is the precise point at which “replace two URLs” became &lt;a href="https://git.linuxserver.pro/chris/hofstede-ip"&gt;hofstede-ip&lt;/a&gt;, a Rust application built with&amp;nbsp;Axum:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Route&lt;/th&gt;
&lt;th&gt;Response&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;HTML&lt;/span&gt; for a browser, otherwise the address as text&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/ip&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;IP&lt;/span&gt; address&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/json&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Full structured result&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/asn&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Autonomous system number&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/provider&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;ISP&lt;/span&gt; or network organisation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/country&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;ISO&lt;/span&gt; country code&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/city&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Approximate GeoIP city&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/healthz&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Internal health and database status&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The root route negotiates&amp;nbsp;on &lt;code&gt;Accept&lt;/code&gt;. A normal browser gets a proper server-rendered page; a command-line client gets the simplest useful&amp;nbsp;answer:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$ curl https://ip.hofstede.it
2a06:9801:1c:1000::42

$ curl https://ip.hofstede.it/asn
AS201379

$ curl https://ip.hofstede.it/json | jq .ip_version
6
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;There is no runtime JavaScript on the page. The application renders the address, protocol verdict, provider, &lt;span class="caps"&gt;ASN&lt;/span&gt;, approximate location, accuracy radius, and the modest collection of facts already volunteered in the &lt;span class="caps"&gt;HTTP&lt;/span&gt; headers: browser family, platform, language, Do Not Track, and Global Privacy Control. I wanted an endpoint for one JavaScript function and somehow responded by building a website with less&amp;nbsp;JavaScript.&lt;/p&gt;
&lt;p&gt;The map is an OpenStreetMap embed loaded lazily when it approaches the viewport. That is called out explicitly because it causes the visitor&amp;#8217;s browser to contact a third party. GeoIP itself remains what it has always been: an approximate network-location hint, not a homing beacon, and the accuracy radius is displayed prominently enough to discourage fantasies of&amp;nbsp;precision.&lt;/p&gt;
&lt;h2 id="one-binary-because-restraint-had-to-appear-somewhere"&gt;One Binary, Because Restraint Had to Appear&amp;nbsp;Somewhere&lt;/h2&gt;
&lt;p&gt;Rust is not required for this job. A shell script&amp;nbsp;behind &lt;code&gt;inetd&lt;/code&gt; could return the address. So could twenty lines of Python, a tiny Go program, Caddy placeholders, nginx, or several sufficiently motivated&amp;nbsp;pigeons.&lt;/p&gt;
&lt;p&gt;I chose Rust because the result is one self-contained FreeBSD binary with no interpreter or application runtime left in production. Axum provides the &lt;span class="caps"&gt;HTTP&lt;/span&gt; routing, Tokio the&amp;nbsp;runtime, &lt;code&gt;maxminddb&lt;/code&gt; reads the local &lt;span class="caps"&gt;MMDB&lt;/span&gt; files,&amp;nbsp;and &lt;code&gt;serde&lt;/code&gt; produces the &lt;span class="caps"&gt;JSON&lt;/span&gt;. The &lt;span class="caps"&gt;HTML&lt;/span&gt; and &lt;span class="caps"&gt;CSS&lt;/span&gt; are compiled into the binary&amp;nbsp;with &lt;code&gt;include_str!&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The production process tree is consequently less dramatic than the source&amp;nbsp;tree:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;USER     PID %CPU %MEM    VSZ   RSS  STAT COMMAND
ipinfo 56973  0.0  0.0  13080  2496 IsJ  daemon: /usr/local/bin/hofstede-ip-env[56974]
ipinfo 56974  0.0  1.0 130932 81576 IJ   /usr/local/sbin/hofstede-ip
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;daemon(8)&lt;/code&gt; supervises the application, restarts it on failure, and gives rc.d a pidfile to work with. The process runs as an&amp;nbsp;unprivileged &lt;code&gt;ipinfo&lt;/code&gt; user. A small wrapper reads the root-controlled environment file before executing the binary because service configuration containing paths and address lists is easier to audit than an increasingly&amp;nbsp;creative &lt;code&gt;command_args&lt;/code&gt; line.&lt;/p&gt;
&lt;p&gt;The application also refuses to start when a configured database is missing or invalid. Running without GeoIP data is fine for development and&amp;nbsp;produces &lt;code&gt;Unknown&lt;/code&gt;; claiming to run with a database while silently failing to open it is&amp;nbsp;not.&lt;/p&gt;
&lt;h2 id="the-reverse-proxy-knows-who-you-are"&gt;The Reverse Proxy Knows Who You&amp;nbsp;Are&lt;/h2&gt;
&lt;p&gt;The public service sits behind my existing Caddy ingress. That creates the only security-sensitive part of an &lt;span class="caps"&gt;IP&lt;/span&gt; echo service: the &lt;span class="caps"&gt;TCP&lt;/span&gt; peer seen by the application is Caddy, not the&amp;nbsp;visitor.&lt;/p&gt;
&lt;p&gt;Caddy passes the original address&amp;nbsp;in &lt;code&gt;X-Forwarded-For&lt;/code&gt;. Trust that header from everybody and the service becomes a form where visitors may type whichever address they would like to be&amp;nbsp;today:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;X-Forwarded-For: 2001:db8::definitely-me
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The application therefore accepts forwarded addresses only when the immediate &lt;span class="caps"&gt;TCP&lt;/span&gt; peer belongs to an explicitly configured proxy network. Anything arriving from another jail, even with a beautifully forged header, is reported as the peer that actually connected. IPv4-mapped IPv6 peers are normalised before the trust check, and an invalid forwarded value falls back to the peer address with a warning in the&amp;nbsp;log.&lt;/p&gt;
&lt;p&gt;In production, the trust list contains the Caddy jail&amp;#8217;s address as&amp;nbsp;a &lt;code&gt;/32&lt;/code&gt;. The application itself binds only&amp;nbsp;to &lt;code&gt;10.254.254.36:8080&lt;/code&gt;, and the firewall permits that port only from the proxy. The same decision is enforced twice: by the network and by the&amp;nbsp;application.&lt;/p&gt;
&lt;p&gt;Caddy terminates &lt;span class="caps"&gt;TLS&lt;/span&gt;, replaces the forwarding headers at the public boundary, performs a health check every 30 seconds, and refuses public access&amp;nbsp;to &lt;code&gt;/healthz&lt;/code&gt;. Personalised responses&amp;nbsp;carry &lt;code&gt;Cache-Control: no-store&lt;/code&gt;, and the service does not log visitor addresses itself. Caddy has its own access-log policy, because claiming that the application does not log something is not the same as claiming that nothing in front of it&amp;nbsp;does.&lt;/p&gt;
&lt;h2 id="naturally-it-needed-its-own-jail"&gt;Naturally, It Needed Its Own&amp;nbsp;Jail&lt;/h2&gt;
&lt;p&gt;The application could have joined an existing jail. It is one binary listening on one port. That line of reasoning had, however, already lost control of the&amp;nbsp;afternoon.&lt;/p&gt;
&lt;p&gt;Instead it runs&amp;nbsp;in &lt;code&gt;ipinfo&lt;/code&gt;, a FreeBSD 15.1 thin jail managed by&amp;nbsp;Bastille:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;[root@radon ~]# bastille list ipinfo
 JID  Name    Boot  Prio  State  Type  IP Address              Release
 15   ipinfo  on    99    Up     thin  10.254.254.36           15.1-RELEASE-p3
                                      2a06:9801:1c:1000::36
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;It is a &lt;span class="caps"&gt;VNET&lt;/span&gt; jail, so it has its own network stack and an epair connecting it to the&amp;nbsp;host&amp;#8217;s &lt;code&gt;bastille0&lt;/code&gt; bridge:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;[root@radon ~]# bastille cmd ipinfo ifconfig vnet0

[ipinfo]:
vnet0: flags=1008843&amp;lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP&amp;gt; mtu 1500
        description: jail interface for bastille0
        inet 10.254.254.36 netmask 0xffffff00 broadcast 10.254.254.255
        inet6 2a06:9801:1c:1000::36 prefixlen 64
        media: Ethernet 10Gbase-T (10Gbase-T &amp;lt;full-duplex&amp;gt;)
        status: active
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The complete request path looks like&amp;nbsp;this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;IPv4 visitor ── A record ─────┐
                              ├── Caddy jail ── HTTP + X-Forwarded-For ── ipinfo jail
IPv6 visitor ── AAAA record ──┘       :443                              :8080
                                                                           |
                                                                  MaxMind MMDBs (ro)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The GeoIP databases already live in the Caddy jail,&amp;nbsp;where &lt;code&gt;geoipupdate&lt;/code&gt; maintains them. They are mounted read-only&amp;nbsp;into &lt;code&gt;ipinfo&lt;/code&gt; with &lt;code&gt;nullfs&lt;/code&gt;. The application loads them into memory at startup, and a host-side update job can restart the service after an atomic database replacement. The account &lt;span class="caps"&gt;ID&lt;/span&gt; and MaxMind licence key never enter the application&amp;nbsp;jail.&lt;/p&gt;
&lt;p&gt;This is not the smallest topology, but it is consistent with the rest of the machine. Caddy is the only public ingress. Application jails are private backends. Each service gets its own filesystem, process namespace, network stack, credentials, and firewall policy. The implementation may be silly in proportion to the original toast; the deployment is at least boring in exactly the way production infrastructure should&amp;nbsp;be.&lt;/p&gt;
&lt;h2 id="three-names-for-two-protocols"&gt;Three Names for Two&amp;nbsp;Protocols&lt;/h2&gt;
&lt;p&gt;The public &lt;span class="caps"&gt;DNS&lt;/span&gt; is where the original detection logic finally becomes independent of a third&amp;nbsp;party:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Name&lt;/th&gt;
&lt;th&gt;Records&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ip.hofstede.it&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A and &lt;span class="caps"&gt;AAAA&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;Normal dual-stack service&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v4.ip.hofstede.it&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;A only&lt;/td&gt;
&lt;td&gt;Force or test IPv4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v6.ip.hofstede.it&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;AAAA&lt;/span&gt; only&lt;/td&gt;
&lt;td&gt;Force or test IPv6&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;All three names point to Caddy, never directly to the application jail. The connection accepted by Caddy determines which address reaches the backend through the trusted forwarding&amp;nbsp;header.&lt;/p&gt;
&lt;p&gt;The website now contains the two lines I wanted in the first&amp;nbsp;place:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;IP_ECHO_URL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;https://ip.hofstede.it/ip&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;IPV6_PROBE_URL&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;https://v6.ip.hofstede.it/ip&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The IPv4-only name is not required by the toast, but once one has built a dual-stack &lt;span class="caps"&gt;IP&lt;/span&gt;-information service, declining to provide explicit protocol-test endpoints would look like an outbreak of restraint at an implausibly late&amp;nbsp;stage.&lt;/p&gt;
&lt;h2 id="the-escalation-in-git-timestamps"&gt;The Escalation, in Git&amp;nbsp;Timestamps&lt;/h2&gt;
&lt;p&gt;Git preserved the incident rather&amp;nbsp;neatly:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;th&gt;Event&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;10:15&lt;/td&gt;
&lt;td&gt;Add the legacy-&lt;span class="caps"&gt;IP&lt;/span&gt; toast using ipify&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:13&lt;/td&gt;
&lt;td&gt;Commit the&amp;nbsp;initial &lt;code&gt;hofstede-ip&lt;/code&gt; service&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;12:17&lt;/td&gt;
&lt;td&gt;Point the website at the self-hosted endpoints&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:16&lt;/td&gt;
&lt;td&gt;Harden proxy handling and add &lt;span class="caps"&gt;HTTP&lt;/span&gt;-level tests&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:17&lt;/td&gt;
&lt;td&gt;Declare this somehow version 0.2.0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;13:27&lt;/td&gt;
&lt;td&gt;Add the new service to the website&amp;#8217;s projects section&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The initial service commit contains 1,982 lines across Rust, &lt;span class="caps"&gt;CSS&lt;/span&gt;, deployment files, documentation, and the dependency lockfile. The original website feature added 159 lines. This means removing a two-line third-party dependency produced more than two thousand lines of owned software before lunch had properly&amp;nbsp;settled.&lt;/p&gt;
&lt;p&gt;That is not a defence of the process. It is simply the kind of arithmetic one should occasionally perform while operating a keyboard&amp;nbsp;unsupervised.&lt;/p&gt;
&lt;h2 id="was-this-overengineering"&gt;Was This&amp;nbsp;Overengineering?&lt;/h2&gt;
&lt;p&gt;Obviously.&lt;/p&gt;
&lt;p&gt;It was also a useful service I did not previously have. I can now&amp;nbsp;type &lt;code&gt;curl https://ip.hofstede.it&lt;/code&gt; from any machine and see the address it uses on the public Internet. I can force either protocol family&amp;nbsp;with &lt;code&gt;v4&lt;/code&gt; and &lt;code&gt;v6&lt;/code&gt;. Scripts get stable plain-text and &lt;span class="caps"&gt;JSON&lt;/span&gt; endpoints under a domain I control. The website no longer tells a third party each time it checks a visitor&amp;#8217;s connection. The deployment reuses the Caddy, Bastille, firewall, &lt;span class="caps"&gt;DNS&lt;/span&gt;, and GeoIP patterns already present in my&amp;nbsp;infrastructure.&lt;/p&gt;
&lt;p&gt;The distinction I would draw is between &lt;strong&gt;needless novelty&lt;/strong&gt; and &lt;strong&gt;enthusiastic disproportion&lt;/strong&gt;. There is nothing novel in the production design. It is the same reverse proxy, the same jail boundary, the same &lt;span class="caps"&gt;VNET&lt;/span&gt; layout, the same read-only database sharing, and the same service supervision I use elsewhere. The ridiculous part is only that all of it was summoned into existence by a 21-rem-wide toast whose purpose is to tease IPv4&amp;nbsp;users.&lt;/p&gt;
&lt;p&gt;At the start of the morning, my website asked ipify for an address. By early afternoon, &lt;a href="https://ip.hofstede.it"&gt;ip.hofstede.it&lt;/a&gt;, &lt;a href="https://v4.ip.hofstede.it"&gt;v4.ip.hofstede.it&lt;/a&gt;, and &lt;a href="https://v6.ip.hofstede.it"&gt;v6.ip.hofstede.it&lt;/a&gt; existed in my own infrastructure, backed by a Rust binary in a dedicated dual-stack FreeBSD&amp;nbsp;jail.&lt;/p&gt;
&lt;p&gt;The toast still looks exactly the&amp;nbsp;same.&lt;/p&gt;
&lt;p&gt;That may be my favourite&amp;nbsp;part. &lt;code&gt;^_^&lt;/code&gt;&lt;/p&gt;</content><category term="Networking"/><category term="ipv6"/><category term="ipv4"/><category term="freebsd"/><category term="bastille"/><category term="jails"/><category term="rust"/><category term="axum"/><category term="caddy"/><category term="maxmind"/><category term="self-hosting"/></entry><entry><title>Running Your Own AS: An IXP over VXLAN, IPv4 on the Overlay, and a Config Spring Clean</title><link href="https://hofstede.it/running-your-own-as-an-ixp-over-vxlan-ipv4-on-the-overlay-and-a-config-spring-clean/" rel="alternate"/><published>2026-07-30T00:00:00+02:00</published><updated>2026-07-30T00:00:00+02:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-07-30:/running-your-own-as-an-ixp-over-vxlan-ipv4-on-the-overlay-and-a-config-spring-clean/</id><summary type="html">&lt;p&gt;Part 5 of the &lt;span class="caps"&gt;AS201379&lt;/span&gt; journey: joining &lt;span class="caps"&gt;EVIX&lt;/span&gt;, an internet exchange that hands out &lt;span class="caps"&gt;VXLAN&lt;/span&gt; tunnels instead of switch ports, carrying a routed IPv4 /29 across the IPv6 overlay with &lt;span class="caps"&gt;PF&lt;/span&gt; policy routing, the dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; routing that makes a &lt;span class="caps"&gt;VPS&lt;/span&gt; work as both a normal host and a routed &lt;span class="caps"&gt;AS&lt;/span&gt; site, moving the downstream MikroTik onto WireGuard, and a long-overdue cleanup of four routers&amp;#8217; worth of accumulated&amp;nbsp;configuration.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-07-30-as201379-part5-evix-vxlan-ipv4.png" title="Part 5: Header image"&gt;&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/h2&gt;
&lt;div class="toc"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#architecture-where-everything-sits-now"&gt;Architecture: Where Everything Sits&amp;nbsp;Now&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#evix-an-exchange-point-with-no-port"&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;: An Exchange Point With No Port&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#what-evix-actually-is"&gt;What &lt;span class="caps"&gt;EVIX&lt;/span&gt; Actually&amp;nbsp;Is&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#vxlan-on-freebsd"&gt;&lt;span class="caps"&gt;VXLAN&lt;/span&gt; on&amp;nbsp;FreeBSD&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#pf-for-a-tunnelled-peering-lan"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt; for a Tunnelled Peering &lt;span class="caps"&gt;LAN&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#frr-two-route-servers-two-asns"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;: Two Route Servers, Two&amp;nbsp;ASNs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#where-evix-sits-in-the-local-preference-hierarchy"&gt;Where &lt;span class="caps"&gt;EVIX&lt;/span&gt; Sits in the Local-Preference&amp;nbsp;Hierarchy&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#ipv4-arrives-sideways"&gt;IPv4 Arrives, Sideways&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#the-situation"&gt;The&amp;nbsp;Situation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#carrying-the-block-across-the-overlay"&gt;Carrying the Block Across the&amp;nbsp;Overlay&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-asymmetry-problem"&gt;The Asymmetry&amp;nbsp;Problem&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-this-buys"&gt;What This&amp;nbsp;Buys&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-far-end-dual-fib-routing-on-radon"&gt;The Far End: Dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; Routing on radon&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#two-routing-tables-one-interface-each"&gt;Two Routing Tables, One Interface&amp;nbsp;Each&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#fib-1-needs-a-complete-map"&gt;&lt;span class="caps"&gt;FIB&lt;/span&gt; 1 Needs a Complete&amp;nbsp;Map&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#three-kinds-of-jail"&gt;Three Kinds of&amp;nbsp;Jail&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#reply-to-on-every-tunnel-ingress-rule"&gt;reply-to on Every Tunnel Ingress&amp;nbsp;Rule&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-two-extras"&gt;The Two&amp;nbsp;Extras&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#downstream-wireguard-for-the-home-router"&gt;Downstream: WireGuard for the Home&amp;nbsp;Router&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-spring-clean"&gt;The Spring Clean&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#one-table-for-every-bgp-peer"&gt;One Table for Every &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Peer&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#deprecating-the-transit-addresses"&gt;Deprecating the Transit&amp;nbsp;Addresses&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#prefix-lists-interface-routes-and-daemons"&gt;Prefix Lists, Interface Routes, and&amp;nbsp;Daemons&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#monitoring-fourteen-sessions-on-one-screen"&gt;Monitoring: Fourteen Sessions on One&amp;nbsp;Screen&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#conclusion"&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#references"&gt;References&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Part 1&lt;/a&gt; set up a single FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; router with two upstreams. &lt;a href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/"&gt;Part 2&lt;/a&gt; added a Vultr edge and iBGP. &lt;a href="https://hofstede.it/running-your-own-as-joining-an-ixp-with-a-third-edge-router/"&gt;Part 3&lt;/a&gt; joined LocIX Düsseldorf with a dedicated third edge. &lt;a href="https://hofstede.it/running-your-own-as-direct-hetzner-peering-a-fourth-edge-and-bringing-the-home-lan-into-the-fabric/"&gt;Part 4&lt;/a&gt; added a fourth edge at iFog, direct peering with Hetzner on FogIXP, and brought the home &lt;span class="caps"&gt;LAN&lt;/span&gt; into the /48. This is Part&amp;nbsp;5.&lt;/p&gt;
&lt;p&gt;Three months later, the network has fourteen &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions and has stopped growing outward. What changed instead is the &lt;em&gt;kind&lt;/em&gt; of thing that got added. The headline feature this time is an exchange point that has no switch port for me to plug into: &lt;a href="https://evix.org/"&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;&lt;/a&gt;, the Experimental Virtual Internet Exchange, delivers its peering &lt;span class="caps"&gt;LAN&lt;/span&gt; as a &lt;span class="caps"&gt;VXLAN&lt;/span&gt; tunnel. My core router terminates that tunnel directly and peers with two route servers on two continents over it, with no additional edge router&amp;nbsp;involved.&lt;/p&gt;
&lt;p&gt;The second change is less glamorous and more useful: &lt;span class="caps"&gt;AS201379&lt;/span&gt; now carries IPv4. Not announced via &lt;span class="caps"&gt;BGP&lt;/span&gt;, but a&amp;nbsp;routed &lt;code&gt;/29&lt;/code&gt; from one of my transit providers, delivered across the same IPv6 overlay the rest of the network runs on, using &lt;span class="caps"&gt;PF&lt;/span&gt; policy routing to keep an intentionally asymmetric path working. This site is served out of&amp;nbsp;that &lt;code&gt;/29&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The third change is the one that took the most hours and produces no screenshots at all: four routers&amp;#8217; worth of configuration got cleaned up. And because the IPv4 block has to actually land somewhere, this part also documents the most complicated firewall in the entire network, which is not on a router at&amp;nbsp;all.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; &lt;span class="caps"&gt;AS201379&lt;/span&gt;&amp;#8217;s&amp;nbsp;prefix &lt;code&gt;2a06:9801:1c::/48&lt;/code&gt; and&amp;nbsp;all &lt;code&gt;hofstede.it&lt;/code&gt; hostnames are shown as-is. So is the routed IPv4&amp;nbsp;block &lt;code&gt;194.28.98.216/29&lt;/code&gt; - the site you are reading this on lives in it, so there is nothing to hide. Public peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; addresses from PeeringDB (&lt;span class="caps"&gt;EVIX&lt;/span&gt;&amp;#8217;s &lt;code&gt;2602:fed2:fff:ffff::/64&lt;/code&gt;,&amp;nbsp;FogIXP&amp;#8217;s &lt;code&gt;2001:7f8:ca:1::/64&lt;/code&gt;,&amp;nbsp;LocIX&amp;#8217;s &lt;code&gt;2a0c:b641:701::/64&lt;/code&gt;) are also shown as-is. Everything else - provider-assigned addresses, tunnel endpoints, trusted-management IPs - has been replaced with &lt;a href="https://www.rfc-editor.org/rfc/rfc5737"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5737&lt;/a&gt; / &lt;a href="https://www.rfc-editor.org/rfc/rfc3849"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 3849&lt;/a&gt; documentation ranges. &lt;span class="caps"&gt;RFC&lt;/span&gt; 1918 addresses on private transport are shown unchanged. Upstream and peer &lt;span class="caps"&gt;AS&lt;/span&gt; numbers are visible in public routing&amp;nbsp;tables.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/h2&gt;
&lt;p&gt;Part 5 adds five things to &lt;span class="caps"&gt;AS201379&lt;/span&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;&lt;/strong&gt;, joined over a &lt;span class="caps"&gt;VXLAN&lt;/span&gt; tunnel terminated on the core router, with route servers in Fremont and&amp;nbsp;Amsterdam&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;a routed&amp;nbsp;IPv4 &lt;code&gt;/29&lt;/code&gt;&lt;/strong&gt;, carried across the IPv6 overlay and policy-routed back out the way it&amp;nbsp;came&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; routing&amp;nbsp;on &lt;code&gt;radon&lt;/code&gt;&lt;/strong&gt;, the host at the far end of that tunnel, which has to be a normal &lt;span class="caps"&gt;VPS&lt;/span&gt; and a routed &lt;span class="caps"&gt;AS&lt;/span&gt; site at the same&amp;nbsp;time&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;WireGuard&lt;/strong&gt; replacing 6to4 for the downstream&amp;nbsp;MikroTik&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;a configuration cleanup&lt;/strong&gt; across all four routers: consolidated &lt;span class="caps"&gt;PF&lt;/span&gt; macros, a single peer table, deprecated provider addresses, trimmed &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;daemons&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Nothing in this list makes the network bigger. All of it makes the network easier to reason&amp;nbsp;about.&lt;/p&gt;
&lt;h2 id="architecture-where-everything-sits-now"&gt;Architecture: Where Everything Sits&amp;nbsp;Now&lt;/h2&gt;
&lt;p&gt;The four-router shape from Part 4 is unchanged. What grew is what hangs&amp;nbsp;off &lt;code&gt;hobgp&lt;/code&gt;, the&amp;nbsp;core.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────────────────────────┐
                    │                Default-Free Zone                 │
                    └──┬───────────┬─────────────┬──────────────┬──────┘
                       │           │             │              │
                   AS209735    AS212895       AS41051         AS137933 / AS209762
                   (Lagrange)  (route64)   (freetransit)      (EVIX RS FMT / AMS)
                       │           │             │              │
                      GRE         GRE           GRE          VXLAN vni 107
                       │           │             │              │
                  ┌────┴───────────┴─────────────┴──────────────┴────┐
                  │                  hobgp (Core)                    │
                  │            FreeBSD + FRR, AS201379               │
                  │            2a06:9801:1c::/48 (origin)            │
                  └─┬──────────┬──────────┬────────────┬─────────────┘
                    │          │          │            │
               iBGP GIF   iBGP GIF   iBGP GIF     WireGuard
                    │          │          │            │
        ┌───────────┴──┐ ┌─────┴──────┐ ┌─┴────────┐ ┌─┴──────────────┐
        │ vtbgp        │ │ lobgp      │ │ ixbgp    │ │ MikroTik       │
        │ Vultr FRA    │ │ Servperso  │ │ iFog ZRH │ │ RouterOS 7.20  │
        │ AS64515      │ │ + LocIX RS │ │ + FogIXP │ │ :6000, :6001   │
        │              │ │ + v4 /29   │ │ + AS24940│ │                │
        └──────────────┘ └────────────┘ └──────────┘ └────────────────┘

Downstream sites tunnelled from hobgp:
   gif0  :1000::/64   radon      FreeBSD hypervisor host (blog, DNS, website)
   gif2  :2000::/64   colo       OPNsense at the Hetzner colo (management)
   gif2  :2001::/64   colo       OPNsense main network, fronting the Proxmox hosts
   gif2  :5000::/64   road       road-warrior network
   gif2  :7000::/64   dn42       DN42 node
   gif3  :3000::/64   mail       mail.linuxserver.pro
   gif4  :4000::/64   bb         burningboard.net (Mastodon)
   wg    :6000::/64   home       home LAN via MikroTik
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Two things are worth reading off that&amp;nbsp;diagram.&lt;/p&gt;
&lt;p&gt;First, &lt;code&gt;hobgp&lt;/code&gt; is now the router with the most eBGP sessions, not the fewest. In Part 2 it was a hub that happened to hold two &lt;span class="caps"&gt;GRE&lt;/span&gt; transits. It now holds three &lt;span class="caps"&gt;GRE&lt;/span&gt; transits plus an &lt;span class="caps"&gt;IXP&lt;/span&gt; fabric, while still terminating every iBGP session and every downstream&amp;nbsp;tunnel.&lt;/p&gt;
&lt;p&gt;Second, the OPNsense at the Hetzner colo has become a router in its own&amp;nbsp;right. &lt;code&gt;gif2&lt;/code&gt; carries four&amp;nbsp;separate &lt;code&gt;/64&lt;/code&gt;s: management, the main colo network that fronts my Proxmox hosts, the road-warrior network, and a &lt;span class="caps"&gt;DN42&lt;/span&gt; node. The actual workloads live behind&amp;nbsp;it. &lt;code&gt;gif0&lt;/code&gt; to &lt;code&gt;radon&lt;/code&gt; is the other end of that spectrum: a single FreeBSD hypervisor host that runs the blog, &lt;span class="caps"&gt;DNS&lt;/span&gt; and my website inside jails and bhyve&amp;nbsp;guests.&lt;/p&gt;
&lt;p&gt;The core router does not know or care about any of this. It has a&amp;nbsp;static &lt;code&gt;/64&lt;/code&gt; route per site and a blackhole for&amp;nbsp;the &lt;code&gt;/48&lt;/code&gt;. Every site is a routing entry, not a special&amp;nbsp;case.&lt;/p&gt;
&lt;h2 id="evix-an-exchange-point-with-no-port"&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;: An Exchange Point With No&amp;nbsp;Port&lt;/h2&gt;
&lt;h3 id="what-evix-actually-is"&gt;What &lt;span class="caps"&gt;EVIX&lt;/span&gt; Actually&amp;nbsp;Is&lt;/h3&gt;
&lt;p&gt;Every &lt;span class="caps"&gt;IXP&lt;/span&gt; in Parts 3 and 4 required a machine physically (or at least virtually) attached to the exchange fabric. LocIX meant a &lt;span class="caps"&gt;VM&lt;/span&gt; at Servperso with a second &lt;span class="caps"&gt;NIC&lt;/span&gt; on the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;. FogIXP meant a &lt;span class="caps"&gt;VM&lt;/span&gt; at iFog with the same arrangement. That is what &amp;#8220;joining an &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;#8221; normally costs: a presence at a facility that has a&amp;nbsp;port.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://evix.org/"&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;&lt;/a&gt; inverts that. It is an experimental, community-run exchange whose fabric is an overlay: members receive a &lt;span class="caps"&gt;VXLAN&lt;/span&gt; tunnel to one of the &lt;span class="caps"&gt;EVIX&lt;/span&gt; VTEPs and terminate it wherever their router happens to be. The peering &lt;span class="caps"&gt;LAN&lt;/span&gt;, &lt;code&gt;2602:fed2:fff:ffff::/64&lt;/code&gt;, is a real shared layer-2 segment. It just happens to be built out of tunnels rather than&amp;nbsp;fibre.&lt;/p&gt;
&lt;p&gt;For a hobby &lt;span class="caps"&gt;AS&lt;/span&gt; this is a meaningful difference. There is no new &lt;span class="caps"&gt;VM&lt;/span&gt;, no new PoP, no monthly port fee, and no second &lt;span class="caps"&gt;NIC&lt;/span&gt; to argue with. My &lt;span class="caps"&gt;VTEP&lt;/span&gt; is in Zürich; my router is in Nuremberg; the exchange does not&amp;nbsp;care.&lt;/p&gt;
&lt;p&gt;The honest caveat is in the name. &lt;span class="caps"&gt;EVIX&lt;/span&gt; is &lt;em&gt;experimental&lt;/em&gt;. The route servers carry a few hundred prefixes rather than a few thousand, and participants are overwhelmingly other hobbyist and research networks. This is not a path to eyeball networks. It is a path to a specific, interesting subset of the internet, and it costs almost nothing to&amp;nbsp;have.&lt;/p&gt;
&lt;h3 id="vxlan-on-freebsd"&gt;&lt;span class="caps"&gt;VXLAN&lt;/span&gt; on&amp;nbsp;FreeBSD&lt;/h3&gt;
&lt;p&gt;FreeBSD has had a&amp;nbsp;native &lt;code&gt;vxlan(4)&lt;/code&gt; interface for years. It is configured entirely&amp;nbsp;through &lt;code&gt;rc.conf&lt;/code&gt; with&amp;nbsp;a &lt;code&gt;create_args_&lt;/code&gt; line:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif if_gre if_wg if_vxlan&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0 gif1 gif2 gif3 gif4 gif5 gif6 gre1 gre2 gre3 vxlan0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# EVIX Zürich VXLAN&lt;/span&gt;
&lt;span class="c1"&gt;# - local endpoint (vtep): 198.51.100.10&lt;/span&gt;
&lt;span class="c1"&gt;# - remote endpoint:       203.0.113.34&lt;/span&gt;
&lt;span class="c1"&gt;# - vni: 107, dstport: 500&lt;/span&gt;
&lt;span class="nv"&gt;create_args_vxlan0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vxlanid 107 vxlanlocal 198.51.100.10 vxlanremote 203.0.113.34 \&lt;/span&gt;
&lt;span class="s2"&gt;    vxlanlocalport 500 vxlanremoteport 500 mtu 1400&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vxlan0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2602:fed2:fff:ffff:6::14 prefixlen 64 -tentative&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vxlan0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;IXP-EVIX-Zurich&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;A few details matter&amp;nbsp;here.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Point-to-point &lt;span class="caps"&gt;VXLAN&lt;/span&gt;, not multicast.&lt;/strong&gt; The usual &lt;span class="caps"&gt;VXLAN&lt;/span&gt; deployment uses a multicast group for &lt;span class="caps"&gt;BUM&lt;/span&gt;&amp;nbsp;traffic. &lt;code&gt;vxlanremote&lt;/code&gt; instead pins a single remote &lt;span class="caps"&gt;VTEP&lt;/span&gt;, which is exactly right for this topology: I have one tunnel, to one &lt;span class="caps"&gt;EVIX&lt;/span&gt; &lt;span class="caps"&gt;VTEP&lt;/span&gt;, and everything else on the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; is reached through&amp;nbsp;it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Port 500.&lt;/strong&gt; &lt;span class="caps"&gt;EVIX&lt;/span&gt; uses &lt;span class="caps"&gt;UDP&lt;/span&gt; port 500 rather than the &lt;span class="caps"&gt;IANA&lt;/span&gt;-assigned 4789. Port 500 is normally &lt;span class="caps"&gt;ISAKMP&lt;/span&gt;, and the practical reason to pick it is that a lot of consumer and hosting &lt;span class="caps"&gt;NAT&lt;/span&gt; gateways treat it as &amp;#8220;&lt;span class="caps"&gt;VPN&lt;/span&gt; traffic&amp;#8221; and pass it unmolested.&amp;nbsp;Both &lt;code&gt;vxlanlocalport&lt;/code&gt; and &lt;code&gt;vxlanremoteport&lt;/code&gt; have to be set; FreeBSD keeps them&amp;nbsp;independent.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;-tentative&lt;/code&gt;.&lt;/strong&gt; &lt;span class="caps"&gt;VXLAN&lt;/span&gt; is an Ethernet interface, so IPv6 duplicate address detection applies. On a peering &lt;span class="caps"&gt;LAN&lt;/span&gt; with dozens of members and a tunnel that takes a moment to come up, &lt;span class="caps"&gt;DAD&lt;/span&gt; can leave the address&amp;nbsp;in &lt;code&gt;tentative&lt;/code&gt; state long enough for &lt;span class="caps"&gt;FRR&lt;/span&gt; to fail its first bind attempt. Disabling it for this address makes boot&amp;nbsp;deterministic.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;MTU&lt;/span&gt; 1400.&lt;/strong&gt; The chain is IPv6 payload inside Ethernet inside &lt;span class="caps"&gt;VXLAN&lt;/span&gt; inside &lt;span class="caps"&gt;UDP&lt;/span&gt; inside IPv4. The overhead is 50 bytes of &lt;span class="caps"&gt;VXLAN&lt;/span&gt; and outer Ethernet on top of 28 bytes of outer IPv4/&lt;span class="caps"&gt;UDP&lt;/span&gt;, and 1400 leaves comfortable room inside Hetzner&amp;#8217;s 1500-byte path without relying on &lt;span class="caps"&gt;PMTUD&lt;/span&gt; working across every member&amp;#8217;s&amp;nbsp;tunnel.&lt;/p&gt;
&lt;p&gt;The result looks like an ordinary Ethernet&amp;nbsp;interface:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;vxlan0: flags=1008843&amp;lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP&amp;gt; metric 0 mtu 1400
        description: IXP-EVIX-Zurich
        ether 58:9c:fc:10:b0:08
        inet6 2602:fed2:fff:ffff:6::14 prefixlen 64
        vxlan vni 107 local 198.51.100.10:500 remote 203.0.113.34:500
        media: Ethernet autoselect (autoselect &amp;lt;full-duplex&amp;gt;)
        status: active
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3 id="pf-for-a-tunnelled-peering-lan"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt; for a Tunnelled Peering &lt;span class="caps"&gt;LAN&lt;/span&gt;&lt;/h3&gt;
&lt;p&gt;Two rules bring the fabric up, and they sit in different halves of the&amp;nbsp;ruleset.&lt;/p&gt;
&lt;p&gt;The outer encapsulation is &lt;span class="caps"&gt;UDP&lt;/span&gt; to a single remote &lt;span class="caps"&gt;VTEP&lt;/span&gt;, and unlike the &lt;span class="caps"&gt;GRE&lt;/span&gt; and proto-41 tunnels elsewhere in this network it is fine to keep state on it, because there is exactly one&amp;nbsp;flow:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;evix_if = &amp;quot;vxlan0&amp;quot;

&lt;span class="gh"&gt;#&lt;/span&gt; Outer UDP encapsulation for VXLAN (port 500)
pass in  quick on $ext_if proto udp from 203.0.113.34 to ($ext_if) port 500 keep state
pass out quick on $ext_if proto udp from ($ext_if) to 203.0.113.34 port 500 keep state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Inside the tunnel, the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; behaves like any other &lt;span class="caps"&gt;IXP&lt;/span&gt; interface, so it gets the same &lt;span class="caps"&gt;MSS&lt;/span&gt; clamp treatment as the&amp;nbsp;rest:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;scrub on { $evix_if } max-mss 1340
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;1340 is deliberately conservative: 1400 minus 40 bytes of IPv6 header minus 20 for &lt;span class="caps"&gt;TCP&lt;/span&gt; options headroom. Peering LANs are where &lt;span class="caps"&gt;MTU&lt;/span&gt; mismatches go to hide, and clamping the &lt;span class="caps"&gt;MSS&lt;/span&gt; on ingress is far cheaper than debugging why exactly one member&amp;#8217;s web server hangs after the &lt;span class="caps"&gt;TLS&lt;/span&gt;&amp;nbsp;handshake.&lt;/p&gt;
&lt;p&gt;The route-server sessions themselves ride the generic &lt;span class="caps"&gt;BGP&lt;/span&gt; rules, because both &lt;span class="caps"&gt;EVIX&lt;/span&gt; route servers are members of&amp;nbsp;the &lt;code&gt;bgp_peers&lt;/code&gt; table (more on that&amp;nbsp;below).&lt;/p&gt;
&lt;h3 id="frr-two-route-servers-two-asns"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;: Two Route Servers, Two&amp;nbsp;ASNs&lt;/h3&gt;
&lt;p&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt; runs route servers in multiple locations, each with its own &lt;span class="caps"&gt;AS&lt;/span&gt; number. I peer with&amp;nbsp;two:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;route-map RM-EVIX-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 250
exit
!
route-map RM-EVIX-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
router bgp 201379
 ! EVIX FMT Route Server (Fremont)
 neighbor 2602:fed2:fff:ffff::1 remote-as 137933
 neighbor 2602:fed2:fff:ffff::1 description IXP-EVIX-RS-FMT
 neighbor 2602:fed2:fff:ffff::1 update-source 2602:fed2:fff:ffff:6::14
 no neighbor 2602:fed2:fff:ffff::1 enforce-first-as
 !
 ! EVIX AMS Route Server (Amsterdam)
 neighbor 2602:fed2:fff:ffff::253 remote-as 209762
 neighbor 2602:fed2:fff:ffff::253 description IXP-EVIX-RS-AMS
 neighbor 2602:fed2:fff:ffff::253 update-source 2602:fed2:fff:ffff:6::14
 no neighbor 2602:fed2:fff:ffff::253 enforce-first-as
 !
 address-family ipv6 unicast
  neighbor 2602:fed2:fff:ffff::1 activate
  neighbor 2602:fed2:fff:ffff::1 soft-reconfiguration inbound
  neighbor 2602:fed2:fff:ffff::1 maximum-prefix 2500 80 restart 30
  neighbor 2602:fed2:fff:ffff::1 route-map RM-EVIX-IN in
  neighbor 2602:fed2:fff:ffff::1 route-map RM-EVIX-OUT out
  neighbor 2602:fed2:fff:ffff::253 activate
  neighbor 2602:fed2:fff:ffff::253 soft-reconfiguration inbound
  neighbor 2602:fed2:fff:ffff::253 maximum-prefix 2500 80 restart 30
  neighbor 2602:fed2:fff:ffff::253 route-map RM-EVIX-IN in
  neighbor 2602:fed2:fff:ffff::253 route-map RM-EVIX-OUT out
 exit-address-family
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Three things differ from the transit&amp;nbsp;sessions.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;update-source&lt;/code&gt; on an eBGP session.&lt;/strong&gt; The peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; address is&amp;nbsp;on &lt;code&gt;vxlan0&lt;/code&gt;, but it is not the address &lt;span class="caps"&gt;FRR&lt;/span&gt; would pick by default for a session whose next hop is reachable over the same interface as several other things. Pinning it removes the ambiguity, and it is the same pattern already used for every iBGP tunnel in this&amp;nbsp;network.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;no ... enforce-first-as&lt;/code&gt;.&lt;/strong&gt; Same reason as FogIXP in Part 4: a transparent route server does not prepend its own &lt;span class="caps"&gt;ASN&lt;/span&gt;, so the first &lt;span class="caps"&gt;AS&lt;/span&gt; in the path is the origin, not the route server. &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;#8217;s default sanity check would reject every&amp;nbsp;update.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;maximum-prefix 2500&lt;/code&gt;, not 350000.&lt;/strong&gt; The transit sessions carry a full table and get a 350k ceiling. An experimental exchange should never send me more than a few hundred prefixes. Setting the ceiling near the expected value rather than near the theoretical maximum is what&amp;nbsp;turns &lt;code&gt;maximum-prefix&lt;/code&gt; from a formality into an actual safety mechanism.&amp;nbsp;The &lt;code&gt;80&lt;/code&gt; is the warning threshold in percent,&amp;nbsp;and &lt;code&gt;restart 30&lt;/code&gt; brings the session back automatically half an hour after a trip rather than leaving it administratively&amp;nbsp;down.&lt;/p&gt;
&lt;p&gt;In practice the two route servers currently hand over 476 and 347 prefixes respectively - about a fifth of what LocIX sends and a tenth of&amp;nbsp;FogIXP.&lt;/p&gt;
&lt;h3 id="where-evix-sits-in-the-local-preference-hierarchy"&gt;Where &lt;span class="caps"&gt;EVIX&lt;/span&gt; Sits in the Local-Preference&amp;nbsp;Hierarchy&lt;/h3&gt;
&lt;p&gt;Local preference is transitive across iBGP, so the values set at the edges in Parts 3 and 4 arrive at the core intact. Adding &lt;span class="caps"&gt;EVIX&lt;/span&gt; at 250 slots it into an existing&amp;nbsp;ladder:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style="text-align: right;"&gt;Local pref&lt;/th&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;th&gt;Rationale&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;301&lt;/td&gt;
&lt;td&gt;Hetzner direct on FogIXP&lt;/td&gt;
&lt;td&gt;direct session with a large network&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;300&lt;/td&gt;
&lt;td&gt;FogIXP and LocIX route servers&lt;/td&gt;
&lt;td&gt;settlement-free exchange routes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;260&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;DTAG&lt;/span&gt; routes learned via Vultr&lt;/td&gt;
&lt;td&gt;traffic engineering from Part 4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;250&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt; route servers&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;exchange routes, but experimental fabric&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;100&lt;/td&gt;
&lt;td&gt;iFog, route64, freetransit, Servperso&lt;/td&gt;
&lt;td&gt;paid and free transit, the default&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align: right;"&gt;50&lt;/td&gt;
&lt;td&gt;Lagrange&lt;/td&gt;
&lt;td&gt;wrong side of the Channel for this network&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt; sits below the two established exchanges and above transit. The reasoning is that an &lt;span class="caps"&gt;EVIX&lt;/span&gt; path is still a direct peering path and worth preferring over transit, but the fabric is an overlay of tunnels operated on a best-effort basis, so it should lose to a real &lt;span class="caps"&gt;IXP&lt;/span&gt; whenever both have a&amp;nbsp;route.&lt;/p&gt;
&lt;p&gt;Lagrange at 50 is the other end of the same idea, and the reason is purely geographic. Every router in &lt;span class="caps"&gt;AS201379&lt;/span&gt; sits in Germany or Switzerland, and Lagrange&amp;#8217;s PoP is on LonAP in London. For traffic to and from central Europe, sending a packet across the Channel and back is the path with the most avoidable latency, so it should never win against a Frankfurt or Zürich exit when one exists. That is not a judgement about the provider; it is a judgement about the&amp;nbsp;map.&lt;/p&gt;
&lt;p&gt;The same geography is the reason the session exists at all. A London PoP is a materially different position on the internet than Frankfurt, and a network with a presence in London has short paths to places a Frankfurt-only network does not - most obviously across the Atlantic. Nothing in the current configuration exploits that yet. When it does, it will look like the &lt;span class="caps"&gt;DTAG&lt;/span&gt; rule from Part 4: an &lt;span class="caps"&gt;AS&lt;/span&gt;-path match plus&amp;nbsp;a &lt;code&gt;match peer&lt;/code&gt;, raising local preference for a specific destination class on this specific session, rather than lifting the baseline for&amp;nbsp;everything.&lt;/p&gt;
&lt;p&gt;Its outbound route-map prepends my own &lt;span class="caps"&gt;ASN&lt;/span&gt; twice, for the same reason the local preference is&amp;nbsp;low:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;route-map RM-LAGRANGE-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
 set as-path prepend 201379 201379
exit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Local preference steers what leaves my network; prepending is the only real lever for what comes back. Both point the same direction&amp;nbsp;here.&lt;/p&gt;
&lt;h2 id="ipv4-arrives-sideways"&gt;IPv4 Arrives,&amp;nbsp;Sideways&lt;/h2&gt;
&lt;h3 id="the-situation"&gt;The&amp;nbsp;Situation&lt;/h3&gt;
&lt;p&gt;&lt;span class="caps"&gt;AS201379&lt;/span&gt; is an IPv6-only &lt;span class="caps"&gt;AS&lt;/span&gt;.&amp;nbsp;The &lt;code&gt;/48&lt;/code&gt; is mine, the &lt;span class="caps"&gt;AS&lt;/span&gt; number is mine, and there is no IPv4 allocation behind it. That is fine for the network itself and increasingly annoying for the services on it, because a meaningful share of the internet still cannot reach an IPv6-only&amp;nbsp;host.&lt;/p&gt;
&lt;p&gt;Servperso, the provider hosting&amp;nbsp;the &lt;code&gt;lobgp&lt;/code&gt; edge from Part 3, routes a small IPv4 block to my &lt;span class="caps"&gt;VM&lt;/span&gt;. It is not my address space and it is not announced by &lt;span class="caps"&gt;AS201379&lt;/span&gt;. It is a static route from their infrastructure to my edge router. Everything after that point is my&amp;nbsp;problem.&lt;/p&gt;
&lt;p&gt;The interesting part is that the block needs to be usable by services that do not live at that&amp;nbsp;edge. &lt;code&gt;radon&lt;/code&gt;, which runs the blog, is at a completely different provider, three tunnel hops&amp;nbsp;away.&lt;/p&gt;
&lt;h3 id="carrying-the-block-across-the-overlay"&gt;Carrying the Block Across the&amp;nbsp;Overlay&lt;/h3&gt;
&lt;p&gt;The path is the same overlay the IPv6 network uses, with IPv4 addresses added to the point-to-point tunnels that need to carry it.&amp;nbsp;On &lt;code&gt;lobgp&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# Tunnel to core, now dual-stack&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.50 198.51.100.10 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 10.255.255.1 10.255.255.2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffd::2 2a06:9801:1c:fffd::1 prefixlen 128&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;hobgp_v4&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_hobgp_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-net 194.28.98.216/29 10.255.255.2&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;And&amp;nbsp;on &lt;code&gt;hobgp&lt;/code&gt;, the block is split up host by host and pointed at the tunnel&amp;nbsp;to &lt;code&gt;radon&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 203.0.113.5 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 10.255.255.5 10.255.255.6&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;radon_v4 testvnet_v4 blog_v4&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_radon_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.217 10.255.255.6&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_testvnet_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.216 10.255.255.6&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_blog_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.219 10.255.255.6&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;10.255.255.0/24&lt;/code&gt; addresses are &lt;span class="caps"&gt;RFC&lt;/span&gt; 1918 link addresses on the &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnels. They carry no traffic of their own and exist purely so the kernel has an IPv4 next hop to point a route at. Routing a&amp;nbsp;public &lt;code&gt;/29&lt;/code&gt; over private point-to-point links is entirely legal and slightly unnerving the first time you do&amp;nbsp;it.&lt;/p&gt;
&lt;h3 id="the-asymmetry-problem"&gt;The Asymmetry&amp;nbsp;Problem&lt;/h3&gt;
&lt;p&gt;Here is where it stops being a routing&amp;nbsp;exercise.&lt;/p&gt;
&lt;p&gt;Inbound packets&amp;nbsp;for &lt;code&gt;194.28.98.219&lt;/code&gt; arrive at Servperso, get routed&amp;nbsp;to &lt;code&gt;lobgp&lt;/code&gt;,&amp;nbsp;cross &lt;code&gt;gif0&lt;/code&gt; to &lt;code&gt;hobgp&lt;/code&gt;,&amp;nbsp;cross &lt;code&gt;gif0&lt;/code&gt; again&amp;nbsp;to &lt;code&gt;radon&lt;/code&gt;, and are delivered. Four hops, all under my control, all&amp;nbsp;fine.&lt;/p&gt;
&lt;p&gt;The reply is the&amp;nbsp;problem. &lt;code&gt;radon&lt;/code&gt; sends it back&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; over the tunnel,&amp;nbsp;and &lt;code&gt;hobgp&lt;/code&gt; looks up the destination - some arbitrary internet address - in its IPv4 routing table. Its IPv4 default route points at Hetzner, its own hosting provider. So the reply leaves through Hetzner, with a source address&amp;nbsp;of &lt;code&gt;194.28.98.219&lt;/code&gt;, from a network that has no business originating that&amp;nbsp;address.&lt;/p&gt;
&lt;p&gt;Every reasonable network drops that packet. Hetzner&amp;#8217;s antispoofing filters see a source address that does not belong to their customer, and the connection dies&amp;nbsp;silently.&lt;/p&gt;
&lt;p&gt;The fix is a &lt;span class="caps"&gt;PF&lt;/span&gt; &lt;code&gt;route-to&lt;/code&gt; rule that overrides the routing table for exactly this&amp;nbsp;traffic:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="gh"&gt;#&lt;/span&gt; Inbound: edge (gif5) to app (gif0)
pass in  quick on gif5 inet from any to 194.28.98.216/29 no state
pass out quick on gif0 inet from any to 194.28.98.216/29 no state
pass out quick on $smallpox_tun inet from any to 194.28.98.220 no state

&lt;span class="gh"&gt;#&lt;/span&gt; Outbound: catch app replies and force them back to the edge
pass in  quick on gif0 route-to (gif5 10.255.255.1) \
    inet from 194.28.98.216/29 to any no state
pass in  quick on $smallpox_tun route-to (gif5 10.255.255.1) \
    inet from 194.28.98.216/29 to any no state
pass out quick on gif5 inet from 194.28.98.216/29 to any no state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;route-to (gif5 10.255.255.1)&lt;/code&gt; says: for packets matching this rule, ignore the routing table and send them&amp;nbsp;out &lt;code&gt;gif5&lt;/code&gt; towards &lt;code&gt;10.255.255.1&lt;/code&gt;, which&amp;nbsp;is &lt;code&gt;lobgp&lt;/code&gt;&lt;span class="quo"&gt;&amp;#8216;&lt;/span&gt;s end of the tunnel. The match condition is the source address, not the destination, which is what makes it work for arbitrary&amp;nbsp;destinations.&lt;/p&gt;
&lt;p&gt;Note that every one of these rules&amp;nbsp;is &lt;code&gt;no state&lt;/code&gt;. Keeping state on a path this asymmetric is how you produce a firewall that works for thirty seconds and then does not. The forward and return packets traverse different interfaces on this router in a way &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s state tracking cannot follow, so state is simply not used. This is the same principle the IPv6 transit rules have followed since Part 2, applied to a case where the asymmetry is deliberate rather than&amp;nbsp;incidental.&lt;/p&gt;
&lt;h3 id="what-this-buys"&gt;What This&amp;nbsp;Buys&lt;/h3&gt;
&lt;p&gt;The site you are reading this on is served&amp;nbsp;from &lt;code&gt;194.28.98.219&lt;/code&gt;, a globally routed IPv4 address delivered to a jail three tunnel hops away from the network that owns it. No proxy, no &lt;span class="caps"&gt;CDN&lt;/span&gt;, no &lt;span class="caps"&gt;NAT&lt;/span&gt; in front of it. The &lt;span class="caps"&gt;AS&lt;/span&gt; carries&amp;nbsp;the &lt;code&gt;/29&lt;/code&gt; the same way it carries&amp;nbsp;the &lt;code&gt;/48&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Structurally it is also a demonstration that the overlay works.&amp;nbsp;The &lt;code&gt;/29&lt;/code&gt; is transported by exactly the same tunnels, in exactly the same topology, as the IPv6 network. Adding a second address family took new routes and one policy-routing rule, not a new&amp;nbsp;network.&lt;/p&gt;
&lt;p&gt;The catch is that the complexity does not disappear, it moves. On the core router it is&amp;nbsp;one &lt;code&gt;route-to&lt;/code&gt; rule. At the far end of the tunnel, it is the most involved firewall configuration in the entire &lt;span class="caps"&gt;AS&lt;/span&gt;.&lt;/p&gt;
&lt;h2 id="the-far-end-dual-fib-routing-on-radon"&gt;The Far End: Dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; Routing on&amp;nbsp;radon&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;radon&lt;/code&gt; is the FreeBSD hypervisor host that runs the blog, &lt;span class="caps"&gt;DNS&lt;/span&gt;, and the rest of the public-facing services in jails. It is also where the IPv4 story gets genuinely awkward, because it has two uplinks that have nothing to do with each&amp;nbsp;other:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;vtnet0&lt;/code&gt;&lt;/strong&gt;, its physical interface at Netcup, with a provider-assigned IPv4 address and a provider default&amp;nbsp;route&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;gif0&lt;/code&gt;&lt;/strong&gt;, the tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;, which is how &lt;span class="caps"&gt;AS201379&lt;/span&gt; reaches&amp;nbsp;it&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Both are real internet uplinks. Neither can be turned off: the tunnel needs the physical interface to exist at all, and some services deliberately want to answer on the Netcup&amp;nbsp;address.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Part 1&lt;/a&gt; introduced the mechanism that makes this survivable: FreeBSD&amp;#8217;s multiple forwarding information bases. Three parts later, it has grown into something worth documenting&amp;nbsp;properly.&lt;/p&gt;
&lt;h3 id="two-routing-tables-one-interface-each"&gt;Two Routing Tables, One Interface&amp;nbsp;Each&lt;/h3&gt;
&lt;p&gt;FreeBSD can run several independent routing tables and assign an interface to one of&amp;nbsp;them. &lt;code&gt;radon&lt;/code&gt; uses&amp;nbsp;two:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# FIB 0: the physical uplink (Netcup)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 203.0.113.5 netmask 255.255.252.0 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;203.0.113.1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:4cc0:2f90::2 prefixlen 64&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fe80::1%vtnet0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# FIB 1: the BGP tunnel&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;bridge0 gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib 1 tunnel 203.0.113.5 198.51.100.10 tunnelfib 0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 10.255.255.6 10.255.255.5&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:ffff::2 2a06:9801:1c:ffff::1 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The single most important line in this file is&amp;nbsp;the &lt;code&gt;gif0&lt;/code&gt; one, and specifically the combination&amp;nbsp;of &lt;code&gt;fib 1&lt;/code&gt; and &lt;code&gt;tunnelfib 0&lt;/code&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;fib 1&lt;/code&gt;&lt;/strong&gt; puts the interface, and therefore the inner payload, in routing table&amp;nbsp;1.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;tunnelfib 0&lt;/code&gt;&lt;/strong&gt; tells the kernel to resolve the &lt;em&gt;outer&lt;/em&gt; IPv4 encapsulation - the proto-41 packets addressed&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; - using routing table&amp;nbsp;0.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Without the split, this is a chicken-and-egg problem. The tunnel&amp;#8217;s own packets need to&amp;nbsp;reach &lt;code&gt;hobgp&lt;/code&gt; over the physical uplink, but if the tunnel interface lives in the same routing table as the physical one, and that table&amp;#8217;s default route points into the tunnel, the encapsulated packets try to route themselves into the tunnel they are supposed to be building. Two FIBs break the loop cleanly: the outer wrapper uses Netcup, the inner payload uses &lt;span class="caps"&gt;AS201379&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;The second-most-important line&amp;nbsp;is &lt;code&gt;-lro -tso&lt;/code&gt;. Large receive offload and &lt;span class="caps"&gt;TCP&lt;/span&gt; segmentation offload on a virtio &lt;span class="caps"&gt;NIC&lt;/span&gt; hand &lt;span class="caps"&gt;PF&lt;/span&gt; packets that do not look like what actually goes on the wire, and the result is that &lt;span class="caps"&gt;NAT&lt;/span&gt; and checksum rewriting silently produce corrupt output. Every FreeBSD router in this series disables both. It is the kind of thing that costs an evening exactly&amp;nbsp;once.&lt;/p&gt;
&lt;h3 id="fib-1-needs-a-complete-map"&gt;&lt;span class="caps"&gt;FIB&lt;/span&gt; 1 Needs a Complete&amp;nbsp;Map&lt;/h3&gt;
&lt;p&gt;A second routing table is not a filter, it is a whole separate routing table, and it starts empty. Anything &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 is expected to reach has to be in&amp;nbsp;it:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib1_default_v4 fib1_jailnet_v4 fib1_host217 fib1_jail219 fib0_jail219&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib1_default fib1_jailnet&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# 1. How FIB 1 reaches the internet&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_fib1_default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-6 default -interface gif0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_fib1_default_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-inet default 10.255.255.5 -fib 1&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# 2. How FIB 1 finds the jail bridge&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_fib1_jailnet&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-6 2a06:9801:1c:1000::/64 -interface bastille0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_fib1_jailnet_v4&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-net 10.254.254.0/24 -interface bastille0 -fib 1&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# 3. Where FIB 1 finds specific public IPv4 addresses&lt;/span&gt;
&lt;span class="nv"&gt;route_fib1_host217&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.217 -interface lo0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_fib1_jail219&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.219 -interface bastille0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_fib0_jail219&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-host 194.28.98.219 -interface bastille0&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Group 2 is the part people forget. If &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 has a default route and nothing else, a packet arriving from the tunnel for a jail on the local bridge follows the default route straight back out the tunnel. The local subnets have to be present in &lt;em&gt;both&lt;/em&gt; tables, or locally destined traffic&amp;nbsp;loops.&lt;/p&gt;
&lt;p&gt;The last two lines are the same host route in both tables, and that is&amp;nbsp;deliberate: &lt;code&gt;194.28.98.219&lt;/code&gt; must be reachable whether the packet arrived over the tunnel (&lt;span class="caps"&gt;FIB&lt;/span&gt; 1) or is being delivered from something in &lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;0.&lt;/p&gt;
&lt;h3 id="three-kinds-of-jail"&gt;Three Kinds of&amp;nbsp;Jail&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;bastille0&lt;/code&gt; bridge carries three different networking paradigms simultaneously, and the difference between them is entirely a matter of policy&amp;nbsp;routing:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Paradigm&lt;/th&gt;
&lt;th&gt;Address&lt;/th&gt;
&lt;th&gt;How it gets out&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span class="caps"&gt;NAT&lt;/span&gt;&amp;#8217;d IPv4&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.254.254.0/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;NAT&lt;/span&gt;&amp;nbsp;to &lt;code&gt;194.28.98.217&lt;/code&gt;, then forced into the tunnel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Routed IPv6&lt;/td&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:1000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;natively routed out the tunnel via &lt;span class="caps"&gt;FIB&lt;/span&gt; 1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Routed IPv4&lt;/td&gt;
&lt;td&gt;&lt;code&gt;194.28.98.219&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;natively routed out the tunnel via &lt;span class="caps"&gt;FIB&lt;/span&gt; 1&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The &lt;span class="caps"&gt;PF&lt;/span&gt; rules that implement this are three lines, one per&amp;nbsp;paradigm:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="gh"&gt;#&lt;/span&gt; IPv6: forced into FIB 1. FIB 1 knows the local bridge route, so jail-to-jail
&lt;span class="gh"&gt;#&lt;/span&gt; traffic loops back locally while internet traffic goes out the tunnel.
pass in quick on bastille0 inet6 from $bgp_net_v6 to any rtable 1 keep state

&lt;span class="gh"&gt;#&lt;/span&gt; IPv4 (NAT&amp;#39;d): deliberately left in FIB 0 so it hits the nat rule on $ext_if
pass in quick on bastille0 inet from &amp;lt;jails_v4&amp;gt; to ! $jail_net keep state

&lt;span class="gh"&gt;#&lt;/span&gt; IPv4 (pure routed): forced straight into FIB 1
pass in quick on bastille0 inet from $frontend_v4 to any rtable 1 keep state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;rtable 1&lt;/code&gt; is &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s instruction to perform the route lookup in a different table. It is the counterpart&amp;nbsp;to &lt;code&gt;route-to&lt;/code&gt;: &lt;code&gt;route-to&lt;/code&gt; names an interface and a next hop explicitly,&amp;nbsp;while &lt;code&gt;rtable&lt;/code&gt; says &amp;#8220;look it up, but over there&amp;#8221;. For jail&amp;nbsp;egress &lt;code&gt;rtable&lt;/code&gt; is the better fit, because &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 already contains the correct answer for every destination, including the local&amp;nbsp;ones.&lt;/p&gt;
&lt;p&gt;The &lt;span class="caps"&gt;NAT&lt;/span&gt;&amp;#8217;d path is the odd one out and the comment in the ruleset earns its place. That traffic is left in &lt;span class="caps"&gt;FIB&lt;/span&gt; 0 on purpose, so it reaches the &lt;span class="caps"&gt;NAT&lt;/span&gt; rule on the physical&amp;nbsp;interface:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;jails_v4&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;host_v4_routed&lt;/span&gt;

&lt;span class="cp"&gt;# The safety net: catch the NAT&amp;#39;d packets on their way out the physical&lt;/span&gt;
&lt;span class="cp"&gt;# interface and forcefully bounce them into the tunnel.&lt;/span&gt;
&lt;span class="n"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;tun_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;tun_gw_v4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;host_v4_routed&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;span class="caps"&gt;NAT&lt;/span&gt; is applied&amp;nbsp;on &lt;code&gt;$ext_if&lt;/code&gt; because that is where &lt;span class="caps"&gt;PF&lt;/span&gt; tracks the state accurately, and the packet is then hijacked into the tunnel by&amp;nbsp;a &lt;code&gt;route-to&lt;/code&gt; rule at the last possible moment. It reads like a hack because it is one, but the alternative - &lt;span class="caps"&gt;NAT&lt;/span&gt; directly on the tunnel interface - produces state entries that do not survive the &lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;boundary.&lt;/p&gt;
&lt;h3 id="reply-to-on-every-tunnel-ingress-rule"&gt;reply-to on Every Tunnel Ingress&amp;nbsp;Rule&lt;/h3&gt;
&lt;p&gt;Inbound traffic has the mirror-image problem to the core router&amp;#8217;s. A packet arriving&amp;nbsp;on &lt;code&gt;gif0&lt;/code&gt; for a jail will, by default, be answered according to whichever routing table the reply lands in, which may well send it out Netcup with a source address from&amp;nbsp;the &lt;code&gt;/29&lt;/code&gt;. Every ingress rule on the tunnel therefore carries an&amp;nbsp;explicit &lt;code&gt;reply-to&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="gh"&gt;#&lt;/span&gt; IPv4 tunnel ingress
pass in quick on $tun_if reply-to ($tun_if $tun_gw_v4) \
    inet proto { tcp, udp } to $frontend_v4 port { 80, 443 } keep state

&lt;span class="gh"&gt;#&lt;/span&gt; IPv6 tunnel ingress
pass in quick on $tun_if reply-to ($tun_if $bgp_hub_ip) \
    inet6 proto { tcp, udp } to $frontend_v6 port { 80, 443 } keep state
pass in quick on $tun_if reply-to ($tun_if $bgp_hub_ip) \
    inet6 proto { tcp, udp } to $powerdns_v6 port 53 keep state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;reply-to&lt;/code&gt; pins the return path of a &lt;em&gt;stateful&lt;/em&gt; connection to a specific interface and next hop. This is the opposite choice from the core router, which&amp;nbsp;uses &lt;code&gt;no state&lt;/code&gt; throughout, and both are correct for their position. The core forwards transit traffic whose two directions genuinely traverse different interfaces, so state is&amp;nbsp;impossible. &lt;code&gt;radon&lt;/code&gt; terminates connections, so state exists and is useful; what it needs is for the reply to ignore the routing table and go back the way the request&amp;nbsp;came.&lt;/p&gt;
&lt;p&gt;The rule of thumb that falls out of this: &lt;strong&gt;forwarders&amp;nbsp;use &lt;code&gt;no state&lt;/code&gt;, endpoints&amp;nbsp;use &lt;code&gt;reply-to&lt;/code&gt;.&lt;/strong&gt; Getting that backwards produces a host that answers the first packet of every connection and then goes&amp;nbsp;quiet.&lt;/p&gt;
&lt;h3 id="the-two-extras"&gt;The Two&amp;nbsp;Extras&lt;/h3&gt;
&lt;p&gt;Two more things on this host are worth mentioning, both of which exist because it has two uplinks rather than&amp;nbsp;one.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;DNS&lt;/span&gt; answers on the Netcup address, on purpose.&lt;/strong&gt; The authoritative nameserver is redirected from the physical interface, not the&amp;nbsp;tunnel:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;rdr&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;udp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;53&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;powerdns_v4&lt;/span&gt;
&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;powerdns_v4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;host_v4_netcup&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The nameserver for a domain should not depend on the availability of the network whose addresses it is serving. Keeping &lt;span class="caps"&gt;DNS&lt;/span&gt; on the provider&amp;#8217;s address and its own &lt;span class="caps"&gt;NAT&lt;/span&gt; rule means a broken tunnel takes the services down but leaves the zone resolvable, which is a considerably better failure mode than the&amp;nbsp;alternative.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;NAT64&lt;/span&gt; for IPv6-only jails.&lt;/strong&gt; Jails that have no IPv4 address at all still occasionally need to reach an IPv4-only service. &lt;span class="caps"&gt;PF&lt;/span&gt; handles that in one&amp;nbsp;rule:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;pass in quick on bastille0 inet6 from $bgp_net_v6 to 64:ff9b::/96 \
    af-to inet from ($ext_if) keep state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;af-to&lt;/code&gt; is &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s address-family translation. Traffic to the well-known &lt;span class="caps"&gt;NAT64&lt;/span&gt;&amp;nbsp;prefix &lt;code&gt;64:ff9b::/96&lt;/code&gt; is rewritten into IPv4 and sent out the physical interface. No Jool, no Tayga, no separate daemon: one rule, and IPv6-only jails get IPv4&amp;nbsp;egress.&lt;/p&gt;
&lt;p&gt;Between the two&amp;nbsp;FIBs, &lt;code&gt;rtable&lt;/code&gt;, &lt;code&gt;route-to&lt;/code&gt;, &lt;code&gt;reply-to&lt;/code&gt; and &lt;code&gt;af-to&lt;/code&gt;, this single host uses more of &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s routing machinery than the four &lt;span class="caps"&gt;BGP&lt;/span&gt; routers combined. That is not an accident of taste. It is what happens when a machine has to be simultaneously a normal &lt;span class="caps"&gt;VPS&lt;/span&gt; at its hosting provider and a routed site inside somebody else&amp;#8217;s autonomous&amp;nbsp;system.&lt;/p&gt;
&lt;h2 id="downstream-wireguard-for-the-home-router"&gt;Downstream: WireGuard for the Home&amp;nbsp;Router&lt;/h2&gt;
&lt;p&gt;Part 4 connected the home MikroTik over a 6to4 tunnel (protocol 41)&amp;nbsp;at &lt;code&gt;2a06:9801:1c:fff0::/64&lt;/code&gt;. That worked, with one structural weakness: protocol 41 is unencrypted, unauthenticated, and gets dropped or mangled by a nontrivial number of consumer ISPs and hotel&amp;nbsp;networks.&lt;/p&gt;
&lt;p&gt;The downstream link now runs over WireGuard&amp;nbsp;instead:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif if_gre if_wg if_vxlan&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;wireguard_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;wireguard_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;wg-smallpox&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;wg-smallpox: flags=10080c1&amp;lt;UP,RUNNING,NOARP,MULTICAST,LOWER_UP&amp;gt; metric 0 mtu 1420
        inet6 2a06:9801:1c:fff4:: prefixlen 127
        groups: wg
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;/127&lt;/code&gt; on the link is the &lt;span class="caps"&gt;RFC&lt;/span&gt; 6164 recommendation for point-to-point IPv6 links, and it is a cleaner fit than&amp;nbsp;the &lt;code&gt;/128&lt;/code&gt;-pair convention the &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnels use. The iBGP session and the routing policy are otherwise identical to Part&amp;nbsp;4:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;ipv6 prefix-list PL-MIKROTIK-NET seq 5 permit 2a06:9801:1c:6000::/64
ipv6 prefix-list PL-MIKROTIK-NET seq 10 permit 2a06:9801:1c:6001::/64
!
route-map RM-DOWNSTREAM-IN permit 10
 match ipv6 address prefix-list PL-MIKROTIK-NET
exit
!
route-map RM-DEFAULT-OUT deny 10
exit
!
router bgp 201379
 neighbor 2a06:9801:1c:fff4::1 remote-as 201379
 neighbor 2a06:9801:1c:fff4::1 description Downstream-MikroTik
 !
 address-family ipv6 unicast
  neighbor 2a06:9801:1c:fff4::1 activate
  neighbor 2a06:9801:1c:fff4::1 default-originate
  neighbor 2a06:9801:1c:fff4::1 route-map RM-DOWNSTREAM-IN in
  neighbor 2a06:9801:1c:fff4::1 route-map RM-DEFAULT-OUT out
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The downstream gets a synthetic default route and nothing else; it is allowed to announce two&amp;nbsp;specific &lt;code&gt;/64&lt;/code&gt;s and nothing else. The second prefix&amp;nbsp;(&lt;code&gt;:6001::/64&lt;/code&gt;) is the reason for the change: a second segment behind the same router, which is much easier to justify adding when the transport is a protocol that survives arbitrary&amp;nbsp;networks.&lt;/p&gt;
&lt;p&gt;&lt;span class="caps"&gt;PF&lt;/span&gt; gets one macro and one&amp;nbsp;clamp:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;smallpox_tun = &amp;quot;wg-smallpox&amp;quot;
wg_peers_v4  = &amp;quot;{ 203.0.113.69, 203.0.113.118 }&amp;quot;

scrub on { $smallpox_tun } max-mss 1360

pass in quick on $ext_if proto udp from $wg_peers_v4 to ($ext_if) port { 51820, 51821 } keep state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;WireGuard&amp;#8217;s own overhead puts the usable &lt;span class="caps"&gt;MTU&lt;/span&gt; at 1420, so the &lt;span class="caps"&gt;MSS&lt;/span&gt; clamp lands at 1360. Unlike &lt;span class="caps"&gt;GRE&lt;/span&gt; and proto-41, the outer flow here is &lt;span class="caps"&gt;UDP&lt;/span&gt; with a real handshake, so keeping state on it is&amp;nbsp;correct.&lt;/p&gt;
&lt;h2 id="the-spring-clean"&gt;The Spring&amp;nbsp;Clean&lt;/h2&gt;
&lt;p&gt;Four routers accumulated four routers&amp;#8217; worth of drift. None of the following changed how a single packet is forwarded. All of it made the configuration possible to&amp;nbsp;read.&lt;/p&gt;
&lt;h3 id="one-table-for-every-bgp-peer"&gt;One Table for Every &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Peer&lt;/h3&gt;
&lt;p&gt;The core router&amp;#8217;s &lt;span class="caps"&gt;PF&lt;/span&gt; ruleset used to carry&amp;nbsp;one &lt;code&gt;pass&lt;/code&gt; rule per &lt;span class="caps"&gt;BGP&lt;/span&gt; session. Peer addresses now live in macros, and the macros feed a single&amp;nbsp;table:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;peer_lagrange&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fd:0fca::1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_route64&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a11:6c7:f00:119::1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_freetransit&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a01:20e:1001:27::1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_vultr&lt;/span&gt;&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fffe::2&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_lobgp&lt;/span&gt;&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fffd::2&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_ixbgp&lt;/span&gt;&lt;span class="w"&gt;       &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fffc::2&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_smallpox&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fff4::1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_evix_fmt&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2602:fed2:fff:ffff::1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_evix_ams&lt;/span&gt;&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2602:fed2:fff:ffff::253&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_lagrange&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_route64&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_freetransit&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_vultr&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_lobgp&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_ixbgp&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_smallpox&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_evix_fmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_evix_ams&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Adding a peer is now one macro and one table entry rather than a pair of rules copy-pasted from the peer above it.&amp;nbsp;The &lt;code&gt;const&lt;/code&gt; keyword makes the table immutable at runtime, so nothing can add an address to it&amp;nbsp;via &lt;code&gt;pfctl&lt;/code&gt; later.&lt;/p&gt;
&lt;h3 id="deprecating-the-transit-addresses"&gt;Deprecating the Transit&amp;nbsp;Addresses&lt;/h3&gt;
&lt;p&gt;The three &lt;span class="caps"&gt;GRE&lt;/span&gt; transit tunnels carry provider-assigned IPv6 addresses. Those addresses exist to bring up a &lt;span class="caps"&gt;BGP&lt;/span&gt; session and for nothing else, so the interfaces are configured with them&amp;nbsp;marked &lt;code&gt;deprecated&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;ifconfig_gre1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 fd:0fca::2 prefixlen 64 deprecated&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre2_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a11:6c7:f00:119::2 prefixlen 64 deprecated&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre3_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a01:20e:1001:27::2/64 prefixlen 64 deprecated&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;deprecated&lt;/code&gt; is an &lt;span class="caps"&gt;RFC&lt;/span&gt; 4862 address state that means &amp;#8220;still valid for existing connections, never select me as a source address for a new one&amp;#8221;. The effect is that anything the router originates picks&amp;nbsp;the &lt;code&gt;/48&lt;/code&gt; loopback instead, without needing a policy rule to make it happen. It complements&amp;nbsp;the &lt;code&gt;nat-to&lt;/code&gt; identity rule from Part 4 rather than replacing it, and that rule now covers the &lt;span class="caps"&gt;VXLAN&lt;/span&gt; interface as&amp;nbsp;well:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;match out on { gre1, gre2, gre3, gif0, gif1, gif2, gif3, gif4, gif5, gif6, vxlan0 } \
    inet6 from 2001:db8:1c19:3ee1::1 to any nat-to 2a06:9801:1c::1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3 id="prefix-lists-interface-routes-and-daemons"&gt;Prefix Lists, Interface Routes, and&amp;nbsp;Daemons&lt;/h3&gt;
&lt;p&gt;Three smaller cleanups worth&amp;nbsp;mentioning:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The bogon list is identical on all four&amp;nbsp;routers.&lt;/strong&gt; &lt;code&gt;PL-BOGONS&lt;/code&gt; ends&amp;nbsp;with &lt;code&gt;deny ::/0 ge 49&lt;/code&gt; followed&amp;nbsp;by &lt;code&gt;permit ::/0 le 48&lt;/code&gt;, which rejects anything longer than&amp;nbsp;a &lt;code&gt;/48&lt;/code&gt; and accepts everything else that survived the preceding twenty-odd lines. Every router runs the same list, byte for byte. Divergence in a filter that is supposed to be universal is the kind of bug that only shows up in one direction of one&amp;nbsp;path.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Interface routes for the iBGP next&amp;nbsp;hops.&lt;/strong&gt; &lt;code&gt;hobgp&lt;/code&gt; carries&amp;nbsp;explicit &lt;code&gt;/128&lt;/code&gt; routes to each edge&amp;#8217;s tunnel&amp;nbsp;address:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;ipv6 route 2a06:9801:1c:fffc::2/128 gif6
ipv6 route 2a06:9801:1c:fffd::2/128 gif5
ipv6 route 2a06:9801:1c:fffe::2/128 gif1
ipv6 route 2a06:9801:1c:fff4::1/128 wg-smallpox
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;These are technically redundant with the connected routes the tunnel interfaces install, and they cost nothing. What they buy is that the iBGP session endpoints are named explicitly in the configuration,&amp;nbsp;so &lt;code&gt;show ipv6 route&lt;/code&gt; answers &amp;#8220;which tunnel is this peer on&amp;#8221; without&amp;nbsp;cross-referencing &lt;code&gt;ifconfig&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Trimmed &lt;span class="caps"&gt;FRR&lt;/span&gt; daemons.&lt;/strong&gt; Every router runs the same short&amp;nbsp;list:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;frr_daemons&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;mgmtd zebra bgpd bfdd staticd&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;No &lt;span class="caps"&gt;OSPF&lt;/span&gt;, no &lt;span class="caps"&gt;RIP&lt;/span&gt;, no &lt;span class="caps"&gt;PIM&lt;/span&gt;, no &lt;span class="caps"&gt;BABEL&lt;/span&gt;. The &lt;span class="caps"&gt;FRR&lt;/span&gt; exporter is scoped to&amp;nbsp;match:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;frr_exporter_args&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;--collector.bgp6 --no-collector.ospf --no-collector.bfd \&lt;/span&gt;
&lt;span class="s2"&gt;    --web.listen-address=[2a06:9801:1c::1]:9342&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Running a routing daemon you do not use is a listening socket and an attack surface for no&amp;nbsp;benefit.&lt;/p&gt;
&lt;h2 id="monitoring-fourteen-sessions-on-one-screen"&gt;Monitoring: Fourteen Sessions on One&amp;nbsp;Screen&lt;/h2&gt;
&lt;p&gt;Every router&amp;nbsp;runs &lt;code&gt;node_exporter&lt;/code&gt; and &lt;code&gt;frr_exporter&lt;/code&gt;, bound to an address inside&amp;nbsp;the &lt;code&gt;/48&lt;/code&gt; and reachable only over the&amp;nbsp;overlay:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;node_exporter_listen_address&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;[2a06:9801:1c::1]:9100&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_exporter_args&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;--collector.bgp6 ... --web.listen-address=[2a06:9801:1c::1]:9342&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;span class="caps"&gt;PF&lt;/span&gt; rules around them follow an allow-then-block pattern that is worth&amp;nbsp;copying:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# Allow Prometheus -&amp;gt; exporters from two specific scrapers&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="n"&gt;a06&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;9801&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="n"&gt;a06&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;9801&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="mi"&gt;25&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;9100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;9342&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;9187&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# Block those ports for the entire network, on every interface&lt;/span&gt;
&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;9100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;9342&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Order matters&amp;nbsp;and &lt;code&gt;quick&lt;/code&gt; matters. The allow rule matches first for the two scrapers and stops evaluation; everything else hits the block. Crucially the block rule sits &lt;em&gt;above&lt;/em&gt; the stateless transit rules, so the exporters are protected not just on the router itself but for every downstream site whose traffic passes through it. A jail three hops away that carelessly&amp;nbsp;binds &lt;code&gt;node_exporter&lt;/code&gt; to a public address is covered by a rule on the core&amp;nbsp;router.&lt;/p&gt;
&lt;p&gt;&lt;img alt="Grafana BGP Status dashboard" src="https://hofstede.it/images/2026-07-30-as201379-part5-grafana-bgp-status.png" title="Fourteen BGP sessions, one dashboard"&gt;&lt;/p&gt;
&lt;p&gt;The dashboard is deliberately boring. The top-left panel is fourteen stat tiles, one per eBGP session across all four routers, each of which is either &lt;span class="caps"&gt;UP&lt;/span&gt; in green or very obviously not. The three graphs underneath are the ones that actually diagnose&amp;nbsp;things:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Full Table Received&lt;/strong&gt; plots received-prefix count per session. The full-table sessions sit flat at roughly 250,000 each; the &lt;span class="caps"&gt;IXP&lt;/span&gt; route servers sit far below. The useful signal is not the absolute number, it is the shape. Every notch in that line is a session that reset and reconverged, and a session that resets repeatedly at the same time of day is a tunnel problem, not a &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;problem.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; Update Rate&lt;/strong&gt; is normally near zero with occasional spikes into the hundreds of updates per second. Those spikes are somebody else&amp;#8217;s network having a bad afternoon, propagated to me. What matters is that they&amp;nbsp;end.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Interface Packet Drops&lt;/strong&gt; is flat at zero, which is what a tunnel-based network wants to see. Drops on a &lt;span class="caps"&gt;GIF&lt;/span&gt; or &lt;span class="caps"&gt;VXLAN&lt;/span&gt; interface usually mean &lt;span class="caps"&gt;MTU&lt;/span&gt;, and &lt;span class="caps"&gt;MTU&lt;/span&gt; problems on an overlay are painful to find after the&amp;nbsp;fact.&lt;/p&gt;
&lt;p&gt;Fourteen sessions is about the point at which &amp;#8220;check whether the sessions are up&amp;#8221; stops being something you can do by&amp;nbsp;hand. &lt;code&gt;vtysh -c 'show bgp ipv6 unicast summary'&lt;/code&gt; on four routers is four &lt;span class="caps"&gt;SSH&lt;/span&gt; sessions and a lot of squinting; one dashboard is a&amp;nbsp;glance.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;An overlay &lt;span class="caps"&gt;IXP&lt;/span&gt; is a legitimate way to join an exchange.&lt;/strong&gt; &lt;span class="caps"&gt;EVIX&lt;/span&gt; gives a small network a real peering &lt;span class="caps"&gt;LAN&lt;/span&gt;, real route servers, and real direct paths without a port, a facility, or a second machine. The prefix counts are modest and the participants are hobbyists, but the mechanics are identical to a physical exchange. If the interesting part of joining an &lt;span class="caps"&gt;IXP&lt;/span&gt; is learning to operate one, the tunnel version teaches exactly the same&amp;nbsp;lesson.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;FreeBSD&amp;#8217;s &lt;code&gt;vxlan(4)&lt;/code&gt; is a&amp;nbsp;three-line &lt;code&gt;rc.conf&lt;/code&gt; entry.&lt;/strong&gt; No userspace daemon, no kernel module&amp;nbsp;beyond &lt;code&gt;if_vxlan&lt;/code&gt;, no configuration file.&amp;nbsp;Between &lt;code&gt;gif&lt;/code&gt;, &lt;code&gt;gre&lt;/code&gt;, &lt;code&gt;wg&lt;/code&gt; and &lt;code&gt;vxlan&lt;/code&gt;, FreeBSD covers every tunnel type this network needs natively, which is a large part of why the network is built on&amp;nbsp;it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;maximum-prefix&lt;/code&gt; is only useful if the number is realistic.&lt;/strong&gt; A 350,000-prefix ceiling on a session that should carry 400 prefixes is not a safety mechanism, it is a comment. Setting it near the expected value is what makes it trip before a misconfigured peer fills the &lt;span class="caps"&gt;RIB&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Asymmetric routing needs stateless filtering and&amp;nbsp;explicit &lt;code&gt;route-to&lt;/code&gt;.&lt;/strong&gt; The&amp;nbsp;IPv4 &lt;code&gt;/29&lt;/code&gt; works because &lt;span class="caps"&gt;PF&lt;/span&gt; is told, in one rule, to ignore the routing table for traffic sourced from that block. Every attempt to solve this with &lt;span class="caps"&gt;NAT&lt;/span&gt; or with stateful rules produces something that appears to work and then fails on the third packet of a long-lived&amp;nbsp;connection.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Forwarders&amp;nbsp;use &lt;code&gt;no state&lt;/code&gt;, endpoints&amp;nbsp;use &lt;code&gt;reply-to&lt;/code&gt;.&lt;/strong&gt; The core router&amp;nbsp;and &lt;code&gt;radon&lt;/code&gt; face the same problem from opposite ends and need opposite answers. A router forwarding transit traffic cannot keep state, because the two directions legitimately traverse different interfaces. A host terminating connections must keep state and pin the return path instead. Applying either rule in the other&amp;#8217;s position produces a firewall that works just long enough to look&amp;nbsp;correct.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;tunnelfib&lt;/code&gt; is the line that makes dual-uplink hosts possible.&lt;/strong&gt; A tunnel interface whose inner payload lives in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 and whose outer encapsulation resolves in &lt;span class="caps"&gt;FIB&lt;/span&gt; 0 is the entire trick behind attaching a &lt;span class="caps"&gt;VPS&lt;/span&gt; at one provider to an autonomous system at another. It is one keyword&amp;nbsp;in &lt;code&gt;rc.conf&lt;/code&gt;, and without it the tunnel tries to route itself through&amp;nbsp;itself.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;deprecated&lt;/code&gt; is an underused address flag.&lt;/strong&gt; One keyword&amp;nbsp;in &lt;code&gt;rc.conf&lt;/code&gt; stops a provider-assigned address from ever being chosen as a source, with no policy routing and no &lt;span class="caps"&gt;NAT&lt;/span&gt; involved. On a router with eleven interfaces and addresses from six different providers, that is worth&amp;nbsp;knowing.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Cleanup is a feature.&lt;/strong&gt; Consolidating peer addresses into a table and a set of macros did not change a single forwarding decision. It did mean that adding &lt;span class="caps"&gt;EVIX&lt;/span&gt; afterwards took two lines instead of six, and that the ruleset can be read top to bottom without cross-referencing. Every configuration cleanup pays for itself at the next change, which is precisely why it never feels urgent at the&amp;nbsp;time.&lt;/p&gt;
&lt;div class="about-cta"&gt;
&lt;p&gt;&lt;strong&gt;Interested in peering with &lt;span class="caps"&gt;AS201379&lt;/span&gt;?&lt;/strong&gt;
More details about the network, including upstreams, IXPs, and policy, live at &lt;a href="https://hofstede.it/as201379.html"&gt;hofstede.it/as201379.html&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;If you&amp;#8217;d like to establish a peering session - over a tunnel, or on the shared fabric at &lt;strong&gt;LocIX Düsseldorf&lt;/strong&gt;, &lt;strong&gt;FogIXP Frankfurt&lt;/strong&gt; or &lt;strong&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt;&lt;/strong&gt; - send a mail to &lt;a href="mailto:peering@hofstede.it"&gt;peering@hofstede.it&lt;/a&gt;. Hobbyists, small ASNs, and experimental setups are explicitly&amp;nbsp;welcome.&lt;/p&gt;
&lt;/div&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;Six months in, the shape of &lt;span class="caps"&gt;AS201379&lt;/span&gt; has stopped changing. Four routers, a core and three edges, a fabric of tunnels, and a set of downstream sites that each get&amp;nbsp;a &lt;code&gt;/64&lt;/code&gt; and their own opinions about firewalls. What changes now is what gets attached to that&amp;nbsp;shape.&lt;/p&gt;
&lt;p&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt; attached an entire internet exchange to it using a &lt;span class="caps"&gt;UDP&lt;/span&gt; socket. The&amp;nbsp;IPv4 &lt;code&gt;/29&lt;/code&gt; attached a second address family using three static routes and one policy rule. The MikroTik moved from one tunnel protocol to another without the routing policy noticing. In each case the network absorbed the change rather than being redesigned around it, which is the actual return on the architecture work from Parts 1 through&amp;nbsp;4.&lt;/p&gt;
&lt;p&gt;The remaining honest assessment: this is still a network with no IPv4 allocation of its own, whose largest peer is a hosting company, and whose total traffic would not register as noise at a real &lt;span class="caps"&gt;IXP&lt;/span&gt;. But it now peers on three exchanges across two continents, carries both address families, and tells me within a minute when any of it breaks. For a project that started as one &lt;span class="caps"&gt;VM&lt;/span&gt; and one &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel, that is a reasonable place to&amp;nbsp;be.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://evix.org/"&gt;&lt;span class="caps"&gt;EVIX&lt;/span&gt; - Experimental Virtual Internet Exchange&lt;/a&gt; - the overlay &lt;span class="caps"&gt;IXP&lt;/span&gt; joined in this&amp;nbsp;article&lt;/li&gt;
&lt;li&gt;&lt;a href="https://man.freebsd.org/cgi/man.cgi?vxlan(4)"&gt;FreeBSD &lt;code&gt;vxlan(4)&lt;/code&gt; manual page&lt;/a&gt; - interface parameters&amp;nbsp;and &lt;code&gt;create_args_&lt;/code&gt; syntax&lt;/li&gt;
&lt;li&gt;&lt;a href="https://man.freebsd.org/cgi/man.cgi?pf.conf(5)"&gt;FreeBSD &lt;code&gt;pf.conf(5)&lt;/code&gt;&lt;/a&gt;&amp;nbsp;- &lt;code&gt;route-to&lt;/code&gt;, &lt;code&gt;scrub max-mss&lt;/code&gt;, tables and&amp;nbsp;macros&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc4862"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 4862&lt;/a&gt; - IPv6 stateless address autoconfiguration, including&amp;nbsp;the &lt;code&gt;deprecated&lt;/code&gt; state&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc6164"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 6164&lt;/a&gt; - using /127 prefixes on inter-router&amp;nbsp;links&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc7348"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 7348&lt;/a&gt; - &lt;span class="caps"&gt;VXLAN&lt;/span&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc6052"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 6052&lt;/a&gt; - the well-known &lt;span class="caps"&gt;NAT64&lt;/span&gt;&amp;nbsp;prefix &lt;code&gt;64:ff9b::/96&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://man.freebsd.org/cgi/man.cgi?setfib(1)"&gt;FreeBSD &lt;code&gt;setfib(1)&lt;/code&gt;&lt;/a&gt; - multiple forwarding information&amp;nbsp;bases&lt;/li&gt;
&lt;li&gt;&lt;a href="https://man.freebsd.org/cgi/man.cgi?gif(4)"&gt;FreeBSD &lt;code&gt;gif(4)&lt;/code&gt;&lt;/a&gt;&amp;nbsp;- &lt;code&gt;tunnelfib&lt;/code&gt; and the inner/outer &lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;split&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.frrouting.org/en/latest/bgp.html"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; Documentation: &lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/a&gt;&amp;nbsp;- &lt;code&gt;maximum-prefix&lt;/code&gt;, &lt;code&gt;enforce-first-as&lt;/code&gt;,&amp;nbsp;route-maps&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/tynany/frr_exporter"&gt;frr_exporter&lt;/a&gt; - the Prometheus exporter behind the&amp;nbsp;dashboard&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.peeringdb.com/"&gt;PeeringDB&lt;/a&gt; - where the &lt;span class="caps"&gt;EVIX&lt;/span&gt;, FogIXP and LocIX peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; addresses come&amp;nbsp;from&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;The thing nobody tells you about running an &lt;span class="caps"&gt;AS&lt;/span&gt; is how much of it is bookkeeping. The &lt;span class="caps"&gt;BGP&lt;/span&gt; part was solved decades ago and &lt;span class="caps"&gt;FRR&lt;/span&gt; implements it well; what actually takes the time is knowing which of eleven interfaces a prefix should leave through, why a route-map clause was added in March, and whether the tunnel that broke last night is the one carrying the thing that matters. Six months of notes,&amp;nbsp;four &lt;code&gt;rc.conf&lt;/code&gt; files, and one dashboard later, most of that is written down. The network is the easy part. Remembering why it is shaped this way is the&amp;nbsp;work.&lt;/p&gt;</content><category term="Networking"/><category term="freebsd"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="ipv4"/><category term="frr"/><category term="pf"/><category term="ixp"/><category term="evix"/><category term="vxlan"/><category term="wireguard"/><category term="policy-routing"/><category term="jails"/><category term="grafana"/></entry><entry><title>IPv6 Foundations: The Internet Protocol You Should Already Be Using</title><link href="https://hofstede.it/ipv6-foundations-the-internet-protocol-you-should-already-be-using/" rel="alternate"/><published>2026-06-08T00:00:00+02:00</published><updated>2026-06-08T00:00:00+02:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-06-08:/ipv6-foundations-the-internet-protocol-you-should-already-be-using/</id><summary type="html">&lt;p&gt;A laid-back tour through the basics of IPv6: how the addresses are built, how to shorten them without losing your mind, how they map onto the IPv4 you already know, and how hosts configure themselves with &lt;span class="caps"&gt;SLAAC&lt;/span&gt;. Plus a short sidebar on &lt;span class="caps"&gt;NDP&lt;/span&gt; and why blocking &lt;span class="caps"&gt;ICMP&lt;/span&gt; on an IPv6 network is a self-inflicted wound. The premise throughout: IPv6 is the current internet protocol, IPv4 is a relic we are still dragging around, and dual-stack is a burden, not a destination. Updated 2026-06-12 with reader Q&amp;amp;A on privacy addresses, banning abusive clients, &lt;span class="caps"&gt;NAT64&lt;/span&gt;/&lt;span class="caps"&gt;DNS64&lt;/span&gt;, and port&amp;nbsp;forwarding.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Update (2026-06-12):&lt;/strong&gt; A reader, Ben, sent in a wonderful list of questions after reading this post, the kind you only get from someone genuinely working through the material. This is the updated version. The &lt;span class="caps"&gt;NDP&lt;/span&gt; and &lt;span class="caps"&gt;ARP&lt;/span&gt; explanations now appear before they are used, the privacy-address and &lt;span class="caps"&gt;RDNSS&lt;/span&gt; parts have grown answers to &amp;#8220;who actually enforces this?&amp;#8221; and &amp;#8220;where does &lt;span class="caps"&gt;DNS&lt;/span&gt; really come from?&amp;#8221;, and there are new sections on banning abusive clients, your router&amp;#8217;s public address and the end of port forwarding, and reaching the IPv4 internet from IPv6-only (&lt;span class="caps"&gt;NAT64&lt;/span&gt;/&lt;span class="caps"&gt;DNS64&lt;/span&gt;). Thanks,&amp;nbsp;Ben.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;img alt="A small router with its gigabit ethernet ports lit up green" src="https://hofstede.it/images/2026-06-08-ipv6-foundations.jpg" title="Every one of those links can have its own /64, and every device behind them a real, globally routable address. That is the part IPv4 cannot give you anymore."&gt;&lt;/p&gt;
&lt;p&gt;Let me start with the thing nobody wants to say out loud: IPv6 is not &amp;#8220;the future of the internet&amp;#8221;. It is the internet. It has been a finished, deployed, standards-track protocol since 1998. Major mobile networks run IPv6-only internally. More than half of the traffic Google sees from many countries is already IPv6. The &amp;#8220;future&amp;#8221; framing is a comfortable lie we tell ourselves so we can keep treating IPv4 as the default and IPv6 as the weird optional thing the network nerd insists&amp;nbsp;on.&lt;/p&gt;
&lt;p&gt;So let me reframe it for this article. IPv6 is the current version of the Internet Protocol. IPv4 is the legacy one. It is a 32-bit address space from 1981 that ran out of room more than a decade ago, kept alive on life support by &lt;span class="caps"&gt;NAT&lt;/span&gt;, carrier-grade &lt;span class="caps"&gt;NAT&lt;/span&gt;, and an aftermarket where people trade /24s like baseball cards. It works, in the way a 40-year-old car works: lovingly, expensively, and only because a lot of people refuse to let it&amp;nbsp;die.&lt;/p&gt;
&lt;p&gt;This is a foundations post. I am going to cover the actual basics: how an IPv6 address is built, how to shorten it, how it maps onto the IPv4 you already understand, and how a host configures itself with &lt;span class="caps"&gt;SLAAC&lt;/span&gt;. There is a short sidebar on &lt;span class="caps"&gt;NDP&lt;/span&gt;, and a small rant about why you must not block &lt;span class="caps"&gt;ICMP&lt;/span&gt;. Keep your shoes off, this is meant to be&amp;nbsp;relaxed.&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/h2&gt;
&lt;div class="toc"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-address-128-bits-written-in-hex"&gt;The address: 128 bits, written in&amp;nbsp;hex&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#shortening-getting-rid-of-the-zeros"&gt;Shortening: getting rid of the&amp;nbsp;zeros&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#structure-the-prefix-and-the-interface"&gt;Structure: the prefix and the interface&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#address-scopes-you-will-meet"&gt;Address scopes you will&amp;nbsp;meet&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-is-the-same-as-ipv4-more-than-you-would-think"&gt;What is the same as IPv4 (more than you would&amp;nbsp;think)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#slaac-how-a-host-configures-itself"&gt;&lt;span class="caps"&gt;SLAAC&lt;/span&gt;: how a host configures itself&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#the-server-side-flip-side-ban-the-64-not-the-address"&gt;The server-side flip side: ban the /64, not the&amp;nbsp;address&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#sidebar-ndp-and-why-icmp-is-not-optional"&gt;Sidebar: &lt;span class="caps"&gt;NDP&lt;/span&gt;, and why &lt;span class="caps"&gt;ICMP&lt;/span&gt; is not&amp;nbsp;optional&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#your-routers-public-address-and-the-end-of-port-forwarding"&gt;Your router&amp;#8217;s &amp;#8220;public address&amp;#8221; and the end of port&amp;nbsp;forwarding&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#dual-stack-is-a-burden-not-a-solution"&gt;Dual-stack is a burden, not a&amp;nbsp;solution&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#reaching-the-ipv4-internet-from-ipv6-only-nat64-and-dns64"&gt;Reaching the IPv4 internet from IPv6-only: &lt;span class="caps"&gt;NAT64&lt;/span&gt; and &lt;span class="caps"&gt;DNS64&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#where-to-go-from-here"&gt;Where to go from&amp;nbsp;here&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;h2 id="the-address-128-bits-written-in-hex"&gt;The address: 128 bits, written in&amp;nbsp;hex&lt;/h2&gt;
&lt;p&gt;An IPv4 address is 32 bits. We write it as four decimal numbers, one per byte, separated by&amp;nbsp;dots: &lt;code&gt;192.0.2.10&lt;/code&gt;. That gives you about 4.3 billion addresses total, which sounded enormous in 1981 and is comically small for a planet with more connected devices than&amp;nbsp;people.&lt;/p&gt;
&lt;p&gt;An IPv6 address is &lt;strong&gt;128 bits&lt;/strong&gt;. That is not &amp;#8220;four times bigger&amp;#8221;, it&amp;nbsp;is &lt;code&gt;2^128&lt;/code&gt; versus &lt;code&gt;2^32&lt;/code&gt;, which is a number so large it stops being meaningful. You get 340 undecillion addresses, give or take. The standing joke, which is only barely a joke, is that there are enough IPv6 addresses to assign one to every atom on the surface of the Earth and still have plenty left over for several more&amp;nbsp;planets.&lt;/p&gt;
&lt;p&gt;Writing 128 bits as decimal bytes would be miserable, so IPv6 uses &lt;strong&gt;hexadecimal&lt;/strong&gt;. The address is split into &lt;strong&gt;eight groups of 16 bits&lt;/strong&gt;, each group written as up to four hex digits, separated by&amp;nbsp;colons:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;2001:0db8:0000:0000:0000:ff00:0042:8309
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That is one full, uncompressed IPv6 address. Eight groups, four hex digits each, colons in between. Each hex digit is 4 bits, four digits is 16 bits, eight groups is 128 bits. The math is tidy even if the result looks intimidating at&amp;nbsp;first.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;2001:db8::/32&lt;/code&gt; is the documentation range, by the way, the IPv6 equivalent&amp;nbsp;of &lt;code&gt;192.0.2.0/24&lt;/code&gt;. I will use it throughout so I am not leaking real&amp;nbsp;addresses.&lt;/p&gt;
&lt;h2 id="shortening-getting-rid-of-the-zeros"&gt;Shortening: getting rid of the&amp;nbsp;zeros&lt;/h2&gt;
&lt;p&gt;Nobody types the full form above, and you should not either. There are exactly two rules for shortening an IPv6 address, and once they click you will read these things at a&amp;nbsp;glance.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rule 1: drop leading zeros in each group.&lt;/strong&gt; Within any single group, leading zeros are&amp;nbsp;noise. &lt;code&gt;0db8&lt;/code&gt; becomes &lt;code&gt;db8&lt;/code&gt;. &lt;code&gt;0000&lt;/code&gt; becomes &lt;code&gt;0&lt;/code&gt;. &lt;code&gt;0042&lt;/code&gt; becomes &lt;code&gt;42&lt;/code&gt;. &lt;code&gt;ff00&lt;/code&gt; stays &lt;code&gt;ff00&lt;/code&gt; because there is nothing leading to drop. Apply that to the address&amp;nbsp;above:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;2001:db8:0:0:0:ff00:42:8309
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Already much friendlier. Note that this is &lt;em&gt;leading&lt;/em&gt; zeros only. You cannot touch trailing or middle zeros inside a group, because the position of a digit is its&amp;nbsp;value. &lt;code&gt;8309&lt;/code&gt; stays &lt;code&gt;8309&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Rule 2: collapse one run of all-zero groups&amp;nbsp;with &lt;code&gt;::&lt;/code&gt;.&lt;/strong&gt; A double colon means &amp;#8220;as many all-zero groups as it takes to make the address 128 bits again&amp;#8221;. Our address has a run of&amp;nbsp;three &lt;code&gt;0&lt;/code&gt; groups in the middle,&amp;nbsp;so:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;2001:db8::ff00:42:8309
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That is the canonical, shortened form. A parser&amp;nbsp;sees &lt;code&gt;::&lt;/code&gt;, counts the groups present on either side (here: two on the left, three on the right, five total), and fills in the missing three groups with zeros to reach eight.&amp;nbsp;Clean.&lt;/p&gt;
&lt;p&gt;The one catch, and it is the only rule that ever trips people up: &lt;strong&gt;you may&amp;nbsp;use &lt;code&gt;::&lt;/code&gt; exactly once per address.&lt;/strong&gt; If you used it twice, the parser could not know how many zero-groups belong to each gap.&amp;nbsp;So &lt;code&gt;2001:db8:0:0:1:0:0:1&lt;/code&gt; collapses&amp;nbsp;to &lt;code&gt;2001:db8::1:0:0:1&lt;/code&gt; or &lt;code&gt;2001:db8:0:0:1::1&lt;/code&gt;, but&amp;nbsp;never &lt;code&gt;2001:db8::1::1&lt;/code&gt;. When there is a choice, convention says collapse the longest run, and if two runs tie, collapse the first&amp;nbsp;one.&lt;/p&gt;
&lt;p&gt;A few useful special cases fall straight out of these&amp;nbsp;rules:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;::1&lt;/code&gt; is the loopback address. The whole thing is zeros except the final bit. This is&amp;nbsp;IPv6&amp;#8217;s &lt;code&gt;127.0.0.1&lt;/code&gt;, except it is a single address instead of an entire wasted&amp;nbsp;/8.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;::&lt;/code&gt; (just the double colon, all zeros) is the unspecified address, used as a source while a host is still figuring out who it&amp;nbsp;is.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;2001:db8::&lt;/code&gt; is &lt;code&gt;2001:0db8:0000:...:0000&lt;/code&gt;, a prefix with everything after it&amp;nbsp;zeroed.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="structure-the-prefix-and-the-interface"&gt;Structure: the prefix and the&amp;nbsp;interface&lt;/h2&gt;
&lt;p&gt;Here is where IPv6 quietly becomes &lt;em&gt;easier&lt;/em&gt; than IPv4, not&amp;nbsp;harder.&lt;/p&gt;
&lt;p&gt;An IPv6 address has two halves. The left half is the &lt;strong&gt;network prefix&lt;/strong&gt; (which subnet you are on), and the right half is the &lt;strong&gt;interface identifier&lt;/strong&gt; (which host you are within that subnet). The slash notation works exactly like &lt;span class="caps"&gt;CIDR&lt;/span&gt; in&amp;nbsp;IPv4: &lt;code&gt;/64&lt;/code&gt; means the first 64 bits are the network and the rest is the&amp;nbsp;host.&lt;/p&gt;
&lt;p&gt;And here is the gift: in the overwhelming majority of cases, the boundary&amp;nbsp;is &lt;code&gt;/64&lt;/code&gt;, full stop. A standard IPv6 subnet is a /64. Not &amp;#8220;depends on how many hosts you have&amp;#8221;, not &amp;#8220;let me get the subnet calculator&amp;#8221;. A /64. Every &lt;span class="caps"&gt;LAN&lt;/span&gt;, every &lt;span class="caps"&gt;VLAN&lt;/span&gt;, every normal segment gets a /64, which&amp;nbsp;contains &lt;code&gt;2^64&lt;/code&gt; addresses, which is roughly 18 quintillion hosts on a single subnet. You will not run&amp;nbsp;out.&lt;/p&gt;
&lt;p&gt;The prefixes you actually deal with as a network&amp;nbsp;operator:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;/64&lt;/strong&gt; - one subnet. The atom of IPv6 addressing. &lt;span class="caps"&gt;SLAAC&lt;/span&gt; (below) &lt;em&gt;requires&lt;/em&gt; a /64 to&amp;nbsp;work.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;/56&lt;/strong&gt; - a typical home or small-site allocation. That is 256 /64 subnets, one prefix handed to your router by your &lt;span class="caps"&gt;ISP&lt;/span&gt;, plenty for VLANs, guest networks, and IoT&amp;nbsp;segregation.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;/48&lt;/strong&gt; - a site allocation. 65,536 /64 subnets. This is what I run for my own &lt;span class="caps"&gt;AS&lt;/span&gt;; the prefix&amp;nbsp;is &lt;code&gt;2a06:9801:1c::/48&lt;/code&gt;, carved into /64s per jail and per &lt;span class="caps"&gt;VLAN&lt;/span&gt;. If you want the &lt;span class="caps"&gt;BGP&lt;/span&gt; side of that story, see &lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Running Your Own &lt;span class="caps"&gt;AS&lt;/span&gt;&lt;/a&gt;.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The mental shift is that you stop hoarding addresses. In IPv4 you agonize over whether a subnet should be a /27 or a /28 because every address is precious. In IPv6 you hand out /64s like business cards and spend your thinking budget on the network &lt;em&gt;design&lt;/em&gt; instead of the&amp;nbsp;arithmetic.&lt;/p&gt;
&lt;h3 id="address-scopes-you-will-meet"&gt;Address scopes you will&amp;nbsp;meet&lt;/h3&gt;
&lt;p&gt;IPv6 leans on the idea that an interface has &lt;em&gt;several&lt;/em&gt; addresses at once, each with a scope. The ones worth&amp;nbsp;knowing:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Link-local&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;fe80::/10&lt;/code&gt;): every IPv6 interface has one, automatically, always. It is only valid on the local link and is never routed. This is the address that &lt;span class="caps"&gt;NDP&lt;/span&gt; (the Neighbor Discovery Protocol, IPv6&amp;#8217;s link-level housekeeping protocol, explained properly in the sidebar below) and your routing protocols actually talk over. If you have ever&amp;nbsp;seen &lt;code&gt;fe80::...%eth0&lt;/code&gt;,&amp;nbsp;the &lt;code&gt;%eth0&lt;/code&gt; is the zone index telling the kernel &lt;em&gt;which&lt;/em&gt; link, because the same link-local address can exist on every&amp;nbsp;interface.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Unique Local Addresses&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;fc00::/7&lt;/code&gt;, in&amp;nbsp;practice &lt;code&gt;fd00::/8&lt;/code&gt;): the IPv6 answer to &lt;span class="caps"&gt;RFC&lt;/span&gt; 1918 private space&amp;nbsp;(&lt;code&gt;10.0.0.0/8&lt;/code&gt; and friends). Routable inside your site, not on the public internet. Useful, but do not build your whole world on&amp;nbsp;them.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Global Unicast&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;2000::/3&lt;/code&gt;): real, publicly routable addresses.&amp;nbsp;The &lt;code&gt;2001:db8::&lt;/code&gt; documentation range lives here, and so does every address your &lt;span class="caps"&gt;ISP&lt;/span&gt; delegates to&amp;nbsp;you.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Multicast&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;ff00::/8&lt;/code&gt;): IPv6 has no broadcast. None. Where IPv4 shouts&amp;nbsp;at &lt;code&gt;255.255.255.255&lt;/code&gt;, IPv6 uses targeted multicast groups, for&amp;nbsp;example &lt;code&gt;ff02::1&lt;/code&gt; (all nodes on the link)&amp;nbsp;and &lt;code&gt;ff02::2&lt;/code&gt; (all routers on the link). This is a genuine improvement: instead of waking every &lt;span class="caps"&gt;NIC&lt;/span&gt; on the segment, you talk only to the machines that signed up to&amp;nbsp;listen.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="what-is-the-same-as-ipv4-more-than-you-would-think"&gt;What is the same as IPv4 (more than you would&amp;nbsp;think)&lt;/h2&gt;
&lt;p&gt;It is easy to look at the colons and the hex and assume IPv6 is an alien protocol. It is not. It is &lt;em&gt;Internet Protocol&lt;/em&gt;, the same job as IPv4, with the address field made sane. Almost everything you know carries straight&amp;nbsp;over:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;It is still routed by longest-prefix match.&lt;/strong&gt; A router looks at the destination, finds the most specific matching prefix in its table, forwards. Same algorithm, wider&amp;nbsp;addresses.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;TCP&lt;/span&gt; and &lt;span class="caps"&gt;UDP&lt;/span&gt; ride on top unchanged.&lt;/strong&gt; A socket is a socket. Your web server, your database, your &lt;span class="caps"&gt;SSH&lt;/span&gt; daemon do not care; they bind to an address and a&amp;nbsp;port. &lt;code&gt;[2001:db8::1]:443&lt;/code&gt; is just an address and a port with brackets so the colons do not get confused with the port&amp;nbsp;separator.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;DNS&lt;/span&gt; works the same&lt;/strong&gt;, you just&amp;nbsp;publish &lt;code&gt;AAAA&lt;/code&gt; records instead of (or&amp;nbsp;alongside) &lt;code&gt;A&lt;/code&gt; records. &lt;code&gt;example.com. AAAA 2001:db8::1&lt;/code&gt;. Resolvers, zones, TTLs, all&amp;nbsp;identical.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;CIDR&lt;/span&gt;, prefixes, and routing tables behave the&amp;nbsp;same.&lt;/strong&gt; &lt;code&gt;/64&lt;/code&gt;, &lt;code&gt;/48&lt;/code&gt;, &lt;code&gt;/32&lt;/code&gt;, default&amp;nbsp;route &lt;code&gt;::/0&lt;/code&gt; instead&amp;nbsp;of &lt;code&gt;0.0.0.0/0&lt;/code&gt;. Same&amp;nbsp;concepts.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The transport security, the application layer, &lt;span class="caps"&gt;TLS&lt;/span&gt;, &lt;span class="caps"&gt;HTTP&lt;/span&gt;, all of it&lt;/strong&gt; sits on top and never knew which &lt;span class="caps"&gt;IP&lt;/span&gt; version carried the&amp;nbsp;packets.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The differences are mostly &lt;em&gt;removals&lt;/em&gt;, and they are removals of bad&amp;nbsp;ideas:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;No &lt;span class="caps"&gt;NAT&lt;/span&gt;.&lt;/strong&gt; Every device can have a real, globally routable address again, the way the internet was supposed to work before we ran out of room. End-to-end connectivity stops being a luxury. (You still want a firewall. A firewall is not &lt;span class="caps"&gt;NAT&lt;/span&gt;, and conflating the two is how a generation came to believe address translation was a security&amp;nbsp;feature.)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;No broadcast&lt;/strong&gt;, as mentioned. Multicast replaces it and does the job more&amp;nbsp;politely.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Routers do not fragment.&lt;/strong&gt; In IPv4, a router could chop up a too-big packet. In IPv6 that is the sender&amp;#8217;s job, via Path &lt;span class="caps"&gt;MTU&lt;/span&gt; Discovery, which leads directly into why you must not block &lt;span class="caps"&gt;ICMP&lt;/span&gt;. Hold that&amp;nbsp;thought.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The header is simpler and fixed-length&lt;/strong&gt; with extension headers bolted on as needed, which makes high-speed forwarding&amp;nbsp;easier.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="slaac-how-a-host-configures-itself"&gt;&lt;span class="caps"&gt;SLAAC&lt;/span&gt;: how a host configures&amp;nbsp;itself&lt;/h2&gt;
&lt;p&gt;This is the part that genuinely delights people the first time they see it. In IPv4-land, a host that wants an address asks a &lt;span class="caps"&gt;DHCP&lt;/span&gt; server, which hands out a lease from a pool it manages, tracks state, and hopes nobody unplugs anything. IPv6 can do that too (DHCPv6), but it also has something better for most cases: &lt;strong&gt;&lt;span class="caps"&gt;SLAAC&lt;/span&gt;, Stateless Address&amp;nbsp;Autoconfiguration.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Here is the whole dance, and notice that there is no server keeping a database&amp;nbsp;anywhere:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The interface comes up and immediately gives itself a &lt;strong&gt;link-local&lt;/strong&gt; address&amp;nbsp;in &lt;code&gt;fe80::/10&lt;/code&gt;. No configuration, no asking. It derives the host part and checks it is unique on the link (Duplicate Address Detection, one of the &lt;span class="caps"&gt;NDP&lt;/span&gt; mechanisms covered in the sidebar&amp;nbsp;below).&lt;/li&gt;
&lt;li&gt;The host sends a &lt;strong&gt;Router Solicitation (&lt;span class="caps"&gt;RS&lt;/span&gt;)&lt;/strong&gt;&amp;nbsp;to &lt;code&gt;ff02::2&lt;/code&gt;, the all-routers multicast group. Translation: &amp;#8220;any routers out there, what is the prefix on this&amp;nbsp;link?&amp;#8221;&lt;/li&gt;
&lt;li&gt;A router replies with a &lt;strong&gt;Router Advertisement (&lt;span class="caps"&gt;RA&lt;/span&gt;)&lt;/strong&gt;. The &lt;span class="caps"&gt;RA&lt;/span&gt; carries&amp;nbsp;the &lt;code&gt;/64&lt;/code&gt; prefix for the link, the default gateway (the router&amp;#8217;s own link-local address), and a set of flags. Routers also send RAs periodically and unprompted, so a host often does not even need to&amp;nbsp;ask.&lt;/li&gt;
&lt;li&gt;The host takes the&amp;nbsp;advertised &lt;code&gt;/64&lt;/code&gt; prefix and generates the host half itself, then bolts the two together into a full global address. No server assigned it. The host &lt;em&gt;made it up&lt;/em&gt; within the prefix it was told to use, and because a /64 has 18 quintillion slots, collisions are a non-issue (and &lt;span class="caps"&gt;DAD&lt;/span&gt; catches them&amp;nbsp;anyway).&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;How the host picks its own interface &lt;span class="caps"&gt;ID&lt;/span&gt; has two&amp;nbsp;schools:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;EUI&lt;/span&gt;-64&lt;/strong&gt;: derive it deterministically from the &lt;span class="caps"&gt;MAC&lt;/span&gt; address. Stable, but it leaks your hardware address and lets you be tracked across networks. Largely out of fashion for client devices for exactly that&amp;nbsp;reason.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Privacy / temporary addresses&lt;/strong&gt; (&lt;span class="caps"&gt;RFC&lt;/span&gt; 4941, and the more modern stable-but-opaque &lt;span class="caps"&gt;RFC&lt;/span&gt; 7217): generate a random-looking interface &lt;span class="caps"&gt;ID&lt;/span&gt;, rotate the temporary ones periodically. This is what your laptop and phone do today. You will routinely see a machine holding several IPv6 addresses at once: a stable one for inbound, a rotating temporary one for outbound. That is working as intended, not a&amp;nbsp;bug.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The privacy-address dance deserves a closer look, because it feels alien the first time you meet it. The key insight is that nobody assigns or tracks these addresses centrally. The device generates them itself, and the device itself decides which address to use as the source when it opens an outbound connection. The router does not need to know which address is &amp;#8220;the outbound one&amp;#8221;; it just routes packets like it routes everything else. In practice the device keeps one stable address (&lt;span class="caps"&gt;RFC&lt;/span&gt; 7217 style: random-looking, but constant on that network) and additionally generates a temporary one (&lt;span class="caps"&gt;RFC&lt;/span&gt; 4941), prefers the temporary one for outbound connections, and replaces it periodically, typically daily, with the old one lingering a while so existing connections do not&amp;nbsp;break.&lt;/p&gt;
&lt;p&gt;And to be clear about the threat model: rotation does not hide you from someone who already knows your current address. That is not what it is for. It exists so that the websites you visit cannot correlate your activity over time and across services using your address as a long-lived identifier, the way the bottom 64 bits would be if they were derived from your &lt;span class="caps"&gt;MAC&lt;/span&gt;. Inbound protection is a separate concern, and that is the firewall&amp;#8217;s job: a sane IPv6 router drops all unsolicited inbound traffic to every address in your network by default, stable and temporary alike, exactly like &lt;span class="caps"&gt;NAT&lt;/span&gt; incidentally did for IPv4. You then open specific holes for the things you want&amp;nbsp;reachable.&lt;/p&gt;
&lt;h3 id="the-server-side-flip-side-ban-the-64-not-the-address"&gt;The server-side flip side: ban the /64, not the&amp;nbsp;address&lt;/h3&gt;
&lt;p&gt;Privacy addresses have a consequence for anyone running a server, a &lt;span class="caps"&gt;WAF&lt;/span&gt;, or fail2ban. Banning a single abusive IPv6 address is mostly pointless, because the client can rotate within its /64 forever. The standard practice is to block the whole /64, since that is one subnet and almost always one household or one server. If abuse persists across neighbouring /64s, escalate to the /56 (a typical home allocation) or the /48 (a typical site). Modern tooling has grown IPv6 support along exactly these lines; fail2ban, for example, can ban by prefix length instead of by single&amp;nbsp;address.&lt;/p&gt;
&lt;p&gt;&lt;span class="dquo"&gt;&amp;#8220;&lt;/span&gt;But where does &lt;span class="caps"&gt;DNS&lt;/span&gt; come from if there is no &lt;span class="caps"&gt;DHCP&lt;/span&gt;?&amp;#8221; There is no magic well-known &lt;span class="caps"&gt;DNS&lt;/span&gt; address baked into the protocol (there were proposals for one long ago; they died). Instead, modern RAs answer the question via the &lt;strong&gt;&lt;span class="caps"&gt;RDNSS&lt;/span&gt;&lt;/strong&gt; option (&lt;span class="caps"&gt;RFC&lt;/span&gt; 8106): the same periodic multicast packet that announces the prefix and the gateway also carries &amp;#8220;and here are your &lt;span class="caps"&gt;DNS&lt;/span&gt; resolvers&amp;#8221;. So it is still configuration on the router, just delivered inside the &lt;span class="caps"&gt;RA&lt;/span&gt; instead of over &lt;span class="caps"&gt;DHCP&lt;/span&gt;, and many consumer routers send an &lt;span class="caps"&gt;RDNSS&lt;/span&gt; option pointing at themselves without ever exposing a knob for it in their &lt;span class="caps"&gt;UI&lt;/span&gt;. Worth knowing: Android famously refuses to implement DHCPv6 at all, so on a network without &lt;span class="caps"&gt;RDNSS&lt;/span&gt;, Android devices simply have no &lt;span class="caps"&gt;DNS&lt;/span&gt;. That stance alone forced a lot of router vendors to support it. The upshot stands: a host can come up on a fresh network and end up with an address, a gateway, and working &lt;span class="caps"&gt;DNS&lt;/span&gt;, having configured &lt;em&gt;everything itself&lt;/em&gt; from a couple of multicast packets. No &lt;span class="caps"&gt;DHCP&lt;/span&gt; server, no lease database, no scope exhaustion at 3am. That is the part worth being a little smug&amp;nbsp;about.&lt;/p&gt;
&lt;h2 id="sidebar-ndp-and-why-icmp-is-not-optional"&gt;Sidebar: &lt;span class="caps"&gt;NDP&lt;/span&gt;, and why &lt;span class="caps"&gt;ICMP&lt;/span&gt; is not&amp;nbsp;optional&lt;/h2&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;The short version:&lt;/strong&gt; In IPv6, &lt;span class="caps"&gt;ICMP&lt;/span&gt; is not a diagnostic afterthought you can firewall away. It is load-bearing. Block it and the network stops working, not &amp;#8220;works worse&amp;#8221;, &lt;em&gt;stops&lt;/em&gt;.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;First, a word on &lt;span class="caps"&gt;ARP&lt;/span&gt; for everyone who comes from a software background rather than a networking one. In IPv4, before a machine can send a packet to another machine on the same network, it needs to know the target&amp;#8217;s hardware (&lt;span class="caps"&gt;MAC&lt;/span&gt;) address, because that is what Ethernet actually delivers frames to. &lt;span class="caps"&gt;ARP&lt;/span&gt; is the small separate protocol that solves this: it shouts &amp;#8220;who has 192.0.2.1?&amp;#8221; as a broadcast to everyone on the wire and waits for the owner to answer. That is the whole job. &lt;span class="caps"&gt;ICMP&lt;/span&gt;, meanwhile, is the optional-feeling thing you use&amp;nbsp;for &lt;code&gt;ping&lt;/code&gt; and that a lot of paranoid admins reflexively block at the firewall &amp;#8220;for&amp;nbsp;security&amp;#8221;.&lt;/p&gt;
&lt;p&gt;IPv6 throws &lt;span class="caps"&gt;ARP&lt;/span&gt; away and replaces it with the &lt;strong&gt;Neighbor Discovery Protocol (&lt;span class="caps"&gt;NDP&lt;/span&gt;)&lt;/strong&gt;, and &lt;span class="caps"&gt;NDP&lt;/span&gt; is built &lt;em&gt;entirely on top of ICMPv6&lt;/em&gt;. Everything I described in the &lt;span class="caps"&gt;SLAAC&lt;/span&gt; section is ICMPv6&amp;nbsp;messages:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Router Solicitation / Router Advertisement&lt;/strong&gt; (the &lt;span class="caps"&gt;RS&lt;/span&gt;/&lt;span class="caps"&gt;RA&lt;/span&gt; from &lt;span class="caps"&gt;SLAAC&lt;/span&gt;) are ICMPv6 types 133 and&amp;nbsp;134.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Neighbor Solicitation / Neighbor Advertisement&lt;/strong&gt; are ICMPv6 types 135 and 136. This is the &lt;span class="caps"&gt;ARP&lt;/span&gt; replacement: &amp;#8220;who&amp;nbsp;has &lt;code&gt;2001:db8::1&lt;/code&gt;?&amp;#8221; answered with &amp;#8220;I do, here is my &lt;span class="caps"&gt;MAC&lt;/span&gt;.&amp;#8221; It rides on multicast instead of broadcast, but functionally it is &lt;span class="caps"&gt;ARP&lt;/span&gt;&amp;#8217;s job done over &lt;span class="caps"&gt;ICMP&lt;/span&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Duplicate Address Detection and Redirects&lt;/strong&gt; are &lt;span class="caps"&gt;NDP&lt;/span&gt;&amp;nbsp;too.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;So when someone copies their old IPv4 firewall habit and writes &amp;#8220;drop all &lt;span class="caps"&gt;ICMP&lt;/span&gt;&amp;#8221; on an IPv6 interface, they have not hardened anything. They have unplugged address resolution, router discovery, and autoconfiguration. The host cannot find its neighbors or its gateway. The network is dead, and the admin spends an afternoon&amp;nbsp;confused.&lt;/p&gt;
&lt;p&gt;It gets one notch worse because of the no-fragmentation-by-routers rule from earlier. IPv6 relies on &lt;strong&gt;Path &lt;span class="caps"&gt;MTU&lt;/span&gt; Discovery&lt;/strong&gt;, where a router that cannot forward an oversized packet sends back an ICMPv6 &amp;#8220;Packet Too Big&amp;#8221; (type 2) message so the sender can shrink its packets. Block that message and you get the classic nightmare: small packets and pings work fine, but large transfers and &lt;span class="caps"&gt;TLS&lt;/span&gt; handshakes hang forever with no obvious cause. It is one of the most maddening failure modes in networking, and it is almost always someone over-filtering&amp;nbsp;ICMPv6.&lt;/p&gt;
&lt;p&gt;The rule is simple. &lt;strong&gt;Do not block ICMPv6 wholesale.&lt;/strong&gt; Permit Neighbor Discovery, permit Router Advertisements where you expect them, permit Packet Too Big, permit the essentials (&lt;span class="caps"&gt;RFC&lt;/span&gt; 4890 spells out exactly which types to allow). Rate-limit if you must, filter specific types thoughtfully, but the blanket &amp;#8220;drop &lt;span class="caps"&gt;ICMP&lt;/span&gt;&amp;#8221; rule that was merely rude on IPv4 is actively self-destructive on IPv6. ICMPv6 is part of the control plane now, not a&amp;nbsp;luxury.&lt;/p&gt;
&lt;h2 id="your-routers-public-address-and-the-end-of-port-forwarding"&gt;Your router&amp;#8217;s &amp;#8220;public address&amp;#8221; and the end of port&amp;nbsp;forwarding&lt;/h2&gt;
&lt;p&gt;A question that comes up constantly from people arriving from IPv4-land: &amp;#8220;my router shows a single public IPv6 address, just like it shows a single public IPv4 address. Can I port forward from it to an internal&amp;nbsp;machine?&amp;#8221;&lt;/p&gt;
&lt;p&gt;The single address your router shows is just its own &lt;span class="caps"&gt;WAN&lt;/span&gt; address, the one it uses for traffic it originates itself. Separately from that, your &lt;span class="caps"&gt;ISP&lt;/span&gt; delegates an entire prefix to it (via DHCPv6 prefix delegation, usually a /56), and the router hands out /64s from that delegation to your &lt;span class="caps"&gt;LAN&lt;/span&gt; segments. Many consumer UIs display the &lt;span class="caps"&gt;WAN&lt;/span&gt; address prominently and the delegated prefix barely or not at all, which is exactly backwards in terms of importance, but here we&amp;nbsp;are.&lt;/p&gt;
&lt;p&gt;And here is the mental flip: port forwarding exists in IPv4 because your internal machines have no public address, so the router must translate, mapping one of its own ports to an internal host. In IPv6 your internal machines already have globally routable addresses, so there is nothing to forward. You do not map router-port-to-internal-host; you simply open a firewall rule that says &amp;#8220;allow inbound &lt;span class="caps"&gt;TCP&lt;/span&gt; 443 to 2001:db8:abcd:1::5&amp;#8221;. Some router UIs still label this &amp;#8220;port forwarding&amp;#8221; out of familiarity, but it is a pinhole, not a translation. That so many consumer devices hide all of this, or reduce IPv6 to a single on/off toggle with no further configuration, is sadly the actual state of consumer networking gear, and it is fair to be annoyed by&amp;nbsp;it.&lt;/p&gt;
&lt;h2 id="dual-stack-is-a-burden-not-a-solution"&gt;Dual-stack is a burden, not a&amp;nbsp;solution&lt;/h2&gt;
&lt;p&gt;The pitch for dual-stack is reasonable on its face: run IPv4 and IPv6 side by side, let clients prefer IPv6, and you have a smooth glide path. In practice, dual-stack is not a destination. It is the most expensive possible state, and we have been parked in it for over a&amp;nbsp;decade.&lt;/p&gt;
&lt;p&gt;Think about what running both actually costs you. Every firewall rule exists twice, in two syntaxes, with two chances to disagree. Every &lt;span class="caps"&gt;DNS&lt;/span&gt; name needs&amp;nbsp;an &lt;code&gt;A&lt;/code&gt; and&amp;nbsp;an &lt;code&gt;AAAA&lt;/code&gt;, and the day they point at different backends is the day you debug a &amp;#8220;works for me&amp;#8221; ticket for three hours. Every monitoring check, every &lt;span class="caps"&gt;ACL&lt;/span&gt;, every load-balancer config, every piece of address-parsing code in your stack has to handle two formats forever. Security review surface doubles. Happy Eyeballs papers over connection failures by racing both stacks, which is clever and also means a half-broken IPv6 path can hide for months because clients silently fall back to&amp;nbsp;IPv4.&lt;/p&gt;
&lt;p&gt;Dual-stack made sense as a &lt;em&gt;transition&lt;/em&gt; mechanism. The problem is that &amp;#8220;transition&amp;#8221; implies you eventually arrive somewhere, and the somewhere is IPv6-only. Running both indefinitely is not having the best of both worlds, it is paying full price for two networks and getting the operational headaches of each. The mobile carriers worked this out years ago, which is why so many of them are IPv6-only at the core with IPv4 stapled on at the edge as a translation service for the legacy internet, rather than the other way&amp;nbsp;around.&lt;/p&gt;
&lt;p&gt;I am not telling you to rip IPv4 out tomorrow; the legacy internet still exists and your users still need to reach it. I am telling you to stop treating IPv4 as the foundation and IPv6 as the add-on. Flip the polarity. Design IPv6-first, make IPv6 the path that works without caveats, and treat IPv4 as the compatibility shim it has become, a thing you provide grudgingly at the edge for the parts of the world that have not caught up&amp;nbsp;yet.&lt;/p&gt;
&lt;h2 id="reaching-the-ipv4-internet-from-ipv6-only-nat64-and-dns64"&gt;Reaching the IPv4 internet from IPv6-only: &lt;span class="caps"&gt;NAT64&lt;/span&gt; and &lt;span class="caps"&gt;DNS64&lt;/span&gt;&lt;/h2&gt;
&lt;p&gt;I keep saying the mobile carriers are IPv6-only internally with IPv4 stapled on at the edge. The mechanism behind that staple is worth knowing, because it answers a question people sense must have an answer: if IPv6 is so much larger, can you not reach the entire IPv4 internet from inside&amp;nbsp;it?&lt;/p&gt;
&lt;p&gt;You can, and the trick is called &lt;strong&gt;&lt;span class="caps"&gt;NAT64&lt;/span&gt;&lt;/strong&gt; plus &lt;strong&gt;&lt;span class="caps"&gt;DNS64&lt;/span&gt;&lt;/strong&gt;. &lt;span class="caps"&gt;DNS64&lt;/span&gt; is a resolver trick: when a client asks for a name that only has an IPv4 address&amp;nbsp;(an &lt;code&gt;A&lt;/code&gt; record but&amp;nbsp;no &lt;code&gt;AAAA&lt;/code&gt;), the resolver synthesizes a fake IPv6 address by embedding the IPv4 address into a special&amp;nbsp;prefix, &lt;code&gt;64:ff9b::/96&lt;/code&gt;. The client happily connects to that synthetic address, the packet lands on a &lt;span class="caps"&gt;NAT64&lt;/span&gt; gateway, and the gateway translates the connection into a real IPv4 one. From the client&amp;#8217;s point of view, the entire IPv4 internet appears as one tiny corner of the IPv6 address space, which only works because v6 is so much&amp;nbsp;larger.&lt;/p&gt;
&lt;p&gt;This is not built into the protocol; you need a &lt;span class="caps"&gt;NAT64&lt;/span&gt; gateway somewhere. ISPs and mobile carriers operate them at scale, and at home you can run &lt;a href="https://www.jool.mx/"&gt;Jool&lt;/a&gt; on a Linux box or use the &lt;span class="caps"&gt;NAT64&lt;/span&gt; modes built into OpenWrt and OPNsense. Combined with &lt;span class="caps"&gt;DNS64&lt;/span&gt; on your resolver, that is all it takes to run an IPv6-only network whose users never notice that half the destinations they reach are still legacy&amp;nbsp;v4.&lt;/p&gt;
&lt;h2 id="where-to-go-from-here"&gt;Where to go from&amp;nbsp;here&lt;/h2&gt;
&lt;p&gt;That is the foundation: 128-bit addresses written in hex, two simple rules for crushing out the zeros,&amp;nbsp;a &lt;code&gt;/64&lt;/code&gt; per subnet so you stop counting hosts, the same routing and &lt;span class="caps"&gt;DNS&lt;/span&gt; and transport you already know, &lt;span class="caps"&gt;SLAAC&lt;/span&gt; so machines configure themselves, and &lt;span class="caps"&gt;NDP&lt;/span&gt;-over-ICMPv6 holding it all together so you really, truly must not block &lt;span class="caps"&gt;ICMP&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;If you want to go deeper, the natural next steps from here are getting your own globally routable space and announcing it, which I covered in &lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Running Your Own &lt;span class="caps"&gt;AS&lt;/span&gt;&lt;/a&gt;, and then firewalling it properly on the platform of your choice. The point of this post was just to knock the intimidation off. IPv6 is not exotic. It is the normal way to do networking, and it has been for a long time. The hex is the only scary part, and now even that is&amp;nbsp;not.&lt;/p&gt;</content><category term="Networking"/><category term="ipv6"/><category term="networking"/><category term="slaac"/><category term="ndp"/><category term="icmp"/><category term="nat64"/><category term="foundations"/></entry><entry><title>Joining DN42: A MikroTik Border, Three WireGuard Peerings, and a FreeBSD Jail in the Hobbyist Internet</title><link href="https://hofstede.it/joining-dn42-a-mikrotik-border-three-wireguard-peerings-and-a-freebsd-jail-in-the-hobbyist-internet/" rel="alternate"/><published>2026-04-24T00:00:00+02:00</published><updated>2026-04-24T00:00:00+02:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-04-24:/joining-dn42-a-mikrotik-border-three-wireguard-peerings-and-a-freebsd-jail-in-the-hobbyist-internet/</id><summary type="html">&lt;p&gt;Building a &lt;span class="caps"&gt;DN42&lt;/span&gt; node from scratch - registering &lt;span class="caps"&gt;AS4242422539&lt;/span&gt; and fdce:73f7:a2dc::/48 in the registry, configuring a MikroTik border router with three WireGuard peerings and &lt;span class="caps"&gt;BGP&lt;/span&gt; filters, running a FreeBSD bastille jail as authoritative &lt;span class="caps"&gt;DNS&lt;/span&gt; for chofstede.dn42, and reaching the rest of the hobbyist mesh in two&amp;nbsp;hops.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-04-24-dn42-mikrotik-wireguard-freebsd.png" title="DN42 node: header image"&gt;&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/h2&gt;
&lt;div class="toc"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-is-dn42"&gt;What is &lt;span class="caps"&gt;DN42&lt;/span&gt;?&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#architecture"&gt;Architecture&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-registry"&gt;The&amp;nbsp;Registry&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-mikrotik-border-router"&gt;The MikroTik Border Router&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#interfaces"&gt;Interfaces&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#bgp"&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#filters"&gt;Filters&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#firewall"&gt;Firewall&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#mss-clamping"&gt;&lt;span class="caps"&gt;MSS&lt;/span&gt;&amp;nbsp;Clamping&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#nat44-and-nat66"&gt;&lt;span class="caps"&gt;NAT44&lt;/span&gt; and &lt;span class="caps"&gt;NAT66&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-native-dn42-vlan"&gt;The Native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-freebsd-service-jail"&gt;The FreeBSD Service Jail&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#a-routeros-nd-quirk"&gt;A RouterOS &lt;span class="caps"&gt;ND&lt;/span&gt;&amp;nbsp;Quirk&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-this-looks-like-from-the-outside"&gt;What This Looks Like From The&amp;nbsp;Outside&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#conclusion"&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#references"&gt;References&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;p&gt;After &lt;a href="https://hofstede.it/running-your-own-as-direct-hetzner-peering-a-fourth-edge-and-bringing-the-home-lan-into-the-fabric/"&gt;a few months of running &lt;span class="caps"&gt;AS201379&lt;/span&gt;&lt;/a&gt; on the public internet, the obvious next experiment was &lt;span class="caps"&gt;DN42&lt;/span&gt; - the parallel, hobbyist-run &lt;span class="caps"&gt;BGP&lt;/span&gt; network that mirrors the structure of the real internet but lives entirely on private address space, glued together by WireGuard tunnels. It runs the same protocols, presents the same operational challenges, and uses many of the same configuration patterns. The difference is that you can break things at three in the morning without anyone losing access to anything&amp;nbsp;important.&lt;/p&gt;
&lt;p&gt;The moment it felt real wasn&amp;#8217;t a routing-table dump or&amp;nbsp;a &lt;code&gt;show ip bgp&lt;/code&gt; summary - it was&amp;nbsp;an &lt;code&gt;https://&lt;/code&gt; &lt;span class="caps"&gt;URL&lt;/span&gt; that worked from somewhere it had no business working from. From a shell on the other side of the &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;nbsp;mesh:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;chofstede@shell-de-fra1:~$ curl -I https://blog.chofstede.dn42
HTTP/1.1 200 OK
Server: nginx/1.28.3
Content-Type: text/html
Content-Length: 12204
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Same content as the public blog&amp;nbsp;at &lt;code&gt;blog.hofstede.it&lt;/code&gt;, completely separate routing path, completely separate &lt;span class="caps"&gt;TLS&lt;/span&gt; chain - the certificate is signed by &lt;a href="https://wiki.dn42.dev/howto/Tls-Certificate"&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s internal &lt;span class="caps"&gt;CA&lt;/span&gt;&lt;/a&gt;, not Let&amp;#8217;s Encrypt, and the request never traversed a single packet of the public internet. Every hop between that shell and my FreeBSD jail rode a WireGuard tunnel between hobbyists who had agreed, via flat files in a git repository, to forward each other&amp;#8217;s&amp;nbsp;packets.&lt;/p&gt;
&lt;p&gt;The path is&amp;nbsp;short:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;chofstede@shell-de-fra1:~$ mtr -rw blog.chofstede.dn42
HOST: shell-de-fra1          Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- fd42:4242:2601:1011::1  0.0%    10    0.5   0.5   0.4   0.6   0.1
  2.|-- fd00:169:254:8::1       0.0%    10    1.8   1.8   1.6   2.3   0.2
  3.|-- router.chofstede.dn42   0.0%    10   25.2  24.9  24.0  26.1   0.6
  4.|-- dn42svc.chofstede.dn42  0.0%    10   25.9  26.6  24.5  32.7   2.6
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Two of the four hops are mine: hop 3 is the MikroTik border router on my home network, hop 4 is the FreeBSD service jail behind it. Hops 1 and 2 belong to the remote shell&amp;#8217;s operator and an intermediate &lt;span class="caps"&gt;DN42&lt;/span&gt; peer - 25 ms across two WireGuard tunnels and a &lt;span class="caps"&gt;BGP&lt;/span&gt;-selected path through the&amp;nbsp;mesh.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; &lt;span class="caps"&gt;DN42&lt;/span&gt; resources are public by design - the registry is a git repository - so &lt;span class="caps"&gt;AS&lt;/span&gt; numbers, &lt;span class="caps"&gt;DN42&lt;/span&gt; prefixes, and &lt;span class="caps"&gt;DN42&lt;/span&gt; hostnames are shown as-is. The WireGuard underlay endpoints are part of the public peering information that &lt;span class="caps"&gt;DN42&lt;/span&gt; operators exchange anyway, so they&amp;#8217;re also real. Only the WireGuard public keys have been&amp;nbsp;redacted.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/h2&gt;
&lt;p&gt;The setup involves four&amp;nbsp;parts:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; registry objects (&lt;span class="caps"&gt;AS&lt;/span&gt;, prefixes, route, mntner, &lt;span class="caps"&gt;DNS&lt;/span&gt;) submitted as a &lt;span class="caps"&gt;PR&lt;/span&gt;&lt;/li&gt;
&lt;li&gt;A MikroTik (RouterOS 7.22.1, running as a bhyve &lt;span class="caps"&gt;CHR&lt;/span&gt;) as the border router, with three WireGuard&amp;nbsp;peerings&lt;/li&gt;
&lt;li&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; on RouterOS with import/export filters and a per-peer local-preference&amp;nbsp;policy&lt;/li&gt;
&lt;li&gt;A FreeBSD bastille jail&amp;nbsp;(&lt;code&gt;dn42svc&lt;/code&gt;) running PowerDNS authoritative&amp;nbsp;for &lt;code&gt;chofstede.dn42&lt;/code&gt; and nginx&amp;nbsp;for &lt;code&gt;blog.chofstede.dn42&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is a stub &lt;span class="caps"&gt;AS&lt;/span&gt;, not a transit. I announce my own prefixes, accept routes from peers, and forward traffic only between the &lt;span class="caps"&gt;DN42&lt;/span&gt; mesh and my own native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt; - never between two &lt;span class="caps"&gt;DN42&lt;/span&gt; peers, and never between &lt;span class="caps"&gt;DN42&lt;/span&gt; and the clearnet uplink. The import/export filters and the firewall policy below are written to enforce that&amp;nbsp;explicitly.&lt;/p&gt;
&lt;h2 id="what-is-dn42"&gt;What is &lt;span class="caps"&gt;DN42&lt;/span&gt;?&lt;/h2&gt;
&lt;p&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; (&amp;#8220;Decentralized Network 42&amp;#8221;) is a virtual network operated by hobbyists. Participants normally self-allocate a private &lt;span class="caps"&gt;ASN&lt;/span&gt; from the &lt;span class="caps"&gt;DN42&lt;/span&gt; native range,&amp;nbsp;currently &lt;code&gt;AS4242420000-AS4242423999&lt;/code&gt;, and reserve prefixes for themselves in the registry. &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s native IPv4 space is&amp;nbsp;primarily &lt;code&gt;172.20.0.0/14&lt;/code&gt;; &lt;code&gt;172.31.0.0/16&lt;/code&gt; is commonly used for transfer and peering links, and additional private ranges can appear through interconnects to networks like ChaosVPN. IPv6 uses the &lt;span class="caps"&gt;ULA&lt;/span&gt;&amp;nbsp;range &lt;code&gt;fd00::/8&lt;/code&gt;. Most links are WireGuard, &lt;span class="caps"&gt;GRE&lt;/span&gt;, or OpenVPN tunnels running over the regular internet. Routing is real &lt;span class="caps"&gt;BGP&lt;/span&gt;, with real route filters and registry-derived route objects that play the same operational role as &lt;span class="caps"&gt;RPKI&lt;/span&gt; ROAs. The &lt;a href="https://git.dn42.dev/dn42/registry"&gt;registry itself&lt;/a&gt; is git-managed: every resource - ASNs, prefixes, &lt;span class="caps"&gt;DNS&lt;/span&gt; delegations, peer objects - is a flat file in a pull&amp;nbsp;request.&lt;/p&gt;
&lt;p&gt;The result is a fully functional parallel internet: it has its own root &lt;span class="caps"&gt;DNS&lt;/span&gt;&amp;nbsp;(&lt;code&gt;.dn42&lt;/code&gt;), looking glasses, &lt;span class="caps"&gt;IRC&lt;/span&gt; servers, IXPs, anycast services, and bridges to ChaosVPN and other hobby networks. Nothing in &lt;span class="caps"&gt;DN42&lt;/span&gt; routes on the public internet, and nothing on the public internet routes into &lt;span class="caps"&gt;DN42&lt;/span&gt;.&lt;/p&gt;
&lt;h2 id="architecture"&gt;Architecture&lt;/h2&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────────────────────────┐
                    │                    DN42 Mesh                     │
                    └──┬─────────────────────┬─────────────────┬───────┘
                       │                     │                 │
                  AS4242420207         AS4242423914       AS4242423035
                  (RoutedBits-FRA1)    (Kioubit-DE2)    (Lare-AS, preferred)
                       │                     │                 │
                  WireGuard               WireGuard         WireGuard
                  :51820 MTU 1360         :51822 MTU 1360   :51823 MTU 1420
                       │                     │                 │
                       └──────────┬──────────┴─────────────────┘
                                  │
                       ┌──────────┴──────────────────────────┐
                       │       MikroTik CHOFSTEDE-DN42-RTR   │
                       │       RouterOS 7.22.1, AS4242422539 │
                       │       172.21.66.193 / fdce:...::1   │
                       └──────┬───────────────┬──────────────┘
                              │               │
                       ether2 (WAN)        ether1 (DN42_LAN)
                              │               │
                       OPNsense Uplink     172.21.66.192/27
                       (192.168.2.0/24)    fdce:73f7:a2dc::/48
                                              │
                                  ┌───────────┴───────────────┐
                                  │   FreeBSD bastille jail   │
                                  │   dn42svc                 │
                                  │   172.21.66.194           │
                                  │   fdce:73f7:a2dc::194     │
                                  │                           │
                                  │   PowerDNS (authoritative)│
                                  │   nginx                   │
                                  └───────────────────────────┘
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The MikroTik sits on the home &lt;span class="caps"&gt;LAN&lt;/span&gt;&amp;nbsp;(&lt;code&gt;ether2&lt;/code&gt; to OPNsense for clearnet uplink) and originates the &lt;span class="caps"&gt;DN42&lt;/span&gt; prefixes&amp;nbsp;on &lt;code&gt;ether1&lt;/code&gt; - a dedicated &lt;span class="caps"&gt;VLAN&lt;/span&gt; that is the &amp;#8220;native&amp;#8221; &lt;span class="caps"&gt;DN42&lt;/span&gt; segment. Anything attached to that &lt;span class="caps"&gt;VLAN&lt;/span&gt; gets a real, registry-allocated &lt;span class="caps"&gt;DN42&lt;/span&gt; address, no &lt;span class="caps"&gt;NAT&lt;/span&gt; involved. The FreeBSD bastille jail on a small server is the first such resident: it gets its own address from the /27 and /48 and serves &lt;span class="caps"&gt;DNS&lt;/span&gt; and &lt;span class="caps"&gt;HTTP&lt;/span&gt; for&amp;nbsp;the &lt;code&gt;chofstede.dn42&lt;/code&gt; zone.&lt;/p&gt;
&lt;p&gt;Three WireGuard tunnels run from the MikroTik to other &lt;span class="caps"&gt;DN42&lt;/span&gt; ASNs. &lt;span class="caps"&gt;BGP&lt;/span&gt; runs on top of those tunnels, exchanging&amp;nbsp;the &lt;code&gt;fd00::/8&lt;/code&gt; and &lt;code&gt;172.20.0.0/14&lt;/code&gt; family of routes that make up the mesh. The home &lt;span class="caps"&gt;LAN&lt;/span&gt; uses &lt;span class="caps"&gt;NAT44&lt;/span&gt; and &lt;span class="caps"&gt;NAT66&lt;/span&gt; to reach &lt;span class="caps"&gt;DN42&lt;/span&gt; from regular consumer addressing - more on that&amp;nbsp;later.&lt;/p&gt;
&lt;h2 id="the-registry"&gt;The&amp;nbsp;Registry&lt;/h2&gt;
&lt;p&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s registry &lt;a href="https://git.dn42.dev/dn42/registry"&gt;lives at git.dn42.dev&lt;/a&gt;. To get on the network, you fork it, add the objects describing yourself, and submit a Pull Request. A schema linter validates the &lt;span class="caps"&gt;PR&lt;/span&gt;; a small group of registry maintainers merges it. There are no fees and no contracts - just a maintainer object authenticated by a &lt;span class="caps"&gt;PGP&lt;/span&gt; key, and a self-allocated chunk of address space that doesn&amp;#8217;t conflict with anything&amp;nbsp;else.&lt;/p&gt;
&lt;p&gt;The minimum set of objects for a routing-capable participant is six files. Mine looks like&amp;nbsp;this:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;data/mntner/CHOFSTEDE-MNT&lt;/code&gt;&lt;/strong&gt; - the maintainer, authenticated by a &lt;span class="caps"&gt;PGP&lt;/span&gt;&amp;nbsp;fingerprint:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mntner:             CHOFSTEDE-MNT
admin-c:            CHOFSTEDE-DN42
tech-c:             CHOFSTEDE-DN42
mnt-by:             CHOFSTEDE-MNT
auth:               pgp-fingerprint F797370E9131BB04D2D339304A64EF24AB2463EA
source:             DN42
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;data/aut-num/AS4242422539&lt;/code&gt;&lt;/strong&gt; - the &lt;span class="caps"&gt;ASN&lt;/span&gt;&amp;nbsp;object:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;aut-num:            AS4242422539
as-name:            CHOFSTEDE-AS
descr:              Christian Hofstede-Kuhn AS
admin-c:            CHOFSTEDE-DN42
tech-c:             CHOFSTEDE-DN42
mnt-by:             CHOFSTEDE-MNT
source:             DN42
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;data/inetnum/172.21.66.192_27&lt;/code&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;code&gt;data/inet6num/fdce:73f7:a2dc::_48&lt;/code&gt;&lt;/strong&gt; - the address blocks I&amp;#8217;ll be announcing. The IPv4 /27 is small on purpose: &lt;span class="caps"&gt;DN42&lt;/span&gt; strongly prefers IPv6, and a /27 is enough for &amp;#8220;a research network&amp;#8221; without tying up significant chunks of the shared&amp;nbsp;/14.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;data/route/172.21.66.192_27&lt;/code&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;code&gt;data/route6/fdce:73f7:a2dc::_48&lt;/code&gt;&lt;/strong&gt; - the route objects that authorize this &lt;span class="caps"&gt;AS&lt;/span&gt; to originate these&amp;nbsp;prefixes:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;route6:             fdce:73f7:a2dc::/48
descr:              DN42 v6 CHOFSTEDE
origin:             AS4242422539
max-length:         48
mnt-by:             CHOFSTEDE-MNT
source:             DN42
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;These play a similar operational role to &lt;span class="caps"&gt;RPKI&lt;/span&gt; ROAs on the public internet - they let peers build import filters that reject any prefix not authorized for a given origin &lt;span class="caps"&gt;AS&lt;/span&gt; - though the mechanism is registry objects validated by tooling, not cryptographically signed ROAs. Most &lt;span class="caps"&gt;DN42&lt;/span&gt; operators run those&amp;nbsp;filters.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;data/dns/chofstede.dn42&lt;/code&gt;&lt;/strong&gt; - the &lt;span class="caps"&gt;DNS&lt;/span&gt; delegation. &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s recursive resolvers use this to forward queries&amp;nbsp;for &lt;code&gt;chofstede.dn42&lt;/code&gt; to the listed&amp;nbsp;nameservers:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;domain:             chofstede.dn42
admin-c:            CHOFSTEDE-DN42
tech-c:             CHOFSTEDE-DN42
mnt-by:             CHOFSTEDE-MNT
nserver:            ns1.chofstede.dn42 172.21.66.194
nserver:            ns1.chofstede.dn42 fdce:73f7:a2dc::194
source:             DN42
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The nameserver address is the FreeBSD jail. Once the &lt;span class="caps"&gt;PR&lt;/span&gt; was merged and the next registry sync ran, queries for anything&amp;nbsp;under &lt;code&gt;.chofstede.dn42&lt;/code&gt; started landing on PowerDNS in that&amp;nbsp;jail.&lt;/p&gt;
&lt;p&gt;The whole submission was a single commit, signed with the &lt;span class="caps"&gt;PGP&lt;/span&gt; key referenced in the maintainer object. The registry&amp;#8217;s &lt;span class="caps"&gt;CI&lt;/span&gt; ran the schema lint, a maintainer reviewed it, and the resources were live a few hours&amp;nbsp;later.&lt;/p&gt;
&lt;h2 id="the-mikrotik-border-router"&gt;The MikroTik Border&amp;nbsp;Router&lt;/h2&gt;
&lt;p&gt;In this setup, the MikroTik isn&amp;#8217;t a physical box - it&amp;#8217;s a RouterOS &lt;span class="caps"&gt;CHR&lt;/span&gt; (Cloud Hosted Router) running as a bhyve guest on a FreeBSD 15 host&amp;nbsp;called &lt;code&gt;voyager&lt;/code&gt;. &lt;span class="caps"&gt;CHR&lt;/span&gt; is MikroTik&amp;#8217;s virtualization-friendly RouterOS image, free up to 1 Mbit/s per interface and licensable for&amp;nbsp;more.&lt;/p&gt;
&lt;p&gt;The bhyve definition is&amp;nbsp;small:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@voyager:~ # cat /zroot/shuttle/routeros/routeros.conf
loader=&amp;quot;uefi&amp;quot;
graphics=&amp;quot;no&amp;quot;
cpu=1
memory=&amp;quot;256m&amp;quot;

disk0_type=&amp;quot;virtio-blk&amp;quot;
disk0_name=&amp;quot;disk0.img&amp;quot;

network0_type=&amp;quot;virtio-net&amp;quot;
network0_switch=&amp;quot;vm-vlan42&amp;quot;   # DN42 (native)
network1_type=&amp;quot;virtio-net&amp;quot;
network1_switch=&amp;quot;vm-vlan20&amp;quot;   # IOT (upstream / OPNsense)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;One vCPU, 256 &lt;span class="caps"&gt;MB&lt;/span&gt; of &lt;span class="caps"&gt;RAM&lt;/span&gt;, two virtio NICs bridged onto two FreeBSD &lt;span class="caps"&gt;VM&lt;/span&gt;-switch instances - one for the native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt;, one for the upstream path to OPNsense. RouterOS sees these&amp;nbsp;as &lt;code&gt;ether1&lt;/code&gt; and &lt;code&gt;ether2&lt;/code&gt; respectively, and the rest of the configuration is identical to what it would be on real hardware. The host&amp;nbsp;runs:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@voyager:~ # freebsd-version
15.0-RELEASE-p4
root@voyager:~ # vm list | grep routeros
routeros  default    uefi       1    256m    -    No       Running (61180)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;That&amp;#8217;s the entire hypervisor footprint of the &lt;span class="caps"&gt;DN42&lt;/span&gt; border. A &lt;span class="caps"&gt;CHR&lt;/span&gt; &lt;span class="caps"&gt;VM&lt;/span&gt; gives me MikroTik&amp;#8217;s mature &lt;span class="caps"&gt;BGP&lt;/span&gt; and firewall implementations without dedicating a piece of physical hardware to it, and the&amp;nbsp;bhyve &lt;code&gt;vm-bhyve&lt;/code&gt; workflow (one config file in &lt;span class="caps"&gt;ZFS&lt;/span&gt;, &lt;code&gt;vm start routeros&lt;/code&gt;) is light enough that I treat the router like any other service on the&amp;nbsp;host.&lt;/p&gt;
&lt;p&gt;The router terminates the WireGuard tunnels, originates the &lt;span class="caps"&gt;DN42&lt;/span&gt; prefixes, and bridges the native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt; to the rest of the &lt;span class="caps"&gt;LAN&lt;/span&gt; with &lt;span class="caps"&gt;NAT&lt;/span&gt;. The total RouterOS configuration is around 100&amp;nbsp;lines.&lt;/p&gt;
&lt;h3 id="interfaces"&gt;Interfaces&lt;/h3&gt;
&lt;p&gt;Three WireGuard interfaces, one for each peer. The MTUs are deliberate: WireGuard adds 80 bytes of overhead over IPv4 underlay (60 over IPv6), so 1500 minus that is the largest tunnel &lt;span class="caps"&gt;MTU&lt;/span&gt; that won&amp;#8217;t fragment. I chose 1360 as a conservative default that survives IPv4 underlays with PPPoE in the path, and 1420 for the Lare peering where the underlay is&amp;nbsp;known-clean:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/interface wireguard
add listen-port=51820 mtu=1360 name=wg-peer1
add listen-port=51822 mtu=1360 name=wg-kioubit
add listen-port=51823 mtu=1420 name=wg-lare

/interface list
add comment=&amp;quot;Uplink to OPNsense/Internet&amp;quot;        name=WAN
add comment=&amp;quot;All WireGuard Peer Interfaces&amp;quot;      name=DN42
add comment=&amp;quot;Native DN42 Subnets (VLAN 42)&amp;quot;      name=DN42_LAN
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The interface lists are RouterOS&amp;#8217;s way of grouping interfaces for firewall rules - &amp;#8220;in-interface-list=&lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8221; matches any of the three tunnels at once, which keeps the firewall&amp;nbsp;short.&lt;/p&gt;
&lt;p&gt;The peer definitions specify which prefixes are allowed inside each tunnel. &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s address ranges are well-defined, so the AllowedIPs list is the same for every&amp;nbsp;peer:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/interface wireguard peers
add interface=wg-peer1 name=peer1 \
    endpoint-address=2600:3c17::2000:83ff:fe31:43bc \
    endpoint-port=52539 persistent-keepalive=25s public-key=&amp;quot;REDACTED&amp;quot; \
    allowed-address=172.20.0.0/14,172.21.0.0/16,172.22.0.0/15,172.31.0.0/16,fd00::/8,fe80::/10

add interface=wg-kioubit name=peer-kioubit \
    endpoint-address=2a01:4f8:c2c:3b65::1 \
    endpoint-port=20055 persistent-keepalive=25s public-key=&amp;quot;REDACTED&amp;quot; \
    allowed-address=172.20.0.0/14,172.21.0.0/16,172.22.0.0/15,172.31.0.0/16,fd00::/8,fe80::/10

add interface=wg-lare name=peer-lare \
    endpoint-address=de01.dn42.lare.cc \
    endpoint-port=22539 persistent-keepalive=25s public-key=&amp;quot;REDACTED&amp;quot; \
    allowed-address=172.20.0.0/14,172.31.0.0/16,fd00::/8,fe80::/10
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;fe80::/10&lt;/code&gt; is in the AllowedIPs list because &lt;span class="caps"&gt;BGP&lt;/span&gt; IPv6 sessions run over link-local addresses on the WireGuard interface - without that, the &lt;span class="caps"&gt;BGP&lt;/span&gt; packets would be silently dropped by WireGuard&amp;#8217;s own &lt;span class="caps"&gt;ACL&lt;/span&gt;. On RouterOS this list is not how routes are selected; &lt;span class="caps"&gt;BGP&lt;/span&gt; does that. It is just the WireGuard peer &lt;span class="caps"&gt;ACL&lt;/span&gt; specifying which source/destination addresses the tunnel may carry, and it has to be wide enough to cover everything &lt;span class="caps"&gt;BGP&lt;/span&gt; will install plus any link-local control&amp;nbsp;traffic.&lt;/p&gt;
&lt;p&gt;The IPv4 numbering for the tunnel endpoints is a small &lt;span class="caps"&gt;DN42&lt;/span&gt; quirk: the link addresses are usually in&amp;nbsp;the &lt;code&gt;172.31.0.0/16&lt;/code&gt; &lt;span class="dquo"&gt;&amp;#8220;&lt;/span&gt;transfer&amp;#8221; range. I&amp;nbsp;assigned &lt;code&gt;172.31.255.1&lt;/code&gt; and &lt;code&gt;172.31.255.2&lt;/code&gt; to &lt;code&gt;wg-kioubit&lt;/code&gt; and &lt;code&gt;wg-peer1&lt;/code&gt; respectively. Lare uses link-local for both v4 and v6 &lt;span class="caps"&gt;BGP&lt;/span&gt;, so no IPv4 transfer address is required&amp;nbsp;there.&lt;/p&gt;
&lt;h3 id="bgp"&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/h3&gt;
&lt;p&gt;Three eBGP sessions, one per peer, plus a couple of route-map-style filter chains for hygiene. The full&amp;nbsp;configuration:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/routing bgp instance
add as=4242422539 name=dn42-instance router-id=172.21.66.193

/routing bgp template
add afi=ip,ipv6 as=4242422539 name=DN42_v6_Template \
    nexthop-choice=force-self routing-table=main \
    output.network=DN42_my_space .network-blackhole=yes

/routing bgp connection
add name=routedbits-fra1-ipv6 instance=dn42-instance \
    afi=ipv6 connect=yes listen=yes templates=DN42_v6_Template \
    local.address=fe80::4242:4225:39%wg-peer1 .role=ebgp \
    remote.address=fe80::207%wg-peer1 .as=4242420207 \
    output.filter-chain=dn42-export .network=DN42_my_space \
    input.filter=dn42-import

add name=routedbits-fra1-ipv4 instance=dn42-instance \
    afi=ip as=4242422539 multihop=yes templates=DN42_v6_Template \
    local.address=172.21.66.193 .role=ebgp \
    remote.address=172.20.19.71 .as=4242420207 \
    output.filter-chain=dn42-export .network=DN42_my_space \
    input.filter=dn42-import

add name=kioubit-de2 instance=dn42-instance templates=DN42_v6_Template \
    local.address=fe80::ade1%wg-kioubit .role=ebgp \
    remote.address=fe80::ade0%wg-kioubit .as=4242423914 \
    output.filter-chain=dn42-export input.filter=dn42-import

add name=lare-dn42 instance=dn42-instance templates=DN42_v6_Template \
    afi=ip,ipv6 connect=yes listen=yes \
    local.address=fe80::4242:4225:39%wg-lare .role=ebgp \
    remote.address=fe80::3035:130%wg-lare .as=4242423035 \
    output.filter-chain=dn42-export .network=DN42_my_space \
    input.filter=lare-in
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;A few things deserve a closer&amp;nbsp;look:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;IPv6 &lt;span class="caps"&gt;BGP&lt;/span&gt; runs over link-local addresses on the tunnel.&lt;/strong&gt;&amp;nbsp;The &lt;code&gt;fe80::4242:4225:39%wg-peer1&lt;/code&gt; syntax is RouterOS&amp;#8217;s zone-id notation. Encoding the &lt;span class="caps"&gt;AS&lt;/span&gt; number in the host part&amp;nbsp;(&lt;code&gt;4242:4225:39&lt;/code&gt; ≈ 4242422539) is convention; it&amp;#8217;s only meaningful to the&amp;nbsp;operator.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;IPv4 &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;needs &lt;code&gt;multihop=yes&lt;/code&gt; and a static route to the peer.&lt;/strong&gt; Unlike the IPv6 sessions where link-local works directly on the WireGuard interface, the IPv4 peers live&amp;nbsp;in &lt;code&gt;172.20.0.0/14&lt;/code&gt; and aren&amp;#8217;t directly attached. The &lt;span class="caps"&gt;BGP&lt;/span&gt; daemon needs a route to the peer&amp;#8217;s address before the session can come up,&amp;nbsp;and &lt;code&gt;multihop=yes&lt;/code&gt; is required because the &lt;span class="caps"&gt;TCP&lt;/span&gt; &lt;span class="caps"&gt;TTL&lt;/span&gt; would otherwise prevent a session that traverses any router in between (which the WireGuard tunnel technically&amp;nbsp;does):&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip route
add comment=&amp;quot;Direct resolution for Lare&amp;quot;  dst-address=172.22.125.130/32 gateway=wg-lare
add comment=&amp;quot;Direct resolution for Peer1&amp;quot; dst-address=172.20.19.71/32   gateway=wg-peer1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Each &lt;code&gt;/32&lt;/code&gt; pins the peer&amp;#8217;s IPv4 address to the correct WireGuard interface, so the &lt;span class="caps"&gt;TCP&lt;/span&gt; session for IPv4 &lt;span class="caps"&gt;BGP&lt;/span&gt; knows where to&amp;nbsp;go.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;force-self&lt;/code&gt; is&amp;nbsp;RouterOS&amp;#8217;s &lt;code&gt;next-hop-self&lt;/code&gt;.&lt;/strong&gt; It ensures that routes I advertise carry my side of the session as the next hop, rather than preserving an unusable or ambiguous next-hop learned from somewhere else. With link-local eBGP this matters: a link-local address is meaningful only with the right interface scope and only for the directly attached neighbor, so forcing the next hop avoids any ambiguity in what the peer or downstream neighbors should&amp;nbsp;resolve.&lt;/p&gt;
&lt;h3 id="filters"&gt;Filters&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;dn42-export&lt;/code&gt; chain is short: only my own prefixes go out, everything else is rejected.&amp;nbsp;The &lt;code&gt;dn42-import&lt;/code&gt; chain accepts anything inside &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s address ranges&amp;nbsp;(&lt;code&gt;fd00::/8&lt;/code&gt; with prefix length&amp;nbsp;44-64, &lt;code&gt;172.20.0.0/14&lt;/code&gt; with prefix length 14-32) but explicitly rejects my own prefixes coming back from a peer - a basic anti-loop measure for cases where the prefix has propagated through the mesh and made its way back to me on a second&amp;nbsp;peering:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/routing filter rule
add chain=dn42-export rule=&amp;quot;if (dst in fdce:73f7:a2dc::/48 &amp;amp;&amp;amp; dst-len == 48) { accept }&amp;quot;
add chain=dn42-export rule=&amp;quot;if (dst in 172.21.66.192/27 &amp;amp;&amp;amp; dst-len == 27) { accept }&amp;quot;
add chain=dn42-export rule=reject

add chain=dn42-import comment=&amp;quot;Reject Own IPv6&amp;quot;  rule=&amp;quot;if (dst in fdce:73f7:a2dc::/48) { reject }&amp;quot;
add chain=dn42-import comment=&amp;quot;Reject Own IPv4&amp;quot;  rule=&amp;quot;if (dst in 172.21.66.192/27) { reject }&amp;quot;
add chain=dn42-import comment=&amp;quot;Accept DN42 IPv6&amp;quot; rule=&amp;quot;if (dst in fd00::/8 &amp;amp;&amp;amp; dst-len in 44-64) { accept }&amp;quot;
add chain=dn42-import                            rule=&amp;quot;if (dst in 172.20.0.0/14 &amp;amp;&amp;amp; dst-len in 14-32) { accept }&amp;quot;
add chain=dn42-import comment=&amp;quot;Default Drop&amp;quot;     rule=reject
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The Lare session uses a small wrapper on top of the import chain to give that peer&amp;#8217;s routes a higher local-preference, then jumps into the standard import&amp;nbsp;logic:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;add chain=lare-in rule=&amp;quot;set bgp-local-pref 200; jump dn42-import&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Lare sits in Frankfurt with low jitter and a peer-friendly operator, and consistently delivers the shortest paths to most other &lt;span class="caps"&gt;DN42&lt;/span&gt; ASNs. Local-pref 200 vs. the default 100 makes its routes win whenever they&amp;#8217;re available, which they almost always are. This is the same trick used&amp;nbsp;on &lt;code&gt;hobgp&lt;/code&gt; in &lt;a href="https://hofstede.it/running-your-own-as-direct-hetzner-peering-a-fourth-edge-and-bringing-the-home-lan-into-the-fabric/"&gt;Part 4 of the &lt;span class="caps"&gt;AS201379&lt;/span&gt; series&lt;/a&gt;, applied here at a much smaller&amp;nbsp;scale.&lt;/p&gt;
&lt;p&gt;Finally, the &lt;span class="caps"&gt;BGP&lt;/span&gt; instance has two blackhole routes for the prefixes I originate, so packets to my own space don&amp;#8217;t follow a default route back to the upstream when no more-specific route is in the&amp;nbsp;table:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip route
add blackhole comment=&amp;quot;DN42 IPv4 Aggregate Origin&amp;quot; dst-address=172.21.66.192/27

/ipv6 route
add blackhole comment=&amp;quot;DN42 Aggregate Origin&amp;quot; dst-address=fdce:73f7:a2dc::/48
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3 id="firewall"&gt;Firewall&lt;/h3&gt;
&lt;p&gt;The firewall is tighter than the public-internet equivalent because the threat model is different: every other &lt;span class="caps"&gt;DN42&lt;/span&gt; participant is theoretically reachable, including the ones whose security posture is &amp;#8220;I am running a hobby network at home.&amp;#8221; So the rule of thumb is to allow only the specific things the router needs, and drop everything else by&amp;nbsp;default.&lt;/p&gt;
&lt;p&gt;The full filter is too long to reproduce; the key sections look like&amp;nbsp;this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip firewall filter
add chain=input  in-interface-list=DN42 protocol=tcp dst-port=179 action=accept \
                 comment=&amp;quot;Accept BGP from DN42 (IPv4)&amp;quot;
add chain=input  in-interface-list=DN42 action=drop  comment=&amp;quot;Drop input from DN42&amp;quot;

add chain=forward in-interface-list=DN42 out-interface-list=DN42_LAN action=accept \
                  comment=&amp;quot;Allow DN42 Mesh to DN42_LAN&amp;quot;
add chain=forward in-interface-list=DN42_LAN out-interface-list=DN42 action=accept \
                  comment=&amp;quot;Allow DN42_LAN to DN42 Mesh&amp;quot;
add chain=forward in-interface-list=DN42 protocol=icmp action=accept \
                  comment=&amp;quot;Allow DN42 Transit ICMPv4&amp;quot;
add chain=forward in-interface-list=DN42 action=drop \
                  comment=&amp;quot;Drop all IPv4 transit from DN42&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Three&amp;nbsp;principles:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Input is closed.&lt;/strong&gt; The router accepts only &lt;span class="caps"&gt;BGP&lt;/span&gt; and &lt;span class="caps"&gt;ICMP&lt;/span&gt; from &lt;span class="caps"&gt;DN42&lt;/span&gt;; everything else - &lt;span class="caps"&gt;SSH&lt;/span&gt;, &lt;span class="caps"&gt;HTTP&lt;/span&gt;, Winbox - is silently dropped on the &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;nbsp;interfaces.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Forwarding is symmetric and explicit.&lt;/strong&gt; Traffic between the native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;LAN&lt;/span&gt;&amp;nbsp;(&lt;code&gt;ether1&lt;/code&gt;) and the mesh is allowed in both directions. Everything else - including transit through my router for traffic that isn&amp;#8217;t destined for my prefix - is dropped, so I&amp;#8217;m not accidentally a transit &lt;span class="caps"&gt;AS&lt;/span&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;WAN&lt;/span&gt;-to-&lt;span class="caps"&gt;DN42&lt;/span&gt; leaks are explicitly blocked&lt;/strong&gt;, both directions. The IPv6 chain&amp;nbsp;has &lt;code&gt;drop chain=forward in-interface-list=DN42 out-interface-list=WAN&lt;/code&gt; and the converse - belt-and-braces against any misconfiguration that would let &lt;span class="caps"&gt;DN42&lt;/span&gt; traffic exit via the clearnet uplink, or vice&amp;nbsp;versa.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id="mss-clamping"&gt;&lt;span class="caps"&gt;MSS&lt;/span&gt;&amp;nbsp;Clamping&lt;/h3&gt;
&lt;p&gt;The 1360-byte WireGuard &lt;span class="caps"&gt;MTU&lt;/span&gt; is small enough that any &lt;span class="caps"&gt;TCP&lt;/span&gt; session through the tunnel needs &lt;span class="caps"&gt;MSS&lt;/span&gt; clamping, otherwise large packets would either fragment or be silently dropped by Path &lt;span class="caps"&gt;MTU&lt;/span&gt; Discovery failures. RouterOS does this with mangle&amp;nbsp;rules:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip firewall mangle
add chain=forward out-interface-list=DN42 protocol=tcp tcp-flags=syn \
    action=change-mss new-mss=1320  comment=&amp;quot;Clamp MSS for DN42 (IPv4)&amp;quot;
add chain=output                    protocol=tcp tcp-flags=syn tcp-mss=!0-1320 \
    action=change-mss new-mss=1320  comment=&amp;quot;Clamp BGP/Output MSS (IPv4)&amp;quot;

/ipv6 firewall mangle
add chain=forward out-interface-list=DN42 protocol=tcp tcp-flags=syn \
    action=change-mss new-mss=1300  comment=&amp;quot;Clamp MSS for DN42&amp;quot;
add chain=output                    protocol=tcp tcp-flags=syn tcp-mss=!0-1300 \
    action=change-mss new-mss=1300  comment=&amp;quot;Clamp BGP MSS Output&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;I use 1320 for IPv4 and 1300 for IPv6 as conservative clamps for a 1360-byte tunnel. They sit below the tunnel &lt;span class="caps"&gt;MTU&lt;/span&gt; after the normal &lt;span class="caps"&gt;IP&lt;/span&gt; and &lt;span class="caps"&gt;TCP&lt;/span&gt; headers are accounted for, and they avoid relying on perfect Path &lt;span class="caps"&gt;MTU&lt;/span&gt; Discovery across a mesh of tunnels.&amp;nbsp;The &lt;code&gt;output&lt;/code&gt; chain rules also clamp the &lt;span class="caps"&gt;MSS&lt;/span&gt; of locally-originated &lt;span class="caps"&gt;TCP&lt;/span&gt; - that&amp;#8217;s what makes &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;#8217;s own session over the tunnel work without falling over on the first large &lt;span class="caps"&gt;UPDATE&lt;/span&gt;&amp;nbsp;message.&lt;/p&gt;
&lt;h3 id="nat44-and-nat66"&gt;&lt;span class="caps"&gt;NAT44&lt;/span&gt; and &lt;span class="caps"&gt;NAT66&lt;/span&gt;&lt;/h3&gt;
&lt;p&gt;The home &lt;span class="caps"&gt;LAN&lt;/span&gt; is a normal consumer network - 192.168.x.x with &lt;span class="caps"&gt;DTAG&lt;/span&gt; IPv6 - and most things on it have no business being in &lt;span class="caps"&gt;DN42&lt;/span&gt; directly. But it&amp;#8217;s useful to be able&amp;nbsp;to &lt;code&gt;ssh&lt;/code&gt; into something at burble.dn42 from a laptop on the regular &lt;span class="caps"&gt;LAN&lt;/span&gt;, so the router&amp;nbsp;NATs:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip firewall nat
add chain=srcnat out-interface-list=DN42 src-address=192.168.1.0/24 \
    action=src-nat to-addresses=172.21.66.193 \
    comment=&amp;quot;NAT44 LAN to DN42&amp;quot;

/ipv6 firewall nat
add chain=srcnat out-interface-list=DN42 src-address=2003:a:1318:5800::/56 \
    action=src-nat to-address=fdce:73f7:a2dc::1/128 \
    comment=&amp;quot;NAT66 LAN to DN42&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;span class="caps"&gt;NAT66&lt;/span&gt; is controversial on the regular IPv6 internet - the &lt;span class="caps"&gt;IETF&lt;/span&gt; has long preferred IPv6 designs that avoid address translation, while acknowledging limited mechanisms such as &lt;a href="https://www.rfc-editor.org/rfc/rfc6296"&gt;NPTv6&lt;/a&gt; for specific multihoming and renumbering cases (see also &lt;a href="https://www.rfc-editor.org/rfc/rfc7157"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 7157&lt;/a&gt; on multihoming without &lt;span class="caps"&gt;NAT&lt;/span&gt;). Inside &lt;span class="caps"&gt;DN42&lt;/span&gt;, where renumbering my entire home &lt;span class="caps"&gt;LAN&lt;/span&gt; into &lt;span class="caps"&gt;DN42&lt;/span&gt; space would be both impractical and wasteful, source-&lt;span class="caps"&gt;NAT&lt;/span&gt; is the pragmatic answer. The home &lt;span class="caps"&gt;LAN&lt;/span&gt; keeps its &lt;span class="caps"&gt;DTAG&lt;/span&gt; addresses, and the router rewrites them to a single &lt;span class="caps"&gt;DN42&lt;/span&gt;-side address on egress. Connections initiated from the &lt;span class="caps"&gt;LAN&lt;/span&gt; work; nothing inbound is reachable, which is exactly what I want for hosts that aren&amp;#8217;t supposed to be in &lt;span class="caps"&gt;DN42&lt;/span&gt; in the first&amp;nbsp;place.&lt;/p&gt;
&lt;p&gt;For things that &lt;em&gt;should&lt;/em&gt; be reachable from &lt;span class="caps"&gt;DN42&lt;/span&gt;, the answer is a different physical interface (the native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt;&amp;nbsp;on &lt;code&gt;ether1&lt;/code&gt;) and an address from my actual /27 and /48 - no &lt;span class="caps"&gt;NAT&lt;/span&gt;, no&amp;nbsp;rewriting.&lt;/p&gt;
&lt;h2 id="the-native-dn42-vlan"&gt;The Native &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt;&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;ether1&lt;/code&gt; is the dedicated native-&lt;span class="caps"&gt;DN42&lt;/span&gt; segment. The router originates the prefix here, and any host that connects to this &lt;span class="caps"&gt;VLAN&lt;/span&gt; gets a real &lt;span class="caps"&gt;DN42&lt;/span&gt; address. The configuration is unremarkable - it&amp;#8217;s just an interface with two&amp;nbsp;addresses:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ip address
add address=172.21.66.193/27 interface=ether1 network=172.21.66.192

/ipv6 address
add address=fdce:73f7:a2dc::1/64 interface=ether1 advertise=no
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;advertise=no&lt;/code&gt; is deliberate. RouterOS&amp;#8217;s IPv6 stack will, by default, send Router Advertisements with the address as the prefix, which would let any host &lt;span class="caps"&gt;SLAAC&lt;/span&gt; into it. That&amp;#8217;s exactly what I want for the home &lt;span class="caps"&gt;LAN&lt;/span&gt;, but for the &lt;span class="caps"&gt;DN42&lt;/span&gt; segment I prefer to assign addresses statically - I want to pick which addresses get registered in &lt;span class="caps"&gt;DNS&lt;/span&gt;, not whatever &lt;span class="caps"&gt;EUI&lt;/span&gt;-64 the host happens to&amp;nbsp;compute.&lt;/p&gt;
&lt;h2 id="the-freebsd-service-jail"&gt;The FreeBSD Service&amp;nbsp;Jail&lt;/h2&gt;
&lt;p&gt;The first resident on the &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt; is a FreeBSD bastille jail&amp;nbsp;named &lt;code&gt;dn42svc&lt;/code&gt;. It runs PowerDNS authoritative for&amp;nbsp;the &lt;code&gt;chofstede.dn42&lt;/code&gt; zone, plus nginx&amp;nbsp;for &lt;code&gt;blog.chofstede.dn42&lt;/code&gt;. The jail itself is unremarkable - it&amp;#8217;s a vnet jail with one network interface&amp;nbsp;(&lt;code&gt;vnet0&lt;/code&gt;) bridged into the &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;VLAN&lt;/span&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@dn42svc:/ # ifconfig vnet0
vnet0: flags=1008843&amp;lt;UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP&amp;gt; metric 0 mtu 1500
        description: jail interface for bridge42
        ether 58:9c:fc:10:27:50
        inet  172.21.66.194 netmask 0xffffffe0 broadcast 172.21.66.223
        inet6 fdce:73f7:a2dc::194 prefixlen 64
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@dn42svc:/ # ps aux | grep -E &amp;#39;pdns|nginx&amp;#39;
root  97962  /usr/local/sbin/pdns_server --daemon --guardian
pdns  97963  /usr/local/sbin/pdns_server-instance (worker)
root  99953  nginx: master process /usr/local/sbin/nginx
www   99954  nginx: worker process
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;PowerDNS is authoritative for three&amp;nbsp;zones:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@dn42svc:/ # pdnsutil list-all-zones
chofstede.dn42
c.d.2.a.7.f.3.7.e.c.d.f.ip6.arpa
192/27.66.21.172.in-addr.arpa
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The forward zone is the obvious one. The two reverse zones are interesting - &lt;span class="caps"&gt;DN42&lt;/span&gt; supports both classless IPv4 reverse delegation (&lt;a href="https://www.rfc-editor.org/rfc/rfc2317"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 2317&lt;/a&gt;, via&amp;nbsp;the &lt;code&gt;192/27.66.21.172.in-addr.arpa&lt;/code&gt; name) and standard IPv6 reverse - and &lt;span class="caps"&gt;DN42&lt;/span&gt; recursive resolvers do the right thing as long as the name servers in&amp;nbsp;your &lt;code&gt;dns/&lt;/code&gt; registry object answer for those&amp;nbsp;zones.&lt;/p&gt;
&lt;p&gt;nginx serves &lt;a href="https://blog.chofstede.dn42"&gt;blog.chofstede.dn42&lt;/a&gt; - the &lt;span class="caps"&gt;URL&lt;/span&gt; from the opening - on the same document root as the public site. Two virtual hosts, two certificates from two completely different CAs (Let&amp;#8217;s Encrypt&amp;nbsp;for &lt;code&gt;blog.hofstede.it&lt;/code&gt;, the &lt;span class="caps"&gt;DN42&lt;/span&gt; internal &lt;span class="caps"&gt;CA&lt;/span&gt;&amp;nbsp;for &lt;code&gt;blog.chofstede.dn42&lt;/code&gt;), one set of files. The &lt;span class="caps"&gt;DN42&lt;/span&gt; cert is renewed by a small script that talks to the &lt;span class="caps"&gt;DN42&lt;/span&gt; &lt;span class="caps"&gt;ACME&lt;/span&gt;-style endpoint described in the &lt;a href="https://wiki.dn42.dev/howto/Tls-Certificate"&gt;&lt;span class="caps"&gt;TLS&lt;/span&gt; howto&lt;/a&gt;; the public cert keeps doing what it has always done. Two front doors to the same&amp;nbsp;building.&lt;/p&gt;
&lt;h3 id="a-routeros-nd-quirk"&gt;A RouterOS &lt;span class="caps"&gt;ND&lt;/span&gt;&amp;nbsp;Quirk&lt;/h3&gt;
&lt;p&gt;There is one interesting edge case in this otherwise simple setup. The router&amp;#8217;s IPv6 neighbor cache for the jail occasionally went stale, even though the jail was clearly up. ICMPv6&amp;nbsp;to &lt;code&gt;fdce:73f7:a2dc::194&lt;/code&gt; would briefly fail until something else triggered a fresh neighbor&amp;nbsp;solicitation.&lt;/p&gt;
&lt;p&gt;The fix is a static neighbor entry on the&amp;nbsp;router:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/ipv6 neighbor
add address=fdce:73f7:a2dc::194 interface=ether1 mac-address=58:9C:FC:10:27:50 \
    comment=&amp;quot;Static ND: RouterOS doesn&amp;#39;t self-probe failed entries&amp;quot;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The comment is the answer: RouterOS&amp;#8217;s neighbor discovery does not aggressively re-probe entries it has marked as failed, so a single missed neighbor advertisement during a busy moment can leave the entry in a state where the router will not retry on its own. A static entry sidesteps the problem entirely. This is the kind of small, undocumented behavior that you only discover by tcpdumping the link when something stops working, and it&amp;#8217;s worth recording in the configuration file with a comment so the next debugger doesn&amp;#8217;t have to rediscover&amp;nbsp;it.&lt;/p&gt;
&lt;h2 id="what-this-looks-like-from-the-outside"&gt;What This Looks Like From The&amp;nbsp;Outside&lt;/h2&gt;
&lt;p&gt;Once everything was up, I checked the connection from outside &lt;span class="caps"&gt;DN42&lt;/span&gt; by visiting one of &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s looking-glass services. The screenshot at the top of this article is from a remote &lt;span class="caps"&gt;DN42&lt;/span&gt; node showing my &lt;span class="caps"&gt;IP&lt;/span&gt;, a working speed test (171/37 Mbps - my home uplink, with WireGuard overhead), and a peer view showing my &lt;span class="caps"&gt;AS&lt;/span&gt; in the &lt;span class="caps"&gt;BGP&lt;/span&gt; table. That&amp;#8217;s a satisfying confirmation that the prefix made it across the mesh and that traffic from a remote &lt;span class="caps"&gt;DN42&lt;/span&gt; node correctly returns through my router and arrives where it&amp;nbsp;should.&lt;/p&gt;
&lt;p&gt;For day-to-day verification, the looking glasses&amp;nbsp;at &lt;code&gt;lg.dn42.dev&lt;/code&gt; and &lt;code&gt;lg.kioubit.dev&lt;/code&gt; are easier to use. They show the &lt;span class="caps"&gt;AS&lt;/span&gt; path of any prefix from their vantage point. Healthy paths through Lare or one of the other peers, no &lt;span class="caps"&gt;AS&lt;/span&gt;-path prepending, and no rejected route notifications: the network is in a stable&amp;nbsp;state.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; is a low-stakes way to actually use &lt;span class="caps"&gt;BGP&lt;/span&gt;.&lt;/strong&gt; Reading about route-maps is one thing; getting an inbound filter wrong and watching peer convergence happen in real time is another. Nothing on the public internet was at risk for any of this, which made it easy to&amp;nbsp;experiment.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The registry workflow is genuinely good.&lt;/strong&gt; Maintaining a network resource as a file in a git repository, signed by a &lt;span class="caps"&gt;PGP&lt;/span&gt; key, reviewed by humans, is the kind of thing the public internet&amp;#8217;s resource provisioning has been trying to evolve into for fifteen years. &lt;span class="caps"&gt;DN42&lt;/span&gt; has been doing it from day&amp;nbsp;one.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Link-local IPv6 &lt;span class="caps"&gt;BGP&lt;/span&gt; on a WireGuard interface just works.&lt;/strong&gt; No transfer network, no manual address allocation between peers, no coordination on which side&amp;nbsp;gets &lt;code&gt;.1&lt;/code&gt; and which side&amp;nbsp;gets &lt;code&gt;.2&lt;/code&gt;. Both sides pick a link-local address (often encoding their &lt;span class="caps"&gt;AS&lt;/span&gt; in the host part), put each other&amp;#8217;s link-local in the &lt;span class="caps"&gt;BGP&lt;/span&gt; neighbor config with the right zone-id, and the session comes up. IPv4 &lt;span class="caps"&gt;BGP&lt;/span&gt; requires more work (multihop, static routes to the peer), which is one more reason to prefer v6 in &lt;span class="caps"&gt;DN42&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;MSS&lt;/span&gt; clamping is not optional on a 1360-byte tunnel.&lt;/strong&gt; Without it, large &lt;span class="caps"&gt;TCP&lt;/span&gt; transfers stall in non-obvious ways - the first few packets work, then everything pauses while the sender tries to figure out the path &lt;span class="caps"&gt;MTU&lt;/span&gt; and gives up.&amp;nbsp;The &lt;code&gt;tcp-mss=!0-1320&lt;/code&gt; condition on the output rule is what makes the &lt;span class="caps"&gt;BGP&lt;/span&gt; session itself survive on the&amp;nbsp;tunnel.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;RouterOS &lt;span class="caps"&gt;BGP&lt;/span&gt; is workable for a small site.&lt;/strong&gt; It&amp;#8217;s verbose compared to &lt;span class="caps"&gt;FRR&lt;/span&gt; and the documentation is patchier, but the model is consistent: instance, template, connection, filter rules. Having a&amp;nbsp;separate &lt;code&gt;lare-in&lt;/code&gt; chain that bumps local-pref before jumping into the standard import chain is exactly the kind of composable filter design that makes per-peer policy&amp;nbsp;easy.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;CHR&lt;/span&gt; on bhyve is a perfectly serviceable border router.&lt;/strong&gt; A 256 &lt;span class="caps"&gt;MB&lt;/span&gt; virtio guest with one vCPU is more than enough for &lt;span class="caps"&gt;DN42&lt;/span&gt;-scale traffic, and running the router as a bhyve &lt;span class="caps"&gt;VM&lt;/span&gt; on the same host as other jails means I get &lt;span class="caps"&gt;ZFS&lt;/span&gt; snapshots, easy backups, and the ability to spin up a second &lt;span class="caps"&gt;CHR&lt;/span&gt; for testing without touching hardware. The free &lt;span class="caps"&gt;CHR&lt;/span&gt; tier&amp;#8217;s bandwidth cap matters for production transit but is irrelevant for a hobby &lt;span class="caps"&gt;AS&lt;/span&gt; - the router moves a few packets per second on a quiet&amp;nbsp;day.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;NAT66&lt;/span&gt; is fine inside &lt;span class="caps"&gt;DN42&lt;/span&gt;, regardless of what &lt;span class="caps"&gt;RFC&lt;/span&gt; 7157 says.&lt;/strong&gt; The constraints are different: there&amp;#8217;s no scarcity argument against IPv6 &lt;span class="caps"&gt;NAT&lt;/span&gt;, but renumbering an entire home &lt;span class="caps"&gt;LAN&lt;/span&gt; into a hobby-network prefix has its own costs. The pragmatic answer is to &lt;span class="caps"&gt;NAT&lt;/span&gt;, give the segments that need real &lt;span class="caps"&gt;DN42&lt;/span&gt; addresses a separate &lt;span class="caps"&gt;VLAN&lt;/span&gt;, and move&amp;nbsp;on.&lt;/p&gt;
&lt;div class="about-cta"&gt;
&lt;p&gt;&lt;strong&gt;Interested in peering on &lt;span class="caps"&gt;DN42&lt;/span&gt;?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;If you&amp;#8217;d like to set up a WireGuard peering with &lt;strong&gt;&lt;span class="caps"&gt;AS4242422539&lt;/span&gt;&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;fdce:73f7:a2dc::/48&lt;/code&gt;, &lt;code&gt;172.21.66.192/27&lt;/code&gt;) - either to test your own &lt;span class="caps"&gt;DN42&lt;/span&gt; setup or just to add another peer to your mesh - send a mail to &lt;a href="mailto:peering@hofstede.it"&gt;peering@hofstede.it&lt;/a&gt; or open an issue on the registry. I have spare WireGuard ports, and new &lt;span class="caps"&gt;DN42&lt;/span&gt; operators are explicitly&amp;nbsp;welcome.&lt;/p&gt;
&lt;/div&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;A MikroTik in a corner of my flat now originates two prefixes into a continental hobby mesh, and a FreeBSD jail behind it serves &lt;span class="caps"&gt;DNS&lt;/span&gt; and &lt;span class="caps"&gt;HTTP&lt;/span&gt; under&amp;nbsp;a &lt;code&gt;.dn42&lt;/code&gt; name. It is not reachable from the public internet, and that is the point: &lt;span class="caps"&gt;DN42&lt;/span&gt; is close enough to the real routing table to teach the operational craft, but small enough that mistakes remain&amp;nbsp;survivable.&lt;/p&gt;
&lt;p&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; is one of those projects where the best way to understand it is to join it. Reading about a parallel &lt;span class="caps"&gt;BGP&lt;/span&gt; network only takes you so far; setting up a node, watching your prefix appear in someone else&amp;#8217;s looking glass, and pinging an anycasted shell on the other side of Europe is what makes it click. The whole thing is small enough to fit in a corner of a home network and large enough to teach you almost everything you&amp;#8217;d want to know about running a real&amp;nbsp;one.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://wiki.dn42.dev/"&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; Wiki&lt;/a&gt; - the canonical entry point: how to register, how to peer, how to run a&amp;nbsp;node&lt;/li&gt;
&lt;li&gt;&lt;a href="https://git.dn42.dev/dn42/registry"&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; Registry&lt;/a&gt; - the git repository where every resource&amp;nbsp;lives&lt;/li&gt;
&lt;li&gt;&lt;a href="https://dn42.us/peers/"&gt;&lt;span class="caps"&gt;DN42&lt;/span&gt; Peer Finder&lt;/a&gt; - matchmaking service for new&amp;nbsp;peers&lt;/li&gt;
&lt;li&gt;&lt;a href="https://lg.kioubit.dev/"&gt;Kioubit&amp;#8217;s Looking Glass&lt;/a&gt; - one of several &lt;span class="caps"&gt;DN42&lt;/span&gt; looking glasses for verifying&amp;nbsp;paths&lt;/li&gt;
&lt;li&gt;&lt;a href="https://help.mikrotik.com/docs/display/ROS/BGP"&gt;MikroTik RouterOS &lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/a&gt; - filter chains, connection templates, route-map&amp;nbsp;equivalents&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc2317"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 2317&lt;/a&gt; - classless &lt;span class="caps"&gt;IN&lt;/span&gt;-&lt;span class="caps"&gt;ADDR&lt;/span&gt;.&lt;span class="caps"&gt;ARPA&lt;/span&gt; delegation, used by &lt;span class="caps"&gt;DN42&lt;/span&gt;&amp;#8217;s reverse-&lt;span class="caps"&gt;DNS&lt;/span&gt;&amp;nbsp;scheme&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc4193"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 4193&lt;/a&gt; - Unique Local IPv6 Unicast Addresses, the canonical reference for&amp;nbsp;the &lt;code&gt;fd00::/8&lt;/code&gt; allocation &lt;span class="caps"&gt;DN42&lt;/span&gt; lives&amp;nbsp;in&lt;/li&gt;
&lt;/ul&gt;</content><category term="Networking"/><category term="dn42"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="mikrotik"/><category term="routeros"/><category term="wireguard"/><category term="freebsd"/><category term="bastille"/><category term="powerdns"/></entry><entry><title>Running Your Own AS: Direct Hetzner Peering, a Fourth Edge, and Bringing the Home LAN into the Fabric</title><link href="https://hofstede.it/running-your-own-as-direct-hetzner-peering-a-fourth-edge-and-bringing-the-home-lan-into-the-fabric/" rel="alternate"/><published>2026-04-16T00:00:00+02:00</published><updated>2026-04-16T00:00:00+02:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-04-16:/running-your-own-as-direct-hetzner-peering-a-fourth-edge-and-bringing-the-home-lan-into-the-fabric/</id><summary type="html">&lt;p&gt;Part 4 of the &lt;span class="caps"&gt;AS201379&lt;/span&gt; journey: adding a fourth FreeBSD edge router at iFog with FogIXP peering, establishing direct &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions with Hetzner, bringing the home network into the &lt;span class="caps"&gt;AS&lt;/span&gt; via an iBGP-speaking MikroTik, and a little traffic engineering to steer Deutsche Telekom traffic over&amp;nbsp;Vultr.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-04-16-as201379-part4-direct-peering-home-network.png" title="Part 4: Header image"&gt;&lt;/p&gt;
&lt;h2 id="table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/h2&gt;
&lt;div class="toc"&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="#table-of-contents"&gt;Table of&amp;nbsp;Contents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#architecture-four-edges-and-a-home-router"&gt;Architecture: Four Edges and a Home&amp;nbsp;Router&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-fourth-edge-ixbgp-at-ifog-and-fogixp"&gt;The Fourth Edge: ixbgp at iFog and FogIXP&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#why-a-fourth-edge"&gt;Why a Fourth&amp;nbsp;Edge&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#network-configuration"&gt;Network&amp;nbsp;Configuration&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#frr-configuration"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#pf-stateless-transit-stateful-control-plane"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt;: Stateless Transit, Stateful Control&amp;nbsp;Plane&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#direct-peering-with-hetzner"&gt;Direct Peering with Hetzner&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#the-path-before"&gt;The Path&amp;nbsp;Before&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-path-now"&gt;The Path&amp;nbsp;Now&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-hetzner-sees"&gt;What Hetzner&amp;nbsp;Sees&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#what-this-buys-in-practice"&gt;What This Buys in&amp;nbsp;Practice&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#bringing-the-home-lan-into-as201379"&gt;Bringing the Home &lt;span class="caps"&gt;LAN&lt;/span&gt; into &lt;span class="caps"&gt;AS201379&lt;/span&gt;&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#the-gap-in-parts-1-3"&gt;The Gap in Parts&amp;nbsp;1-3&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#mikrotik-configuration"&gt;MikroTik&amp;nbsp;Configuration&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-core-side-configuration"&gt;The Core-Side&amp;nbsp;Configuration&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#traffic-engineering-steering-dtag-via-vultr"&gt;Traffic Engineering: Steering &lt;span class="caps"&gt;DTAG&lt;/span&gt; via Vultr&lt;/a&gt;&lt;ul&gt;
&lt;li&gt;&lt;a href="#the-observation"&gt;The&amp;nbsp;Observation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-implementation"&gt;The&amp;nbsp;Implementation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#the-effect"&gt;The&amp;nbsp;Effect&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;a href="#hub-hygiene-one-ip-for-traceroutes"&gt;Hub Hygiene: One &lt;span class="caps"&gt;IP&lt;/span&gt; for&amp;nbsp;Traceroutes&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#downstream-sites-pi-addressing-for-friends-and-services"&gt;Downstream Sites: &lt;span class="caps"&gt;PI&lt;/span&gt; Addressing for Friends and&amp;nbsp;Services&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#conclusion"&gt;Conclusion&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="#references"&gt;References&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/div&gt;
&lt;p&gt;&lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Part 1&lt;/a&gt; set up a single FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; router with two upstream providers. &lt;a href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/"&gt;Part 2&lt;/a&gt; added a Vultr edge with native peering and tied both routers together with iBGP. &lt;a href="https://hofstede.it/running-your-own-as-joining-an-ixp-with-a-third-edge-router/"&gt;Part 3&lt;/a&gt; joined LocIX Düsseldorf with a dedicated third edge router. This is Part&amp;nbsp;4.&lt;/p&gt;
&lt;p&gt;A few months of operating a multi-PoP &lt;span class="caps"&gt;BGP&lt;/span&gt; network produces a shopping list. I wanted direct peering with networks that move real traffic, a fourth edge in a new PoP, and my own IPv6 space on the home &lt;span class="caps"&gt;LAN&lt;/span&gt; instead of &lt;span class="caps"&gt;ISP&lt;/span&gt;-assigned addressing. This article covers the changes that made that&amp;nbsp;happen.&lt;/p&gt;
&lt;p&gt;The headline, if I had to pick one, is&amp;nbsp;two &lt;code&gt;mtr&lt;/code&gt; traces. First, from a nettest jail on my home &lt;span class="caps"&gt;LAN&lt;/span&gt; to Hetzner&amp;#8217;s&amp;nbsp;network:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;root@nettest:~ # mtr -rwz hetzner.com
HOST: nettest                                Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS201379      2a06:9801:1c:6000::1       0.0%    10    0.2   0.2   0.1   0.2   0.0
  2. AS201379      2a06:9801:1c:fff0::1       0.0%    10    3.2   3.2   2.9   3.4   0.2
  3. AS201379      ixbgp.edge.hofstede.it     0.0%    10    6.9   7.0   6.8   7.3   0.2
  4. AS???         2001:7f8:ca:1:0:2:4940:1   0.0%    10    7.6   7.6   7.3   7.9   0.2
  5. AS24940       core11.nbg1.hetzner.com    0.0%    10   10.9  10.7  10.5  10.9   0.1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Hop 1 is the home &lt;span class="caps"&gt;LAN&lt;/span&gt; gateway - a MikroTik announcing a /64 from my /48 via iBGP. Hop 2 is the iBGP tunnel to the core router. Hop 3 is the new fourth edge&amp;nbsp;(&lt;code&gt;ixbgp&lt;/code&gt;) at the iFog datacenter in Zürich. Hop 4 is Hetzner&amp;#8217;s peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; address on FogIXP in Frankfurt. &lt;strong&gt;That&amp;#8217;s a direct &lt;span class="caps"&gt;BGP&lt;/span&gt; session between &lt;span class="caps"&gt;AS201379&lt;/span&gt; and &lt;span class="caps"&gt;AS24940&lt;/span&gt;&lt;/strong&gt;. Hop 5 is Hetzner&amp;#8217;s backbone. Five hops, 10 ms, and everything in the middle is either mine or&amp;nbsp;Hetzner.&lt;/p&gt;
&lt;p&gt;The return path, from a Hetzner machine back&amp;nbsp;to &lt;code&gt;blog.hofstede.it&lt;/code&gt;, completes the&amp;nbsp;picture:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;[root@krypton ~]# mtr -rwz blog.hofstede.it
HOST: krypton                              Loss%   Snt   Last   Avg  Best  Wrst StDev
  4. AS???    2001:7f8:ca:1:0:20:1379:1     0.0%    10    5.1   5.1   5.1   5.2   0.0
  5. AS201379 ifog-gw.core.hofstede.it      0.0%    10    9.2  14.3   8.9  61.1  16.4
  6. AS201379 radon.server.hofstede.it      0.0%    10    9.9  10.2   9.7  10.9   0.3
  7. AS201379 hofstede.it                   0.0%    10   10.0  10.3   9.9  11.1   0.4
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Hop 4 is &lt;span class="caps"&gt;AS201379&lt;/span&gt;&amp;#8217;s own peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; address on FogIXP - Hetzner hands traffic for my prefix directly to my router. From there it&amp;#8217;s three hops inside &lt;span class="caps"&gt;AS201379&lt;/span&gt; to the jail serving the blog. No intermediate transit &lt;span class="caps"&gt;AS&lt;/span&gt;, no &lt;span class="caps"&gt;DE&lt;/span&gt;-&lt;span class="caps"&gt;CIX&lt;/span&gt; hop, no shared upstream - just a direct handoff at the&amp;nbsp;exchange.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; &lt;span class="caps"&gt;AS201379&lt;/span&gt;&amp;#8217;s&amp;nbsp;prefix &lt;code&gt;2a06:9801:1c::/48&lt;/code&gt; and&amp;nbsp;all &lt;code&gt;hofstede.it&lt;/code&gt; hostnames are shown as-is. Public peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; addresses from PeeringDB&amp;nbsp;(FogIXP&amp;#8217;s &lt;code&gt;2001:7f8:ca:1::/64&lt;/code&gt;,&amp;nbsp;LocIX&amp;#8217;s &lt;code&gt;185.1.155.0/24&lt;/code&gt; and &lt;code&gt;2a0c:b641:701::/64&lt;/code&gt;) are also shown as-is. Everything else - provider-assigned addresses, tunnel endpoints, trusted-management IPs - has been replaced with &lt;a href="https://www.rfc-editor.org/rfc/rfc5737"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5737&lt;/a&gt; / &lt;a href="https://www.rfc-editor.org/rfc/rfc3849"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 3849&lt;/a&gt; documentation ranges. Upstream and peer &lt;span class="caps"&gt;AS&lt;/span&gt; numbers are visible in public routing&amp;nbsp;tables.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="at-a-glance"&gt;At a&amp;nbsp;Glance&lt;/h2&gt;
&lt;p&gt;Part 4 adds three things to &lt;span class="caps"&gt;AS201379&lt;/span&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;a fourth FreeBSD edge router at iFog Zürich&amp;nbsp;(&lt;code&gt;ixbgp&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;a direct &lt;span class="caps"&gt;BGP&lt;/span&gt; peering session with Hetzner on&amp;nbsp;FogIXP&lt;/li&gt;
&lt;li&gt;a MikroTik home router speaking iBGP, bringing the home &lt;span class="caps"&gt;LAN&lt;/span&gt; into the&amp;nbsp;/48&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The result is lower-latency paths to Hetzner, more control over outbound policy, and stable provider-independent IPv6 at&amp;nbsp;home.&lt;/p&gt;
&lt;h2 id="architecture-four-edges-and-a-home-router"&gt;Architecture: Four Edges and a Home&amp;nbsp;Router&lt;/h2&gt;
&lt;p&gt;The topology grew in two dimensions. A fourth FreeBSD edge router&amp;nbsp;(&lt;code&gt;ixbgp&lt;/code&gt;) joins the existing three at a new PoP.&amp;nbsp;Below &lt;code&gt;hobgp&lt;/code&gt;, a new downstream site appears: the home &lt;span class="caps"&gt;LAN&lt;/span&gt;, reachable via an iBGP-speaking&amp;nbsp;MikroTik.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────────────────────────┐
                    │                Default-Free Zone                 │
                    └──┬──────────┬──────────────┬──────────────┬──────┘
                       │          │              │              │
                   AS209533   AS209735       AS212895         AS34927
                   (iFog free) (Lagrange)    (route64)       (iFog paid)
                       │          │              │              │
                    GRE          GRE            GRE            eBGP over vtnet0
                       │          │              │              │      +
                       │          │              │              │  FogIXP RS1/2/3 (AS47498)
                       │          │              │              │      +
                       │          │              │              │  AS24940 (Hetzner, direct)
                  ┌────┴──────────┴──────────────┴────┐   ┌─────┴────────────────────────┐
                  │             hobgp (Core)          │   │       ixbgp (Edge - iFog)    │
                  │       FreeBSD + FRR, AS201379     │◄──┤  FreeBSD + FRR, AS201379     │
                  │       2a06:9801:1c::/48           │iBGP  FogIXP peering LAN          │
                  └─┬───────────┬─────────────┬───────┘   └──────────────────────────────┘
                    │           │             │
               iBGP GIF   iBGP GIF       iBGP 6to4
                    │           │             │
            ┌───────┴───────┐ ┌─┴──────────┐ ┌┴────────────────────────────────┐
            │ vtbgp (Vultr) │ │lobgp(LocIX)│ │ MikroTik (Home LAN)             │
            │ Native AS64515│ │Servperso   │ │ RouterOS 7.20, AS201379         │
            │               │ │+ LocIX RS  │ │ Announces 2a06:9801:1c:6000::/64│
            └───────────────┘ └────────────┘ └─────────────────────────────────┘

Downstream tunnels from hobgp (provider-independent IPv6 for services):
   :1000::/64  radon      (blog, DNS, Gemini, fedi comments)
   :2000::/64  colo       (Hetzner colo OPNsense)
   :3000::/64  mail       (mail.linuxserver.pro)
   :4000::/64  bb         (burningboard.net)
   :5000::/64  road       (road-warrior network)
   :6000::/64  home       (home LAN via MikroTik)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;hobgp&lt;/code&gt; at Hetzner Nuremberg remains the hub from Part 2 - the core router that every edge speaks iBGP to, and every downstream site tunnels&amp;nbsp;through. &lt;code&gt;ixbgp&lt;/code&gt; is the new announcement point at iFog Zürich with a direct link to FogIXP Frankfurt. The home network is a downstream site on the same tunnel pattern as radon, but runs iBGP instead of static&amp;nbsp;routing.&lt;/p&gt;
&lt;p&gt;Downstream /64s ride &lt;span class="caps"&gt;GIF&lt;/span&gt; or 6to4 tunnels rather than native transport because most hosting providers don&amp;#8217;t let customers announce foreign prefixes on their infrastructure. A tunnel gives each site a direct layer-3 path back to a router that can originate and route the&amp;nbsp;prefix.&lt;/p&gt;
&lt;h2 id="the-fourth-edge-ixbgp-at-ifog-and-fogixp"&gt;The Fourth Edge: ixbgp at iFog and&amp;nbsp;FogIXP&lt;/h2&gt;
&lt;h3 id="why-a-fourth-edge"&gt;Why a Fourth&amp;nbsp;Edge&lt;/h3&gt;
&lt;p&gt;The free BGPTunnel service I used in Parts 1-3 is a great on-ramp, but it&amp;#8217;s operated as a best-effort community service. Pairing it with a paid iFog &lt;span class="caps"&gt;BGP&lt;/span&gt; port gives me a real transit contract and something the free tunnel does not: a &lt;span class="caps"&gt;VM&lt;/span&gt; with a second &lt;span class="caps"&gt;NIC&lt;/span&gt; directly on FogIXP&amp;#8217;s peering &lt;span class="caps"&gt;LAN&lt;/span&gt;. FogIXP is a smaller &lt;span class="caps"&gt;IXP&lt;/span&gt; than LocIX but has a different participant mix - notably, a direct Hetzner presence on that&amp;nbsp;fabric.&lt;/p&gt;
&lt;p&gt;The architecture&amp;nbsp;mirrors &lt;code&gt;lobgp&lt;/code&gt; from Part 3: two physical interfaces (internet-facing and peering-&lt;span class="caps"&gt;LAN&lt;/span&gt;-facing), an iBGP &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel back&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;, route-maps per peer. The main addition is the direct Hetzner&amp;nbsp;session.&lt;/p&gt;
&lt;h3 id="network-configuration"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;/etc/rc.conf&lt;/code&gt; on &lt;code&gt;ixbgp&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;ixbgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Internet side (iFog)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 198.51.100.176 netmask 255.255.255.0 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;198.51.100.1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:1030::1229 prefixlen 48 -rxcsum6 -tso6&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:1030::1&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# FogIXP peering LAN (direct L2)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet1&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;up -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:7f8:ca:1::20:1379:1 prefixlen 64 -rxcsum6 -tso6&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# iBGP tunnel to core&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.176 198.51.100.10 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffc::2 2a06:9801:1c:fffc::1 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Uplink-to-hobgp&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;myblock&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_myblock&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48 -interface gif0&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;pf_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_daemons&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;mgmtd zebra bgpd bfdd staticd&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The peering-&lt;span class="caps"&gt;LAN&lt;/span&gt;&amp;nbsp;address &lt;code&gt;2001:7f8:ca:1::20:1379:1&lt;/code&gt; encodes my &lt;span class="caps"&gt;ASN&lt;/span&gt; in the interface &lt;span class="caps"&gt;ID&lt;/span&gt;&amp;nbsp;(&lt;code&gt;20:1379&lt;/code&gt; → &lt;span class="caps"&gt;AS201379&lt;/span&gt;), which is a common convention on &lt;span class="caps"&gt;IXP&lt;/span&gt; LANs. FogIXP assigns these statically;&amp;nbsp;the &lt;code&gt;/64&lt;/code&gt; is the peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; subnet shared by every&amp;nbsp;participant.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;ipv6_route_myblock="2a06:9801:1c::/48 -interface gif0"&lt;/code&gt; is the same trick as&amp;nbsp;on &lt;code&gt;lobgp&lt;/code&gt;: any traffic arriving&amp;nbsp;at &lt;code&gt;ixbgp&lt;/code&gt; for the /48 gets forwarded through the iBGP tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;. &lt;code&gt;ixbgp&lt;/code&gt; originates the route, but forwarding still happens&amp;nbsp;through &lt;code&gt;hobgp&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id="frr-configuration"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;Five &lt;span class="caps"&gt;BGP&lt;/span&gt; neighbors: three FogIXP route servers, the Hetzner direct peer, the paid iFog transit, and iBGP back to the core. The excerpt below omits the full bogon and hygiene policy for brevity (see Part 1 for the&amp;nbsp;complete &lt;code&gt;PL-BOGONS&lt;/code&gt; list); only the per-peer local-preference differences matter&amp;nbsp;here.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;frr version 10.5.1
hostname ixbgp
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
!
! [PL-BOGONS: same as Part 1, trimmed for brevity]
!
route-map RM-IXP-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 300
exit
!
route-map RM-HETZNER-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 301
exit
!
route-map RM-IFOG-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 100
exit
!
route-map RM-IXP-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-HETZNER-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IFOG-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IBGP-OUT permit 10
exit
!
router bgp 201379
 bgp router-id 198.51.100.176
 no bgp default ipv4-unicast
 no bgp enforce-first-as
 neighbor 2001:db8:1030::1 remote-as 34927
 neighbor 2001:db8:1030::1 description Transit-iFog-Paid
 neighbor 2a06:9801:1c:fffc::1 remote-as 201379
 neighbor 2a06:9801:1c:fffc::1 description Core-Hetzner
 neighbor 2a06:9801:1c:fffc::1 update-source 2a06:9801:1c:fffc::2
 neighbor 2001:7f8:ca:1::111 remote-as 47498
 neighbor 2001:7f8:ca:1::111 description FogIXP-RS1
 neighbor 2001:7f8:ca:1::222 remote-as 47498
 neighbor 2001:7f8:ca:1::222 description FogIXP-RS2
 neighbor 2001:7f8:ca:1::333 remote-as 47498
 neighbor 2001:7f8:ca:1::333 description FogIXP-RS3
 neighbor 2001:7f8:ca:1:0:2:4940:1 remote-as 24940
 neighbor 2001:7f8:ca:1:0:2:4940:1 description Hetzner-Peer
 !
 address-family ipv6 unicast
  neighbor 2001:db8:1030::1 activate
  neighbor 2001:db8:1030::1 soft-reconfiguration inbound
  neighbor 2001:db8:1030::1 route-map RM-IFOG-IN in
  neighbor 2001:db8:1030::1 route-map RM-IFOG-OUT out
  neighbor 2a06:9801:1c:fffc::1 activate
  neighbor 2a06:9801:1c:fffc::1 next-hop-self
  neighbor 2a06:9801:1c:fffc::1 route-map RM-IBGP-OUT out
  neighbor 2001:7f8:ca:1::111 activate
  neighbor 2001:7f8:ca:1::111 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::111 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::111 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1::222 activate
  neighbor 2001:7f8:ca:1::222 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::222 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::222 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1::333 activate
  neighbor 2001:7f8:ca:1::333 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1::333 route-map RM-IXP-IN in
  neighbor 2001:7f8:ca:1::333 route-map RM-IXP-OUT out
  neighbor 2001:7f8:ca:1:0:2:4940:1 activate
  neighbor 2001:7f8:ca:1:0:2:4940:1 soft-reconfiguration inbound
  neighbor 2001:7f8:ca:1:0:2:4940:1 route-map RM-HETZNER-IN in
  neighbor 2001:7f8:ca:1:0:2:4940:1 route-map RM-HETZNER-OUT out
 exit-address-family
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The local-preference scheme is&amp;nbsp;deliberate:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Hetzner direct (&lt;span class="caps"&gt;LP&lt;/span&gt; 301)&lt;/strong&gt; - highest. A direct session with a network the size of Hetzner is preferable to&amp;nbsp;transit.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;FogIXP route servers (&lt;span class="caps"&gt;LP&lt;/span&gt; 300)&lt;/strong&gt; - next. Routes learned via the exchange are free to use and generally&amp;nbsp;short.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;iFog paid transit (&lt;span class="caps"&gt;LP&lt;/span&gt; 100)&lt;/strong&gt; - lowest. Transit is the fallback, so it gets the lowest local&amp;nbsp;preference.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The one-point gap is deliberate: if I learn a Hetzner route both directly and via the route servers, the direct session wins. That keeps path selection predictable and avoids relying on route-server propagation&amp;nbsp;behavior.&lt;/p&gt;
&lt;p&gt;FogIXP&amp;#8217;s route servers are &amp;#8220;transparent&amp;#8221; - they don&amp;#8217;t prepend their own &lt;span class="caps"&gt;AS&lt;/span&gt; onto forwarded routes, so the first &lt;span class="caps"&gt;AS&lt;/span&gt; in the path is the actual origin, not the route server itself. &lt;span class="caps"&gt;FRR&lt;/span&gt; checks for this by default, so route-server sessions usually&amp;nbsp;need &lt;code&gt;no bgp enforce-first-as&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id="pf-stateless-transit-stateful-control-plane"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt;: Stateless Transit, Stateful Control&amp;nbsp;Plane&lt;/h3&gt;
&lt;p&gt;The &lt;span class="caps"&gt;PF&lt;/span&gt; configuration&amp;nbsp;on &lt;code&gt;ixbgp&lt;/code&gt; follows the same pattern established in Parts 2 and 3. The tunnel encapsulation must be stateless (otherwise &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s state table interferes with asymmetric paths), while control-plane &lt;span class="caps"&gt;TCP&lt;/span&gt; sessions must be stateful (&lt;span class="caps"&gt;TCP&lt;/span&gt; needs state to track three-way handshakes). The key&amp;nbsp;bits:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;ext_if    = &amp;quot;vtnet0&amp;quot;
ixp_if    = &amp;quot;vtnet1&amp;quot;
hobgp_tun = &amp;quot;gif0&amp;quot;

my_network_v6 = &amp;quot;2a06:9801:1c::/48&amp;quot;
my_router_ip  = &amp;quot;2a06:9801:1c:fffc::2&amp;quot;

&lt;span class="gh"&gt;#&lt;/span&gt; Outer tunnel encapsulation: stateless
pass in  quick on $ext_if proto { 41, ipencap } from 198.51.100.10 to ($ext_if) no state
pass out quick on $ext_if proto { 41, ipencap } from ($ext_if) to 198.51.100.10 no state

&lt;span class="gh"&gt;#&lt;/span&gt; eBGP sessions: stateful
pass in quick on $ext_if proto tcp from 2001:db8:1030::1 to ($ext_if) port 179 keep state
pass in quick on $ixp_if proto tcp from 2001:7f8:ca:1::/64 to ($ixp_if) port 179 keep state

&lt;span class="gh"&gt;#&lt;/span&gt; Transit data plane: stateless (essential for DSR and inner path asymmetry)
pass in  quick inet6 from any to $my_network_v6 no state
pass out quick inet6 from any to $my_network_v6 no state
pass in  quick inet6 from $my_network_v6 to any no state
pass out quick inet6 from $my_network_v6 to any no state

&lt;span class="gh"&gt;#&lt;/span&gt; IXP best practice: don&amp;#39;t leak multicast onto the peering LAN
block out quick on $ixp_if to { 224.0.0.0/4, 255.255.255.255, ff02::/16 }
block out quick on $ixp_if proto { igmp, ospf, pim, vrrp, gre }

&lt;span class="gh"&gt;#&lt;/span&gt; Monitoring: allow-list from trusted scrapers only
pass  in quick inet6 proto tcp from { 2a06:9801:1c:2000::21, 2a06:9801:1c:2000::25 } \
    to $my_router_ip port { 9100, 9342 } keep state
block in quick proto tcp from any to any port { 9100, 9342 }
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The routing-protocol block rule is &lt;span class="caps"&gt;IXP&lt;/span&gt;-specific: &lt;span class="caps"&gt;OSPF&lt;/span&gt; hellos and the like have no business leaking onto a shared peering &lt;span class="caps"&gt;LAN&lt;/span&gt;, and most IXPs explicitly require participants to block them. The multicast block rule is the same principle - anything that&amp;#8217;s not unicast peering traffic gets dropped before it hits the&amp;nbsp;fabric.&lt;/p&gt;
&lt;h2 id="direct-peering-with-hetzner"&gt;Direct Peering with&amp;nbsp;Hetzner&lt;/h2&gt;
&lt;h3 id="the-path-before"&gt;The Path&amp;nbsp;Before&lt;/h3&gt;
&lt;p&gt;Before the FogIXP session, traffic from my home network to a Hetzner server&amp;nbsp;went:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;home MikroTik → hobgp → iFog BGPTunnel → iFog&amp;#39;s upstream → DE-CIX fabric
  → AS24940 Hetzner core → target
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Four transit hops after leaving my network, round-trip time in the 30-40 ms range despite Frankfurt and Nuremberg being ~180 km&amp;nbsp;apart.&lt;/p&gt;
&lt;h3 id="the-path-now"&gt;The Path&amp;nbsp;Now&lt;/h3&gt;
&lt;p&gt;Post-FogIXP:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;home MikroTik → hobgp → ixbgp (Zürich) → FogIXP fabric → Hetzner backbone → target
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;One exchange point, one peering session, one handoff.&amp;nbsp;The &lt;code&gt;mtr&lt;/code&gt; at the top of this article shows ~10 ms end-to-end - not because the physical distance changed, but because the routing got&amp;nbsp;shorter.&lt;/p&gt;
&lt;h3 id="what-hetzner-sees"&gt;What Hetzner&amp;nbsp;Sees&lt;/h3&gt;
&lt;p&gt;Hetzner&amp;#8217;s routers now&amp;nbsp;learn &lt;code&gt;2a06:9801:1c::/48&lt;/code&gt; directly from &lt;span class="caps"&gt;AS201379&lt;/span&gt;, with no transit &lt;span class="caps"&gt;AS&lt;/span&gt; in between. From any Hetzner-hosted machine, traffic to my prefix exits through the Hetzner-side peering session at FogIXP and lands directly&amp;nbsp;on &lt;code&gt;ixbgp&lt;/code&gt;, which forwards over iBGP&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;bgp.tools&lt;/code&gt; shows the &lt;span class="caps"&gt;AS&lt;/span&gt; path diversity: the prefix is still visible via iFog free, Lagrange, route64, Vultr, and the LocIX route servers, and now via FogIXP including the Hetzner direct session. Traffic originating inside Hetzner generally prefers the direct path; everyone else picks based on their own&amp;nbsp;policy.&lt;/p&gt;
&lt;h3 id="what-this-buys-in-practice"&gt;What This Buys in&amp;nbsp;Practice&lt;/h3&gt;
&lt;p&gt;The practical benefit is not just lower latency to one &lt;span class="caps"&gt;VPS&lt;/span&gt;. Because the /48 is mine and announced by &lt;span class="caps"&gt;AS201379&lt;/span&gt;, every service behind it inherits the direct peering&amp;nbsp;automatically.&lt;/p&gt;
&lt;h2 id="bringing-the-home-lan-into-as201379"&gt;Bringing the Home &lt;span class="caps"&gt;LAN&lt;/span&gt; into &lt;span class="caps"&gt;AS201379&lt;/span&gt;&lt;/h2&gt;
&lt;h3 id="the-gap-in-parts-1-3"&gt;The Gap in Parts&amp;nbsp;1-3&lt;/h3&gt;
&lt;p&gt;Parts 1-3 built an &lt;span class="caps"&gt;AS&lt;/span&gt; that serves my public-facing infrastructure. Downstream servers&amp;nbsp;like &lt;code&gt;radon&lt;/code&gt; or the colo OPNsense get their own /64 routed natively via &lt;span class="caps"&gt;GIF&lt;/span&gt;. My home network, however, used Deutsche Telekom&amp;#8217;s provider-assigned IPv6 -&amp;nbsp;a &lt;code&gt;/56&lt;/code&gt; from &lt;span class="caps"&gt;DTAG&lt;/span&gt; that changes on every reconnect and is useless for inbound&amp;nbsp;connectivity.&lt;/p&gt;
&lt;p&gt;The fix is conceptually identical to&amp;nbsp;what &lt;code&gt;radon&lt;/code&gt; already does: allocate a /64 from my /48, tunnel it to a router at the endpoint, announce it from there. The difference is that the home router is a MikroTik, not FreeBSD, and MikroTik&amp;#8217;s tunnel implementation is 6to4&amp;nbsp;(&lt;code&gt;interface 6to4&lt;/code&gt;) rather than &lt;span class="caps"&gt;GIF&lt;/span&gt;. The iBGP session plugs the home router into the existing core&amp;nbsp;router.&lt;/p&gt;
&lt;h3 id="mikrotik-configuration"&gt;MikroTik&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;RouterOS 7.20 has a usable &lt;span class="caps"&gt;BGP&lt;/span&gt; implementation. The tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; is a 6to4 interface (IPv6-in-IPv4, protocol 41), and iBGP runs over&amp;nbsp;it:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;/interface 6to4
add local-address=203.0.113.86 mtu=1480 name=tun-hobgp \
    remote-address=198.51.100.10

/ipv6 address
add address=2a06:9801:1c:fff0::2/128 advertise=no interface=tun-hobgp
add address=2a06:9801:1c:6000::1 interface=lan

/ipv6 route
add dst-address=2a06:9801:1c:fff0::1/128 gateway=tun-hobgp
add blackhole dst-address=2a06:9801:1c:6000::/64

/routing bgp instance
add as=201379 name=default router-id=203.0.113.86

/routing bgp template
add as=201379 name=ibgp-hobgp

/routing bgp connection
add afi=ipv6 disabled=no instance=default \
    local.address=2a06:9801:1c:fff0::2 .role=ibgp \
    remote.address=2a06:9801:1c:fff0::1 .as=201379 \
    name=hobgp templates=ibgp-hobgp \
    output.filter-chain=bgp-out .redistribute=connected,static

/routing filter rule
add chain=bgp-out rule=&amp;quot;if (dst == 2a06:9801:1c:6000::/64) { accept }&amp;quot;
add chain=bgp-out rule=reject
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Three things are worth calling&amp;nbsp;out:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The blackhole route for the /64.&lt;/strong&gt; Same idea as&amp;nbsp;the &lt;code&gt;-reject&lt;/code&gt; route&amp;nbsp;on &lt;code&gt;hobgp&lt;/code&gt; for the /48: if the MikroTik gets traffic for an address in the /64 that isn&amp;#8217;t assigned, it drops it locally rather than following the default route back out (which would cause a loop). On RouterOS this is expressed as a specific blackhole&amp;nbsp;route.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The output&amp;nbsp;filter.&lt;/strong&gt; &lt;code&gt;bgp-out&lt;/code&gt; only permits the /64 that belongs to this site. Even&amp;nbsp;though &lt;code&gt;redistribute=connected,static&lt;/code&gt; would otherwise announce every connected subnet (including &lt;span class="caps"&gt;LAN&lt;/span&gt; &lt;span class="caps"&gt;RFC&lt;/span&gt; 1918 ranges), the filter strips everything that&amp;nbsp;isn&amp;#8217;t &lt;code&gt;2a06:9801:1c:6000::/64&lt;/code&gt;. This is the MikroTik equivalent of&amp;nbsp;the &lt;code&gt;PL-MY-NET&lt;/code&gt; prefix lists on the FreeBSD&amp;nbsp;edges.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;advertise=no&lt;/code&gt; on the tunnel address.&lt;/strong&gt; The tunnel link&amp;nbsp;address &lt;code&gt;2a06:9801:1c:fff0::2&lt;/code&gt; is&amp;nbsp;a &lt;code&gt;/128&lt;/code&gt; point-to-point - it should not be announced to the &lt;span class="caps"&gt;LAN&lt;/span&gt; as something hosts can &lt;span class="caps"&gt;SLAAC&lt;/span&gt;&amp;nbsp;from.&lt;/p&gt;
&lt;h3 id="the-core-side-configuration"&gt;The Core-Side&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;On &lt;code&gt;hobgp&lt;/code&gt;, the MikroTik is a downstream iBGP peer with a different routing policy than the other edges. The other three edges receive the full table&amp;nbsp;via &lt;code&gt;RM-IBGP-OUT permit 10 | match PL-MY-NET&lt;/code&gt; - they only get our prefix, since that&amp;#8217;s the only thing they need to advertise. The home router gets a default route and nothing&amp;nbsp;else:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;ipv6 prefix-list PL-MIKROTIK-NET seq 5 permit 2a06:9801:1c:6000::/64
!
route-map RM-DOWNSTREAM-IN permit 10
 match ipv6 address prefix-list PL-MIKROTIK-NET
exit
route-map RM-DOWNSTREAM-IN deny 20
exit
!
route-map RM-DEFAULT-OUT deny 10
exit
!
router bgp 201379
 ...
 neighbor 2a06:9801:1c:fff0::2 remote-as 201379
 neighbor 2a06:9801:1c:fff0::2 description Downstream-MikroTik-Lab
 neighbor 2a06:9801:1c:fff0::2 update-source 2a06:9801:1c:fff0::1
 !
 address-family ipv6 unicast
  neighbor 2a06:9801:1c:fff0::2 activate
  neighbor 2a06:9801:1c:fff0::2 default-originate
  neighbor 2a06:9801:1c:fff0::2 soft-reconfiguration inbound
  neighbor 2a06:9801:1c:fff0::2 route-map RM-DOWNSTREAM-IN in
  neighbor 2a06:9801:1c:fff0::2 route-map RM-DEFAULT-OUT out
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;default-originate&lt;/code&gt; generates a&amp;nbsp;synthetic &lt;code&gt;::/0&lt;/code&gt; route and sends it to the&amp;nbsp;MikroTik. &lt;code&gt;RM-DEFAULT-OUT deny 10&lt;/code&gt; prevents any other prefix from being advertised. The inbound filter&amp;nbsp;(&lt;code&gt;RM-DOWNSTREAM-IN&lt;/code&gt;) only&amp;nbsp;accepts &lt;code&gt;2a06:9801:1c:6000::/64&lt;/code&gt; - a safety measure against the home router accidentally announcing something it shouldn&amp;#8217;t. Combined with the MikroTik&amp;#8217;s own outbound filter, this means the worst-case scenario of a misconfigured home lab is a correctly-announced /64 or nothing - never route&amp;nbsp;leakage.&lt;/p&gt;
&lt;p&gt;The result is that a &lt;span class="caps"&gt;PC&lt;/span&gt; on the home &lt;span class="caps"&gt;LAN&lt;/span&gt; now &lt;span class="caps"&gt;SLAAC&lt;/span&gt;-configures&amp;nbsp;from &lt;code&gt;2a06:9801:1c:6000::/64&lt;/code&gt;, keeps a stable globally routable address across &lt;span class="caps"&gt;ISP&lt;/span&gt; reconnects, and exits through the &lt;span class="caps"&gt;AS&lt;/span&gt; like any other downstream site. The home &lt;span class="caps"&gt;LAN&lt;/span&gt; is now just another site in &lt;span class="caps"&gt;AS201379&lt;/span&gt;, indistinguishable from radon&amp;#8217;s /64 as far as the rest of the internet is&amp;nbsp;concerned.&lt;/p&gt;
&lt;h2 id="traffic-engineering-steering-dtag-via-vultr"&gt;Traffic Engineering: Steering &lt;span class="caps"&gt;DTAG&lt;/span&gt; via&amp;nbsp;Vultr&lt;/h2&gt;
&lt;h3 id="the-observation"&gt;The&amp;nbsp;Observation&lt;/h3&gt;
&lt;p&gt;Deutsche Telekom (&lt;span class="caps"&gt;AS3320&lt;/span&gt;) is the largest German eyeball network. A large share of blog readers reach my services over &lt;span class="caps"&gt;DTAG&lt;/span&gt;&amp;#8217;s network, so the return path&amp;nbsp;matters.&lt;/p&gt;
&lt;p&gt;With four edge routers, I have four potential ways to reach &lt;span class="caps"&gt;DTAG&lt;/span&gt;. Watching the &lt;span class="caps"&gt;AS&lt;/span&gt; paths that show up&amp;nbsp;on &lt;code&gt;bgp.tools&lt;/code&gt; and&amp;nbsp;in &lt;code&gt;show ipv6 bgp&lt;/code&gt; on &lt;code&gt;hobgp&lt;/code&gt;, the Vultr edge consistently learned shorter paths to &lt;span class="caps"&gt;DTAG&lt;/span&gt; than the others - Vultr&amp;#8217;s upstream connectivity into &lt;span class="caps"&gt;DTAG&lt;/span&gt; is short and direct in ways that the transit providers&amp;nbsp;aren&amp;#8217;t.&lt;/p&gt;
&lt;h3 id="the-implementation"&gt;The&amp;nbsp;Implementation&lt;/h3&gt;
&lt;p&gt;The tool is a &lt;span class="caps"&gt;BGP&lt;/span&gt; &lt;span class="caps"&gt;AS&lt;/span&gt;-path access list combined with a route-map that matches on both the &lt;span class="caps"&gt;AS&lt;/span&gt;-path and the learning peer.&amp;nbsp;On &lt;code&gt;hobgp&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;bgp as-path access-list AS-DTAG seq 5 permit _3320$

route-map RM-IBGP-IN permit 5
 match as-path AS-DTAG
 match peer 2a06:9801:1c:fffe::2
 set local-preference 260
exit
route-map RM-IBGP-IN permit 10
exit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Reading this top to&amp;nbsp;bottom:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;bgp as-path access-list AS-DTAG&lt;/code&gt; matches any &lt;span class="caps"&gt;AS&lt;/span&gt; path that ends&amp;nbsp;in &lt;code&gt;3320&lt;/code&gt; (the &lt;code&gt;$&lt;/code&gt; anchors to the end of the &lt;span class="caps"&gt;AS&lt;/span&gt; path). That means &amp;#8220;routes originated by &lt;span class="caps"&gt;DTAG&lt;/span&gt;&amp;#8221; - &lt;span class="caps"&gt;DTAG&lt;/span&gt; is the terminal &lt;span class="caps"&gt;AS&lt;/span&gt;, not a&amp;nbsp;transit.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;route-map RM-IBGP-IN permit 5&lt;/code&gt; has two match conditions: the path must&amp;nbsp;match &lt;code&gt;AS-DTAG&lt;/code&gt;, and the update must have come&amp;nbsp;from &lt;code&gt;2a06:9801:1c:fffe::2&lt;/code&gt; - the Vultr edge router. Both must be&amp;nbsp;true.&lt;/li&gt;
&lt;li&gt;Matching routes get local-preference 260. Higher local-pref wins in &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;#8217;s selection algorithm, so these routes are preferred over the same prefix learned via iFog, Lagrange, route64, LocIX, or FogIXP&amp;nbsp;direct.&lt;/li&gt;
&lt;li&gt;&lt;code&gt;route-map RM-IBGP-IN permit 10&lt;/code&gt; is the catch-all - any route that didn&amp;#8217;t match clause 5 falls through and gets accepted with default&amp;nbsp;attributes.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The &lt;code&gt;match peer&lt;/code&gt; clause is essential. Without it, the rule would match routes to &lt;span class="caps"&gt;DTAG&lt;/span&gt; learned from &lt;em&gt;any&lt;/em&gt; iBGP peer, which isn&amp;#8217;t what I want. Vultr happens to have the short path; the other edges don&amp;#8217;t. The policy is specifically &amp;#8220;prefer Vultr&amp;#8217;s version of &lt;span class="caps"&gt;DTAG&lt;/span&gt; routes, if it has&amp;nbsp;one.&amp;#8221;&lt;/p&gt;
&lt;h3 id="the-effect"&gt;The&amp;nbsp;Effect&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;show ipv6 bgp X:X:X::/32&lt;/code&gt; for a &lt;span class="caps"&gt;DTAG&lt;/span&gt;-customer prefix shows multiple paths - from iFog, LocIX, Vultr, and so on. Before the route-map, &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;#8217;s tiebreaker (&lt;span class="caps"&gt;AS&lt;/span&gt; path length, then &lt;span class="caps"&gt;MED&lt;/span&gt;, then &lt;span class="caps"&gt;BGP&lt;/span&gt; origin) picked whichever was shortest at that moment. After the route-map, if Vultr has a path for a &lt;span class="caps"&gt;DTAG&lt;/span&gt; destination, Vultr wins - because 260 &amp;gt; 100&amp;nbsp;(default).&lt;/p&gt;
&lt;p&gt;Inbound traffic is out of my control (that&amp;#8217;s determined by the announcing side), but outbound traffic from my servers to &lt;span class="caps"&gt;DTAG&lt;/span&gt; customers now consistently exits through Vultr Frankfurt. Latency improvement varies by destination but averages around 5 ms lower than the transit paths, and jitter drops because the path no longer changes based on transient &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;convergence.&lt;/p&gt;
&lt;p&gt;This kind of traffic engineering was technically possible before, but not especially useful - with one or two edges, there&amp;#8217;s nothing to steer between. With four, &amp;#8220;which path goes where&amp;#8221; becomes a legitimate question, and &lt;span class="caps"&gt;AS&lt;/span&gt;-path regex gives a precise&amp;nbsp;answer.&lt;/p&gt;
&lt;h2 id="hub-hygiene-one-ip-for-traceroutes"&gt;Hub Hygiene: One &lt;span class="caps"&gt;IP&lt;/span&gt; for&amp;nbsp;Traceroutes&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;hobgp&lt;/code&gt; has many interfaces. When a packet transits the router, the source address used for any router-originated reply (&lt;span class="caps"&gt;ICMP&lt;/span&gt; time-exceeded from a traceroute, for instance) depends on which interface is closest in the routing table. The result was&amp;nbsp;that &lt;code&gt;hobgp&lt;/code&gt; appeared under different addresses&amp;nbsp;in &lt;code&gt;mtr&lt;/code&gt; output depending on the flow&amp;nbsp;direction.&lt;/p&gt;
&lt;p&gt;The fix is a &lt;span class="caps"&gt;PF&lt;/span&gt; &lt;code&gt;match&lt;/code&gt; rule that rewrites router-originated IPv6 traffic to a stable loopback alias from the&amp;nbsp;/48:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;ifconfig_lo0_alias0=&amp;quot;inet6 2a06:9801:1c::1 prefixlen 64&amp;quot;

match out on { gre0, gre1, gre2, gif0, gif1, gif2, gif3, gif4, gif5, gif6 } \
    inet6 from 2001:db8:1c19:3ee1::1 to any nat-to 2a06:9801:1c::1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;code&gt;match&lt;/code&gt; is a non-terminating &lt;span class="caps"&gt;PF&lt;/span&gt; action - it modifies packets without making a pass/block decision. Now traceroutes&amp;nbsp;through &lt;code&gt;hobgp&lt;/code&gt; always&amp;nbsp;show &lt;code&gt;2a06:9801:1c::1&lt;/code&gt; (which resolves&amp;nbsp;to &lt;code&gt;hofstede.it&lt;/code&gt;) regardless of which tunnel the forward packet took, and the Hetzner-assigned address stays off public&amp;nbsp;traceroutes.&lt;/p&gt;
&lt;h2 id="downstream-sites-pi-addressing-for-friends-and-services"&gt;Downstream Sites: &lt;span class="caps"&gt;PI&lt;/span&gt; Addressing for Friends and&amp;nbsp;Services&lt;/h2&gt;
&lt;p&gt;A side effect of having a /48 and a working &lt;span class="caps"&gt;BGP&lt;/span&gt; infrastructure is that you can offer stable, provider-independent IPv6 addressing to services you run for friends or that live in separate failure domains. The current&amp;nbsp;allocation:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Prefix&lt;/th&gt;
&lt;th&gt;Site&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;loopback on hobgp&lt;/td&gt;
&lt;td&gt;router identity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:1000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;radon&lt;/td&gt;
&lt;td&gt;blog, &lt;span class="caps"&gt;DNS&lt;/span&gt;, Gemini, fediverse comments&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:2000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;colo OPNsense&lt;/td&gt;
&lt;td&gt;Hetzner colo test network&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:3000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;mail.linuxserver.pro&lt;/td&gt;
&lt;td&gt;mail server&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:4000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;burningboard.net&lt;/td&gt;
&lt;td&gt;Mastodon instance&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:5000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;road-warrior&lt;/td&gt;
&lt;td&gt;laptop on the move&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:6000::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;home &lt;span class="caps"&gt;LAN&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;residential network&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;2a06:9801:1c:fff*::/64&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;infra&lt;/td&gt;
&lt;td&gt;point-to-point tunnel link addresses&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The&amp;nbsp;high &lt;code&gt;:fff*&lt;/code&gt; ranges are reserved for infrastructure and point-to-point&amp;nbsp;links: &lt;code&gt;:ffff::/64&lt;/code&gt; for the radon &lt;span class="caps"&gt;GIF&lt;/span&gt;, &lt;code&gt;:fffe::/64&lt;/code&gt; for the Vultr iBGP&amp;nbsp;link, &lt;code&gt;:fffd::/64&lt;/code&gt; for the LocIX&amp;nbsp;edge, &lt;code&gt;:fffc::/64&lt;/code&gt; for the FogIXP&amp;nbsp;edge, &lt;code&gt;:fff0::/64&lt;/code&gt; for the MikroTik. Keeping link addresses out of the customer-facing ranges means the useful space&amp;nbsp;(&lt;code&gt;:1000::&lt;/code&gt; through &lt;code&gt;:6000::&lt;/code&gt;) is contiguous and easy to reason&amp;nbsp;about.&lt;/p&gt;
&lt;p&gt;Every site runs its own services, has its own firewall, and is operationally independent. What they share is an address prefix that doesn&amp;#8217;t change if their hoster does - and that, more than anything else, is why I did all of&amp;nbsp;this.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Direct peering is worth the setup effort.&lt;/strong&gt; A peering session with a large network replaces an entire transit path with a single handoff. The operational cost is one more &lt;span class="caps"&gt;BGP&lt;/span&gt; neighbor and a handful of route-map entries; the benefit is shorter, more predictable paths for a large chunk of real&amp;nbsp;traffic.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;RouterOS &lt;span class="caps"&gt;BGP&lt;/span&gt; is good enough for a downstream site.&lt;/strong&gt; The MikroTik iBGP session has been stable since it was brought up. The configuration is verbose compared to &lt;span class="caps"&gt;FRR&lt;/span&gt; but not surprising - prefix-list equivalents, route-maps, the blackhole for the /64. If the home router is what you already have, &lt;span class="caps"&gt;BGP&lt;/span&gt; on it is a reasonable&amp;nbsp;answer.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Two-condition route-maps are a precision&amp;nbsp;instrument.&lt;/strong&gt; &lt;code&gt;match as-path&lt;/code&gt; plus &lt;code&gt;match peer&lt;/code&gt; lets you say exactly what you mean: &amp;#8220;prefer this specific path for this specific destination class.&amp;#8221; It&amp;#8217;s conceptually a small jump from single-condition filters, but it unlocks meaningful traffic engineering the moment you have more than two&amp;nbsp;edges.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;A consistent router identity helps more than it should.&lt;/strong&gt;&amp;nbsp;Rewriting &lt;code&gt;hobgp&lt;/code&gt;&lt;span class="quo"&gt;&amp;#8216;&lt;/span&gt;s source address to the /48 loopback is five lines of &lt;span class="caps"&gt;PF&lt;/span&gt;, but having a single hostname appear for the hub in every traceroute - regardless of interface - makes debugging paths significantly easier. It also keeps provider addresses out of public&amp;nbsp;traceroutes.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The &amp;#8220;announce at the edge, forward at the core&amp;#8221; pattern scales.&lt;/strong&gt; Each new edge is the same recipe: two interfaces or one plus a tunnel, &lt;span class="caps"&gt;FRR&lt;/span&gt; with per-peer route-maps, &lt;span class="caps"&gt;PF&lt;/span&gt; with stateless transit rules, an iBGP tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;. The core router does the work. Adding a fourth edge was faster than adding the second one, because the pattern was already&amp;nbsp;established.&lt;/p&gt;
&lt;div class="about-cta"&gt;
&lt;p&gt;&lt;strong&gt;Interested in peering with &lt;span class="caps"&gt;AS201379&lt;/span&gt;?&lt;/strong&gt;
More details about the network, including upstreams, IXPs, and policy, live at &lt;a href="https://hofstede.it/as201379.html"&gt;hofstede.it/as201379.html&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;If you&amp;#8217;d like to establish a peering session - either over a tunnel or on the shared fabric at &lt;strong&gt;LocIX Düsseldorf&lt;/strong&gt; or &lt;strong&gt;FogIXP Frankfurt&lt;/strong&gt; - send a mail to &lt;a href="mailto:peering@hofstede.it"&gt;peering@hofstede.it&lt;/a&gt;. Hobbyists, small ASNs, and experimental setups are explicitly&amp;nbsp;welcome.&lt;/p&gt;
&lt;/div&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;Four months in, &lt;span class="caps"&gt;AS201379&lt;/span&gt; has started to feel less like a project and more like a very small &lt;span class="caps"&gt;ISP&lt;/span&gt;: a core router, multiple edges in different PoPs, direct peering, traffic engineering, and downstream sites with stable addressing. It is still, by any real metric, tiny. But the difference between &amp;#8220;tiny&amp;#8221; and &amp;#8220;zero&amp;#8221; is categorical - Hetzner now learns my prefix directly over a peering session, and a &lt;span class="caps"&gt;PC&lt;/span&gt; on my home &lt;span class="caps"&gt;LAN&lt;/span&gt; now introduces itself to the internet with an address from my own&amp;nbsp;prefix.&lt;/p&gt;
&lt;p&gt;Whether there is a Part 5 depends on what breaks, what improves, or what new opportunity shows up next. So far, every version of this network has felt complete right up until the moment it&amp;nbsp;wasn&amp;#8217;t.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://fogixp.net/"&gt;FogIXP - Frankfurt Open Exchange&lt;/a&gt; - the &lt;span class="caps"&gt;IXP&lt;/span&gt; used for this&amp;nbsp;expansion&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.locix.online/"&gt;LocIX Düsseldorf&lt;/a&gt; - the &lt;span class="caps"&gt;IXP&lt;/span&gt; from Part&amp;nbsp;3&lt;/li&gt;
&lt;li&gt;&lt;a href="https://bgp.tools/"&gt;bgp.tools&lt;/a&gt; - looking glass and &lt;span class="caps"&gt;AS&lt;/span&gt;-path&amp;nbsp;visibility&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.frrouting.org/en/latest/bgp.html"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; Documentation: &lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/a&gt; - route-maps, &lt;span class="caps"&gt;AS&lt;/span&gt;-path&amp;nbsp;access-lists, &lt;code&gt;match peer&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://help.mikrotik.com/docs/display/ROS/BGP"&gt;MikroTik RouterOS &lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/a&gt; - filter chains, connection&amp;nbsp;templates&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.manrs.org/"&gt;&lt;span class="caps"&gt;MANRS&lt;/span&gt; Routing Security&lt;/a&gt; - the filtering and peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; hygiene principles applied on every&amp;nbsp;edge&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.peeringdb.com/"&gt;PeeringDB&lt;/a&gt; - where FogIXP and LocIX peering-&lt;span class="caps"&gt;LAN&lt;/span&gt; addresses come&amp;nbsp;from&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; is one of those protocols where you never really finish. There&amp;#8217;s always one more peer to add, one more route-map clause to tune, one more site to bring into the fabric. The satisfying part is that each addition compounds: every new edge improves paths for every existing service, and every new downstream inherits the entire backbone&amp;#8217;s routing decisions for free. Four edges and six downstream sites ago, this was one router and one &lt;span class="caps"&gt;VPS&lt;/span&gt;. The architecture was the hard part. Everything since has been variations on the same&amp;nbsp;theme.&lt;/p&gt;</content><category term="Networking"/><category term="freebsd"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="frr"/><category term="pf"/><category term="ibgp"/><category term="ixp"/><category term="mikrotik"/><category term="routeros"/></entry><entry><title>Running Your Own AS: Joining an IXP with a Third Edge Router</title><link href="https://hofstede.it/running-your-own-as-joining-an-ixp-with-a-third-edge-router/" rel="alternate"/><published>2026-03-21T00:00:00+01:00</published><updated>2026-03-21T00:00:00+01:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-03-21:/running-your-own-as-joining-an-ixp-with-a-third-edge-router/</id><summary type="html">&lt;p&gt;Connecting &lt;span class="caps"&gt;AS201379&lt;/span&gt; to LocIX Düsseldorf via a dedicated edge router - adding direct exchange point peering to the existing multi-homed &lt;span class="caps"&gt;BGP&lt;/span&gt; setup, completing a three-router FreeBSD infrastructure with transit, native peering, and &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;nbsp;connectivity.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Part 1&lt;/a&gt; covered running a single &lt;span class="caps"&gt;BGP&lt;/span&gt; router with two upstream providers. &lt;a href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/"&gt;Part 2&lt;/a&gt; added a second Point of Presence with Vultr native peering and iBGP. This is Part 3: connecting to an Internet Exchange&amp;nbsp;Point.&lt;/p&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-03-21-joining-an-ixp-locix-bgp-freebsd.png" title="Joining an IXP: Header image"&gt;&lt;/p&gt;
&lt;p&gt;An &lt;span class="caps"&gt;IXP&lt;/span&gt; changes the game. Instead of sending traffic through transit providers and their upstreams, networks at an exchange peer directly over a shared switching fabric. The result is shorter paths, lower latency, and fewer&amp;nbsp;intermediaries.&lt;/p&gt;
&lt;p&gt;This article documents connecting my &lt;span class="caps"&gt;AS201379&lt;/span&gt; to &lt;a href="https://www.locix.online/"&gt;LocIX Düsseldorf&lt;/a&gt; - a community-run &lt;span class="caps"&gt;IXP&lt;/span&gt; with over 350 participants - using a dedicated FreeBSD edge router, and the iBGP plumbing that ties it back into the existing infrastructure. This article assumes familiarity with &lt;span class="caps"&gt;BGP&lt;/span&gt;, iBGP, and basic FreeBSD networking as covered in the previous&amp;nbsp;parts.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; All provider-assigned and transit-provider &lt;span class="caps"&gt;IP&lt;/span&gt; addresses have been replaced with &lt;a href="https://www.rfc-editor.org/rfc/rfc5737"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5737&lt;/a&gt; / &lt;a href="https://www.rfc-editor.org/rfc/rfc3849"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 3849&lt;/a&gt; documentation ranges. &lt;span class="caps"&gt;AS201379&lt;/span&gt;, its prefix 2a06:9801:1c::/48, and all &lt;span class="caps"&gt;IXP&lt;/span&gt;-facing addresses (which are publicly visible in peering databases) are shown as-is. LocIX route server addresses and the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; prefix are equally&amp;nbsp;public.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="why-join-an-ixp"&gt;Why Join an &lt;span class="caps"&gt;IXP&lt;/span&gt;?&lt;/h2&gt;
&lt;p&gt;Transit providers create indirect paths: even if two networks are geographically close, their traffic may traverse multiple intermediary ASes. At an &lt;span class="caps"&gt;IXP&lt;/span&gt;, those intermediaries disappear - networks exchange traffic directly over a shared fabric. The practical impact is&amp;nbsp;measurable:&lt;/p&gt;
&lt;p&gt;One example is traffic between my &lt;span class="caps"&gt;AS201379&lt;/span&gt; and&amp;nbsp;meerfarbig.net:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Without IXP (via transit):
  hobgp → iFog → DE-CIX fabric → meerfarbig.net
  3+ hops, 20-30ms

With IXP (direct peering):
  hobgp → lobgp → LocIX fabric → meerfarbig.net
  2 hops, ~9ms
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Beyond latency, &lt;span class="caps"&gt;IXP&lt;/span&gt; peering provides route diversity that transit alone cannot match. Route servers at the exchange announce routes from every participant. A single &lt;span class="caps"&gt;BGP&lt;/span&gt; session to the route server gives me access to routes from hundreds of networks - routes I won&amp;#8217;t see through transit because those networks either peer-lock certain prefixes or prefer to exchange&amp;nbsp;locally.&lt;/p&gt;
&lt;h2 id="architecture-the-three-router-triangle"&gt;Architecture: The Three-Router&amp;nbsp;Triangle&lt;/h2&gt;
&lt;p&gt;My third router&amp;nbsp;(&lt;code&gt;lobgp&lt;/code&gt;) connects to LocIX through a hosting provider with direct layer-2 access to the exchange fabric. It peers with LocIX&amp;#8217;s route servers over the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;, announces my /48, and sends everything it learns back&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; via iBGP over a &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel. The resulting topology looks like&amp;nbsp;this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────────────────────────┐
                    │                Default-Free Zone                 │
                    └──┬─────────────┬──────────────┬──────────────────┘
                       │             │              │
                   AS209533      AS209735       AS212895
                   (iFog)        (Lagrange)     (route64)
                       │             │              │
                  GRE tunnel    GRE tunnel     GRE tunnel
                       │             │              │
                  ┌────┴─────────────┴──────────────┴────────────┐
                  │                  hobgp (Core)                │
                  │          FreeBSD + FRR, AS201379             │
                  │          2a06:9801:1c::/48                   │
                  └────────────┬────────────────┬────────────────┘
                               │                │
                          GIF tunnel        GIF tunnel
                          (iBGP)            (iBGP)
                               │                │
  ┌────────────────────────────┴──┐   ┌─────────┴─────────────────────┐
  │         vtbgp (Edge)          │   │         lobgp (Edge)          │
  │  FreeBSD + FRR, AS201379      │   │  FreeBSD + FRR, AS201379      │
  │  Native BGP → AS64515 (Vultr) │   │  LocIX Peering LAN            │
  └───────────────────────────────┘   │  → RS1/RS2 (AS202409)         │
                                      │  → Transit AS34872 (Servperso)│
                                      └───────────────────────────────┘
                                                   │
                                          LocIX Düsseldorf
                                    ┌──────────────┴──────────────┐
                                    │  350+ participants          │
                                    │  Direct L2 peering fabric   │
                                    └─────────────────────────────┘
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The design maintains the hub-and-spoke model from Part&amp;nbsp;2: &lt;code&gt;hobgp&lt;/code&gt; remains the forwarding hub. Traffic arriving&amp;nbsp;at &lt;code&gt;lobgp&lt;/code&gt; from the &lt;span class="caps"&gt;IXP&lt;/span&gt; is forwarded through the iBGP tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;, which handles routing to downstream&amp;nbsp;servers. &lt;code&gt;lobgp&lt;/code&gt; announces my prefix to the exchange and passes learned routes&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; for path&amp;nbsp;selection.&lt;/p&gt;
&lt;h3 id="getting-ixp-access"&gt;Getting &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;nbsp;Access&lt;/h3&gt;
&lt;p&gt;LocIX is a community-run exchange, which makes it accessible to smaller networks. The connection itself requires a &lt;span class="caps"&gt;VM&lt;/span&gt; with a network interface on the LocIX peering &lt;span class="caps"&gt;LAN&lt;/span&gt; - a layer-2 &lt;span class="caps"&gt;VLAN&lt;/span&gt; that reaches the exchange&amp;nbsp;fabric.&lt;/p&gt;
&lt;p&gt;Several hosting providers offer VMs with LocIX connectivity. I use &lt;a href="https://www.servperso.net/"&gt;Servperso&lt;/a&gt;, which provides VMs in Düsseldorf with a dedicated interface connected to the LocIX peering &lt;span class="caps"&gt;VLAN&lt;/span&gt;. The &lt;span class="caps"&gt;VM&lt;/span&gt; gets two network&amp;nbsp;interfaces: &lt;code&gt;vtnet1&lt;/code&gt; for regular internet connectivity (transit from Servperso&amp;#8217;s own &lt;span class="caps"&gt;AS34872&lt;/span&gt;),&amp;nbsp;and &lt;code&gt;vtnet0&lt;/code&gt; directly on the LocIX peering &lt;span class="caps"&gt;LAN&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;After signing up with LocIX and receiving the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; assignment, the setup is straightforward: configuring the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; addresses, peering with the route servers, and announcing my&amp;nbsp;prefix.&lt;/p&gt;
&lt;h2 id="the-ixp-edge-router-lobgp"&gt;The &lt;span class="caps"&gt;IXP&lt;/span&gt; Edge Router:&amp;nbsp;lobgp&lt;/h2&gt;
&lt;h3 id="network-configuration"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;/etc/rc.conf&lt;/code&gt; configures two physical interfaces and a &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel back to the&amp;nbsp;core:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;lobgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Internet connectivity (Servperso)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet1&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 198.51.100.50 netmask 255.255.255.224 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;198.51.100.62&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:b640::50/112 -rxcsum6 -tso6&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:b640::ffff&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# LocIX Peering LAN (direct L2)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 185.1.155.23/24 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a0c:b641:701::a5:20:1379:1 prefixlen 64 -rxcsum6 -tso6&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF tunnel to core (hobgp) - runs over IPv6&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 tunnel 2001:db8:b640::50 2001:db8:1c19::1 mtu 1460&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffd::2 2a06:9801:1c:fffd::1 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Uplink-to-hobgp&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Routing&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;myblock&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_myblock&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48 -interface gif0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Services&lt;/span&gt;
&lt;span class="nv"&gt;pf_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;sshd_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Several things are different&amp;nbsp;from &lt;code&gt;vtbgp&lt;/code&gt; in Part&amp;nbsp;2:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Two physical interfaces&lt;/strong&gt; instead of&amp;nbsp;one. &lt;code&gt;vtnet1&lt;/code&gt; provides regular internet&amp;nbsp;connectivity; &lt;code&gt;vtnet0&lt;/code&gt; connects directly to the LocIX peering &lt;span class="caps"&gt;LAN&lt;/span&gt;. This separation is important - peering &lt;span class="caps"&gt;LAN&lt;/span&gt; traffic should never traverse the internet-facing&amp;nbsp;interface.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel runs over IPv6&lt;/strong&gt;, not IPv4. Both Servperso and Hetzner provide native IPv6, so the encapsulation&amp;nbsp;uses &lt;code&gt;inet6 tunnel&lt;/code&gt; to wrap IPv6-in-IPv6. The tunnel &lt;span class="caps"&gt;MTU&lt;/span&gt; is set to 1460 - the standard 1500-byte Ethernet &lt;span class="caps"&gt;MTU&lt;/span&gt; minus the 40-byte outer IPv6 header added by &lt;span class="caps"&gt;GIF&lt;/span&gt;. To avoid fragmentation, &lt;span class="caps"&gt;PF&lt;/span&gt; clamps &lt;span class="caps"&gt;TCP&lt;/span&gt; &lt;span class="caps"&gt;MSS&lt;/span&gt; to 1400 bytes&amp;nbsp;on &lt;code&gt;$hobgp_tun&lt;/code&gt;, ensuring segments fit cleanly inside the tunnel after the inner IPv6 header (40 bytes) and &lt;span class="caps"&gt;TCP&lt;/span&gt; header (20 bytes) are accounted&amp;nbsp;for.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The peering &lt;span class="caps"&gt;LAN&lt;/span&gt; addresses&lt;/strong&gt; are publicly&amp;nbsp;registered. &lt;code&gt;185.1.155.23&lt;/code&gt; (IPv4)&amp;nbsp;and &lt;code&gt;2a0c:b641:701::a5:20:1379:1&lt;/code&gt; (IPv6) are assigned by LocIX and visible in the PeeringDB entry. These aren&amp;#8217;t documentation-range replacements - they&amp;#8217;re the real addresses, because hiding them would be counterproductive for anyone wanting to&amp;nbsp;peer.&lt;/p&gt;
&lt;p&gt;Like &lt;code&gt;vtbgp&lt;/code&gt;, the static&amp;nbsp;route &lt;code&gt;ipv6_route_myblock&lt;/code&gt; points the entire /48 at gif0, so any traffic arriving for my prefix gets forwarded through the tunnel to my core&amp;nbsp;router &lt;code&gt;hobgp&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id="frr-configuration"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;The &lt;span class="caps"&gt;FRR&lt;/span&gt; config has four &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions: two to LocIX route servers, one to Servperso for transit, and one iBGP back&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;frr version 10.5.1
frr defaults traditional
hostname lobgp
log syslog informational
service integrated-vtysh-config
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
!
! [PL-BOGONS: denies reserved/invalid prefixes, permits ::/0 le 48 - see Part 1]
!
route-map RM-IXP-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 200
exit
!
route-map RM-IXP-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-SERVPERSO-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 100
exit
!
route-map RM-SERVPERSO-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IBGP-OUT permit 10
exit
!
router bgp 201379
 bgp router-id 185.1.155.23
 no bgp enforce-first-as
 no bgp default ipv4-unicast
 neighbor 2a06:9801:1c:fffd::1 remote-as 201379
 neighbor 2a06:9801:1c:fffd::1 description Core-hobgp
 neighbor 2a06:9801:1c:fffd::1 update-source 2a06:9801:1c:fffd::2
 neighbor 2a0c:b641:701:0:a5:20:2409:1 remote-as 202409
 neighbor 2a0c:b641:701:0:a5:20:2409:1 description LocIX-RS1-v6
 neighbor 2a0c:b641:701:0:a5:20:2409:2 remote-as 202409
 neighbor 2a0c:b641:701:0:a5:20:2409:2 description LocIX-RS2-v6
 neighbor 2001:db8:b640::ffff remote-as 34872
 neighbor 2001:db8:b640::ffff description Transit-Servperso-v6
 !
 address-family ipv6 unicast
  neighbor 2a06:9801:1c:fffd::1 activate
  neighbor 2a06:9801:1c:fffd::1 next-hop-self
  neighbor 2a06:9801:1c:fffd::1 route-map RM-IBGP-OUT out
  neighbor 2a0c:b641:701:0:a5:20:2409:1 activate
  neighbor 2a0c:b641:701:0:a5:20:2409:1 soft-reconfiguration inbound
  neighbor 2a0c:b641:701:0:a5:20:2409:1 route-map RM-IXP-IN in
  neighbor 2a0c:b641:701:0:a5:20:2409:1 route-map RM-IXP-OUT out
  neighbor 2a0c:b641:701:0:a5:20:2409:2 activate
  neighbor 2a0c:b641:701:0:a5:20:2409:2 soft-reconfiguration inbound
  neighbor 2a0c:b641:701:0:a5:20:2409:2 route-map RM-IXP-IN in
  neighbor 2a0c:b641:701:0:a5:20:2409:2 route-map RM-IXP-OUT out
  neighbor 2001:db8:b640::ffff activate
  neighbor 2001:db8:b640::ffff soft-reconfiguration inbound
  neighbor 2001:db8:b640::ffff route-map RM-SERVPERSO-IN in
  neighbor 2001:db8:b640::ffff route-map RM-SERVPERSO-OUT out
 exit-address-family
exit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h4 id="route-servers-one-session-hundreds-of-routes"&gt;Route Servers: One Session, Hundreds of&amp;nbsp;Routes&lt;/h4&gt;
&lt;p&gt;&lt;span class="caps"&gt;IXP&lt;/span&gt; route servers are the key simplification. Instead of establishing individual &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions with every network at the exchange (which could mean hundreds of sessions), you peer with the exchange&amp;#8217;s route servers. LocIX runs two route servers (&lt;span class="caps"&gt;AS202409&lt;/span&gt;) for redundancy. Each route server collects routes from all participants and re-announces them to everyone. You can think of the route server as a fan-out mechanism: one &lt;span class="caps"&gt;BGP&lt;/span&gt; session in, hundreds of routes&amp;nbsp;out.&lt;/p&gt;
&lt;p&gt;The two route server sessions deliver approximately 1,800 unique prefixes each - routes from the ~350 networks present at LocIX. These are routes you typically wouldn&amp;#8217;t see through transit, or would see with a longer &lt;span class="caps"&gt;AS&lt;/span&gt;-path. At the exchange, they&amp;#8217;re direct: one hop across the peering&amp;nbsp;fabric.&lt;/p&gt;
&lt;h4 id="no-bgp-enforce-first-as-route-server-compatibility"&gt;&lt;code&gt;no bgp enforce-first-as&lt;/code&gt;: Route Server&amp;nbsp;Compatibility&lt;/h4&gt;
&lt;p&gt;Route servers are transparent &lt;span class="caps"&gt;BGP&lt;/span&gt; speakers. They don&amp;#8217;t insert their own &lt;span class="caps"&gt;AS&lt;/span&gt; into the &lt;span class="caps"&gt;AS&lt;/span&gt;-path of re-announced routes. When &lt;span class="caps"&gt;RS1&lt;/span&gt; (&lt;span class="caps"&gt;AS202409&lt;/span&gt;) sends you a route originally announced by, say, &lt;span class="caps"&gt;AS13335&lt;/span&gt; (Cloudflare), the &lt;span class="caps"&gt;AS&lt;/span&gt;-path still starts&amp;nbsp;with &lt;code&gt;13335&lt;/code&gt; -&amp;nbsp;not &lt;code&gt;202409 13335&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Standard &lt;span class="caps"&gt;BGP&lt;/span&gt; behavior rejects routes where the first &lt;span class="caps"&gt;AS&lt;/span&gt; in the path doesn&amp;#8217;t match the neighbor&amp;#8217;s configured remote-&lt;span class="caps"&gt;AS&lt;/span&gt;. Since the route server&amp;#8217;s &lt;span class="caps"&gt;AS&lt;/span&gt; (202409) isn&amp;#8217;t in the path, &lt;span class="caps"&gt;FRR&lt;/span&gt; would reject every route the server&amp;nbsp;sends. &lt;code&gt;no bgp enforce-first-as&lt;/code&gt; disables this check, which is required for route server&amp;nbsp;peering.&lt;/p&gt;
&lt;p&gt;This is safe in this context: route servers at established IXPs perform their own &lt;span class="caps"&gt;RPKI&lt;/span&gt; and &lt;span class="caps"&gt;IRR&lt;/span&gt; validation. In practice, you&amp;#8217;re trusting their filtering - just as you trust a transit provider&amp;#8217;s ingress filtering - only applied at a different point in the&amp;nbsp;path.&lt;/p&gt;
&lt;h4 id="local-preference-ixp-routes-win"&gt;Local Preference: &lt;span class="caps"&gt;IXP&lt;/span&gt; Routes&amp;nbsp;Win&lt;/h4&gt;
&lt;p&gt;The&amp;nbsp;route-map &lt;code&gt;RM-IXP-IN&lt;/code&gt; sets local-preference 200 on routes learned from the route servers.&amp;nbsp;On &lt;code&gt;hobgp&lt;/code&gt;, the iBGP-learned routes&amp;nbsp;from &lt;code&gt;lobgp&lt;/code&gt; carry this &lt;span class="caps"&gt;LP&lt;/span&gt; value. The preference hierarchy across the entire network&amp;nbsp;becomes:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Source&lt;/th&gt;
&lt;th&gt;Local Preference&lt;/th&gt;
&lt;th&gt;Path&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;LocIX route servers (via lobgp)&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;IXP&lt;/span&gt; direct peering&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vultr (via vtbgp)&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;Vultr native net&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;iFog BGPTunnel&lt;/td&gt;
&lt;td&gt;150&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;GRE&lt;/span&gt; transit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lagrange / route64 / Servperso&lt;/td&gt;
&lt;td&gt;100&lt;/td&gt;
&lt;td&gt;&lt;span class="caps"&gt;GRE&lt;/span&gt; transit (backup)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;LocIX and Vultr share &lt;span class="caps"&gt;LP&lt;/span&gt; 200 intentionally. When a destination is reachable via both (&lt;span class="caps"&gt;LP&lt;/span&gt; tied), &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;#8217;s tiebreakers apply. The most important one here is &lt;span class="caps"&gt;AS&lt;/span&gt;-path length: a route learned directly from a peer at LocIX carries that peer&amp;#8217;s &lt;span class="caps"&gt;AS&lt;/span&gt;-path as-is, while the same route via Vultr traverses Vultr&amp;#8217;s upstream chain, adding &lt;span class="caps"&gt;AS&lt;/span&gt; hops. Rather than artificially preferring one over the other, the tie lets &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;#8217;s path-length comparison pick the objectively shorter&amp;nbsp;route.&lt;/p&gt;
&lt;p&gt;The Servperso transit&amp;nbsp;(&lt;code&gt;RM-SERVPERSO-IN&lt;/code&gt;, &lt;span class="caps"&gt;LP&lt;/span&gt; 100) serves as a fallback. If the tunnel to hobgp fails or hobgp is down, lobgp still has full internet connectivity through Servperso and can continue forwarding traffic for our&amp;nbsp;/48.&lt;/p&gt;
&lt;h4 id="next-hop-self-on-the-ibgp-session"&gt;&lt;code&gt;next-hop-self&lt;/code&gt; on the iBGP&amp;nbsp;Session&lt;/h4&gt;
&lt;p&gt;Same principle&amp;nbsp;as &lt;code&gt;vtbgp&lt;/code&gt; in Part 2. Routes learned from the LocIX route servers have next-hops on the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;&amp;nbsp;(e.g., &lt;code&gt;2a0c:b641:701::a5:20:13335:1&lt;/code&gt; for&amp;nbsp;Cloudflare). &lt;code&gt;hobgp&lt;/code&gt; can&amp;#8217;t reach those addresses - they&amp;#8217;re on a &lt;span class="caps"&gt;VLAN&lt;/span&gt; in Düsseldorf that hobgp has no path&amp;nbsp;to. &lt;code&gt;next-hop-self&lt;/code&gt; rewrites all next-hops&amp;nbsp;to &lt;code&gt;2a06:9801:1c:fffd::2&lt;/code&gt; (lobgp&amp;#8217;s tunnel address), making the routes actionable for hobgp: &amp;#8220;send traffic down the iBGP tunnel to lobgp, which will forward it across the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;.&amp;#8221;&lt;/p&gt;
&lt;h3 id="pf-protecting-the-ixp-edge"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt;: Protecting the &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;nbsp;Edge&lt;/h3&gt;
&lt;p&gt;The peering &lt;span class="caps"&gt;LAN&lt;/span&gt; is a shared medium. Dozens of networks on the same layer-2 segment means your edge router&amp;#8217;s interfaces are exposed to &lt;span class="caps"&gt;ARP&lt;/span&gt;/&lt;span class="caps"&gt;NDP&lt;/span&gt; from every participant. The firewall must protect both the router itself and the peering&amp;nbsp;infrastructure:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vtnet1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vtnet0&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;hobgp_tun&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;my_router_ip&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fffd::2&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;gw_hobgp_v6&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:1c19::1&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;rs1_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a0c:b641:701::a5:20:2409:1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;rs2_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a0c:b641:701::a5:20:2409:2&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;peer_servperso&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:b640::ffff&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;rs1_v6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;rs2_v6&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_servperso&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;skip&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;lo0&lt;/span&gt;
&lt;span class="n"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;policy&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;drop&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;fragment&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;reassemble&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;hobgp_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;mss&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1400&lt;/span&gt;

&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;log&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;
&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bogons&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;
&lt;span class="n"&gt;antispoof&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# --- Control Plane ---&lt;/span&gt;

&lt;span class="c1"&gt;# ICMPv6 on external and IXP interfaces&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ipv6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;icmp6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;type&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;neighbrsol&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;neighbradv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;echoreq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ipv6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# Outer tunnel: IPv6-in-IPv6 encapsulation, stateless&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ipv6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;gw_hobgp_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ipv6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;gw_hobgp_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# iBGP session over tunnel&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;hobgp_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="n"&gt;a06&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;9801&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;fffd&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;hobgp_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="n"&gt;a06&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;9801&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;fffd&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# eBGP sessions on peering LAN (route servers + Servperso transit)&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# ARP/ICMP on peering LAN for neighbor discovery&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ixp_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# --- Data Plane ---&lt;/span&gt;

&lt;span class="c1"&gt;# Router&amp;#39;s own outbound&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_router_ip&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# Transit: stateless for asymmetric routing&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;no&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# Default outbound fallback&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Compared&amp;nbsp;to &lt;code&gt;vtbgp&lt;/code&gt;, the &lt;span class="caps"&gt;IXP&lt;/span&gt; edge firewall adds a few important&amp;nbsp;constraints:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; is locked to the route server addresses.&lt;/strong&gt; On the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;, only the LocIX route servers and the Servperso transit peer are allowed to establish &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions. Random participants can&amp;#8217;t initiate unsolicited &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions to our&amp;nbsp;router.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;ICMPv6 is permitted on the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;.&lt;/strong&gt; &lt;span class="caps"&gt;NDP&lt;/span&gt; (Neighbor Discovery Protocol) must work for the router to resolve &lt;span class="caps"&gt;MAC&lt;/span&gt; addresses of peers on the shared segment. Without this, the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; interface is&amp;nbsp;dead.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;IPv4 &lt;span class="caps"&gt;ICMP&lt;/span&gt; is explicitly allowed on the peering &lt;span class="caps"&gt;LAN&lt;/span&gt;.&lt;/strong&gt; Even though this is primarily an IPv6 setup, the peering &lt;span class="caps"&gt;LAN&lt;/span&gt; carries IPv4 &lt;span class="caps"&gt;ARP&lt;/span&gt; and &lt;span class="caps"&gt;ICMP&lt;/span&gt; for dual-stack participants. Allowing &lt;span class="caps"&gt;ICMP&lt;/span&gt; keeps diagnostics&amp;nbsp;working.&lt;/p&gt;
&lt;h2 id="core-router-changes-adding-the-second-ibgp-peer"&gt;Core Router Changes: Adding the Second iBGP&amp;nbsp;Peer&lt;/h2&gt;
&lt;p&gt;On &lt;code&gt;hobgp&lt;/code&gt;,&amp;nbsp;adding &lt;code&gt;lobgp&lt;/code&gt; means a new &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel and a new iBGP session. The additions&amp;nbsp;to &lt;code&gt;rc.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# GIF tunnel to lobgp (LocIX via Servperso) - IPv6-in-IPv6&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif5&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 tunnel 2001:db8:1c19::1 2001:db8:b640::50 mtu 1460&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif5_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffd::1 2a06:9801:1c:fffd::2 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif5_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tunnel-to-lobgp&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;And the &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;additions:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;neighbor 2a06:9801:1c:fffd::2 remote-as 201379
neighbor 2a06:9801:1c:fffd::2 description Edge-Servperso-IXP
neighbor 2a06:9801:1c:fffd::2 update-source 2a06:9801:1c:fffd::1
!
address-family ipv6 unicast
 neighbor 2a06:9801:1c:fffd::2 activate
 neighbor 2a06:9801:1c:fffd::2 soft-reconfiguration inbound
 neighbor 2a06:9801:1c:fffd::2 route-map RM-IBGP-IN in
 neighbor 2a06:9801:1c:fffd::2 route-map RM-IBGP-OUT out
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;RM-IBGP-IN&lt;/code&gt; and &lt;code&gt;RM-IBGP-OUT&lt;/code&gt; route-maps are shared with&amp;nbsp;the &lt;code&gt;vtbgp&lt;/code&gt; iBGP session - no new policy needed. The core router&amp;#8217;s firewall also needs the tunnel endpoint added to its permitted peers, and the gif5 tunnel added to &lt;span class="caps"&gt;MSS&lt;/span&gt; clamping&amp;nbsp;rules.&lt;/p&gt;
&lt;p&gt;With&amp;nbsp;this, &lt;code&gt;hobgp&lt;/code&gt; now has five &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions: three eBGP upstreams (iFog, Lagrange, route64) and two iBGP peers (vtbgp for Vultr, lobgp for&amp;nbsp;LocIX).&lt;/p&gt;
&lt;h2 id="verification"&gt;Verification&lt;/h2&gt;
&lt;h3 id="bgp-sessions"&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Sessions&lt;/h3&gt;
&lt;p&gt;First, confirm that all &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions are established.&amp;nbsp;On &lt;code&gt;lobgp&lt;/code&gt;, &lt;code&gt;vtysh -c 'show bgp ipv6 summary'&lt;/code&gt; shows:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Neighbor                     V    AS   MsgRcvd   MsgSent  Up/Down   PfxRcd  Desc
2a06:9801:1c:fffd::1         4  201379    4034   1792956  2d19h10m       1  Core-hobgp
2001:db8:b640::ffff          4   34872 2512958      4034  2d19h10m  238115  Transit-Servperso
2a0c:b641:701:0:a5:20:2409:1 4  202409  123879      4035  2d19h10m    1859  LocIX-RS1-v6
2a0c:b641:701:0:a5:20:2409:2 4  202409  123882      4035  2d19h10m    1859  LocIX-RS2-v6
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The route servers each contribute ~1,859 prefixes - the routes announced by other LocIX participants. Servperso provides a full &lt;span class="caps"&gt;DFZ&lt;/span&gt; view (~238K prefixes) as transit backup. The iBGP session to hobgp sends exactly 1 prefix outbound: our&amp;nbsp;/48.&lt;/p&gt;
&lt;p&gt;On &lt;code&gt;hobgp&lt;/code&gt;, the five-peer summary shows the combined&amp;nbsp;view:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Neighbor             V    AS   MsgRcvd   MsgSent  Up/Down   PfxRcd  Desc
2001:db8:300::1      4  209533 5096936     33805  10:02:36  239576  Upstream-iFog-FRA
fd00:ca::1           4  209735 4777022      5636  3d21h52m  237342  Upstream-Lagrange-UK
2001:db8:400::1      4  212895 1907099      5636  3d21h52m  243233  Upstream-Route64-FRA
2a06:9801:1c:fffd::2 4  201379 2762876      5622  2d19h09m  238120  Edge-Servperso-IXP
2a06:9801:1c:fffe::2 4  201379 2396813      5640  3d21h48m  233048  Edge-Vultr-Frankfurt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The vtbgp peer&amp;nbsp;(&lt;code&gt;2a06:9801:1c:fffe::2&lt;/code&gt;) is the Vultr edge router from &lt;a href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/"&gt;Part 2&lt;/a&gt;. The new lobgp iBGP session&amp;nbsp;(&lt;code&gt;2a06:9801:1c:fffd::2&lt;/code&gt;) delivers ~238K prefixes to hobgp - the combination of LocIX routes and Servperso&amp;#8217;s full table, merged and forwarded via iBGP. The &lt;span class="caps"&gt;IXP&lt;/span&gt; routes carry &lt;span class="caps"&gt;LP&lt;/span&gt; 200 and win path selection for destinations that are reachable via the&amp;nbsp;exchange.&lt;/p&gt;
&lt;h3 id="the-ixp-difference-real-traceroutes"&gt;The &lt;span class="caps"&gt;IXP&lt;/span&gt; Difference: Real&amp;nbsp;Traceroutes&lt;/h3&gt;
&lt;p&gt;The real test is measuring what the exchange connection actually changes. Here&amp;#8217;s a traceroute to &lt;a href="https://meerfarbig.net"&gt;meerfarbig&lt;/a&gt;, a German &lt;span class="caps"&gt;ISP&lt;/span&gt; that peers with my &lt;span class="caps"&gt;AS201379&lt;/span&gt; at&amp;nbsp;LocIX:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;HOST: hobgp                             Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- lobgp.edge.hofstede.it             0.0%    10    8.3   8.8   8.2  11.4   1.0
  2.|-- ae0.800.mx240.dus1.meerfarbig.net  0.0%    10    9.5  14.4   9.4  29.2   6.2
  3.|-- 2a00:f820:11::45                   0.0%    10    9.4   9.6   9.4  10.0   0.2
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Three hops&amp;nbsp;total. &lt;code&gt;hobgp&lt;/code&gt; sends&amp;nbsp;to &lt;code&gt;lobgp&lt;/code&gt; through the iBGP tunnel (hop&amp;nbsp;1). &lt;code&gt;lobgp&lt;/code&gt; forwards across the LocIX peering fabric to meerfarbig&amp;#8217;s edge&amp;nbsp;router &lt;code&gt;ae0.800.mx240.dus1&lt;/code&gt; in Düsseldorf (hop 2) - this is where the &lt;span class="caps"&gt;ICMP&lt;/span&gt; &lt;span class="caps"&gt;TTL&lt;/span&gt;-exceeded reply comes from, confirming the packet entered meerfarbig&amp;#8217;s network directly from the exchange. Hop 3&amp;nbsp;(&lt;code&gt;2a00:f820:11::45&lt;/code&gt;) is the final destination, one hop deeper inside meerfarbig&amp;#8217;s infrastructure. Total latency: under 10ms. Without the &lt;span class="caps"&gt;IXP&lt;/span&gt;, this traffic would traverse iFog or route64, adding hops through transit infrastructure and roughly doubling the&amp;nbsp;latency.&lt;/p&gt;
&lt;p&gt;This is direct peering in practice. No transit &lt;span class="caps"&gt;AS&lt;/span&gt; in the path. The packet crosses one switching fabric and enters the peer&amp;#8217;s network at their edge router - from there, it&amp;#8217;s their internal routing to the&amp;nbsp;destination.&lt;/p&gt;
&lt;h2 id="peering-policy"&gt;Peering&amp;nbsp;Policy&lt;/h2&gt;
&lt;p&gt;With &lt;span class="caps"&gt;IXP&lt;/span&gt; access comes a peering policy. My &lt;span class="caps"&gt;AS201379&lt;/span&gt; operates an &lt;strong&gt;open peering policy&lt;/strong&gt; - I&amp;#8217;m happy to peer with anyone at LocIX or any other exchange point, with no traffic requirements or filtering restrictions beyond basic hygiene (valid &lt;span class="caps"&gt;RPKI&lt;/span&gt;, prefix&amp;nbsp;limits).&lt;/p&gt;
&lt;p&gt;The full peering policy, contact details, and current peering information are published at &lt;a href="https://hofstede.it/as201379.html"&gt;hofstede.it/as201379.html&lt;/a&gt;.&lt;/p&gt;
&lt;h2 id="whats-next-ipv4"&gt;What&amp;#8217;s Next:&amp;nbsp;IPv4&lt;/h2&gt;
&lt;p&gt;The entire setup so far is IPv6-only. Every tunnel, every &lt;span class="caps"&gt;BGP&lt;/span&gt; session, every announced prefix is IPv6. This was deliberate: IPv6 address space is cheap and easy to obtain, making it realistic for a hobby &lt;span class="caps"&gt;AS&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;The next step is adding IPv4. A sponsoring &lt;span class="caps"&gt;LIR&lt;/span&gt; will provide me a small IPv4 prefix, which opens up the setup to the ~40% of internet traffic that still runs on IPv4. Adding IPv4 means dual-stack on the existing routers, IPv4 tunnel overlays in parallel with the IPv6 ones, and &lt;span class="caps"&gt;PF&lt;/span&gt; rules for both address families. The three-router topology stays the same; it just gains a second address&amp;nbsp;family.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Route servers are the &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;#8217;s killer feature for small networks.&lt;/strong&gt; Without them, joining an &lt;span class="caps"&gt;IXP&lt;/span&gt; would mean manually negotiating &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions with individual participants - operationally intractable for a hobby network. Two &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions give you access to every participant&amp;#8217;s routes - the exchange handles policy enforcement and&amp;nbsp;validation.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;no bgp enforce-first-as&lt;/code&gt; is required for route server peering.&lt;/strong&gt; This is a common gotcha. Route servers are transparent - they don&amp;#8217;t insert their own &lt;span class="caps"&gt;AS&lt;/span&gt; into the path. Without disabling the first-&lt;span class="caps"&gt;AS&lt;/span&gt; check, &lt;span class="caps"&gt;FRR&lt;/span&gt; rejects every route the server&amp;nbsp;sends.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;The peering &lt;span class="caps"&gt;LAN&lt;/span&gt; is a hostile environment.&lt;/strong&gt; Dozens of networks on the same L2 segment means your router is directly exposed. Lock &lt;span class="caps"&gt;BGP&lt;/span&gt; to known peers, allow only essential ICMPv6/&lt;span class="caps"&gt;NDP&lt;/span&gt;, and drop everything else. The peering &lt;span class="caps"&gt;LAN&lt;/span&gt; is not a trusted&amp;nbsp;network.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;IXP&lt;/span&gt; routes and transit routes complement each other.&lt;/strong&gt; The ~1,800 prefixes from LocIX&amp;#8217;s route servers are a subset of the ~240K full table. For those 1,800 destinations, the &lt;span class="caps"&gt;IXP&lt;/span&gt; path is dramatically shorter and faster. For everything else, transit providers fill in. Setting &lt;span class="caps"&gt;LP&lt;/span&gt; 200 on &lt;span class="caps"&gt;IXP&lt;/span&gt; routes ensures they always win when&amp;nbsp;available.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;IPv6-in-IPv6 tunnels work well for iBGP links.&lt;/strong&gt; When both endpoints have native IPv6, there&amp;#8217;s no reason to wrap IPv6 traffic in IPv4. The &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel between lobgp and hobgp&amp;nbsp;uses &lt;code&gt;inet6 tunnel&lt;/code&gt; and avoids the IPv4 dependency&amp;nbsp;entirely.&lt;/p&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;Adding an &lt;span class="caps"&gt;IXP&lt;/span&gt; connection completes the final piece of a small but functional internet infrastructure: transit provides global reachability, native Vultr peering provides reach into Vultr-connected networks, and the &lt;span class="caps"&gt;IXP&lt;/span&gt; provides direct paths to every other participant at the exchange. Three routers, three distinct connectivity roles - each solving a problem the others&amp;nbsp;can&amp;#8217;t.&lt;/p&gt;
&lt;p&gt;The three-router topology is surprisingly manageable. Each router has a clear&amp;nbsp;role: &lt;code&gt;hobgp&lt;/code&gt; is the forwarding hub with upstream&amp;nbsp;transit, &lt;code&gt;vtbgp&lt;/code&gt; announces into Vultr&amp;#8217;s network,&amp;nbsp;and &lt;code&gt;lobgp&lt;/code&gt; peers at the exchange. iBGP ties them together, and local-preference ensures traffic takes the best available path&amp;nbsp;automatically.&lt;/p&gt;
&lt;p&gt;The practical impact is measurable in every traceroute. Destinations reachable via LocIX are one hop away instead of three or four. That&amp;#8217;s not a theoretical improvement - it&amp;#8217;s 10ms instead of 25ms, visible in every&amp;nbsp;connection.&lt;/p&gt;
&lt;p&gt;For anyone running a hobby &lt;span class="caps"&gt;AS&lt;/span&gt; and wondering whether &lt;span class="caps"&gt;IXP&lt;/span&gt; connectivity is worth the extra router: it is. The route diversity alone justifies the cost, and watching your traffic take a direct two-hop path to a network that used to be five hops away through transit is exactly the kind of thing that makes this hobby worth&amp;nbsp;pursuing.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.locix.online/"&gt;LocIX - Community Internet&amp;nbsp;Exchange&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.peeringdb.com/ix/2084"&gt;PeeringDB - LocIX&amp;nbsp;Düsseldorf&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.servperso.net/"&gt;Servperso - Düsseldorf Hosting with &lt;span class="caps"&gt;IXP&lt;/span&gt;&amp;nbsp;Access&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://hofstede.it/as201379.html"&gt;&lt;span class="caps"&gt;AS201379&lt;/span&gt; Peering&amp;nbsp;Policy&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.frrouting.org/en/latest/bgp.html"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; Documentation: Route Servers and&amp;nbsp;enforce-first-as&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://bgp.tools/"&gt;bgp.tools - &lt;span class="caps"&gt;BGP&lt;/span&gt; Looking&amp;nbsp;Glass&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;Part 1: Running Your Own &lt;span class="caps"&gt;AS&lt;/span&gt;&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/"&gt;Part 2: Going&amp;nbsp;Multi-Homed&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;Three articles, three evolutionary steps, and the same fundamental toolset: FreeBSD, &lt;span class="caps"&gt;FRR&lt;/span&gt;, &lt;span class="caps"&gt;PF&lt;/span&gt;, and tunnels. The internet doesn&amp;#8217;t care that &lt;span class="caps"&gt;AS201379&lt;/span&gt; runs on three small virtual machines. It sees valid routes, clean filters, and responsive peering. From a single router with one upstream to a three-PoP network with direct exchange peering - the architecture scales because the protocols&amp;nbsp;do.&lt;/p&gt;</content><category term="Networking"/><category term="freebsd"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="frr"/><category term="pf"/><category term="ixp"/><category term="locix"/><category term="ibgp"/></entry><entry><title>Running Your Own AS: Going Multi-Homed with iBGP and three Transits</title><link href="https://hofstede.it/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/" rel="alternate"/><published>2026-02-26T00:00:00+01:00</published><updated>2026-02-26T00:00:00+01:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-02-26:/running-your-own-as-going-multi-homed-with-ibgp-and-three-transits/</id><summary type="html">&lt;p&gt;Expanding a single &lt;span class="caps"&gt;BGP&lt;/span&gt; router into a two-PoP distributed network: adding a Vultr edge router with native &lt;span class="caps"&gt;BGP&lt;/span&gt; peering, three upstream &lt;span class="caps"&gt;GRE&lt;/span&gt; providers and iBGP to tie it together - plus the stateless &lt;span class="caps"&gt;PF&lt;/span&gt; rules that make transit routing actually&amp;nbsp;work.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-02-26-going-multihomed-bgp-freebsd.png" title="Going Multi-Homed: Header image"&gt;&lt;/p&gt;
&lt;p&gt;The &lt;a href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/"&gt;previous article&lt;/a&gt; covered the basics: obtain an &lt;span class="caps"&gt;AS&lt;/span&gt; number and IPv6 prefix, build a single FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; router with two upstream providers, and tunnel the prefix to downstream servers. That setup works. But it has a property that feels wrong once you&amp;#8217;ve operated it for a while: all your traffic converges on one point. One machine, one location, two upstreams. Everything in, everything out, through the same&amp;nbsp;box.&lt;/p&gt;
&lt;p&gt;Multi-homing addresses this at multiple levels. More upstream providers give inbound path diversity. A second Point of Presence at a different network means traffic from different parts of the internet can enter through different doors, and you can engineer which traffic uses which path. This article documents the evolution from that single-router setup to a distributed network: two FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; routers connected by iBGP and three upstream &lt;span class="caps"&gt;GRE&lt;/span&gt;-based transit&amp;nbsp;providers.&lt;/p&gt;
&lt;p&gt;The routing concepts involved - iBGP, local-preference, next-hop-self, stateless transit filtering - are the same ones any real network operator deals with. The hardware is two virtual&amp;nbsp;machines.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; All provider-assigned &lt;span class="caps"&gt;IP&lt;/span&gt; addresses, tunnel endpoints, and management IPs have been replaced with &lt;a href="https://www.rfc-editor.org/rfc/rfc5737"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5737&lt;/a&gt; / &lt;a href="https://www.rfc-editor.org/rfc/rfc3849"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 3849&lt;/a&gt; documentation ranges. &lt;span class="caps"&gt;AS201379&lt;/span&gt; and the prefix 2a06:9801:1c::/48 are real public &lt;span class="caps"&gt;BGP&lt;/span&gt; resources and shown as-is. Upstream &lt;span class="caps"&gt;AS&lt;/span&gt; numbers (&lt;span class="caps"&gt;AS209533&lt;/span&gt;, &lt;span class="caps"&gt;AS209735&lt;/span&gt;, &lt;span class="caps"&gt;AS212895&lt;/span&gt;, &lt;span class="caps"&gt;AS64515&lt;/span&gt;) are equally visible in public routing&amp;nbsp;tables.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="why-add-more"&gt;Why Add&amp;nbsp;More?&lt;/h2&gt;
&lt;p&gt;The original two-upstream setup had one failure domain and one point of path selection. Three practical problems motivated the&amp;nbsp;expansion:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Route diversity.&lt;/strong&gt; Different parts of the internet have different peering arrangements. A network that transits heavily through Vultr&amp;#8217;s infrastructure prefers paths that show up in Vultr&amp;#8217;s &lt;span class="caps"&gt;BGP&lt;/span&gt;. A network at &lt;span class="caps"&gt;DE&lt;/span&gt;-&lt;span class="caps"&gt;CIX&lt;/span&gt; Frankfurt prefers the shorter path through a Frankfurt exchange point. With one upstream set, you can only be optimally reachable for the networks whose traffic happens to prefer your specific&amp;nbsp;providers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Single point of failure.&lt;/strong&gt; One router means one failure domain. A reboot or misconfiguration takes down all reachability&amp;nbsp;simultaneously.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Traffic engineering granularity.&lt;/strong&gt; &lt;span class="caps"&gt;AS&lt;/span&gt;-path prepending lets you make one path look less preferred, but only when the receiving network actually has a choice of paths. Multiple announcement points give that&amp;nbsp;choice.&lt;/p&gt;
&lt;h2 id="architecture"&gt;Architecture&lt;/h2&gt;
&lt;p&gt;The setup has two tiers: a core router&amp;nbsp;(&lt;code&gt;hobgp&lt;/code&gt;) at a Hetzner facility in Nuremberg that handles three &lt;span class="caps"&gt;GRE&lt;/span&gt; upstreams and an edge router&amp;nbsp;(&lt;code&gt;vtbgp&lt;/code&gt;) on Vultr in Frankfurt that announces our prefix natively into Vultr&amp;#8217;s network. The two routers are connected by a &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel running&amp;nbsp;iBGP.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────────────────────────┐
                    │                Default-Free Zone                  │
                    └──┬─────────────┬──────────────┬──────────────────┘
                       │             │              │
                   AS209533      AS209735       AS212895
                   (iFog)        (Lagrange)     (route64)
                       │             │              │
                  GRE tunnel    GRE tunnel     GRE tunnel
                       │             │              │
                  ┌────┴─────────────┴──────────────┴────────────┐
                  │                  hobgp (Core)                 │
                  │          FreeBSD + FRR, AS201379              │
                  │          2a06:9801:1c::/48                    │
                  └──────────────────────┬────────────────────────┘
                                         │
                                    GIF tunnel (iBGP)
                                         │
                  ┌──────────────────────┴────────────────────────┐
                  │                  vtbgp (Edge)                  │
                  │          FreeBSD + FRR, AS201379              │
                  │          Native BGP → AS64515 (Vultr)          │
                  └───────────────────────────────────────────────┘
                                         │
                                     AS64515
                                      (Vultr)
                                         │
                              ┌──────────┴──────────┐
                              │   Default-Free Zone   │
                              └───────────────────────┘

hobgp also tunnels downstream (as in the previous article):
                           ┌────────────┐
                           │   radon    │
                           │  :1000::/64│
                           └────────────┘
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The key routing properties of this&amp;nbsp;design: &lt;code&gt;hobgp&lt;/code&gt; is the actual forwarding hub - all traffic to and from downstream servers flows through&amp;nbsp;it. &lt;code&gt;vtbgp&lt;/code&gt; is an announcement-only node; traffic that arrives there is immediately sent through the iBGP tunnel&amp;nbsp;to &lt;code&gt;hobgp&lt;/code&gt; for forwarding. This asymmetry is intentional and efficient: Vultr&amp;#8217;s infrastructure sees our prefix announced locally, which shortens the path for Vultr-connected networks, without&amp;nbsp;requiring &lt;code&gt;vtbgp&lt;/code&gt; to carry the full routing table or forward arbitrary internet&amp;nbsp;traffic.&lt;/p&gt;
&lt;h2 id="the-core-router-hobgp"&gt;The Core Router:&amp;nbsp;hobgp&lt;/h2&gt;
&lt;h3 id="network-configuration"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;rc.conf&lt;/code&gt; now manages three &lt;span class="caps"&gt;GRE&lt;/span&gt; upstreams and a &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel to the edge router, in addition to the downstream server tunnels from the original&amp;nbsp;setup:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;hobgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif if_gre&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Physical interface (Hetzner)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 198.51.100.10/32 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:1c19::1/64&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;198.51.100.1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fe80::1%vtnet0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Loopback: router&amp;#39;s address within our /48&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_lo0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c::1 prefixlen 64&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Tunnel interfaces&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0 gif1 gre0 gre1 gre2&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GRE to iFog (BGPTunnel.com free transit)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 198.51.100.44&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:300::2 prefixlen 126&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Transit-iFog-Frankfurt BGPTunnel&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GRE to Lagrange (UK transit, backup path)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre1&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 198.51.100.45&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 fd00:ca::2 prefixlen 64&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre1_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Transit-Lagrange-UK&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GRE to route64.org (Frankfurt transit)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre2&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 198.51.100.46&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre2_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:400::2 prefixlen 64&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre2_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Transit-route64-Frankfurt&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF to downstream VPS (radon)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 203.0.113.10 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:ffff::1 2a06:9801:1c:ffff::2 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tunnel-to-Radon-Server&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF to Vultr edge router (vtbgp) - iBGP link&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 203.0.113.20 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffe::1 2a06:9801:1c:fffe::2 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tunnel-to-Vultr-Edge&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Routing&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;myblock cloud vtbgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_myblock&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48 -reject&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_cloud&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:1000::/64 2a06:9801:1c:ffff::2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_vtbgp&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:fffe::2/128 -interface gif1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The iBGP link (gif1) uses addresses from our own /48&amp;nbsp;(&lt;code&gt;2a06:9801:1c:fffe::/64&lt;/code&gt;). This is convenient - those addresses are already routable within our infrastructure, so the iBGP session doesn&amp;#8217;t depend on any provider-assigned addresses being&amp;nbsp;reachable.&lt;/p&gt;
&lt;h3 id="frr-configuration"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;The &lt;span class="caps"&gt;FRR&lt;/span&gt; configuration grows to four &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions. The substantive additions over the original are local-preference assignment on inbound route-maps, and the iBGP session to&amp;nbsp;vtbgp:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;frr version 10.5.1
frr defaults traditional
hostname hobgp
log syslog informational
service integrated-vtysh-config
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
!
! [PL-BOGONS: same comprehensive bogon filter as original - see previous article]
!
route-map RM-IFOG-BGPTUNNEL-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
 set local-preference 150
exit
!
route-map RM-IFOG-BGPTUNNEL-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-LAGRANGE-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
exit
!
route-map RM-LAGRANGE-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
 set as-path prepend 201379 201379
exit
!
route-map RM-ROUTE64-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
exit
!
route-map RM-ROUTE64-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-IBGP-IN permit 10
 set local-preference 200
exit
!
route-map RM-IBGP-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
ipv6 route 2a06:9801:1c::/48 blackhole
ipv6 route 2a06:9801:1c:fffe::2/128 gif1
!
router bgp 201379
 bgp router-id 198.51.100.10
 no bgp default ipv4-unicast
 neighbor 2001:db8:300::1 remote-as 209533
 neighbor 2001:db8:300::1 description Upstream-iFog-Frankfurt
 neighbor 2001:db8:300::1 update-source 2001:db8:300::2
 neighbor fd00:ca::1 remote-as 209735
 neighbor fd00:ca::1 description Upstream-Lagrange-UK
 neighbor fd00:ca::1 update-source fd00:ca::2
 neighbor 2001:db8:400::1 remote-as 212895
 neighbor 2001:db8:400::1 description Upstream-Route64-FRA
 neighbor 2001:db8:400::1 update-source 2001:db8:400::2
 neighbor 2a06:9801:1c:fffe::2 remote-as 201379
 neighbor 2a06:9801:1c:fffe::2 description Edge-Vultr-Frankfurt
 neighbor 2a06:9801:1c:fffe::2 update-source 2a06:9801:1c:fffe::1
 !
 address-family ipv6 unicast
  network 2a06:9801:1c::/48
  neighbor 2001:db8:300::1 activate
  neighbor 2001:db8:300::1 soft-reconfiguration inbound
  neighbor 2001:db8:300::1 maximum-prefix 250000 90 restart 30
  neighbor 2001:db8:300::1 route-map RM-IFOG-BGPTUNNEL-IN in
  neighbor 2001:db8:300::1 route-map RM-IFOG-BGPTUNNEL-OUT out
  neighbor fd00:ca::1 activate
  neighbor fd00:ca::1 soft-reconfiguration inbound
  neighbor fd00:ca::1 maximum-prefix 250000 90 restart 30
  neighbor fd00:ca::1 route-map RM-LAGRANGE-IN in
  neighbor fd00:ca::1 route-map RM-LAGRANGE-OUT out
  neighbor 2001:db8:400::1 activate
  neighbor 2001:db8:400::1 soft-reconfiguration inbound
  neighbor 2001:db8:400::1 maximum-prefix 250000 90 restart 30
  neighbor 2001:db8:400::1 route-map RM-ROUTE64-IN in
  neighbor 2001:db8:400::1 route-map RM-ROUTE64-OUT out
  neighbor 2a06:9801:1c:fffe::2 activate
  neighbor 2a06:9801:1c:fffe::2 soft-reconfiguration inbound
  neighbor 2a06:9801:1c:fffe::2 route-map RM-IBGP-IN in
  neighbor 2a06:9801:1c:fffe::2 route-map RM-IBGP-OUT out
 exit-address-family
exit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h4 id="local-preference-the-outbound-traffic-hierarchy"&gt;Local Preference: The Outbound Traffic&amp;nbsp;Hierarchy&lt;/h4&gt;
&lt;p&gt;Local-preference (&lt;span class="caps"&gt;LP&lt;/span&gt;) is the primary mechanism for outbound path selection within an &lt;span class="caps"&gt;AS&lt;/span&gt;. &lt;span class="caps"&gt;BGP&lt;/span&gt; evaluates &lt;span class="caps"&gt;LP&lt;/span&gt; before &lt;span class="caps"&gt;AS&lt;/span&gt;-path length, making it the dominant decision factor. The hierarchy here&amp;nbsp;is:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Neighbor&lt;/th&gt;
&lt;th&gt;Local Preference&lt;/th&gt;
&lt;th&gt;Role&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;vtbgp (iBGP/Vultr)&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;td&gt;Preferred exit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;iFog BGPTunnel&lt;/td&gt;
&lt;td&gt;150&lt;/td&gt;
&lt;td&gt;Secondary exit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lagrange&lt;/td&gt;
&lt;td&gt;100 (default)&lt;/td&gt;
&lt;td&gt;Tertiary&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;route64&lt;/td&gt;
&lt;td&gt;100 (default)&lt;/td&gt;
&lt;td&gt;Tertiary&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;With &lt;span class="caps"&gt;LP&lt;/span&gt; 200 on iBGP-learned routes, hobgp prefers to send outbound traffic through vtbgp and into Vultr&amp;#8217;s network wherever vtbgp has a route. iFog serves as backup when vtbgp lacks a route or the tunnel is down. Lagrange and route64 act as last resort, carrying traffic to parts of the internet not well-served by the other&amp;nbsp;two.&lt;/p&gt;
&lt;p&gt;For inbound traffic - what the rest of the internet chooses - the tool is different: you control what you announce and how attractive you make&amp;nbsp;it. &lt;code&gt;RM-LAGRANGE-OUT&lt;/code&gt; adds two extra prepends&amp;nbsp;(&lt;code&gt;set as-path prepend 201379 201379&lt;/code&gt;), making our prefix appear three &lt;span class="caps"&gt;AS&lt;/span&gt;-hops away via Lagrange. Most networks see a shorter path via iFog or Vultr and prefer those, demoting Lagrange to a true&amp;nbsp;backup.&lt;/p&gt;
&lt;h4 id="a-note-on-free-transit"&gt;A Note on Free&amp;nbsp;Transit&lt;/h4&gt;
&lt;p&gt;Both iFog&amp;#8217;s &lt;a href="https://bgptunnel.com"&gt;BGPTunnel.com&lt;/a&gt; service and &lt;a href="https://route64.org"&gt;route64.org&lt;/a&gt; offer free &lt;span class="caps"&gt;BGP&lt;/span&gt; transit aimed at hobbyists and researchers. BGPTunnel provides a &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel to iFog&amp;#8217;s Frankfurt infrastructure and announces your prefix to their peers. route64 works similarly. Neither carries SLAs suitable for production, but for learning and redundancy in a hobby setup they&amp;#8217;re genuinely useful. The community around these services is active and documentation is&amp;nbsp;good.&lt;/p&gt;
&lt;h2 id="the-edge-router-vtbgp"&gt;The Edge Router:&amp;nbsp;vtbgp&lt;/h2&gt;
&lt;p&gt;The second router runs on Vultr in Frankfurt. Its role is narrow: announce 2a06:9801:1c::/48 into Vultr&amp;#8217;s network and pass routing information back to hobgp via iBGP. It does not forward arbitrary internet&amp;nbsp;traffic.&lt;/p&gt;
&lt;h3 id="why-vultr"&gt;Why&amp;nbsp;Vultr?&lt;/h3&gt;
&lt;p&gt;Vultr offers a &lt;span class="caps"&gt;BGP&lt;/span&gt; service where customers announce their own prefixes through Vultr&amp;#8217;s infrastructure. Vultr&amp;#8217;s network (&lt;span class="caps"&gt;AS20473&lt;/span&gt; for customer connectivity, &lt;span class="caps"&gt;AS64515&lt;/span&gt; for &lt;span class="caps"&gt;BGP&lt;/span&gt; peering sessions) has good upstream diversity and significant peering at major exchanges. Networks that peer with or transit through Vultr have a direct path to a locally-announced prefix, resulting in measurably shorter routes than if they had to reach across to an independent transit&amp;nbsp;provider.&lt;/p&gt;
&lt;h3 id="network-configuration_1"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vtbgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif tcpmd5&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Physical interface (Vultr-assigned)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 203.0.113.20/23 -rxcsum -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:f480::20/64 -rxcsum6 -tso6 accept_rtadv&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;203.0.113.1&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF tunnel back to core (hobgp)&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 203.0.113.20 198.51.100.10 mtu 1480&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:fffe::2 2a06:9801:1c:fffe::1 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Uplink-to-hobgp&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Routing&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;myblock vultrbgp&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_myblock&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48 -interface gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_vultrbgp&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:19f0:ffff::1 fe80::1%vtnet0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Services&lt;/span&gt;
&lt;span class="nv"&gt;pf_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;sshd_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipsec_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;rtsold_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;rtsold_flags&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-aF&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Three details stand&amp;nbsp;out:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;ipv6_route_myblock&lt;/code&gt;&lt;/strong&gt; points the entire /48 at gif0. Any traffic for our prefix arriving at vtbgp via Vultr&amp;#8217;s infrastructure gets forwarded through the tunnel to hobgp, which handles actual routing to downstream&amp;nbsp;servers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;ipv6_route_vultrbgp&lt;/code&gt;&lt;/strong&gt; is a host route to Vultr&amp;#8217;s &lt;span class="caps"&gt;BGP&lt;/span&gt; peering address&amp;nbsp;(&lt;code&gt;2001:19f0:ffff::1&lt;/code&gt;) via the link-local next-hop. Vultr&amp;#8217;s peering address isn&amp;#8217;t directly connected - it&amp;#8217;s reachable via Vultr&amp;#8217;s own routing - so this static route tells the kernel where to send packets destined for it. &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;uses &lt;code&gt;ebgp-multihop 2&lt;/code&gt; to allow the &lt;span class="caps"&gt;BGP&lt;/span&gt; session to traverse that extra&amp;nbsp;hop.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;rtsold&lt;/code&gt;&lt;/strong&gt; handles &lt;span class="caps"&gt;SLAAC&lt;/span&gt; for vtbgp&amp;#8217;s provider-assigned IPv6 on vtnet0, receiving the gateway via router&amp;nbsp;advertisements.&lt;/p&gt;
&lt;p&gt;A reader might wonder how vtbgp reaches the internet for its own traffic - software updates, &lt;span class="caps"&gt;DNS&lt;/span&gt;, the &lt;span class="caps"&gt;BGP&lt;/span&gt; session setup itself - given that it deliberately doesn&amp;#8217;t install the 240,000 &lt;span class="caps"&gt;BGP&lt;/span&gt; routes into the kernel. The answer is the&amp;nbsp;IPv4 &lt;code&gt;defaultrouter="203.0.113.1"&lt;/code&gt; in &lt;code&gt;rc.conf&lt;/code&gt;. vtbgp&amp;#8217;s own outbound traffic uses a plain static default route provided by Vultr, completely independent of the &lt;span class="caps"&gt;BGP&lt;/span&gt; routing table. The &lt;span class="caps"&gt;BGP&lt;/span&gt; table exists solely for &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;#8217;s own decision-making and for propagation to hobgp via&amp;nbsp;iBGP.&lt;/p&gt;
&lt;h3 id="tcp-md5-authentication"&gt;&lt;span class="caps"&gt;TCP&lt;/span&gt;-&lt;span class="caps"&gt;MD5&lt;/span&gt;&amp;nbsp;Authentication&lt;/h3&gt;
&lt;p&gt;Vultr requires &lt;span class="caps"&gt;TCP&lt;/span&gt;-&lt;span class="caps"&gt;MD5&lt;/span&gt; authentication on &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions. This is decades-old, &lt;span class="caps"&gt;MD5&lt;/span&gt; is not modern, and it&amp;#8217;s standard across the industry - practically every carrier requires it for &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions. It prevents trivial session disruption from third parties who don&amp;#8217;t know the shared&amp;nbsp;key.&lt;/p&gt;
&lt;p&gt;On FreeBSD, &lt;span class="caps"&gt;TCP&lt;/span&gt;-&lt;span class="caps"&gt;MD5&lt;/span&gt; is implemented via the IPsec subsystem. The security associations live&amp;nbsp;in &lt;code&gt;/etc/ipsec.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;add 2001:db8:f480::20 2001:19f0:ffff::1 tcp 0x1000 -A tcp-md5 &amp;quot;your-shared-secret&amp;quot;;
add 2001:19f0:ffff::1 2001:db8:f480::20 tcp 0x1000 -A tcp-md5 &amp;quot;your-shared-secret&amp;quot;;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;span class="caps"&gt;SPI&lt;/span&gt; value (0x1000) is fixed and matches what &lt;span class="caps"&gt;FRR&lt;/span&gt; expects.&amp;nbsp;With &lt;code&gt;ipsec_enable="YES"&lt;/code&gt; in &lt;code&gt;rc.conf&lt;/code&gt; and the IPsec SAs loaded, &lt;span class="caps"&gt;FRR&lt;/span&gt; picks up the authentication automatically&amp;nbsp;when &lt;code&gt;password&lt;/code&gt; is set on the neighbor. The shared secret comes from Vultr&amp;#8217;s &lt;span class="caps"&gt;BGP&lt;/span&gt; configuration&amp;nbsp;panel.&lt;/p&gt;
&lt;h3 id="frr-configuration_1"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;frr version 10.5.1
frr defaults traditional
hostname vtbgp
log syslog informational
service integrated-vtysh-config
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
! [PL-BOGONS: same bogon filter as hobgp]
!
route-map RM-VULTR-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
exit
!
route-map RM-VULTR-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
exit
!
route-map RM-IBGP-OUT permit 10
exit
!
route-map RM-KERNEL-DENY deny 10
exit
!
router bgp 201379
 bgp router-id 203.0.113.20
 no bgp default ipv4-unicast
 neighbor 2001:19f0:ffff::1 remote-as 64515
 neighbor 2001:19f0:ffff::1 description Vultr-Core
 neighbor 2001:19f0:ffff::1 password your-shared-secret
 neighbor 2001:19f0:ffff::1 ebgp-multihop 2
 neighbor 2a06:9801:1c:fffe::1 remote-as 201379
 neighbor 2a06:9801:1c:fffe::1 description Core-hobgp
 neighbor 2a06:9801:1c:fffe::1 update-source 2a06:9801:1c:fffe::2
 !
 address-family ipv6 unicast
  neighbor 2001:19f0:ffff::1 activate
  neighbor 2001:19f0:ffff::1 soft-reconfiguration inbound
  neighbor 2001:19f0:ffff::1 route-map RM-VULTR-IN in
  neighbor 2001:19f0:ffff::1 route-map RM-VULTR-OUT out
  neighbor 2a06:9801:1c:fffe::1 activate
  neighbor 2a06:9801:1c:fffe::1 next-hop-self
  neighbor 2a06:9801:1c:fffe::1 route-map RM-IBGP-OUT out
 exit-address-family
exit
!
ipv6 protocol bgp route-map RM-KERNEL-DENY
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Two things here deserve careful&amp;nbsp;explanation.&lt;/p&gt;
&lt;h4 id="rm-kernel-deny-keep-the-routing-table-clean"&gt;&lt;code&gt;RM-KERNEL-DENY&lt;/code&gt;: Keep the Routing Table&amp;nbsp;Clean&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;ipv6 protocol bgp route-map RM-KERNEL-DENY&lt;/code&gt; applies a deny-all route-map to every &lt;span class="caps"&gt;BGP&lt;/span&gt; route before &lt;span class="caps"&gt;FRR&lt;/span&gt; tries to install it into the kernel&amp;#8217;s forwarding table. The result: &lt;span class="caps"&gt;FRR&lt;/span&gt; maintains its full &lt;span class="caps"&gt;BGP&lt;/span&gt; table internally (~240,000 routes), but none of those routes appear in the&amp;nbsp;kernel.&lt;/p&gt;
&lt;p&gt;This is intentional. vtbgp&amp;#8217;s job is to &lt;strong&gt;announce&lt;/strong&gt; our prefix, not to &lt;strong&gt;forward arbitrary internet traffic&lt;/strong&gt;. The only forwarding it does is for traffic arriving destined for 2a06:9801:1c::/48, which is handled by the static route pointing at gif0. Pushing 240,000 routes into the kernel would be wasteful memory consumption with no&amp;nbsp;benefit.&lt;/p&gt;
&lt;p&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; still uses its internal table for its own purposes and still sends routes to hobgp via iBGP - they just never touch the kernel&amp;#8217;s &lt;span class="caps"&gt;FIB&lt;/span&gt; on&amp;nbsp;vtbgp.&lt;/p&gt;
&lt;p&gt;A note for readers familiar with &lt;span class="caps"&gt;FRR&lt;/span&gt; internals: the standard layer for controlling kernel route installation is Zebra, &lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;#8217;s routing daemon that mediates between the protocol daemons (&lt;span class="caps"&gt;BGP&lt;/span&gt;, &lt;span class="caps"&gt;OSPF&lt;/span&gt;, etc.) and the &lt;span class="caps"&gt;OS&lt;/span&gt;. An alternative&amp;nbsp;to &lt;code&gt;RM-KERNEL-DENY&lt;/code&gt; is configuring Zebra directly to not install &lt;span class="caps"&gt;BGP&lt;/span&gt; routes into the kernel. Both approaches achieve the same&amp;nbsp;result; &lt;code&gt;RM-KERNEL-DENY&lt;/code&gt; intercepts routes at the &lt;span class="caps"&gt;BGP&lt;/span&gt; level before they reach Zebra at all. At this scale the distinction is academic, but Zebra is the canonical control point for &lt;span class="caps"&gt;RIB&lt;/span&gt;-to-&lt;span class="caps"&gt;FIB&lt;/span&gt; separation in production&amp;nbsp;deployments.&lt;/p&gt;
&lt;h4 id="next-hop-self-making-ibgp-routes-usable"&gt;&lt;code&gt;next-hop-self&lt;/code&gt;: Making iBGP Routes&amp;nbsp;Usable&lt;/h4&gt;
&lt;p&gt;When vtbgp receives a route from Vultr&amp;nbsp;(next-hop: &lt;code&gt;2001:19f0:ffff::1&lt;/code&gt;) and re-advertises it to hobgp via iBGP, the standard iBGP behavior is to preserve the original next-hop. hobgp can&amp;#8217;t&amp;nbsp;reach &lt;code&gt;2001:19f0:ffff::1&lt;/code&gt; directly - that&amp;#8217;s inside Vultr&amp;#8217;s network, reachable from vtbgp but not from&amp;nbsp;hobgp.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;next-hop-self&lt;/code&gt; rewrites the next-hop to vtbgp&amp;#8217;s own address&amp;nbsp;(&lt;code&gt;2a06:9801:1c:fffe::2&lt;/code&gt;) before sending routes to hobgp. hobgp reaches that address via gif1, so the routes become actionable: &amp;#8220;to get to this destination, send traffic down the iBGP tunnel to vtbgp.&amp;#8221;&amp;nbsp;Without &lt;code&gt;next-hop-self&lt;/code&gt;, hobgp would receive routes with unreachable next-hops and ignore&amp;nbsp;them.&lt;/p&gt;
&lt;h2 id="ibgp-connecting-the-two-routers"&gt;iBGP: Connecting the Two&amp;nbsp;Routers&lt;/h2&gt;
&lt;p&gt;iBGP (internal &lt;span class="caps"&gt;BGP&lt;/span&gt;) runs between routers in the same &lt;span class="caps"&gt;AS&lt;/span&gt;. It has different semantics from eBGP in two important&amp;nbsp;ways:&lt;/p&gt;
&lt;p&gt;First, the &lt;strong&gt;split-horizon rule&lt;/strong&gt;: routes learned via iBGP are not re-advertised to other iBGP peers. For a two-router setup this doesn&amp;#8217;t matter - each router hears from the other directly. For larger networks with three or more routers, you&amp;#8217;d need route reflection or a full mesh; with two, a single session is a complete&amp;nbsp;solution.&lt;/p&gt;
&lt;p&gt;Second, &lt;strong&gt;local-preference propagation&lt;/strong&gt;: &lt;span class="caps"&gt;LP&lt;/span&gt; is carried across iBGP sessions. Routes that vtbgp receives from Vultr carry &lt;span class="caps"&gt;LP&lt;/span&gt; 100 (default) until&amp;nbsp;hobgp&amp;#8217;s &lt;code&gt;RM-IBGP-IN&lt;/code&gt; overrides them to 200. This means hobgp&amp;#8217;s outbound preference for the Vultr path is configured entirely on hobgp - vtbgp doesn&amp;#8217;t need to know anything about the preference&amp;nbsp;hierarchy.&lt;/p&gt;
&lt;p&gt;The session itself is mechanically simple. On hobgp, vtbgp is a neighbor&amp;nbsp;with &lt;code&gt;remote-as 201379&lt;/code&gt; (our own &lt;span class="caps"&gt;AS&lt;/span&gt; number). &lt;span class="caps"&gt;FRR&lt;/span&gt; recognizes same-&lt;span class="caps"&gt;AS&lt;/span&gt; neighbors as iBGP peers automatically. The session runs over the gif1/gif0 tunnel pair, authenticated by virtue of the tunnel endpoints being within our own address&amp;nbsp;space.&lt;/p&gt;
&lt;h2 id="pf-on-a-transit-router-the-stateless-requirement"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt; on a Transit Router: The Stateless&amp;nbsp;Requirement&lt;/h2&gt;
&lt;p&gt;Operating a transit router exposes a fundamental tension in stateful packet filtering. On a server, stateful rules are ideal: track &lt;span class="caps"&gt;TCP&lt;/span&gt; connections, match replies to established state, drop unsolicited packets. On a transit router, stateful rules for through-traffic break&amp;nbsp;things.&lt;/p&gt;
&lt;p&gt;The reason is &lt;strong&gt;asymmetric routing&lt;/strong&gt;. When hobgp receives a packet on gre0 (via iFog) destined for a host behind gif0, &lt;span class="caps"&gt;PF&lt;/span&gt; creates a state entry associating that flow with gre0. The destination host responds, the packet traverses gif0 back to hobgp, and the best exit path might be gre1 (Lagrange) - different from the arrival interface. &lt;span class="caps"&gt;PF&lt;/span&gt; sees the reply on gre1 with no matching state on that interface and drops&amp;nbsp;it.&lt;/p&gt;
&lt;p&gt;This isn&amp;#8217;t an edge case. Path asymmetry is the norm on the public internet. A request and its reply routinely traverse different providers, different exchange points, different physical&amp;nbsp;paths.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Request: Client ──&amp;gt; [iFog / gre0] ──&amp;gt; hobgp ──&amp;gt; [gif0] ──&amp;gt; Server
                                                               │
Reply:   Client &amp;lt;── [Lagrange / gre1] &amp;lt;── hobgp &amp;lt;── [gif0] &amp;lt;──┘
                    ^^^^^^^^^^^^^^^^^^^
                    PF drops it: state was created on gre0, reply arrives on gre1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The solution is to keep transit traffic&amp;nbsp;stateless:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="gh"&gt;#&lt;/span&gt; Transit traffic: no state, no asymmetry problems
pass in quick inet6 from any to $my_network_v6 no state
pass out quick inet6 from any to $my_network_v6 no state
pass in quick inet6 from $my_network_v6 to any no state
pass out quick inet6 from $my_network_v6 to any no state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;These four rules cover all IPv6 traffic to or from&amp;nbsp;2a06:9801:1c::/48. &lt;code&gt;no state&lt;/code&gt; means &lt;span class="caps"&gt;PF&lt;/span&gt; allows the packets without creating any tracking entry. Each packet is evaluated independently on its own&amp;nbsp;merits.&lt;/p&gt;
&lt;p&gt;But not everything can be stateless. The router itself makes outbound connections - &lt;span class="caps"&gt;BGP&lt;/span&gt; keepalives, &lt;span class="caps"&gt;DNS&lt;/span&gt; queries, software updates - and those need state tracking to match replies. The control plane (&lt;span class="caps"&gt;SSH&lt;/span&gt;, &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions) also needs state. The solution is a three-tier &lt;span class="caps"&gt;PF&lt;/span&gt; structure where ordering determines which rule&amp;nbsp;wins:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="gh"&gt;#&lt;/span&gt; Tier 1: Control plane - stateful, locked to known peers
pass in quick proto tcp from &amp;lt;bgp_peers&amp;gt; to any port 179 keep state
pass out quick proto tcp from any to &amp;lt;bgp_peers&amp;gt; port 179 keep state

&lt;span class="gh"&gt;#&lt;/span&gt; Tier 2: Router&amp;#39;s own traffic - stateful for outbound connectivity
pass in  quick inet6 from any to $my_router_ip keep state
pass out quick inet6 from $my_router_ip to any keep state

&lt;span class="gh"&gt;#&lt;/span&gt; Tier 3: Transit traffic - stateless for asymmetric routing
pass in quick inet6 from any to $my_network_v6 no state
pass out quick inet6 from any to $my_network_v6 no state
pass in quick inet6 from $my_network_v6 to any no state
pass out quick inet6 from $my_network_v6 to any no state
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;span class="caps"&gt;PF&lt;/span&gt;&amp;nbsp;evaluates &lt;code&gt;quick&lt;/code&gt; rules in order and stops at the first match. Traffic&amp;nbsp;to &lt;code&gt;$my_router_ip&lt;/code&gt; (2a06:9801:1c::1, the router&amp;#8217;s address within our prefix) matches Tier 2 and gets state tracking. Everything else in the /48 falls through to Tier 3 and flows&amp;nbsp;stateless.&lt;/p&gt;
&lt;h3 id="nat-for-router-originated-traffic"&gt;&lt;span class="caps"&gt;NAT&lt;/span&gt; for Router-Originated&amp;nbsp;Traffic&lt;/h3&gt;
&lt;p&gt;There&amp;#8217;s a related problem: hobgp&amp;#8217;s physical interface has a Hetzner-assigned IPv6 address&amp;nbsp;(&lt;code&gt;2001:db8:1c19::1&lt;/code&gt;). When the router sends traffic out a &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel using that address as the source, it&amp;#8217;s sending from an address with no relation to our announced prefix. Transit providers may find this confusing; more practically, return traffic to that address may take a completely different path or simply&amp;nbsp;fail.&lt;/p&gt;
&lt;p&gt;The fix is &lt;span class="caps"&gt;NAT&lt;/span&gt; on the &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel interfaces, translating non-/48 source addresses to the router&amp;#8217;s &lt;span class="caps"&gt;PI&lt;/span&gt;&amp;nbsp;address:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_ifog&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_router_ip&lt;/span&gt;
&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_lagrange&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_router_ip&lt;/span&gt;
&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre2&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;peer_route64&lt;/span&gt;&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_router_ip&lt;/span&gt;
&lt;span class="n"&gt;nat&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gif1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;any&lt;/span&gt;&lt;span class="w"&gt;             &lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_router_ip&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Reading the first rule: packets exiting gre0, with a source address not in our /48, destined for anywhere other than the iFog &lt;span class="caps"&gt;BGP&lt;/span&gt; peer itself, get their source rewritten&amp;nbsp;to &lt;code&gt;$my_router_ip&lt;/code&gt; (2a06:9801:1c::1). The peer exclusion is critical - &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions must use the tunnel link address, not the NATed address, for the session to work. Everything else the router originates - traceroutes, pings, package fetches - appears on the internet as 2a06:9801:1c::1, an address inside our announced&amp;nbsp;prefix.&lt;/p&gt;
&lt;h2 id="verification"&gt;Verification&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;vtysh -c 'show bgp ipv6 summary'&lt;/code&gt; on hobgp shows four sessions in established&amp;nbsp;state:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;Neighbor             V    AS   MsgRcvd   MsgSent  Up/Down   State/PfxRcd  Desc
2001:db8:300::1      4  209533  1873391    10601  16:21:35       237902    Upstream-iFog-FRA
fd00:ca::1           4  209735  1267858     1771  1d05h26m       239352    Upstream-Lagrange-UK
2001:db8:400::1      4  212895   609199     1771  1d05h26m       238542    Upstream-Route64-FRA
2a06:9801:1c:fffe::2 4  201379  1370565     1771  1d05h26m       229257    Edge-Vultr-Frankfurt
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Four peers, three carrying the full &lt;span class="caps"&gt;DFZ&lt;/span&gt; table (~237-239K prefixes each), one iBGP peer with a partial Vultr view (~229K prefixes). All in established&amp;nbsp;state.&lt;/p&gt;
&lt;p&gt;The real test is whether different networks actually use different paths. &lt;span class="caps"&gt;MTR&lt;/span&gt; traces from three vantage points answer this&amp;nbsp;directly.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;From Deutsche Telekom (heavy Vultr&amp;nbsp;peering):&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;HOST: client.example.com                                    Loss%  Snt  Last   Avg
  ...
  9.|-- constantcompany-ic-375791.ip.twelve99-cust.net       0.0%   10  25.9  26.7
 10.|-- ethernetae1-sr2.fkt3.constant.com                    0.0%   10  23.4  29.6
 11.|-- 2001:19f0:6c00:154::33                               0.0%   10  24.1  23.7
 12.|-- vtbgp.example.com                                    0.0%   10  26.8  26.3
 13.|-- hobgp.example.com                                    0.0%   10  27.1  26.6
 14.|-- radon.example.com                                    0.0%   10  27.9  31.2
 15.|-- 2a06:9801:1c:1000::10                                0.0%   10  27.1  30.4
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Hops 9-11 are Constant Company infrastructure - that&amp;#8217;s Vultr&amp;#8217;s upstream. The traffic enters via vtbgp (hop 12), crosses the iBGP tunnel to hobgp (hop 13), and reaches the downstream server. The Vultr announcement is doing exactly what it should for traffic from &lt;span class="caps"&gt;DTAG&lt;/span&gt;-connected&amp;nbsp;networks.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;From Netcup (via &lt;span class="caps"&gt;AORTA&lt;/span&gt;/Vienna Internet&amp;nbsp;Exchange):&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;HOST: server.example.com                                    Loss%  Snt  Last   Avg
  ...
  9.|-- 2001:7f8:15e::64                                     0.0%   10  16.7  16.7
 10.|-- 2a0d:5440:1::f                                       0.0%   10  15.8  15.9
 11.|-- hobgp.example.com                                    0.0%   10  19.9  19.6
 12.|-- radon.example.com                                    0.0%   10  20.7  21.4
 13.|-- 2a06:9801:1c:1000::10                                0.0%   10  20.8  22.0
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Hop 9&amp;nbsp;(&lt;code&gt;2001:7f8:15e::64&lt;/code&gt;) is an internet exchange peering &lt;span class="caps"&gt;LAN&lt;/span&gt; address&amp;nbsp;(&lt;code&gt;2001:7f8::/29&lt;/code&gt; is &lt;span class="caps"&gt;IANA&lt;/span&gt;-allocated &lt;span class="caps"&gt;IXP&lt;/span&gt; space). Hop 10&amp;nbsp;(&lt;code&gt;2a0d:5440:1::f&lt;/code&gt;) belongs to route64 (&lt;span class="caps"&gt;AS212895&lt;/span&gt;). The traffic entered via route64 at an exchange point and arrived directly at hobgp - no vtbgp&amp;nbsp;involved.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;From another Hetzner server&amp;nbsp;(Falkenstein):&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;HOST: server3.example.com                                   Loss%  Snt  Last   Avg
  ...
  8.|-- 2001:7f8:11c:1:0:20:9533:1                          0.0%   10   5.5   5.6
  9.|-- hobgp.example.com                                    0.0%   10   9.5   9.6
 10.|-- radon.example.com                                    0.0%   10  10.3  16.4
 11.|-- 2a06:9801:1c:1000::10                                0.0%   10  10.5  11.3
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Hop 8 is a &lt;span class="caps"&gt;DE&lt;/span&gt;-&lt;span class="caps"&gt;CIX&lt;/span&gt; Frankfurt switching address&amp;nbsp;(the &lt;code&gt;2001:7f8:11c&lt;/code&gt; range belongs to &lt;span class="caps"&gt;DE&lt;/span&gt;-&lt;span class="caps"&gt;CIX&lt;/span&gt;). Traffic exits &lt;span class="caps"&gt;DE&lt;/span&gt;-&lt;span class="caps"&gt;CIX&lt;/span&gt; and hits hobgp in single-digit milliseconds - iFog&amp;#8217;s Frankfurt presence at the same exchange. The prefix is reachable from Hetzner in under&amp;nbsp;10ms.&lt;/p&gt;
&lt;p&gt;Three different paths, three different entry points. Each path is geometrically shorter for its respective traffic class than routing everything through a single upstream would&amp;nbsp;be.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Meta:&lt;/strong&gt; The destination address in every trace above&amp;nbsp;- &lt;code&gt;2a06:9801:1c:1000::10&lt;/code&gt; - is this blog. If you&amp;#8217;re reading this article over IPv6, you just arrived here via one of those&amp;nbsp;paths.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="what-this-costs"&gt;What This&amp;nbsp;Costs&lt;/h2&gt;
&lt;p&gt;Running your own IPv6 &lt;span class="caps"&gt;AS&lt;/span&gt; is surprisingly inexpensive. Here are the actual numbers for this setup, with monthly equivalents in&amp;nbsp;euros.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Item&lt;/th&gt;
&lt;th&gt;Actual cost&lt;/th&gt;
&lt;th&gt;Monthly (approx. €)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span class="caps"&gt;ASN&lt;/span&gt; registration (one-time)&lt;/td&gt;
&lt;td&gt;£15&lt;/td&gt;
&lt;td&gt;–&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span class="caps"&gt;ASN&lt;/span&gt; annual fee&lt;/td&gt;
&lt;td&gt;£54.99/year&lt;/td&gt;
&lt;td&gt;€5.50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;span class="caps"&gt;PA&lt;/span&gt; /48 IPv6 block&lt;/td&gt;
&lt;td&gt;£7/year&lt;/td&gt;
&lt;td&gt;€0.70&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hetzner &lt;span class="caps"&gt;CX23&lt;/span&gt; (core router, hobgp)&lt;/td&gt;
&lt;td&gt;€4.15/month&lt;/td&gt;
&lt;td&gt;€4.15&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vultr vc2-1c-1gb (edge router, vtbgp)&lt;/td&gt;
&lt;td&gt;$6.00/month&lt;/td&gt;
&lt;td&gt;€5.50&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lagrange Cloud transit (100 Mbps)&lt;/td&gt;
&lt;td&gt;£2/month&lt;/td&gt;
&lt;td&gt;€2.40&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;iFog BGPTunnel.com (100 Mbps)&lt;/td&gt;
&lt;td&gt;Free (non-commercial)&lt;/td&gt;
&lt;td&gt;€0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;route64.org (200 Mbps)&lt;/td&gt;
&lt;td&gt;Free&lt;/td&gt;
&lt;td&gt;€0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Vultr &lt;span class="caps"&gt;BGP&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;Included in &lt;span class="caps"&gt;VPS&lt;/span&gt;&lt;/td&gt;
&lt;td&gt;€0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Total recurring&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;≈ €18/month&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;That is less than many people pay for a single mid-range &lt;span class="caps"&gt;VPS&lt;/span&gt; - and it buys a redundant multi-homed autonomous system with 1.4 Gbps of available transit&amp;nbsp;capacity.&lt;/p&gt;
&lt;p&gt;The reason IPv6-only is so accessible comes down to address space economics. An IPv4 &lt;span class="caps"&gt;PA&lt;/span&gt; block costs thousands of euros per year through a sponsoring &lt;span class="caps"&gt;LIR&lt;/span&gt; and requires documented justification. An IPv6 /48 costs €8. The transit situation is similarly skewed: iFog and route64 offer free &lt;span class="caps"&gt;GRE&lt;/span&gt;-based transit specifically because IPv6-only hobbyists place negligible load on their networks. The free tier is a real service, not a&amp;nbsp;trial.&lt;/p&gt;
&lt;p&gt;The Vultr edge router is the largest recurring cost after the &lt;span class="caps"&gt;ASN&lt;/span&gt; fees, and it&amp;#8217;s paying for something specific: native &lt;span class="caps"&gt;BGP&lt;/span&gt; peering with Vultr&amp;#8217;s infrastructure, which reaches networks that simply don&amp;#8217;t have a short path to any other upstream in this setup. If path diversity from Vultr-connected networks isn&amp;#8217;t a priority, the edge router is optional and the setup collapses back to a single core router at under €5/month in compute&amp;nbsp;costs.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Stateless transit rules are not optional.&lt;/strong&gt; Getting this wrong produces subtle failures: &lt;span class="caps"&gt;ICMP&lt;/span&gt; works but &lt;span class="caps"&gt;TCP&lt;/span&gt; stalls, small packets flow freely but large ones don&amp;#8217;t, connections from one direction succeed while the other direction silently drops. When troubleshooting transit routing on &lt;span class="caps"&gt;PF&lt;/span&gt;, the first question is always whether state tracking is interfering with asymmetric&amp;nbsp;paths.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;iBGP between two routers is straightforward.&lt;/strong&gt; The iBGP full-mesh requirement sounds complicated, but with exactly two routers it&amp;#8217;s trivially satisfied by one session on each side. The main configuration concern&amp;nbsp;is &lt;code&gt;next-hop-self&lt;/code&gt;, which is required whenever the iBGP peers can&amp;#8217;t directly reach the eBGP next-hops learned from&amp;nbsp;upstream.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;RM-KERNEL-DENY&lt;/code&gt; is useful for announcement-only nodes.&lt;/strong&gt; A router whose job is to announce a prefix and forward traffic for that prefix back through a tunnel doesn&amp;#8217;t need 240,000 kernel routes. Keeping the kernel&amp;#8217;s &lt;span class="caps"&gt;RIB&lt;/span&gt; clean saves memory, avoids confusing interactions with static routes, and makes the router&amp;#8217;s actual forwarding intent&amp;nbsp;explicit.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Vultr&amp;#8217;s &lt;span class="caps"&gt;BGP&lt;/span&gt; service reaches a meaningfully different part of the internet.&lt;/strong&gt; The &lt;span class="caps"&gt;DTAG&lt;/span&gt; traceroute shows traffic entering via Vultr&amp;#8217;s Constant Company upstream - a path that simply doesn&amp;#8217;t exist via iFog or route64. The two sets of upstreams complement each other. For traffic where both would work, local-preference makes the decision; for traffic where only one path exists, the choice is made for&amp;nbsp;you.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Free transit providers provide real diversity.&lt;/strong&gt; BGPTunnel.com, route64.org, and similar services offer transit that&amp;#8217;s genuinely usable for learning and redundancy. Three upstreams with different Frankfurt exchange presence means real path diversity without transit&amp;nbsp;fees.&lt;/p&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;The gap between a single-router &lt;span class="caps"&gt;BGP&lt;/span&gt; setup and a distributed multi-homed network is mostly conceptual, not computational. The same FreeBSD tools - &lt;span class="caps"&gt;FRR&lt;/span&gt;, &lt;span class="caps"&gt;PF&lt;/span&gt;, &lt;span class="caps"&gt;GIF&lt;/span&gt;/&lt;span class="caps"&gt;GRE&lt;/span&gt; tunnels - handle the expanded setup. iBGP adds one session. The third upstream adds another &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel and two route-maps. The edge router requires &lt;span class="caps"&gt;TCP&lt;/span&gt;-&lt;span class="caps"&gt;MD5&lt;/span&gt; and a careful decision to keep routes out of the&amp;nbsp;kernel.&lt;/p&gt;
&lt;p&gt;What changes is what the setup can do. Three entry points mean inbound traffic takes geometrically shorter paths for a larger portion of the internet. Local-preference makes outbound path selection explicit and tunable. The iBGP architecture means either router can be rebooted without taking down the other&amp;#8217;s upstream&amp;nbsp;sessions.&lt;/p&gt;
&lt;p&gt;The next logical evolution would be geographic diversity - a PoP in a different region with genuinely independent upstream connectivity. But Frankfurt with three providers and two PoPs is already a substantial improvement over a single machine with one vendor&amp;#8217;s connectivity, and the lessons from building it transfer directly to any larger&amp;nbsp;scale.&lt;/p&gt;
&lt;p&gt;Each of those traceroutes ends the same way: hop 11 or hop 13 or hop 15, an address in 2a06:9801:1c::/48. The path that gets it there varies by two thousand kilometres and half a dozen autonomous systems, chosen automatically by the routing protocol. That&amp;#8217;s what multi-homing looks like when it&amp;#8217;s&amp;nbsp;working.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://bgptunnel.com/"&gt;BGPTunnel.com - Free &lt;span class="caps"&gt;BGP&lt;/span&gt; Tunnels via&amp;nbsp;iFog&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://route64.org/"&gt;route64.org - Free IPv6 &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Transit&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.vultr.com/configuring-bgp-on-vultr"&gt;Vultr &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.frrouting.org/en/latest/bgp.html"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; Documentation: iBGP and&amp;nbsp;next-hop-self&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.freebsd.org/en/books/handbook/firewalls/#pf-stateful"&gt;FreeBSD Handbook: &lt;span class="caps"&gt;PF&lt;/span&gt; Stateful&amp;nbsp;Tracking&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc2385"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 2385 - Protection of &lt;span class="caps"&gt;BGP&lt;/span&gt; Sessions via &lt;span class="caps"&gt;TCP&lt;/span&gt; &lt;span class="caps"&gt;MD5&lt;/span&gt;&amp;nbsp;Signature&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://bgp.tools/"&gt;bgp.tools - &lt;span class="caps"&gt;BGP&lt;/span&gt; Looking&amp;nbsp;Glass&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;</content><category term="Networking"/><category term="freebsd"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="frr"/><category term="pf"/><category term="ibgp"/><category term="multihoming"/></entry><entry><title>Running Your Own AS: BGP on FreeBSD with FRR, GRE Tunnels, and Policy Routing</title><link href="https://hofstede.it/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/" rel="alternate"/><published>2026-02-08T00:00:00+01:00</published><updated>2026-02-08T00:00:00+01:00</updated><author><name>Larvitz</name></author><id>tag:blog.hofstede.it,2026-02-08:/running-your-own-as-bgp-on-freebsd-with-frr-gre-tunnels-and-policy-routing/</id><summary type="html">&lt;p&gt;How I obtained my own &lt;span class="caps"&gt;AS&lt;/span&gt; number and IPv6 prefix, set up a FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; router with &lt;span class="caps"&gt;FRR&lt;/span&gt;, and built a tunnel overlay to bring globally routable addresses to servers that already have provider-assigned IPv6 - using dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; policy routing to make both&amp;nbsp;coexist.&lt;/p&gt;</summary><content type="html">&lt;hr&gt;
&lt;p&gt;&lt;img alt="Logo" src="https://hofstede.it/images/2026-02-08-running-your-own-as-bgp-freebsd.png" title="Running Your Own AS: Header image"&gt;&lt;/p&gt;
&lt;p&gt;Running your own Autonomous System on the public internet sounds like something reserved for ISPs and large enterprises. It&amp;#8217;s not. With sponsoring LIRs making &lt;span class="caps"&gt;AS&lt;/span&gt; numbers and IPv6 prefixes accessible to individuals, and FreeBSD providing the routing tools to make it work, you can announce your own address space to the Default-Free Zone from a single virtual&amp;nbsp;machine.&lt;/p&gt;
&lt;p&gt;This article walks through the complete setup: obtaining resources from &lt;span class="caps"&gt;RIPE&lt;/span&gt; via a sponsoring &lt;span class="caps"&gt;LIR&lt;/span&gt;, configuring a FreeBSD &lt;span class="caps"&gt;BGP&lt;/span&gt; router with &lt;span class="caps"&gt;FRR&lt;/span&gt;, building &lt;span class="caps"&gt;GRE&lt;/span&gt;/&lt;span class="caps"&gt;GIF&lt;/span&gt; tunnels to distribute prefixes to remote servers, and solving the routing challenge that arises when a server needs to speak from two different IPv6 address spaces&amp;nbsp;simultaneously.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note on addresses:&lt;/strong&gt; All provider-assigned &lt;span class="caps"&gt;IP&lt;/span&gt; addresses, hostnames, and management IPs in this article have been replaced with &lt;a href="https://www.rfc-editor.org/rfc/rfc5737"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5737&lt;/a&gt; / &lt;a href="https://www.rfc-editor.org/rfc/rfc3849"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 3849&lt;/a&gt; documentation ranges. My own &lt;span class="caps"&gt;AS&lt;/span&gt; number (&lt;span class="caps"&gt;AS201379&lt;/span&gt;) and prefix (2a06:9801:1c::/48) are public &lt;span class="caps"&gt;BGP&lt;/span&gt; resources and shown as-is. The upstream &lt;span class="caps"&gt;AS&lt;/span&gt; numbers (&lt;span class="caps"&gt;AS34927&lt;/span&gt;, &lt;span class="caps"&gt;AS209735&lt;/span&gt;) are equally visible in public routing&amp;nbsp;tables.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id="why-run-your-own-as"&gt;Why Run Your Own &lt;span class="caps"&gt;AS&lt;/span&gt;?&lt;/h2&gt;
&lt;p&gt;Provider-assigned IPv6 addresses are tied to that provider. Move to a different hoster and your addresses change - along with &lt;span class="caps"&gt;DNS&lt;/span&gt; records, firewall rules, reputation, and every system that references them. With your own &lt;span class="caps"&gt;AS&lt;/span&gt; and prefix, your addresses follow you. Migrate a server, update a tunnel endpoint, and traffic flows again without touching a single service&amp;nbsp;configuration.&lt;/p&gt;
&lt;p&gt;There are also less practical reasons. Understanding &lt;span class="caps"&gt;BGP&lt;/span&gt; transforms how you think about internet routing. Watching your prefix propagate through the &lt;span class="caps"&gt;DFZ&lt;/span&gt; and appear on looking glasses worldwide is genuinely satisfying. And if you run services across multiple providers, having provider-independent addressing simplifies the architecture&amp;nbsp;considerably.&lt;/p&gt;
&lt;h2 id="obtaining-resources"&gt;Obtaining&amp;nbsp;Resources&lt;/h2&gt;
&lt;p&gt;To announce prefixes on the internet, you need two things from a Regional Internet Registry (in Europe, that&amp;#8217;s &lt;span class="caps"&gt;RIPE&lt;/span&gt; &lt;span class="caps"&gt;NCC&lt;/span&gt;):&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;An &lt;span class="caps"&gt;AS&lt;/span&gt; number&lt;/strong&gt; - your identity in &lt;span class="caps"&gt;BGP&lt;/span&gt;. Mine is &lt;span class="caps"&gt;AS201379&lt;/span&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;An IPv6 prefix&lt;/strong&gt; - the address space you&amp;#8217;ll announce. I received&amp;nbsp;2a06:9801:1c::/48.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;As an individual, you don&amp;#8217;t need to become a &lt;span class="caps"&gt;RIPE&lt;/span&gt; member (which involves fees and bureaucracy). Instead, you work with a &lt;strong&gt;sponsoring &lt;span class="caps"&gt;LIR&lt;/span&gt;&lt;/strong&gt; - an existing &lt;span class="caps"&gt;RIPE&lt;/span&gt; member who sponsors your resource registration. Several LIRs cater to hobbyists and small operators. The process typically&amp;nbsp;involves:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Filling out a request form with your intended use&amp;nbsp;case&lt;/li&gt;
&lt;li&gt;Creating the appropriate &lt;span class="caps"&gt;RIPE&lt;/span&gt; database objects (aut-num, inet6num,&amp;nbsp;route6)&lt;/li&gt;
&lt;li&gt;Setting up &lt;span class="caps"&gt;RPKI&lt;/span&gt; ROAs (Route Origin Authorizations) to cryptographically bind your prefix to your &lt;span class="caps"&gt;AS&lt;/span&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Once the paperwork is done, you need upstream connectivity - someone willing to carry your &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions and announce your routes to the rest of the&amp;nbsp;internet.&lt;/p&gt;
&lt;h2 id="architecture-overview"&gt;Architecture&amp;nbsp;Overview&lt;/h2&gt;
&lt;p&gt;The setup involves two tiers: a &lt;span class="caps"&gt;BGP&lt;/span&gt; router that peers with upstream providers, and downstream servers that receive tunneled subnets from the router&amp;#8217;s&amp;nbsp;/48.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;                    ┌──────────────────────────────┐
                    │     Default-Free Zone         │
                    └──────┬──────────────┬─────────┘
                           │              │
                    AS34927 (iFog)   AS209735 (Lagrange)
                           │              │
                      GRE tunnel     Direct peering
                           │              │
                    ┌──────┴──────────────┴─────────┐
                    │    router01 (BGP Router)       │
                    │     FreeBSD + FRR              │
                    │     AS201379                   │
                    │     2a06:9801:1c::/48          │
                    └──────┬──────────────┬─────────┘
                           │              │
                      GIF tunnel     GIF tunnel
                      (proto 41)     (proto 41)
                           │              │
                    ┌──────┴───┐   ┌──────┴──────────┐
                    │  vps01   │   │  dcgw01          │
                    │  VPS     │   │  DC OPNsense     │
                    │  :1000:: │   │  :2000::/62      │
                    │  /64     │   │                   │
                    └──────────┘   └──────────────────┘
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;span class="caps"&gt;BGP&lt;/span&gt; router&amp;nbsp;(&lt;code&gt;router01&lt;/code&gt;) announces 2a06:9801:1c::/48 to two upstream providers and maintains a blackhole route for the aggregate. Individual /64s (and a /62 for my Colocation datacenter) are tunneled to downstream servers via &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnels (IPv6-in-IPv4 encapsulation). Each server receives real, globally routable addresses from my prefix while keeping its existing provider-assigned IPv6 fully&amp;nbsp;operational.&lt;/p&gt;
&lt;h2 id="the-bgp-router"&gt;The &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;Router&lt;/h2&gt;
&lt;p&gt;The router runs on a FreeBSD &lt;span class="caps"&gt;VM&lt;/span&gt; at a colocation facility with direct connectivity to two upstream networks. Let&amp;#8217;s walk through each&amp;nbsp;layer.&lt;/p&gt;
&lt;h3 id="network-configuration"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;The&amp;nbsp;router&amp;#8217;s &lt;code&gt;/etc/rc.conf&lt;/code&gt; sets up the physical interface, tunnel interfaces, and static&amp;nbsp;routes:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;router01&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Security&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Physical interface&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 198.51.100.10/24 -rxcsum -txcsum -rxcsum6 -txcsum6 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:100::96/64&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;198.51.100.1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:100::1&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Loopback alias for originated prefix&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_lo0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c::1 prefixlen 64&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Tunnel interfaces&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0 gif1 gre0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kld_list&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;if_gif if_gre&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GRE Tunnel to transit provider (iFog)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 198.51.100.44&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:300::2 2001:db8:300::1 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gre0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Transit-iFog&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF Tunnel to VPS (vps01)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 203.0.113.10&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:ffff::1 2a06:9801:1c:ffff::2 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tunnel-to-VPS&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_cloud&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:1000::/64 2a06:9801:1c:ffff::2&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF Tunnel to datacenter firewall (dcgw01)&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;tunnel 198.51.100.10 192.0.2.50&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:ffff::3 2a06:9801:1c:ffff::4 prefixlen 128&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif1_descr&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tunnel-to-Datacenter&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_dc&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c:2000::/62 2a06:9801:1c:ffff::4&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Blackhole route for the aggregate + downstream routes&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;myblock cloud dc&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_route_myblock&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48 -reject&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Services&lt;/span&gt;
&lt;span class="nv"&gt;pf_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;pflog_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;frr_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;zfs_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;sshd_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;A few things worth&amp;nbsp;explaining:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;The blackhole route&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;-reject&lt;/code&gt; for the /48) is essential. Without it, traffic for unassigned subnets within your prefix would follow the default route back to the upstream, creating a routing loop. The blackhole ensures unrouted traffic is dropped&amp;nbsp;locally.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Point-to-point tunnel addresses&lt;/strong&gt; use /128 prefixes on&amp;nbsp;the &lt;code&gt;2a06:9801:1c:ffff::/64&lt;/code&gt; link subnet. Each tunnel gets a pair of addresses from this&amp;nbsp;range.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Downstream routes&lt;/strong&gt; point specific subnets at the far end of each tunnel. The /64 for the &lt;span class="caps"&gt;VPS&lt;/span&gt; and /62 for the datacenter are routed to their respective tunnel&amp;nbsp;endpoints.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;GRE&lt;/span&gt; vs &lt;span class="caps"&gt;GIF&lt;/span&gt;&lt;/strong&gt;: The iFog peering uses &lt;span class="caps"&gt;GRE&lt;/span&gt; because that&amp;#8217;s what the provider requires. The downstream tunnels use &lt;span class="caps"&gt;GIF&lt;/span&gt; (protocol 41, IPv6-in-IPv4) which is simpler and has less&amp;nbsp;overhead.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id="frr-configuration"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt; (Free Range Routing) handles the &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions. The configuration lives&amp;nbsp;at &lt;code&gt;/usr/local/etc/frr/frr.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;frr version 10.5.1
frr defaults traditional
hostname router01
log syslog informational
service integrated-vtysh-config
!
ipv6 prefix-list PL-MY-NET seq 5 permit 2a06:9801:1c::/48
!
ipv6 prefix-list PL-BOGONS seq 5 deny ::/0 le 7
ipv6 prefix-list PL-BOGONS seq 10 deny ::/8
ipv6 prefix-list PL-BOGONS seq 15 deny 100::/8
ipv6 prefix-list PL-BOGONS seq 20 deny 200::/7
ipv6 prefix-list PL-BOGONS seq 25 deny 400::/6
ipv6 prefix-list PL-BOGONS seq 30 deny 800::/5
ipv6 prefix-list PL-BOGONS seq 35 deny 1000::/4
ipv6 prefix-list PL-BOGONS seq 40 deny 4000::/3
ipv6 prefix-list PL-BOGONS seq 45 deny 6000::/3
ipv6 prefix-list PL-BOGONS seq 50 deny 8000::/3
ipv6 prefix-list PL-BOGONS seq 55 deny a000::/3
ipv6 prefix-list PL-BOGONS seq 60 deny c000::/3
ipv6 prefix-list PL-BOGONS seq 65 deny e000::/4
ipv6 prefix-list PL-BOGONS seq 70 deny f000::/5
ipv6 prefix-list PL-BOGONS seq 75 deny f800::/6
ipv6 prefix-list PL-BOGONS seq 80 deny fc00::/7
ipv6 prefix-list PL-BOGONS seq 85 deny fe80::/10
ipv6 prefix-list PL-BOGONS seq 90 deny fec0::/10
ipv6 prefix-list PL-BOGONS seq 95 deny ff00::/8
ipv6 prefix-list PL-BOGONS seq 100 deny 2a06:9801:1c::/48
ipv6 prefix-list PL-BOGONS seq 105 deny ::/0 ge 49
ipv6 prefix-list PL-BOGONS seq 110 permit ::/0 le 48
!
route-map RM-IFOG-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
 set community 34927:9501 34927:9301 additive
exit
!
route-map RM-LAGRANGE-OUT permit 10
 match ipv6 address prefix-list PL-MY-NET
 set as-path prepend 201379 201379
exit
!
route-map RM-IFOG-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
exit
!
route-map RM-LAGRANGE-IN permit 10
 match ipv6 address prefix-list PL-BOGONS
exit
!
ipv6 route 2a06:9801:1c::/48 blackhole
!
router bgp 201379
 bgp router-id 198.51.100.10
 no bgp default ipv4-unicast
 neighbor 2001:db8:300::1 remote-as 34927
 neighbor 2001:db8:300::1 description Upstream-iFog
 neighbor 2001:db8:300::1 ttl-security hops 1
 neighbor 2001:db8:300::1 update-source gre0
 neighbor 2001:db8:100::ff remote-as 209735
 neighbor 2001:db8:100::ff description Upstream-Lagrange
 neighbor 2001:db8:100::ff ttl-security hops 1
 neighbor 2001:db8:100::ff update-source 2001:db8:100::96
 !
 address-family ipv6 unicast
  network 2a06:9801:1c::/48
  neighbor 2001:db8:300::1 activate
  neighbor 2001:db8:300::1 soft-reconfiguration inbound
  neighbor 2001:db8:300::1 maximum-prefix 250000 90 restart 30
  neighbor 2001:db8:300::1 route-map RM-IFOG-IN in
  neighbor 2001:db8:300::1 route-map RM-IFOG-OUT out
  neighbor 2001:db8:100::ff activate
  neighbor 2001:db8:100::ff soft-reconfiguration inbound
  neighbor 2001:db8:100::ff maximum-prefix 250000 90 restart 30
  neighbor 2001:db8:100::ff route-map RM-LAGRANGE-IN in
  neighbor 2001:db8:100::ff route-map RM-LAGRANGE-OUT out
 exit-address-family
exit
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;There&amp;#8217;s a lot happening here. Let me break down the key design&amp;nbsp;decisions.&lt;/p&gt;
&lt;h4 id="prefix-lists"&gt;Prefix&amp;nbsp;Lists&lt;/h4&gt;
&lt;p&gt;Two prefix lists control what gets sent and&amp;nbsp;received:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;PL&lt;/span&gt;-&lt;span class="caps"&gt;MY&lt;/span&gt;-&lt;span class="caps"&gt;NET&lt;/span&gt;&lt;/strong&gt;: Matches only our /48. Used in outbound route-maps to ensure we only ever announce our own&amp;nbsp;prefix.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;PL&lt;/span&gt;-&lt;span class="caps"&gt;BOGONS&lt;/span&gt;&lt;/strong&gt;: A comprehensive bogon filter for inbound routes. This rejects non-routable address space (link-local, &lt;span class="caps"&gt;ULA&lt;/span&gt;, multicast, documentation ranges), our own prefix (to prevent loops), and anything more specific than a /48 or less specific than a /8. The&amp;nbsp;final &lt;code&gt;permit ::/0 le 48&lt;/code&gt; at the end accepts everything that survived the deny&amp;nbsp;rules.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The bogon filter deserves emphasis. Accepting bad routes from peers can cause anything from black-holed traffic to becoming an unwitting participant in route hijacks. Filter aggressively on&amp;nbsp;inbound.&lt;/p&gt;
&lt;h4 id="route-maps"&gt;Route&amp;nbsp;Maps&lt;/h4&gt;
&lt;p&gt;Each peer gets its own pair of inbound/outbound route&amp;nbsp;maps:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Outbound to iFog&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;RM-IFOG-OUT&lt;/code&gt;): Announces our /48 with &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;communities &lt;code&gt;34927:9501&lt;/code&gt; and &lt;code&gt;34927:9301&lt;/code&gt;. These are iFog-specific communities that control route propagation - in this case, requesting announcement to specific peering&amp;nbsp;partners.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Outbound to Lagrange&lt;/strong&gt;&amp;nbsp;(&lt;code&gt;RM-LAGRANGE-OUT&lt;/code&gt;): Announces our /48 with &lt;span class="caps"&gt;AS&lt;/span&gt;-path prepending (adds our &lt;span class="caps"&gt;ASN&lt;/span&gt; twice). This makes the Lagrange path appear longer to the rest of the internet, steering inbound traffic to prefer the iFog path. Useful for traffic engineering when one upstream has better&amp;nbsp;connectivity.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Inbound from both&lt;/strong&gt;: Apply the bogon filter to reject garbage&amp;nbsp;routes.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id="bgp-session-details"&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; Session&amp;nbsp;Details&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;no bgp default ipv4-unicast&lt;/code&gt;&lt;/strong&gt;: We&amp;#8217;re IPv6-only. Don&amp;#8217;t activate IPv4 address family by&amp;nbsp;default.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;ttl-security hops 1&lt;/code&gt;&lt;/strong&gt;: &lt;span class="caps"&gt;GTSM&lt;/span&gt; (Generalized &lt;span class="caps"&gt;TTL&lt;/span&gt; Security Mechanism) - reject &lt;span class="caps"&gt;BGP&lt;/span&gt; packets with &lt;span class="caps"&gt;TTL&lt;/span&gt; less than 254. This prevents remote attacks on the &lt;span class="caps"&gt;BGP&lt;/span&gt; session since only directly connected peers can send packets with &lt;span class="caps"&gt;TTL&lt;/span&gt;&amp;nbsp;255.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;soft-reconfiguration inbound&lt;/code&gt;&lt;/strong&gt;: Store received routes before applying filters. This lets you change inbound policy without resetting the &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;session.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;maximum-prefix 250000 90 restart 30&lt;/code&gt;&lt;/strong&gt;: Safety valve. If a peer sends more than 250,000 prefixes (or 90% of that as a warning), tear down the session and retry after 30 minutes. Protects against route leaks from&amp;nbsp;upstream.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id="firewall-on-the-router"&gt;Firewall on the&amp;nbsp;Router&lt;/h3&gt;
&lt;p&gt;The &lt;span class="caps"&gt;BGP&lt;/span&gt; router&amp;#8217;s &lt;span class="caps"&gt;PF&lt;/span&gt; configuration protects the control plane while allowing data plane&amp;nbsp;forwarding:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# --- Macros ---&lt;/span&gt;
&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vtnet0&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;dc_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif1&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;vps_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;gif0&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;trusted_ipv4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;{ 198.51.100.100, 198.51.100.101 }&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;trusted_ipv6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;{ 2001:db8:ffff:1::/64, 2001:db8:ffff:2::/64 }&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;bgp_peers_v4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;{ 198.51.100.20 }&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;bgp_peers_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;{ 2001:db8:100::ff }&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;ifog_gre_endpoint&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;198.51.100.44&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;ifog_bgp_peer&lt;/span&gt;&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2001:db8:300::1&amp;quot;&lt;/span&gt;

&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2a06:9801:1c::/48&amp;quot;&lt;/span&gt;
&lt;span class="n"&gt;vps_v4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;203.0.113.10&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# --- Tables ---&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bruteforce&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;persist&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;trusted_v4&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;trusted_ipv4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;trusted_v6&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;trusted_ipv6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v4&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v6&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="n"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bogons&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;const&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;10.0&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;172.16&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="mf"&gt;192.168&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mf"&gt;169.254&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="mf"&gt;0.0&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;96&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;fc00&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;fec0&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ff00&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# --- Options ---&lt;/span&gt;
&lt;span class="n"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;skip&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;lo0&lt;/span&gt;
&lt;span class="n"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;policy&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;drop&lt;/span&gt;
&lt;span class="n"&gt;set&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;loginterface&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;

&lt;span class="c1"&gt;# --- Scrub ---&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;fragment&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;reassemble&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;vps_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;mss&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1440&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;dc_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;mss&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1140&lt;/span&gt;
&lt;span class="n"&gt;scrub&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;mss&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1400&lt;/span&gt;

&lt;span class="c1"&gt;# --- Filtering ---&lt;/span&gt;
&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;log&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;
&lt;span class="n"&gt;block&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bogons&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;
&lt;span class="n"&gt;antispoof&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# --- Control Plane ---&lt;/span&gt;

&lt;span class="c1"&gt;# SSH from trusted sources only&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;trusted_v4&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;S&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;SA&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;rate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="n"&gt;overload&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bruteforce&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;flush&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;global&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;trusted_v6&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;22&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;S&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;SA&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nb"&gt;max&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;src&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;rate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;     &lt;/span&gt;&lt;span class="n"&gt;overload&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bruteforce&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;flush&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;global&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# BGP (TCP 179) - strictly limited to known peers&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v4&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;S&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;SA&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="n"&gt;bgp_peers_v6&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;flags&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;S&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;SA&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# GRE tunnel from iFog&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ifog_gre_endpoint&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;tcp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ifog_bgp_peer&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;179&lt;/span&gt;

&lt;span class="c1"&gt;# ICMPv6: essential for NDP, PMTUD, and diagnostics&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ipv6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;icmp6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;type&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;echoreq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;echorep&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;neighbrsol&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;neighbradv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="n"&gt;toobig&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;timex&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;paramprob&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;routersol&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;icmp&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;type&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;echoreq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;unreach&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;timex&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# --- Data Plane ---&lt;/span&gt;

&lt;span class="c1"&gt;# Inbound traffic destined for our prefix&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;gre0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# Return traffic from downstream tunnels&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;vps_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;dc_tun&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;my_network_v6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;

&lt;span class="c1"&gt;# GIF tunnel encapsulation (proto 41) from downstream endpoints&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="ow"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;41&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;vps_v4&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;$&lt;/span&gt;&lt;span class="n"&gt;ext_if&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# Outbound&lt;/span&gt;
&lt;span class="k"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;out&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The firewall cleanly separates &lt;strong&gt;control plane&lt;/strong&gt; (&lt;span class="caps"&gt;SSH&lt;/span&gt;, &lt;span class="caps"&gt;BGP&lt;/span&gt; sessions) from &lt;strong&gt;data plane&lt;/strong&gt; (forwarded traffic). The control plane rules are strict: &lt;span class="caps"&gt;BGP&lt;/span&gt; is locked to known peer addresses, &lt;span class="caps"&gt;SSH&lt;/span&gt; to trusted management IPs. The data plane rules are simpler since the router just needs to forward packets between upstreams and downstream&amp;nbsp;tunnels.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;block in quick on $ext_if from { &amp;lt;bogons&amp;gt;, $my_network_v6 }&lt;/code&gt; rule is important - it drops packets claiming to come from our own prefix arriving on the external interface. If someone on the internet spoofs a source address from our range, this catches it before it enters the forwarding&amp;nbsp;path.&lt;/p&gt;
&lt;p&gt;Note the per-tunnel &lt;span class="caps"&gt;MSS&lt;/span&gt; clamping in the scrub section. Each tunnel has different overhead (&lt;span class="caps"&gt;GRE&lt;/span&gt; adds more headers than &lt;span class="caps"&gt;GIF&lt;/span&gt;), so the &lt;span class="caps"&gt;MSS&lt;/span&gt; values differ. Getting this wrong causes mysterious connection stalls with large&amp;nbsp;packets.&lt;/p&gt;
&lt;h2 id="the-downstream-server-dual-stack-with-policy-routing"&gt;The Downstream Server: Dual-Stack with Policy&amp;nbsp;Routing&lt;/h2&gt;
&lt;p&gt;This is where things get interesting. The &lt;span class="caps"&gt;VPS&lt;/span&gt;&amp;nbsp;(&lt;code&gt;vps01&lt;/code&gt;) already has provider-assigned IPv6 from its hoster. Jails on this server use addresses from both address&amp;nbsp;spaces:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Provider IPv6&lt;/strong&gt; (2001:db8:200:0:1000::/68) - the hoster&amp;#8217;s addresses, NATed to the&amp;nbsp;host&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;span class="caps"&gt;BGP&lt;/span&gt; IPv6&lt;/strong&gt; (2a06:9801:1c:1000::/64) - our own prefix, routed natively via the &lt;span class="caps"&gt;GIF&lt;/span&gt;&amp;nbsp;tunnel&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Private IPv4&lt;/strong&gt; (10.254.254.0/24) - NATed to the host&amp;#8217;s public&amp;nbsp;IPv4&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The challenge: when a jail sends traffic from its &lt;span class="caps"&gt;BGP&lt;/span&gt; address (2a06:&amp;#8230;), that traffic must exit through the &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel to the &lt;span class="caps"&gt;BGP&lt;/span&gt; router - not through the default route to the &lt;span class="caps"&gt;VPS&lt;/span&gt; provider, where it would be dropped as spoofed. But traffic from the provider address must continue using the normal default&amp;nbsp;route.&lt;/p&gt;
&lt;p&gt;The solution is &lt;strong&gt;dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; policy routing&lt;/strong&gt; - FreeBSD&amp;#8217;s implementation of multiple routing&amp;nbsp;tables.&lt;/p&gt;
&lt;h3 id="how-dual-fib-works"&gt;How Dual-&lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;Works&lt;/h3&gt;
&lt;p&gt;FreeBSD supports multiple routing tables called FIBs (Forwarding Information Bases). Each &lt;span class="caps"&gt;FIB&lt;/span&gt; is an independent routing table with its own default route and entries. Interfaces and &lt;span class="caps"&gt;PF&lt;/span&gt; rules can assign traffic to a specific &lt;span class="caps"&gt;FIB&lt;/span&gt;, and the kernel consults the appropriate table when&amp;nbsp;forwarding.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;FIB 0 (default):
  default --&amp;gt; vtnet0 --&amp;gt; VPS provider upstream
  Used by: host traffic, provider-addressed jail traffic

FIB 1:
  default --&amp;gt; gif0 --&amp;gt; BGP router (router01)
  Used by: BGP-addressed jail traffic (2a06:9801:1c::/48)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3 id="network-configuration_1"&gt;Network&amp;nbsp;Configuration&lt;/h3&gt;
&lt;p&gt;Here&amp;#8217;s the relevant portion of the&amp;nbsp;server&amp;#8217;s &lt;code&gt;/etc/rc.conf&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;hostname&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;vps01.example.com&amp;quot;&lt;/span&gt;

&lt;span class="nv"&gt;kern_securelevel_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;kern_securelevel&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;2&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Primary interface - provider IPv4 and IPv6&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 203.0.113.10 netmask 255.255.252.0 -lro -tso&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_vtnet0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:200::2 prefixlen 68&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;203.0.113.1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_defaultrouter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fe80::1%vtnet0&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Jail bridge - three address spaces&lt;/span&gt;
&lt;span class="nv"&gt;cloned_interfaces&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;bridge0 gif0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_bridge0_name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;bastille0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_bastille0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet 10.254.254.1/24&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_bastille0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2001:db8:200:0:1000::1 prefixlen 68&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_bastille0_alias0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:1000::1 prefixlen 64&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# GIF tunnel to BGP router - assigned to FIB 1&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib 1 tunnel 203.0.113.10 198.51.100.10 tunnelfib 0&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ifconfig_gif0_ipv6&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;inet6 2a06:9801:1c:ffff::2 2a06:9801:1c:ffff::1 prefixlen 128&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# Enable forwarding&lt;/span&gt;
&lt;span class="nv"&gt;gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;ipv6_gateway_enable&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;YES&amp;quot;&lt;/span&gt;

&lt;span class="c1"&gt;# FIB 1 routing table entries&lt;/span&gt;
&lt;span class="nv"&gt;static_routes&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib1default jailleak bgplink&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_fib1default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-6 default -interface gif0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_jailleak&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-6 2001:db8:200:0:1000::/68 -interface bastille0 -fib 1&amp;quot;&lt;/span&gt;
&lt;span class="nv"&gt;route_bgplink&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;-6 2a06:9801:1c:1000::/64 -interface bastille0 -fib 1&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel configuration deserves a closer&amp;nbsp;look:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nv"&gt;ifconfig_gif0&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;fib 1 tunnel 203.0.113.10 198.51.100.10 tunnelfib 0&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This single line contains two critical&amp;nbsp;directives:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;fib 1&lt;/code&gt;&lt;/strong&gt;: The tunnel interface itself lives in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1. Traffic arriving on gif0 and traffic routed out gif0 consults routing table&amp;nbsp;1.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;tunnelfib 0&lt;/code&gt;&lt;/strong&gt;: But the outer IPv4 encapsulation (the 203.0.113.10 &amp;#8212;&amp;gt; 198.51.100.10 wrapper) uses &lt;span class="caps"&gt;FIB&lt;/span&gt; 0. This is essential - the IPv4 path to the &lt;span class="caps"&gt;BGP&lt;/span&gt; router goes through the provider&amp;#8217;s default route in &lt;span class="caps"&gt;FIB&lt;/span&gt; 0.&amp;nbsp;Without &lt;code&gt;tunnelfib 0&lt;/code&gt;, the encapsulated packets would try to use &lt;span class="caps"&gt;FIB&lt;/span&gt; 1&amp;#8217;s default route (which points at gif0 itself), creating a recursive&amp;nbsp;loop.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The three static routes in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 complete the&amp;nbsp;picture:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;fib1default&lt;/code&gt;&lt;/strong&gt;: Default route in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 exits through gif0 to the &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;router&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;jailleak&lt;/code&gt;&lt;/strong&gt;: Tells &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 that the provider&amp;#8217;s jail subnet is reachable via bastille0 (without this, return traffic in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 for jails&amp;#8217; provider addresses would try to exit through&amp;nbsp;gif0)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;bgplink&lt;/code&gt;&lt;/strong&gt;: Same for the &lt;span class="caps"&gt;BGP&lt;/span&gt; jail subnet - &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 needs to know these addresses are local on&amp;nbsp;bastille0&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id="pf-the-routing-glue"&gt;&lt;span class="caps"&gt;PF&lt;/span&gt;: The Routing&amp;nbsp;Glue&lt;/h3&gt;
&lt;p&gt;&lt;span class="caps"&gt;PF&lt;/span&gt; is where the address-based routing decision happens. When a jail sends a packet from a &lt;span class="caps"&gt;BGP&lt;/span&gt; address, &lt;span class="caps"&gt;PF&lt;/span&gt; assigns it to &lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;1:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="err"&gt;#&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;BGP&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;addressed&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;jail&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;traffic&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;--&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;force&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;into&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;routing&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;table&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;exits&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;via&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;gif0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nx"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;bastille0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;bgp_net&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;rtable&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;rtable 1&lt;/code&gt; directive is the key. It tells &lt;span class="caps"&gt;PF&lt;/span&gt; to route matching packets using &lt;span class="caps"&gt;FIB&lt;/span&gt; 1 instead of the default &lt;span class="caps"&gt;FIB&lt;/span&gt; 0. Since &lt;span class="caps"&gt;FIB&lt;/span&gt; 1&amp;#8217;s default route points out gif0 to the &lt;span class="caps"&gt;BGP&lt;/span&gt; router, these packets get encapsulated and sent to router01, which then forwards them to the internet with the correct source&amp;nbsp;address.&lt;/p&gt;
&lt;p&gt;For traffic arriving on the tunnel destined for jails, &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;nbsp;uses &lt;code&gt;reply-to&lt;/code&gt; to ensure return traffic takes the same&amp;nbsp;path:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="err"&gt;#&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Inbound&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;BGP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;traffic&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;reply&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ensures&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;responses&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;exit&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;via&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;gif0&lt;/span&gt;
&lt;span class="nx"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;tun_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;reply&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;tun_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;bgp_hub_ip&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nx"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;bgp_net&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;

&lt;span class="err"&gt;#&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;BGP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ICMPv6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;also&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;needs&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;reply&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;correct&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;path&lt;/span&gt;
&lt;span class="nx"&gt;pass&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;quick&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;tun_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;reply&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;tun_if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;bgp_hub_ip&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;inet6&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;proto&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ipv6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nx"&gt;icmp&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nx"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;any&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;to&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;$&lt;/span&gt;&lt;span class="nx"&gt;bgp_net&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nx"&gt;icmp6&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="k"&gt;type&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;echoreq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;echorep&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;toobig&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;timex&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;paramprob&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;\
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nx"&gt;keep&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;state&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Without &lt;code&gt;reply-to&lt;/code&gt;, the kernel would consult &lt;span class="caps"&gt;FIB&lt;/span&gt; 0 for return traffic (since the jail itself isn&amp;#8217;t in &lt;span class="caps"&gt;FIB&lt;/span&gt; 1), and replies to &lt;span class="caps"&gt;BGP&lt;/span&gt;-addressed connections would exit through vtnet0 with the wrong source routing - getting dropped as spoofed by the provider.&amp;nbsp;The &lt;code&gt;reply-to&lt;/code&gt; directive forces &lt;span class="caps"&gt;PF&lt;/span&gt; to send reply packets back out the interface they arrived on, to the specified&amp;nbsp;next-hop.&lt;/p&gt;
&lt;h3 id="the-complete-picture"&gt;The Complete&amp;nbsp;Picture&lt;/h3&gt;
&lt;p&gt;Here&amp;#8217;s how a request to a &lt;span class="caps"&gt;BGP&lt;/span&gt;-addressed jail service&amp;nbsp;flows:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt; 1. Client sends packet to 2a06:9801:1c:1000::10 (web jail)
 2. Packet traverses the internet, reaching AS201379 via iFog or Lagrange
 3. router01 forwards it through gif0 tunnel to vps01
 4. vps01 receives proto 41 on vtnet0, decapsulates --&amp;gt; gif0
 5. PF matches: reply-to ($tun_if $bgp_hub_ip), creates state
 6. Packet forwarded to bastille0 --&amp;gt; jail
 7. Jail responds, packet exits on bastille0
 8. PF&amp;#39;s state table triggers reply-to: send via gif0 to bgp_hub_ip
 9. gif0 encapsulates (proto 41) using FIB 0 to reach router01
10. router01 receives, forwards to upstream --&amp;gt; internet --&amp;gt; client
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;And for outbound connections initiated by the jail using its &lt;span class="caps"&gt;BGP&lt;/span&gt;&amp;nbsp;address:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;1. Jail sends packet from 2a06:9801:1c:1000::10
2. Packet arrives on bastille0
3. PF matches: &amp;quot;from $bgp_net --&amp;gt; rtable 1&amp;quot;
4. Kernel routes via FIB 1 --&amp;gt; default route --&amp;gt; gif0
5. gif0 encapsulates using FIB 0 --&amp;gt; vtnet0 --&amp;gt; router01
6. router01 receives, forwards to internet (source: 2a06:9801:1c:1000::10)
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Meanwhile, the exact same jail can communicate using its provider address through the normal default route in &lt;span class="caps"&gt;FIB&lt;/span&gt; 0, with &lt;span class="caps"&gt;NAT&lt;/span&gt; to the host&amp;#8217;s address. Both address spaces coexist on the same interface, differentiated purely by &lt;span class="caps"&gt;PF&lt;/span&gt; rules and &lt;span class="caps"&gt;FIB&lt;/span&gt;&amp;nbsp;selection.&lt;/p&gt;
&lt;h2 id="verification"&gt;Verification&lt;/h2&gt;
&lt;p&gt;Once everything is running, verification is straightforward. From inside a jail with both&amp;nbsp;addresses:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# Traffic from the provider address - NATed through the hoster&lt;/span&gt;
root@caddy:~&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="c1"&gt;# curl --interface 2001:db8:200:0:1000::10 https://ifconfig.co&lt;/span&gt;
&lt;span class="m"&gt;2001&lt;/span&gt;:db8:200::2

&lt;span class="c1"&gt;# Traffic from the BGP address - routed natively through the tunnel&lt;/span&gt;
root@caddy:~&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="c1"&gt;# curl --interface 2a06:9801:1c:1000::10 https://ifconfig.co&lt;/span&gt;
2a06:9801:1c:1000::10
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The first request shows the host&amp;#8217;s NATed provider address. The second shows the jail&amp;#8217;s real &lt;span class="caps"&gt;BGP&lt;/span&gt; address - confirming the packet traversed the tunnel and reached the internet through &lt;span class="caps"&gt;AS201379&lt;/span&gt;.&lt;/p&gt;
&lt;p&gt;A traceroute from an external host confirms the &lt;span class="caps"&gt;BGP&lt;/span&gt; path is&amp;nbsp;working:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;$ mtr -rw 2a06:9801:1c:1000::10
HOST:                              Loss%   Snt   Last   Avg  Best  Wrst StDev
 1.|-- [local-gateway]               0.0%    10    2.6   5.5   2.6  14.1   3.6
    ...
 9.|-- [transit-provider-edge]       0.0%    10   33.8  46.2  33.8  81.2  19.0
10.|-- [ifog-peering-fabric]         0.0%    10   33.5  46.7  33.5  87.0  18.6
11.|-- 2001:db8:300::2               0.0%    10   44.3  59.1  41.9 136.7  33.2
12.|-- 2a06:9801:1c:ffff::2         0.0%    10   72.7  98.9  68.8 198.8  42.4
13.|-- 2a06:9801:1c:1000::10        0.0%    10  164.1  83.1  63.5 164.1  33.4
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Traffic enters via the transit provider (hops 9-10), traverses the &lt;span class="caps"&gt;GRE&lt;/span&gt; tunnel to router01 (hop 11), then the &lt;span class="caps"&gt;GIF&lt;/span&gt; tunnel to vps01 (hop 12,&amp;nbsp;the &lt;code&gt;2a06:9801:1c:ffff::2&lt;/code&gt; link address), and finally reaches the jail (hop 13). The prefix also shows up correctly on bgp.tools as active and originated by &lt;span class="caps"&gt;AS201379&lt;/span&gt; with both upstreams&amp;nbsp;visible.&lt;/p&gt;
&lt;h2 id="lessons-learned"&gt;Lessons&amp;nbsp;Learned&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;MSS&lt;/span&gt; clamping is non-negotiable with tunnels.&lt;/strong&gt; Every layer of encapsulation eats into the &lt;span class="caps"&gt;MTU&lt;/span&gt;. &lt;span class="caps"&gt;GIF&lt;/span&gt; adds 20 bytes (IPv4 header) to every packet. &lt;span class="caps"&gt;GRE&lt;/span&gt; adds more. If you don&amp;#8217;t clamp the &lt;span class="caps"&gt;TCP&lt;/span&gt; &lt;span class="caps"&gt;MSS&lt;/span&gt;, large packets get fragmented or dropped, causing mysterious failures where small requests work but large transfers stall.&amp;nbsp;Set &lt;code&gt;max-mss&lt;/code&gt; in &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s scrub rules for every tunnel interface, calculated as: &lt;span class="caps"&gt;MTU&lt;/span&gt; minus IPv6 header (40 bytes) minus &lt;span class="caps"&gt;TCP&lt;/span&gt; header (20&amp;nbsp;bytes).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;span class="caps"&gt;FIB&lt;/span&gt; separation is cleaner than source-based routing hacks.&lt;/strong&gt; FreeBSD&amp;#8217;s multi-&lt;span class="caps"&gt;FIB&lt;/span&gt; support is a first-class feature.&amp;nbsp;Using &lt;code&gt;rtable&lt;/code&gt; in &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;nbsp;and &lt;code&gt;fib&lt;/code&gt;/&lt;code&gt;tunnelfib&lt;/code&gt; on interfaces gives you full control over which routing table handles which traffic. It&amp;#8217;s conceptually cleaner and more debuggable than alternatives like ip6tables &lt;span class="caps"&gt;MARK&lt;/span&gt; targets on&amp;nbsp;Linux.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Bogon filtering matters even for small networks.&lt;/strong&gt; The internet is full of misconfigurations and occasional malice. Filtering inbound routes prevents your router from accepting nonsense that could black-hole traffic or worse. The cost is a few lines of configuration; the protection is&amp;nbsp;real.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;reply-to&lt;/code&gt; solves asymmetric routing.&lt;/strong&gt; When traffic can arrive on multiple interfaces, the kernel&amp;#8217;s default &lt;span class="caps"&gt;FIB&lt;/span&gt; selection for return traffic may choose the wrong path. &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s &lt;code&gt;reply-to&lt;/code&gt; directive forces replies back out the arrival interface, which is exactly what you need for tunnel overlay&amp;nbsp;setups.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Start with two upstreams.&lt;/strong&gt; A single upstream means zero redundancy and no ability to do traffic engineering. Two upstreams give you failover and the ability to prefer one path over the other using &lt;span class="caps"&gt;AS&lt;/span&gt;-path prepending or communities. The operational complexity increase is&amp;nbsp;minimal.&lt;/p&gt;
&lt;h2 id="conclusion"&gt;Conclusion&lt;/h2&gt;
&lt;p&gt;Running your own &lt;span class="caps"&gt;AS&lt;/span&gt; on the internet is more accessible than most people assume. The barrier isn&amp;#8217;t technical complexity - it&amp;#8217;s knowing that the option exists. A FreeBSD &lt;span class="caps"&gt;VM&lt;/span&gt;, &lt;span class="caps"&gt;FRR&lt;/span&gt;, a couple of tunnels, and some careful &lt;span class="caps"&gt;PF&lt;/span&gt; rules give you provider-independent addressing, real &lt;span class="caps"&gt;BGP&lt;/span&gt; peering, and a deeper understanding of how the internet actually&amp;nbsp;works.&lt;/p&gt;
&lt;p&gt;The dual-&lt;span class="caps"&gt;FIB&lt;/span&gt; approach on the downstream server is the piece I&amp;#8217;m most satisfied with. It elegantly solves the &amp;#8220;two address spaces, one server&amp;#8221; problem without hacks: &lt;span class="caps"&gt;BGP&lt;/span&gt; traffic takes the tunnel, provider traffic takes the default route, and &lt;span class="caps"&gt;PF&lt;/span&gt;&amp;#8217;s &lt;code&gt;rtable&lt;/code&gt; directive makes the decision based purely on source address. Both paths coexist transparently, and the jails don&amp;#8217;t need to know anything about the routing&amp;nbsp;underneath.&lt;/p&gt;
&lt;p&gt;Is it overkill for a blog? Absolutely. But the same infrastructure carries every service I run, and having addresses that survive provider migrations has already paid for itself in operational simplicity. Besides, there&amp;#8217;s something deeply satisfying about seeing your own &lt;span class="caps"&gt;AS&lt;/span&gt; number show up in a&amp;nbsp;traceroute.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id="references"&gt;References&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.ripe.net/manage-ips-and-asns/resource-management/number-resources/"&gt;&lt;span class="caps"&gt;RIPE&lt;/span&gt; &lt;span class="caps"&gt;NCC&lt;/span&gt; - Requesting&amp;nbsp;Resources&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.frrouting.org/en/latest/"&gt;&lt;span class="caps"&gt;FRR&lt;/span&gt;&amp;nbsp;Documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.freebsd.org/en/books/handbook/firewalls/"&gt;FreeBSD Handbook: Firewalls (&lt;span class="caps"&gt;PF&lt;/span&gt;)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://man.freebsd.org/cgi/man.cgi?query=setfib"&gt;FreeBSD&amp;nbsp;setfib(1)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://bgp.tools/"&gt;bgp.tools&lt;/a&gt; - &lt;span class="caps"&gt;BGP&lt;/span&gt; looking glass and&amp;nbsp;analytics&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.ripe.net/manage-ips-and-asns/resource-management/rpki/"&gt;&lt;span class="caps"&gt;RIPE&lt;/span&gt; &lt;span class="caps"&gt;RPKI&lt;/span&gt;&amp;nbsp;Documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.rfc-editor.org/rfc/rfc5082"&gt;&lt;span class="caps"&gt;RFC&lt;/span&gt; 5082 - &lt;span class="caps"&gt;GTSM&lt;/span&gt; (&lt;span class="caps"&gt;TTL&lt;/span&gt;&amp;nbsp;Security)&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;p&gt;The internet is a network of networks, and now you&amp;#8217;re one of them. There&amp;#8217;s a certain elegance in participating in the same routing protocol that glues together every network on the planet - from your single /48 all the way up to the Tier 1 carriers. &lt;span class="caps"&gt;BGP&lt;/span&gt; doesn&amp;#8217;t care about your size. It just cares that your routes are valid, your filters are clean, and your packets know where to&amp;nbsp;go.&lt;/p&gt;</content><category term="Networking"/><category term="freebsd"/><category term="bgp"/><category term="networking"/><category term="ipv6"/><category term="frr"/><category term="pf"/><category term="tunneling"/></entry></feed>