All posts

Default route or a full BGP table

BIRD 2.14 held 1,176,120 IPv4 routes in 173 MiB. A default route took 2.6 MiB and the same next hop. A 2 GiB cap held the table.

Written by NodeRings Inc.

NodeRings Inc. writes these notes. The company runs the marketplace and the agent that stays on the provider’s side of the hypervisor.

Every route in this session pointed at one next hop. BIRD 2.14 imported 1,176,120 IPv4 routes from a single peer. Three restarts left the process at 177,152 KiB, 177,204 KiB, and 177,232 KiB. The middle value is 177,204 KiB, which is 173 MiB. The same daemon, importing only 0.0.0.0/0, sat at 2,652 KiB, 2,696 KiB, and 2,772 KiB. The middle value is 2,696 KiB, which is 2.6 MiB.

The CIDR Report lists 1,088,687 IPv4 prefixes in its table history, and 1,088,384 in the aggregation summary. The potaroo BGP table report uses that second figure for the active FIB. The BIRD table is 87,433 prefixes above the table-history number. It is one path for each prefix in the RIPE RIS rrc00 RIB, the prefixes that collector's peers had sent. The smaller number is the active FIB count those two reports publish.

Default routeFull IPv4 table
Routes BIRD kept11,176,120
Resident set, middle of three starts2,696 KiB177,204 KiB
Seconds from process start to a settled count2.0411.5
Routes exported00
Next hop192.0.2.54192.0.2.54
Two pairs of bars. Route count is 1 against 1,176,120. Resident set is 2,696 KiB against 177,204 KiB.
Middle of three starts. Teal is the default. Purple is the full IPv4 import.

What BIRD used for the full table

show memory on the loaded process split the cost into 107.0 MB of routing table and 53.6 MB of route attributes, 160.6 MB in total, plus 9.9 MB of overhead. The resident set sits a little above that accounting. Both numbers are the full table. The 173 MiB figure is the kernel's charge for the process, from the middle of the three restarts.

The clock starts when the process starts, so the seconds include the session coming up. The full table settled in 11.2, 11.5, and 12.3 seconds. The default settled in 2.04, 2.95, and 2.04 seconds. The middle of each set of three is the number in the table.

birdc show route count
birdc show memory
birdc show route count prints 1176120 routes in master4 and 0 in master6. show memory totals 160.6 MB plus 9.9 MB of overhead.
BIRD 2.14 after the full IPv4 import. IPv6 was not in this feed, so master6 stayed empty.

A 2 GiB cap, with swap turned off for that process, still reached 1,176,120 routes. systemd reported MemoryCurrent=180154368 against MemoryMax=2147483648. That current figure is 172 MiB. The process was still running, and the host IPv4 route count was 5 before the session and 5 after it. BIRD had no kernel protocol, so none of the million routes were installed in the forwarding table.

ps prints 179160 KiB for the BIRD process. free shows 48973 MiB of RAM and swap at 0. A 2 GiB memory cap reports 180154368 bytes in use.
The 179,160 KiB line is BIRD under the 2 GiB cap. The used column on the host also includes the program that replayed the RIB.

That other program was GoBGP 4.11.0. It read the RIB and spoke the session. By the end of the restarts it had grown to 3,588,436 KiB, about 3.4 GiB. A VPS running BIRD in this role is the 173 MiB process. The 3.4 GiB belonged to the program that held the dump.

One peer, and what it sent

The session was eBGP between 192.0.2.53 (AS 65053, BIRD) and 192.0.2.54 (AS 65054, GoBGP), on a veth pair in two network namespaces.

BIRD came from the Ubuntu package bird2 2.14-1build2. Installing it runs ucf to place /usr/share/bird2/bird.conf at /etc/bird/bird.conf, adds the bird user, and starts bird.service. The unit's check is bird -p, then bird -f as that user. birdc is the shell the measurements used.

apt-get install bird2
systemctl enable --now bird
birdc configure
birdc show route count
birdc show memory

birdc configure reloads /etc/bird/bird.conf after the protocol block below replaces the sample's empty policy. The sample starts the daemon and imports nothing. The block is what filled master4.

The BIRD config that imported the table was this:

protocol bgp upstream {
    local 192.0.2.53 as 65053;
    neighbor 192.0.2.54 as 65054;
    ipv4 {
        import all;
        export none;
    };
}

GoBGP was given the rrc00 RIB with --only-best, --no-ipv6, and --nexthop 192.0.2.54. In GoBGP 4.11.0 the best-path flag keeps the first path stored for the prefix. The next hop on the wire was the lab peer, which is the next hop BIRD accepted. IPv6 stayed out, and master6 stayed at 0 routes.

1.1.1.0/24 arrived with AS path 65054 34549 13335, communities (34549,200) and (34549,10000), and an aggregator at AS13335. 8.8.8.0/24 arrived with AS path 65054 34549 15169. The leading 65054 is the lab speaker. The communities and the aggregator are what filled the 53.6 MB of route attributes.

The RIB also contained 0.0.0.0/0, with AS path 65054 44393 64073 174. A collected full feed can include a default because some network announced one. That route is a different object from a session whose only job is to hand you the default your own upstream generated.

When the extra routes change an exit

Take the full table when a second exit exists and a policy will use the extra prefixes. With the one peer in this lab, 0.0.0.0/0 and 1.1.1.0/24 both used 192.0.2.54. The extra prefixes changed the size of the table. With one next hop, there was no second exit to prefer.

A second session to a second network is what gives a prefix a choice of exit. Two sessions to the same provider keep the session up when one of them drops, and the packets still leave through that provider when both next hops are theirs.

A default learned from the session is withdrawn when that session closes. A static default in the kernel stays, and the more-specifics from the closed session are gone, so traffic keeps using the static route. That is the case where the full table shows you the upstream vanished. This lab left the routes out of the kernel, so that withdraw was not timed.

GetVPS puts the same choice on a VPS panel. Some locations offer a default and no full table. Receiving the full table does not make your own prefix easier to reach. Reachability is the export. This session exported nothing, because the output filter was reject. A session that should announce your prefix needs that prefix allowed out, whether the import is a default or the full table. Nothing in this import checked a ROA. A route in the table is a route the peer sent.

How many IPv4 prefixes are in the global table?

The CIDR Report's table history lists 1,088,687. The active FIB figure on that report, and on the potaroo report, is 1,088,384. The BIRD session held 1,176,120, one path per prefix from the rrc00 RIB.

Will a 2 GB VPS hold a full IPv4 table in BIRD?

This process did. BIRD 2.14, one IPv4 peer, 1,176,120 routes, 173 MiB resident, under a 2 GiB cap with swap off for that process. The parser that read the dump was a different process, and that one used 3.4 GiB. Size the daemon that will keep running after the import.

The 173 MiB is the table. The service the routes exist to reach still has to fit beside it. The cap test had nothing else in the cgroup.

Does importing the full table announce your prefix?

This one exported 0 routes. import all fills your table. export none sends nothing. The prefix you are allowed to originate belongs in the export, and it belongs there in either mode. GetVPS makes the same split: the route mode is what they send you, and your announcement is a separate decision.

Does a full table reject a hijack?

This session had no RPKI check. Importing 1,176,120 routes accepted 1,176,120 routes. An invalid origin would have landed in the table with the rest. The drop happens in a filter you configure, or in a filter your upstream applied before the route was sent.

Does NodeRings hand a VM a BGP session?

IPAM assigns a static IPv4, a static IPv6, and a gateway. That gateway is a default. The page does not describe a BGP neighbor, a full table, or an ASN on the guest.