blog/ posts/ freeRtr SR ISIS EVPN iBGP-IX-peering-fabric
← FRRouting SR IS-IS EVPN iBGP-IX-peering-fabric | posts | FRRouting routing protocol debugging using terminal monitor →
  1. Network topology
  2. Configuration
    1. IS-IS
      1. P router
      2. PE router
    2. BFD
      1. P router
      2. PE router
    3. Full IS-IS BFD configs
    4. Verification
      1. IS-IS interface
      2. IS-IS neighbor
      3. BFD neighbor
      4. IP route
      5. segrout
      6. IS-IS spf 2
    5. BGP
      1. PE router
    6. BGP EVPN PE configs
    7. Verification
      1. BGP neighbor
      2. BGP evpn database
      3. bridge
  3. Convergence test
  4. Summary
  5. See also
  6. References

The topic sound really great, and it really is for me too. But at the bottom it is about building a switch using 6 routers. At the beginning of February 2024 I have been visiting the Cisco Live 2024 in Amsterdam. The sessions visited were only SR - Segement Routing focused. The Service Provider sessions are all good and technical to the bottom, mostly protocol and hardware focused, more protocol. Long term solution orientation. Visited one session presented by Emerson Moura - BRKSP-2275 - Rethink your Edge Routing Architecture. Emerson talked about new alternative networking designs in operation that are possible using SR or SRv6. If easy enough design is used vendor agnostic solutions based on SR or SRv6, ar possible for service providers. It is possible to replace highly redundant, chassis-based routers to, protocol based, cheaper, one HU pizza-box routers,running SR, you need at least 5 or 6 of them, but all are independent from the control plane of view. Nowadays LACP using separate control plane is possible, i did not know of that. SRvR using ELAN - RFC draft. All protocol based using SRv6 on IOS-XR and works. It is a technical use case. Separating functions into distinct areas and moving away from chassis design to protocol based design to bunch of protocol rich and capable 1 height unit SR routers. For running many SR routers and operating them a smart design necessary.

This is a netlab in GNS3 made using currently available and implemented technologies and protocols using the free freeRtr router. This will be a attempt to take a already very well written and full of interesting SR MPLS network designs article following document here., written by Phil Bedard.

The solution is made using freeRtr. To be able to configure and verify protocols used following freeRtr examples were based on following test scenarios:

📘 Note
The freeRtr examples linked above might change and become unavailable during time, new features and tests are added constantly. In such case select the protocol example shown here in a handy overview.

Configuring a EVPN BGP IX peering, based on the xrdoc.io blog entry using freeRtr free router implementation. This is a 1-st approach creating a networking solution without having deep insight into freeRtr or experience at all. The IPv6 part in the freeRtr configuration is without any functionality. The pmsitun BGP template configuration is nice to have, you need to decide if your network has a use-case for it. Read the RFC reference put in the section. The reason having this unnecessary part here is, configuration is taken primarily from the both linked examples above. Leave that parts out to make your netlab more easy to configure.

The protocols setting used in this freeRtr IX-peering solution:

IPv4 and IPv6 (IP dual-stack):

  • VRF v1
  • IS-IS
  • SR
    • label index 1 - 20
    • v4 label base - 16000
    • v6 label base - 20000
  • BFD
    • 100ms 100ms 3

IPv4:

  • BGP EVPN
    • bridge1
    • bvi1
    • vxlan1
  • internal BGP - AS65000

No BGP route reflector used in example. Internal BGP full-mesh network consisting of:

  • PE11
  • PE12
  • PE13
  • PE14

Network topology

The freeRtr appliances have 4 ports allocated eth1 - eth4. A fabric network design is a specific

  • P1, P2
  • PE1, PE2, PE3, PE4

Network topology with IP addressing:

      lo                  lo                   lo
  192.0.2.11           192.0.2.1            192.0.2.13
  +-------+            +-------+            +-------+
  +4     1+------------+1     3+------------+1     4+
  | PE11  |            |  P1   |            |  PE13 |
  |      2+-+  +-------+2     4+-+  +-------+2      |
  +-------+ |  |       +-------+ |  |       +-------+
            |  |                 |  |
            +--(-----+           +--(-----+
               |     |              |     |
  +-------+    |     | +-------+    |     | +-------+
  |      1+----+     +-+1     3+----+     +-+1      |
  | PE12  |            |  P2   |            |  PE14 |
  +4     2+------------+2     4+------------+2     4+
  +-------+            +-------+            +-------+
     lo                   lo                   lo
  192.0.2.12           192.0.2.2            192.0.2.14
  • +4 - physical ethernet port number

Configuration

The goal is to configure a routed fabric, routing MAC address only using BGP EVPN. For customer routers (CE) the connection type is like connected to a switch.

The generic approach is to configure in first step the IS-IS routed network and route only the loopback interfaces. First goal is to establish the reachalisty of the loopack interfaces. loopback1 loopback2.

In the second setup step configure the BGP EVPN on the PE routers only.

The BGP EVPN is setup only on IPv4, while the IS-IS configured to be running dual-stack.

IS-IS

Using freeRtr implementation IS-IS is setup per IP address family. This is the IS-IS configuration only. The interfaces facing to the PE routers are configured as sub-interfaces. Each IS-IS instance is handled per IP address family. All routers are IS-IS level2 only.

For the protocol configuration overview, the P and PE routers have following configuration specifics:

Routing protocol instance IP address family, per process id:

  • isis4 1
  • isis6 1

IP loopback per address family:

  • IPv4 - loopback1
  • IPv6 - loopback2

Sub-interface per IP address family:

  • IPv4 - eth1.1
  • IPv6 - eth1.2

SPF specific configuration:

config
router isis4 1
 level2 spf-ecmp
 ecmp

P router

All configurations for all routers are at the end of the section. This is only the lengthy configuration of one of the P1 routers:

config
!
vrf definition v1
 rd 1:1
 exit
!
hostname P1
!
router isis4 1
 vrf v1
 net-id 49.0001.1111.1111.1111.00
 traffeng 192.0.2.1
 is-type level2
 segrout 20 base 16000
 level2 spf-ecmp
 level2 segrout 
 justadvert loopback1
 ecmp
 exit
!
router isis6 1
 vrf for v1
 net-id 49.0001.1111.1111.1111.00
 is-type level2
 traffeng 192.0.2.1
 segrout 20 base 20000
 level2 spf-ecmp
 level2 segrout
 justadvert loopback2
 ecmp
 exit
!
interface loopback1
 vrf forwarding v1
 ipv4 address 192.0.2.1 255.255.255.255
 router isis4 1 ena
 router isis4 1 segrout index 1
 router isis4 1 segrout node
 exit
!
interface loopback2
 vrf forwarding v1
 ipv6 address 2001:db8:1111:: ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
 router isis6 1 ena
 router isis6 1 segrout index 1
 router isis6 1 segrout node
 exit
!
int eth1.1
 vrf forwarding v1
 ipv4 address 1.0.0.1 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 no shutdown
 exit
!
int eth1.2
 vrf for v1
 ipv6 addr fe80:1:11::1 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
int eth2.1
 vrf forwarding v1
 ipv4 address 1.0.0.5 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 exit
!
int eth2.2
 vrf for v1
 ipv6 addr fe80:1:12::1 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
int eth3.1
 vrf forwarding v1
 ipv4 address 1.0.0.9 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 no shutdown
 exit
!
int eth3.2
 vrf for v1
 ipv6 addr fe80:1:13::1 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
int eth4.1
 vrf forwarding v1
 ipv4 address 1.0.0.13 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 no shutdown
 exit
!
int eth4.2
 vrf for v1
 ipv6 addr fe80:1:14::1 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
end
wr

PE router

This is the IS-IS configuration for the PE11 router:

config
hostname PE11
!
vrf definition v1
 rd 1:1
 exit
!
router isis4 1
 vrf v1
 net-id 49.0001.1011.1111.1111.00
 traffeng 192.0.2.11
 is-type level2
 segrout 20 base 16000
 level2 spf-ecmp
 level2 segrout
 justadvert loopback1
 ecmp
 exit
!
router isis6 1
 vrf v1
 net-id 49.0001.1011.1111.1111.00
 is-type level2
 traffeng 192.0.2.11
 segrout 20 base 20000
 level2 spf-ecmp 
 level2 segrout
 justadvert loopback2
 ecmp
 exit
!
interface loopback1
 vrf forwarding v1
 ipv4 address 192.0.2.11 255.255.255.255
 router isis4 1 ena
 router isis4 1 segrout index 11
 router isis4 1 segrout node
 no shutdown
 exit
!
interface loopback2
 vrf for v1
 ipv6 address 2001:db8:1011:: ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff
 router isis6 1 ena
 router isis6 1 segrout index 11
 router isis6 1 segrout node
 exit
!
int eth1.1
 vrf forwarding v1
 ipv4 address 1.0.0.2 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 no shutdown
 exit
!
int eth1.2
 vrf for v1
 ipv6 addr fe80:1:11::2 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
int eth2.1
 vrf forwarding v1
 ipv4 address 1.0.0.18 255.255.255.252
 mpls enable
 router isis4 1 enable
 router isis4 1 bfd
 no shutdown
 exit
!
int eth2.2
 vrf for v1
 ipv6 addr fe80:2:11::2 ffff:ffff:ffff:ffff::
 mpls enable
 router isis6 1 ena
 router isis6 1 bfd
 exit
!
end
wr

BFD

To not to configure to many things at once, here the BFD specific part for each IP address family. This configuration sets the BFD timers and enables BFD. Without setting BFD timers the enabled BFD sessions will not come up.

P router

Configuration for the P routers:

!P router configuration
!
config
int eth1.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
 exit
int eth1.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
exit
int eth2.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
exit
int eth2.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
exit
int eth3.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
 exit
int eth3.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
exit
int eth4.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
 exit
int eth4.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
end
wr

PE router

PE router configuration:

!PE router
config
int eth1.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
 exit
int eth1.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
exit
int eth2.1
 ipv4 bfd 100 100 3
 router isis4 1 bfd
exit
int eth2.2
 ipv6 bfd 100 100 3
 router isis6 1 bfd
end
wr

Full IS-IS BFD configs

The full IS-IS and SR MPLS configurations for the routers in the network topology:

Verification

Verification steps to assure configuration and routers are working.

IS-IS interface

Show commands to display the interfaces:

show ipv4 isis 1 interface

For the IPv6 address family:

show ipv6 isis 1 interface

Notice the loopack interface does not have neighbors, it is set to passive.

P1#show ipv4 isis 1 interface
interface    neighbors
loopback1    0
ethernet1.1  1
ethernet2.1  1
ethernet3.1  1
ethernet4.1  1

P1#show ipv6 isis 1 interface
interface    neighbors
loopback2    0
ethernet1.2  1
ethernet2.2  1
ethernet3.2  1
ethernet4.2  1

IS-IS neighbor

Show command:

show ipv4 isis 1 neighbor

Notice the IPv4 neighbours on the ethX.1 sub-interface:

P1#show ipv4 isis 1 neighbor
interface    level  routerid        ip address  other address  state  uptime
ethernet1.1  2      1011.1111.1111  1.0.0.2     ::             up     00:26:48
ethernet2.1  2      1012.1111.1111  1.0.0.6     ::             up     01:42:20
ethernet3.1  2      1013.1111.1111  1.0.0.10    ::             up     00:10:25
ethernet4.1  2      1014.1111.1111  1.0.0.14    ::             up     00:37:44

Show command:

show ipv6 isis 1 neighbor

Command output for IPv6 the neighbouring routers globally using the ethX.2 sub-interface:

P1#show ipv6 isis 1 neighbor
interface    level  routerid        ip address    other address  state  uptime
ethernet1.2  2      1011.1111.1111  fe80:1:11::2  ::             up     00:29:04
ethernet2.2  2      1012.1111.1111  fe80:1:12::2  ::             up     01:44:36
ethernet3.2  2      1013.1111.1111  fe80:1:13::2  ::             up     00:12:41
ethernet4.2  2      1014.1111.1111  fe80:1:14::2  ::             up     01:44:39

BFD neighbor

Verify BFD neigbors on point to point links

show ipv4 bfd v1 neighbor

Show bfd neighbors, notice the timeout setting in the command line output and the clients columns:

P1#show ipv4 bfd v1 neighbor
interface    address   state  timeout  uptime    clients
ethernet1.1  1.0.0.2   up     300      00:33:39  isis
ethernet2.1  1.0.0.6   up     300      01:54:39  isis
ethernet3.1  1.0.0.10  up     300      00:00:53  isis
ethernet4.1  1.0.0.14  up     300      00:39:07  isis

Command line output for IPv6:

P1#show ipv6 bfd v1 neighbor
interface    address       state  timeout  uptime    clients
ethernet1.2  fe80:1:11::2  up     300      00:34:07  isis
ethernet2.2  fe80:1:12::2  up     300      01:55:06  isis
ethernet3.2  fe80:1:13::2  up     300      00:01:20  isis
ethernet4.2  fe80:1:14::2  up     300      00:39:34  isis

IP route

Show command to display the IPv4 routing table. freeRtr has out of the box no default routing table. Or more specific it is needed to set the according routing table to show the ip route output, in example v1:

show ipv4 route v1

Command line output:

P1#show ipv4 route v1
typ  prefix         metric  iface        hop       time
C    1.0.0.0/30     0/0     ethernet1.1  null      01:50:14
LOC  1.0.0.1/32     0/1     ethernet1.1  null      01:50:14
C    1.0.0.4/30     0/0     ethernet2.1  null      01:50:13
LOC  1.0.0.5/32     0/1     ethernet2.1  null      01:50:13
C    1.0.0.8/30     0/0     ethernet3.1  null      01:50:13
LOC  1.0.0.9/32     0/1     ethernet3.1  null      01:50:13
C    1.0.0.12/30    0/0     ethernet4.1  null      01:50:12
LOC  1.0.0.13/32    0/1     ethernet4.1  null      01:50:12
I    1.0.0.16/30    115/20  ethernet1.1  1.0.0.2   00:29:02
I    1.0.0.20/30    115/20  ethernet2.1  1.0.0.6   01:50:01
I    1.0.0.24/30    115/20  ethernet3.1  1.0.0.10  00:12:39
I    1.0.0.28/30    115/20  ethernet4.1  1.0.0.14  00:34:29
C    192.0.2.1/32   0/0     loopback1    null      01:50:15
I    192.0.2.2/32   115/30  ethernet1.1  1.0.0.2   00:29:02
I    192.0.2.11/32  115/20  ethernet1.1  1.0.0.2   00:29:02
I    192.0.2.12/32  115/20  ethernet2.1  1.0.0.6   01:50:01
I    192.0.2.13/32  115/20  ethernet3.1  1.0.0.10  00:12:39
I    192.0.2.14/32  115/20  ethernet4.1  1.0.0.14  00:34:29

Routing table output for the IPv6 v1 table.

show ipv6 route v1

Command line output verifying the configuration is working.

P1#show ipv6 route v1
typ  prefix               metric  iface        hop           time
I    2001:db8:1011::/128  115/20  ethernet1.2  fe80:1:11::2  00:28:06
I    2001:db8:1012::/128  115/20  ethernet2.2  fe80:1:12::2  01:49:05
I    2001:db8:1013::/128  115/20  ethernet3.2  fe80:1:13::2  00:11:43
I    2001:db8:1014::/128  115/20  ethernet4.2  fe80:1:14::2  00:33:34
C    2001:db8:1111::/128  0/0     loopback2    null          01:49:19
C    fe80:1:11::/64       0/0     ethernet1.2  null          01:49:18
LOC  fe80:1:11::1/128     0/1     ethernet1.2  null          01:49:18
C    fe80:1:12::/64       0/0     ethernet2.2  null          01:49:18
LOC  fe80:1:12::1/128     0/1     ethernet2.2  null          01:49:18
C    fe80:1:13::/64       0/0     ethernet3.2  null          01:49:17
LOC  fe80:1:13::1/128     0/1     ethernet3.2  null          01:49:17
C    fe80:1:14::/64       0/0     ethernet4.2  null          01:49:16
LOC  fe80:1:14::1/128     0/1     ethernet4.2  null          01:49:16
I    fe80:2:11::/64       115/20  ethernet1.2  fe80:1:11::2  00:28:06
I    fe80:2:12::/64       115/20  ethernet2.2  fe80:1:12::2  01:49:05
I    fe80:2:13::/64       115/20  ethernet3.2  fe80:1:13::2  00:11:43
I    fe80:2:14::/64       115/20  ethernet4.2  fe80:1:14::2  00:33:34

Note, the IPv6 configuration does not have any function in this freeRtr IX-fabric. The BGP EVPN configuration part relies on reachability of the IPv4 loopback1 addresses.

segrout

Display the configured and used labels for prefixes using the 

show ipv4 segrout v1

command. Output form the P1 router

P1#show ipv4 segrout v1
prefix         index  base   oldbase
192.0.2.2/32   2      16000  16000
192.0.2.2/32   2      16000  16000
192.0.2.2/32   2      16000  16000
192.0.2.2/32   2      16000  16000
192.0.2.11/32  11     16000  16000
192.0.2.12/32  12     16000  16000
192.0.2.13/32  13     16000  16000
192.0.2.14/32  14     16000  16000

Command output from the PE11 router:

PE11#show ipv4 segrout v1
prefix         index  base   oldbase
192.0.2.1/32   1      16000  16000
192.0.2.2/32   2      16000  16000
192.0.2.12/32  12     16000  16000
192.0.2.12/32  12     16000  16000
192.0.2.13/32  13     16000  16000
192.0.2.13/32  13     16000  16000
192.0.2.14/32  14     16000  16000
192.0.2.14/32  14     16000  16000

IS-IS spf 2

To verify ISIS segment routing working use following commands:

show ipv4 isis 1 spf 0

Command output from the P1 router:

P1#show ipv4 isis 1 spf 0
category     value
reach        PE11 PE12 PE13 PE14 P1 P2
reachable    6
unreach
unreachable  0
stub
segrou       PE11=11 PE12=12 PE13=13 PE14=14 P1=1 P2=2
nosegrou
bier
nobier       PE11 PE12 PE13 PE14 P1 P2
topostr      PE11,true,1,2,3 PE12,true,1,2,3 PE13,true,1,2,3 PE14,true,1,2,3 P1,true,1,4,5 P2,true,1,4,5
topoid       1b2e1c62
last         2024-02-28 13:31:40 (00:00:50 ago)
fill         5
calc         0
table        0
run          146

when  ago  time  topoid  unreach

Other protocols like the multicast replacement - B.I.E.R. keep working also using freeRtr, but that is not used and configured here in example as shown in the command output above.

BGP

The BGP EVPN setup is applied only to the PE routers. The P routers are only for the reachability of the loopback interfaces. aka smart-edge dumb-core. the core or P routers only forward as quick as possible. The edge

The BGP EVPN template has one specific setting pmsitun, that I did not see used on other implementations. The explanation is found in the RFC6541 - BGP Encodings and Procedures for Multicast in MPLS/BGP IP VPNs. Thanks to vxwarlock in libera.chat. for pointing this out.

PE router

Fixed most easy example, one bridge:

  • iBGP - all AS65000
  • bridge 1 - one bridge only
  • bvi1 - (not configured)
  • vxlan100 - BGP encapsulation
  • vrf v1
  • BGP
    • address family EVPN
    • encapsulation vxlan

The bvi1 interface is left in its default configuration without IP addressing.

freeRtr EVPN BGP PE router configuration:

config
!
hostname PE11
!
bridge 1
 rd 1:1
 rt-import 1:1
 rt-export 1:1
 mac-learn
 exit
!
vrf definition v1
 rd 1:1
 labelmode per-prefix
 exit
!
router bgp4 1
 vrf v1
 router-id 192.0.2.11
 address-family evpn
 !
 template iBGP local-as 65000
 template iBGP remote-as 65000
 template iBGP address-family evpn
 template iBGP update-source loopback0
 template iBGP send-community standard extended
 template iBGP pmsitun
 !
 neighbor 192.0.2.12 template iBGP
 !
 neighbor 192.0.2.13 template iBGP
 !
 neighbor 192.0.2.14 template iBGP
 !
 afi-evpn 100 bridge-group 1
 afi-evpn 100 encapsulation vxlan
 afi-evpn 100 update-source loopback0
 !
 exit
!
interface ethernet4
 description CE-access-port
 bridge-group 1
 no shutdown
end
wr

BGP EVPN PE configs

The PE BGP configurations for the routers in the network topology:

Verification

BGP configuration verification procedure.

BGP neighbor

Verify the bgp neighbor status:

PE11#show ipv4 bgp 1 summary
neighbor    as     ready  learn  sent  uptime
192.0.2.12  65000  yes    2      2     01:12:03
192.0.2.13  65000  yes    2      2     07:18:22
192.0.2.14  65000  yes    2      2     07:18:22

All 4 PE routers are connected.

BGP evpn database

Now to verify which addressess are already routed across the IX network use:

show ipv4 bgp 1 evpn database

This is a state of a converged network. CE routers are already connected to the PE routers

PE11#show ipv4 bgp 1 evpn database
prefix                     hop         metric       aspath
200::aabb:cc00:300#:: 1:1  192.0.2.11  0/0/0/0
200::aabb:cc00:400#:: 1:1  192.0.2.12  200/100/0/0
200::aabb:cc00:500#:: 1:1  192.0.2.13  200/100/0/0
200::aabb:cc00:600#:: 1:1  192.0.2.14  200/100/0/0
300::#192.0.2.14 1:1       192.0.2.14  200/100/0/0
300::#192.0.2.13 1:1       192.0.2.13  200/100/0/0
300::#192.0.2.12 1:1       192.0.2.12  200/100/0/0
300::#192.0.2.11 1:1       192.0.2.11  0/0/0/0

bridge

The show bridge command one of the few commands where vxlan output is shown. There might be other commands showing different vxlan settings I am not aware of. This command will only be successfully on the PE routers, where the MAC bridge is configured:

show bridge 1

Output on the PE11 router, showing statistics to the bridge bgp neighbors and routed MAC addresses:

PE11#show bridge 1
iface                fwd   phys   tx    rx    drop  tx       rx      drop  grp
brprt bvi
ethernet4            true  true   7793  6473  0     1201729  648404  0
vxlan to 192.0.2.14  true  false  2743  3125  0     330602   491909  0
vxlan to 192.0.2.12  true  false  3170  3558  0     374088   541175  0
vxlan to 192.0.2.13  true  false  720   1110  0     124150   262161  0

addr            iface                static  time      tx    rx    drop  tx      rx      drop
aabb.cc00.0100  ethernet4            false   00:00:16  4621  6473  0     471868  648404  0
aabb.cc00.0200  vxlan to 192.0.2.12  false   00:00:16  2504  3558  0     255742  498479  0
aabb.cc00.0300  vxlan to 192.0.2.13  false   00:01:16  54    1110  0     5804    248841  0
aabb.cc00.0400  vxlan to 192.0.2.14  false   00:00:16  2077  3125  0     212256  454409  0

The ethernet4 interface is the port facing the customer edge (CE) router. Check the addr entry of the output.

Routing MAC addresses using BGP. Building a big switch.

Convergence test

This is a random ICMP test for IPv4. The CE routers, here CE1 - CE4. are connected to the PE routers. For the CE routers perspective the network topology looks like this:

  +-----+     +-----+     +-----+     +-----+
  |     |     |     |     |     |     |     |
  | CE1 |     | CE2 |     | CE3 |     | CE4 |
  |     |     |     |     |     |     |     |
  +--0--+     +--0--+     +--0--+     +--0--+
     |           |           |           |
     4-----------4-----------4-----------4

The CE routers are connected to ethernet4 PE interface. That interface is in bridge-group 1.

For the test, on a fully converged network topology,

1) The P1 router is powered off (power outage simulation) 2) Result the IX peering network converges 3) The P1 is powered on during power outage 4) Result the IX peering network converges

ICMP echo for the interface IP on the subnet assigned to each CE on the eth0 interface 192.168.1.0/29 IP subnet. All CE routers are running OSPFv2 and the interface setting is to ip ospf point-to-multipoint for IPv4 and IPv6, identical setting. However the interface IP address is directly connected.

CE1 - ping 192.168.1.4

Success rate is 99 percent (1998/2000), round-trip min/avg/max = 27/32/92 ms

CE2 - ping 192.168.1.1

Success rate is 99 percent (1998/2000), round-trip min/avg/max = 26/33/80 ms

CE3 - ping 192.168.1.2

Success rate is 99 percent (1999/2000), round-trip min/avg/max = 29/32/80 ms

CE4 - ping 192.168.1.3

Success rate is 99 percent (1999/2000), round-trip min/avg/max = 25/31/80 ms

ICMPv6 echo for the routed loopback IPv6 address of the router. IPv6 OSPF dynamic routing configured on CE routers. The target IP 2001:db8:203:113:1:: is the routed loopback.

CE1 - ping 2001:DB8:203:113:4::

Success rate is 99 percent (1998/2000), round-trip min/avg/max = 26/33/72 ms

CE2 - ping 2001:DB8:203:113:1::

Success rate is 99 percent (1998/2000), round-trip min/avg/max = 25/33/45 ms

CE3 - ping 2001:DB8:203:113:2::

Success rate is 99 percent (1999/2000), round-trip min/avg/max = 27/31/48 ms

CE4 - ping 2001:DB8:203:113:3::

Success rate is 99 percent (1999/2000), round-trip min/avg/max = 20/36/72 ms

This is the convergence packet loss meassured during for shut down P1 then power on P1. 1,5 IP packets lost on the whole system. The convergence results for the MAC routing fabric are, for me, very impressive. It is all over very good. Much more than I expected from a free available implementation, in any sense. Amazing router this freeRtr.

Summary

freeRtr is capable and swiss networking knife router appliance. Testing this given scenario proved a stable solution. Using BFD improves convergence time of routing protocols drastically. This is a real use-case for fast converging "system", and the results are impressive.

Using testing hardware like in CERN-APS Networks BF2556X-1T using the p4 TOFINO NPU - would be nice to see in production or pilot deployment. If the BFD could be offloaded to the NIC integrated circuits. It would be interesting test such setup in real. Building a giant fabric MAN network or state wide network.

My personal opinion about freeRtr. freeRtr is a solid networking router implementation, capable of providing hundreds of services out of the box. It is stable in netlab environment and has a extensive protocol stack support, it is usable as following server types just by using the configuration CLI, and these few listed below are a small part of the long list of available servers:

  • HTTP
  • DHCPv6
  • TFTP
  • DNS
  • SYSLOG
  • RADIUS
  • TACACS
  • NTP
  • FTP
  • SCP/SSH
  • TELNET

See here for a full list of all full list of the available server configurations.

Quick setup of services or servers for netlab for spontaneous situational needs.

See also

References