Segment Routing, in short called SR too, has been implemented in a range of network operating systems. First successful working implementation happened in year 2014. I have been curious about the state of SR in FRR since some time but did not have the right router appliance that would be easy enough to setup and easy to maintain. Now this is possible, the blog article is over here. Building a small FRRouting router in less than 10 minutes.
The FRR implementation lacks at the moment lots of functionality, many things are not working so do not await anything fancy from it. But it is good enough for a networking lab to see how things are supposed to work, and what how things are actually implemented in FRR router daemon.
The SR implementation in FRR is a demo tech presentation using the OSPF process. It shows it has been put work and first effort to get segment routing running in FRRouting daemon.
This is an experimental implementation of segment routing in FRR. It was never meant for use in production networks.
In a previous blog entry a setup routine is explained, how to create a small FRR virtual router appliance for networking labs. This particular FRR appliance is used in explained examples.
Following steps explained below are necessary be applied to the router appliance additionally to get segment routing working. SR works in FRR only for IPv4. SR relies on the MPLS linux kernel modules to be available and loaded in the kernel.
Prerequisites
The setup steps explained below are need to get SR working. SR will use the MPLS data plane to distribute label information. MPLS linux modules are needed for this operation and a correct setting for interfaces to be used, and the maximal amount of available labels.
Use a current linux version in the Router appliance. Linux version used in this lab:
It is recommended to use the most recent FRR routing version, here it is version 7.2:
Hello, this is FRRouting (version 7.2). Copyright 1996-2005 Kunihiro Ishiguro, et al.
Load the necessary linux kernel MPLS modules:
modprobe mpls_router
modprobe mpls_gso
modprobe mpls_iptunnel
Verify kernel modules are loaded using the lsmod | grep mpls command:
mpls_iptunnel 16384 0 mpls_gso 16384 0 mpls_router 28672 1 mpls_iptunnel ip_tunnel 24576 1 mpls_router
Append these 3 kernel modules to the /etc/modules-load.d/modules.conf configuration file. This ensures modules are loaded on each system start.
mpls_router mpls_gso mpls_iptunnel
Activate MPLS for all 5 interfaces in the FRR appliance, and set the maximal available platform MPLS label stack to 1048575. Label with the label ID=1048575 will be the last possible label for this platform.
sysctl -w net.mpls.conf.eth0.input=1
sysctl -w net.mpls.conf.eth1.input=1
sysctl -w net.mpls.conf.eth2.input=1
sysctl -w net.mpls.conf.eth3.input=1
sysctl -w net.mpls.conf.lo.input=1
sysctl -w net.mpls.platform_labels=1048575
For permanent setting across reboots, create following file /etc/sysctl.d/91-mpls.conf
net.mpls.conf.lo.input=1 net.mpls.conf.eth0.input=1 net.mpls.conf.eth1.input=1 net.mpls.conf.eth2.input=1 net.mpls.conf.eth3.input=1 net.mpls.platform_labels=1048575
The FRR router appliance is now prepared to be used as a template for the SR networking lab. It is safe to poweroff the appliance. Create a new router template in GNS3 with 4 ethernet interfaces using this special setup appliance as explained in steps above.
Network topology setup
3 routers. Full mesh network topology. All routers are directly connected.
FRR SR OSPF network topology with IP adressing:
lo lo
192.0.2.1 192.0.2.2
+-------+ +-------+
| | eth0 eth0 | |
| FRR1 |-----------------------------------| FRR2 |
| | .1/30 .2/30 | |
+-------+ +-------+
eth2 | .10/30 .5/30 | eth1
| |
| |
| |
| |
| |
| |
| |
| +-------+ |
| .9/30 | | .6/30 |
+-----------------| FRR3 |-----------------+
eth2 | | eth1
+-------+
lo
192.0.2.3
Setup this topology in the network simulator application.
Router configuration
Current FRR SR implementation uses the OSPF SPF algorithm to calculate the network topology. The configuration consists of 2 parts. * 1st part - Configure OSPF on routers to get IP reachability in the network topology * 2nd part - Enable specific FRR OSPF features to enable and configure SR
OSPF process
- Create OSPF router process and assingn a router-id. The router-id is identical with the IP of the loopback lo interface.
- Create loopback address and announce its network to area 0.
- Configure IP point-to-point links between network routers use (/30) transfer networks, and announce prefixes to area 0
Initial router configuration for FRR1:
conf t
!
router ospf
ospf router-id 192.0.2.1
exit
!
interface lo
ip address 192.0.2.1/32
ip ospf area 0
exit
!
interface eth0
ip address 10.0.0.1/30
ip ospf area 0
ip ospf network point-to-point
no shut
exit
!
interface eth1
ip address 10.0.0.10/30
ip ospf area 0
ip ospf network point-to-point
no shut
end
!
Configuration steps seen above need to be adjusted to match correct settings for router FRR2 and router FRR3. The initial OSPF configuration for all 3 FRR networking lab routers is available below:
After configuration, verify if the loopback IP addresses of FRR2 192.0.2.2 and FRR3 192.0.2.3 are in the routing table. The last 2 entries:
FRR1# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
O 10.0.0.0/30 [110/100] is directly connected, eth0, 00:00:59
C>* 10.0.0.0/30 is directly connected, eth0, 01:06:36
O>* 10.0.0.4/30 [110/200] via 10.0.0.2, eth0, 00:00:42
* via 10.0.0.9, eth2, 00:00:42
O 10.0.0.8/30 [110/100] is directly connected, eth2, 00:00:45
C>* 10.0.0.8/30 is directly connected, eth2, 01:06:36
O 192.0.2.1/32 [110/0] is directly connected, lo, 00:00:42
C>* 192.0.2.1/32 is directly connected, lo, 01:06:36
O>* 192.0.2.2/32 [110/100] via 10.0.0.2, eth0, 00:00:45
O>* 192.0.2.3/32 [110/100] via 10.0.0.9, eth2, 00:00:44
Routers FRR2 and FRR3 loopack IP address are in the routing table. This demonstrates functioning IP reachability in network.
SR Segment Routing
The SR uses the OSPF Opaque LSA Option for routing information flooding within the backbone area.
The FRR OSPF router process needs to have the capability opaque command configured. SR uses 3 different OPAQUE LSA's in OSPF to carry the information about:
- Router Information:: flood the SR capabilities of the node.
- node-msd and
- segment-routing global-block 16000 23999
- supported algorithm, the only available and the default is SPF
- Extended Link: flood the Adjaceny and Lan Adjacency Segment Identifier
- Extended Prefix: flood the Prefix Segment Identifier
Here in example the default SRGB Segment Routing Global Block is used. This is the block 16000 - 23999. A great presentation from Clarence Filsfils about the SRGB can found on the official segment routing website.
To enable flooding of the LSA's to the OSPF area use the router-info area command is used. This configuration example is for the router FRR1, identified by the router-address setting.
conf t
!
router ospf
capability opaque
mpls-te on
mpls-te router-address 192.0.2.1
router-info area
segment-routing on
segment-routing global-block 16000 23999
segment-routing node-msd 8
segment-routing prefix 192.0.2.1/32 index 1021
end
FRR2 and FRR3 configuration should be adjusted to match the network topology setup. All router fully working configurations are available below:
Verification
Following commands are used to show the routing protocol information:
sh ip route
Show the converged IP network:
FRR1# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
O 10.0.0.0/30 [110/100] is directly connected, eth0, 01:37:50
C>* 10.0.0.0/30 is directly connected, eth0, 01:37:51
O>* 10.0.0.4/30 [110/200] via 10.0.0.2, eth0, 01:35:28
* via 10.0.0.9, eth2, 01:35:28
O 10.0.0.8/30 [110/100] is directly connected, eth2, 01:37:50
C>* 10.0.0.8/30 is directly connected, eth2, 01:37:51
O 192.0.2.1/32 [110/0] is directly connected, lo, 01:37:50
C>* 192.0.2.1/32 is directly connected, lo, 01:37:51
O>* 192.0.2.2/32 [110/100] via 10.0.0.2, eth0, 01:35:38
O>* 192.0.2.3/32 [110/100] via 10.0.0.9, eth2, 01:35:28
show mpls table
Display MPLS label distribution, and take a look at the Inbound Label ID:
FRR1# show mpls table
Inbound Outbound
Label Type Nexthop Label
-------- ------- --------------- --------
17002 SR 10.0.0.2 implicit-null
50004 SR 10.0.0.9 implicit-null
50005 SR 10.0.0.9 implicit-null
50006 SR 10.0.0.2 implicit-null
50007 SR 10.0.0.2 implicit-null
show ip ospf
Show the running OSPF process and verify the OpaqCapability flag is set to enabled, notice the output of opaque area LSA 12 in the last line of the output:
FRR1# sh ip ospf
OSPF Routing Process, Router ID: 192.0.2.1
Supports only single TOS (TOS0) routes
This implementation conforms to RFC2328
RFC1583Compatibility flag is disabled
OpaqueCapability flag is enabled
Initial SPF scheduling delay 0 millisec(s)
Minimum hold time between consecutive SPFs 50 millisec(s)
Maximum hold time between consecutive SPFs 5000 millisec(s)
Hold time multiplier is currently 2
SPF algorithm last executed 25m45s ago
Last SPF duration 763 usecs
SPF timer is inactive
LSA minimum interval 5000 msecs
LSA minimum arrival 1000 msecs
Write Multiplier set to 20
Refresh timer 10 secs
Number of external LSA 0. Checksum Sum 0x00000000
Number of opaque AS LSA 0. Checksum Sum 0x00000000
Number of areas attached to this router: 1
Area ID: 0.0.0.0 (Backbone)
Number of interfaces in this area: Total: 3, Active: 3
Number of fully adjacent neighbors in this area: 2
Area has no authentication
SPF algorithm executed 9 times
Number of LSA 15
Number of router LSA 3. Checksum Sum 0x0001ca89
Number of network LSA 0. Checksum Sum 0x00000000
Number of summary LSA 0. Checksum Sum 0x00000000
Number of ASBR summary LSA 0. Checksum Sum 0x00000000
Number of NSSA LSA 0. Checksum Sum 0x00000000
Number of opaque link LSA 0. Checksum Sum 0x00000000
Number of opaque area LSA 12. Checksum Sum 0x00067bc2
show ip ospf database
Check the OSPF database. It shows 3 router link states, having 5 links each. Notify the Area-Local Opaque-LSA section displayed:
FRR1# sh ip ospf database
OSPF Router with ID (192.0.2.1)
Router Link States (Area 0.0.0.0)
Link ID ADV Router Age Seq# CkSum Link count
192.0.2.1 192.0.2.1 679 0x80000009 0x5919 5
192.0.2.2 192.0.2.2 681 0x80000009 0x7305 5
192.0.2.3 192.0.2.3 683 0x80000005 0xfe6b 5
Area-Local Opaque-LSA (Area 0.0.0.0)
Opaque-Type/Id ADV Router Age Seq# CkSum
4.0.0.0 192.0.2.1 674 0x80000001 0xba06
4.0.0.0 192.0.2.2 733 0x80000001 0xb40b
4.0.0.0 192.0.2.3 733 0x80000001 0xae10
7.0.0.1 192.0.2.1 682 0x80000001 0x6490
7.0.0.1 192.0.2.2 733 0x80000001 0x866b
7.0.0.1 192.0.2.3 733 0x80000001 0xa846
8.0.0.1 192.0.2.3 733 0x80000001 0x65a6
8.0.0.2 192.0.2.3 678 0x80000001 0x0bf4
8.0.0.5 192.0.2.1 674 0x80000001 0xe429
8.0.0.5 192.0.2.2 675 0x80000001 0xb05d
8.0.0.6 192.0.2.1 682 0x80000001 0x5ea1
8.0.0.6 192.0.2.2 733 0x80000001 0x679f
show ip ospf database segment-routing
It took me some time to find the command to verify SR operation. Following commands are currently available:
CR Carriage Return adv-router Advertising SR node json JavaScript Object Notation self-originate Self-originated SR node
Display self originated IP prefixes. Notify the SRGB size of 8000 labels and its first label 16000.
FRR1# show ip ospf database segment-routing self-originate
OSPF Segment Routing database for ID 192.0.2.1
SR-Node: 192.0.2.1 SRGB (Size/Label): 8000/16000 Algorithm(s): SPF MSD: 8
Prefix or Link Label In Label Out Node or Adj. SID Interface Nexthop
------------------ -------- --------- --------------------- --------- ---------------
192.0.2.1/32 0 0 SR Pfx (idx 1) lo 192.0.2.1
10.0.0.10/32 50005 pop SR Adj. (lbl 50005) eth2 10.0.0.9
10.0.0.10/32 50004 pop SR Adj. (lbl 50004) eth2 10.0.0.9
10.0.0.1/32 50007 pop SR Adj. (lbl 50007) eth0 10.0.0.2
10.0.0.1/32 50006 pop SR Adj. (lbl 50006) eth0 10.0.0.2
Show all prefixes in the OSPF SR DB of FRR1. Verify the MSD is set to 8, notice the algorithm(s) preset SPF:
FRR1# sh ip ospf database segment-routing
OSPF Segment Routing database for ID 192.0.2.1
SR-Node: 192.0.2.1 SRGB (Size/Label): 8000/16000 Algorithm(s): SPF MSD: 8
Prefix or Link Label In Label Out Node or Adj. SID Interface Nexthop
------------------ -------- --------- --------------------- --------- ---------------
192.0.2.1/32 0 0 SR Pfx (idx 1) lo 192.0.2.1
10.0.0.10/32 50005 pop SR Adj. (lbl 50005) eth2 10.0.0.9
10.0.0.10/32 50004 pop SR Adj. (lbl 50004) eth2 10.0.0.9
10.0.0.1/32 50007 pop SR Adj. (lbl 50007) eth0 10.0.0.2
10.0.0.1/32 50006 pop SR Adj. (lbl 50006) eth0 10.0.0.2
SR-Node: 192.0.2.2 SRGB (Size/Label): 8000/16000 Algorithm(s): SPF MSD: 8
Prefix or Link Label In Label Out Node or Adj. SID Interface Nexthop
------------------ -------- --------- --------------------- --------- ---------------
192.0.2.2/32 17022 pop SR Pfx (idx 2) eth0 10.0.0.2
SR-Node: 192.0.2.3 SRGB (Size/Label): 8000/16000 Algorithm(s): SPF MSD: 8
Prefix or Link Label In Label Out Node or Adj. SID Interface Nexthop
------------------ -------- --------- --------------------- --------- ---------------
192.0.2.3/32 17023 pop SR Pfx (idx 3) eth2 10.0.0.9
The MPLS label distribution is done using SR segment routing. LDP is not used. While using IPv4 with SR then SR is the control plane and MPLS is the data plane.
For the IPv6 SR the data plane is IPv6. Segment routing support in the linux kernel is already working verification using the dmesg command:
At this moment FRR lacks the control plane like SR working with IPv6. The FRR ospf6 routing daemon has not the needed functionality in the code:
FRR1(config)# router ospf6
FRR1(config-ospf6)# ?
area OSPF6 area parameters
auto-cost Calculate OSPF interface cost according to bandwidth
distance Administrative distance
end End current mode and change to enable mode
exit Exit current mode and down to previous mode
find Find CLI command matching a regular expression
interface Enable routing on an IPv6 interface
list Print command list
log-adjacency-changes Log changes in adjacency state
no Negate a command or set its defaults
ospf6 Open Shortest Path First (OSPF) for IPv6
output Direct vtysh output to file
quit Exit current mode and down to previous mode
redistribute Redistribute
stub-router Make router a stub router
timers Adjust routing timers
Just out of curiosity isisd routing protocol lacks SR too:
FRR1(config)# router isis 1
FRR1(config-router)# ?
area-password Configure the authentication password for an area
default-information Control distribution of default information
domain-password Set the authentication password for a routing domain
end End current mode and change to enable mode
exit Exit current mode and down to previous mode
find Find CLI command matching a regular expression
hostname Dynamic hostname for IS-IS
is-type IS Level for this routing process (OSI only)
list Print command list
log-adjacency-changes Log changes in adjacency state
lsp-gen-interval Minimum interval between regenerating same LSP
lsp-mtu Configure the maximum size of generated LSPs
lsp-refresh-interval LSP refresh interval
max-lsp-lifetime Maximum LSP lifetime
metric-style Use old-style (ISO 10589) or new-style packet formats
mpls-te MPLS-TE specific commands
net A Network Entity Title for this process (OSI only)
no Negate a command or set its defaults
output Direct vtysh output to file
purge-originator Use the RFC 6232 purge-originator
quit Exit current mode and down to previous mode
redistribute Redistribute information from another routing protocol
set-attached-bit Set attached bit to identify as L1/L2 router for inter-area traffic
set-overload-bit Set overload bit to avoid any transit traffic
spf-delay-ietf IETF SPF delay algorithm
spf-interval Minimum interval between SPF calculations
topology Configure IS-IS topologie
The SR for isisd might prove easier to implement:
FRR3(config-router)# topology ?
ipv4-mgmt IPv4 management topology
ipv4-multicast IPv4 multicast topology
ipv4-unicast IPv4 unicast topology
ipv6-dstsrc IPv6 dst-src topology
ipv6-mgmt IPv6 management topology
ipv6-multicast IPv6 multicast topology
ipv6-unicast IPv6 unicast topology
OSPF is written on top of IP. This is the reason why FRR and many other popular network operating system implementations, have a separate OSPF router processes for each IP address family. FRR has ospf for IP and ospf6 for IPv6.
While implementing a solution for isisd. Solving the problem once and for both address families, IP and IPv6. The control plane the IS-IS routing protocol does not depend on the data plane, here IP and IPv6.
It would be a great project to code the SRv6 support for FRR. An opportunity to learn more about segment routing, IPv6, and its ospf, ospf6 and isis routing daemons. Learn about linux routing to gain programming experience while implementing the solution.
Someone has to make a first step to get things going.
References
- FRR - official OSPF SR documentation
- Segment Routing
- Segment Routing Global Block - SRGB
- RFC5250 - The OSPF Opaque LSA Option
- RFC8402 - Segment Routing Architecture
- RFC8287 - Label Switched Path (LSP) Ping/Traceroute for Segment Routing (SR) IGP-Prefix and IGP-Adjacency Segment Identifiers (SIDs) with MPLS Data Planes
- IS-IS - Routing Protocol - Wikipedia
- IS-IS and OSPF Difference Discussions