*mydirectindustry

Catalogs
Nokia
Collapsed Spine EVPN VXLAN

Collapsed Spine EVPN VXLAN

Product catalog summary

Executive Summary
Nokia Validated Designs (NVDs) offer validated recommendations for deploying Nokia's portfolio across various market segments. This document outlines a collapsed spine EVPN VXLAN design for a single-site, single-tiered data center architecture, using EVPN as the control plane and VXLAN as the data plane.

Reference Architecture Overview
Design Considerations and Components
The collapsed spine topology reduces the number of devices and their capacity requirements in data centers. The document details the components and technology choices involved in this design.

Network Deployment
High-Level Design
The document provides a high-level design overview, platform positioning, and traffic patterns for deploying the network.

Feature Configuration
This section covers the configuration of various features such as underlay with IPv6 link-local addressing, BGP for underlay and overlay routes, MTU, BFD, LLDP, and different types of interfaces including Layer 2 and Layer 3 server-facing interfaces.

Test Summary
The document includes a feature validation matrix and traffic convergence results to ensure the design's reliability and performance.

EDA Integration
This section describes the EDA architecture and workflows for deploying the fabric, including node onboarding, ASN pool creation, and fabric creation.

Validation
Network and EDA validation processes are detailed to ensure the design meets the required standards.

Automation and Orchestration
The document discusses the use of digital twins with Containerlab for automation and orchestration purposes.

Overview: The document discusses a modern data center design using a collapsed spine architecture, which integrates legacy Layer 2 switches into a scalable network infrastructure. This design is cost-effective and focuses on power utilization, allowing for future expansion into a 3-stage Clos architecture.

Design Considerations: The collapsed spine architecture merges the spines and leafs of a 3-stage Clos fabric into a single layer. It uses point-to-point Layer 3 interfaces with IPv6 link-local addresses for connectivity, and a single BGP session is established using these addresses. The spines act as VXLAN tunnel endpoints (VTEPs), and ToR switches are multihomed to the spines.

Operational Workflow: The deployment process involves gathering customer requirements, converting them into an intent, and orchestrating this intent using Nokia's Event Driven Automation (EDA). EDA facilitates zero-touch provisioning and provides telemetry options for continuous integration and deployment.

Network Deployment: The network uses a collapsed spine fabric with BGP EVPN as the control plane and VXLAN for data encapsulation. Integrated routing and bridging (IRB) interfaces are configured on the spines, serving as default gateways for servers. The design supports both asymmetric and symmetric routing models.

Platform Positioning: Nokia platforms like the 7220-IXR-D3L and 7220-IXR-D2L are positioned for different roles within the fabric. The document provides a visual depiction and a table listing the platforms and their roles.

Traffic Patterns: The document describes various traffic patterns validated in the collapsed spine EVPN VXLAN NVD, including Layer 2 and Layer 3 interfaces, and Ethernet segment LAGs. It explains packet flows for different scenarios, such as local bias forwarding and VXLAN encapsulation.

Feature Configuration: The configuration includes using IPv6 link-local addressing for point-to-point interfaces between spines, mapping interfaces to the default network-instance, and configuring BGP for underlay and overlay routes. The document provides configuration snippets for these setups.

Overview: This document provides technical configurations and guidelines for implementing a Collapsed Spine EVPN VXLAN network using Nokia equipment. It includes detailed configurations for BGP, MTU, BFD, LLDP, Layer 2 interfaces, and Ethernet segments.

1. BGP Configuration: The document outlines BGP settings for multipath routing, allowing multiple AS paths with a maximum of 64 paths for eBGP and iBGP. EVPN settings include inter-AS VPN and rapid updates. IPv4 and IPv6 unicast configurations are enabled with specific path limits and next-hop advertisements.

2. MTU Configuration: System-wide MTUs are set to accommodate larger packets due to VXLAN encapsulation. Default MTU values are specified for ports, Layer 2, and IP.

3. BFD Configuration: Bidirectional Forwarding Detection is enabled for fast failover in BGP configurations, with specific intervals and detection multipliers set for subinterfaces.

4. LLDP Configuration: Link Layer Discovery Protocol is enabled on multiple interfaces for neighbor discovery.

5. Layer 2 Interfaces: Configurations for untagged and tagged Layer 2 server-facing interfaces are provided, including subinterface settings for VLAN tagging and IP addressing.

6. All-active ES-based LAG: The document describes configurations for all-active Ethernet segments, supporting multihoming with multiple VTEPs. It includes LAG interface settings and Ethernet segment parameters.

7. Single-active ES-based LAG: Single-active Ethernet segments are configured for scenarios where only one link should be active. The document details LAG interface settings, Ethernet segment configurations, and DF election parameters.

Conclusion: The document provides comprehensive configurations for setting up a robust and efficient EVPN VXLAN network, focusing on redundancy, failover, and efficient traffic management.

Ethernet Segment Configuration: The document outlines the configuration of Ethernet segments on standby VTEP, detailing the admin state, ESI, multi-homing mode, and interface configurations. It highlights the preference algorithm with a preference value of 500 and non-revertive capabilities.

LAG Interface State: The LAG interface state on active and standby VTEP is described, including LACP state, interval, mode, system ID, and operational state. The active VTEP shows an operational state of 'up', while the standby VTEP is 'down'.

LAG on ToR Switches: Configuration of LAG interfaces on ToR switches is provided, including interface descriptions, admin state, VLAN tagging, and LACP settings. The document specifies the use of LACP with a fast interval and active mode.

Layer 3 Server-Facing Interfaces: The document discusses the deployment of Layer 3 server-facing interfaces for cloud-native environments, enabling end-to-end routing. It includes configuration examples with IPv4/IPv6 addresses and VLAN tagging.

IRB Interfaces: Configuration of IRB subinterfaces on spine1 is detailed, including IP MTU, IPv4/IPv6 addresses, ARP settings, and anycast gateway configurations.

VXLAN Tunnels: The creation of VXLAN tunnels as tunnel interfaces on SR Linux is explained, with examples of bridged and routed tunnel configurations. These are associated with network instances for MAC and IP VRFs.

MAC VRFs: The document describes the configuration of MAC VRFs for Layer 2 isolation, including options for overlay ECMP, ARP/ND advertisement, and duplicate MAC detection.

IP VRFs: Configuration of IP VRFs for Layer 3 isolation is provided, with details on associated IRB subinterfaces, VXLAN interfaces, and BGP EVPN settings.

Feature Validation Matrix: A matrix summarizing feature validation for SRL v25.3.2 and EDA v25.4.1 is included, listing features like IPv6 addressing, BGP peering, and telemetry.

Traffic Convergence Results: The document presents traffic convergence metrics for various scenarios, such as link shutdown, BGP reset, and ToR uplink failure, with convergence times measured in seconds.

EDA Integration: An overview of Nokia's Event Driven Automation (EDA) platform architecture is provided, highlighting its deployment on Kubernetes and its cloud-native capabilities.

Overview: The document provides a detailed description of the EDA/K8s tech stack, focusing on the deployment and management of Kubernetes pods within a network infrastructure. It includes a list of active pods, their namespaces, and functionalities, as well as procedures for onboarding nodes using Zero Touch Provisioning (ZTP).

Key Sections:

1. EDA/K8s Tech Stack: The document lists various Kubernetes pods across different namespaces such as cert-manager, eda-system, kube-system, metallb-system, and rook-ceph. Each pod is described with its status, readiness, and age.

2. Pod Functionalities: Key pods and their roles are outlined, including:

eda-asvr: Artifact server for storing common artifacts.
eda-bsvr: Bootstrap server for node onboarding and lifecycle management.
eda-ce: Configuration engine for managing application dependencies and intents.
eda-npp: Responsible for schema validation and device communication.
eda-api: REST API server for user access and GUI consumption.
eda-toolbox: Provides tools for EDA transactions and topology generation.

3. EDA Workflow: The document describes a high-level workflow for deploying a prescriptive collapsed spine NVD using EDA UI or Kubernetes manifest files. It includes steps for onboarding nodes and deploying the fabric, with references to figures demonstrating the process.

4. EDA Onboarding with ZTP: Details the process of onboarding fabric nodes via ZTP, automating the deployment of Nokia SR Linux nodes. The workflow includes connecting nodes to the OOB infrastructure and pushing configurations based on user intent. Console logs from a Nokia 7220 IXR-D3L are provided as an example.

Figures and Examples: The document includes figures illustrating the EDA workflow and ZTP process, as well as examples of console logs during node onboarding.

Overview: The document provides a detailed technical guide on deploying an EDA-orchestrated collapsed spine EVPN VXLAN fabric using SR Linux version 25.3.2. It includes specifications, procedures, and manifest files necessary for the deployment and management of the network infrastructure.

1. Event Logs and ZTP Process: The document begins with event logs detailing a chassis reboot and the Zero Touch Provisioning (ZTP) process. The ZTP process involves obtaining a DHCP lease, updating the hostname, fetching, and executing a provisioning script, and successfully starting the SR Linux process.

2. EDA Kubernetes Workflow: This section describes the use of manifest files for deploying the network fabric. It emphasizes the importance of referencing the correct EDA namespace, specifically the '2-way-collapsed-spine' namespace.

3. EDA Artifacts for SR Linux: Artifacts are created for the target SR Linux version, including manifest files for the .bin image, md5 hash file, and YAML zip file. These artifacts are crucial for the deployment process.

4. Subnet Allocation: A manifest file is created to instantiate an IPv4/IPv6 subnet pool for managing SR Linux fabric nodes. This pool is used during ZTP to assign IP addresses to management interfaces.

5. EDA Node Group and User Creation: Node groups determine service access levels, while node users manage access to fabric nodes. The document provides manifest examples for creating node groups and users.

6. EDA Node Profile: The node profile facilitates node onboarding, including authentication details, DHCP scope, and image version checks. It ensures nodes are onboarded with the correct configurations.

7. Custom Resource Modifications: The init-base custom resource is modified to ensure configurations are saved to startup whenever EDA commits changes via gNMI.

8. TopoNode Custom Resource: Nodes are onboarded using the TopoNode custom resource, which includes metadata labels, node profile names, platforms, and serial numbers.

9. ASN and IP Pool Allocation: ASN pools are created for fabric deployment, and IP allocation pools are established for system0 addresses, which are used as VTEP source addresses.

10. Interface and Link Creation: Interfaces and links between nodes are instantiated using manifest files. These configurations are essential for establishing connectivity within the network fabric.

Fabric Creation (Underlay and Overlay)
The document outlines the orchestration of a collapsed spine EVPN VXLAN fabric using EDA with IPv6 link-local addressing and MP-BGP peering. Key inputs include IP and ASN pools, label selectors, and interswitch links. BFD is enabled for fast failover on point-to-point interfaces.

Bridge Domain Creation
Bridge domains are instantiated as MAC VRFs on SR Linux nodes. Two types are created: EVPN VXLAN BDs for collapsed spine nodes and simple BDs for ToR switches. EVPN VXLAN BDs map to VXLAN VNI and EVPN instance, enabling EVPN learning.

IRB Interfaces
IRB interfaces facilitate routing between Layer 2 VNIs in EVPN VXLAN deployments. The document provides a manifest for creating IRB interfaces, highlighting the importance of enabling Layer 3 proxy-ARP for stretched VLANs.

IP VRF Creation
IP VRFs are created with a 1:1 mapping to Layer 3 VNIs and EVIs. VRF creation is based on bridge domains referencing the IP VRF.

VLAN Creation
The document demonstrates VLAN creation for both untagged and tagged Layer 2 deployments, with label selectors determining interface deployment.

Routed Interfaces
Routed interfaces are created using a manifest, specifying ARP timeout, interface, IP MTU, and IP addresses.

Namespace in EDA
EDA allows logical separation of resources into namespaces, which must be referenced in manifest files.

EDA Configlets
Configlets allow user-defined configurations, such as ARP/ND advertisement, system-wide MTU settings, and system logging for SR Linux subsystems.

Custom Topology View
The topology manifest can be updated to include labels for ToR switches, allowing accurate visualization of the collapsed spine design.

EDA Workflows via UI
Node profiles, ASN pools, and IP pool creation are managed through the EDA UI, facilitating node onboarding and resource allocation.

IP Pool Creation: IP pools can be created by navigating to Main → IP Addresses in the EDA UI. Figures illustrate the creation and listing of IP pools.

Node Onboarding: Nodes are onboarded into the fabric and can be managed via Main → Nodes. The UI provides a topology view of these nodes.

Fabric Creation: Fabrics are instantiated by navigating to Main → Fabrics, which configures fabric nodes based on label selectors.

Bridge Domains: These are instantiated as MAC VRFs on fabric nodes and can be created via Main → Virtual Networks → Bridge Domains.

IRB Interfaces: Act as default gateways using an anycast, distributed gateway model. They are created via Main → Virtual Networks → IRB Interfaces.

IP VRFs: Used for multitenancy and Layer 3 isolation, created via Main → Virtual Networks → Routers.

VLANs: Created by navigating to Main → Virtual Networks → VLANs.

Configlets: Allow for custom configurations and are created via Main → Configuration → Configlets.

Network Validation: Includes underlay and overlay validation using IPv6 link-local addressing and BGP dynamic discovery. BFD is used for fast-failover, and its session state can be confirmed through specific commands.

Link Aggregation: Interface and LAG states can be viewed using specific commands, with details on LACP-enabled LAGs provided.

Ethernet Segments: DF and non-DF status can be determined per-VRF, with operational states and peer information detailed.

MAC VRFs and MAC Address Learning: The bridge table per MAC VRF can be viewed using specific commands, detailing MAC addresses, destinations, and types.

Overview: This document provides technical details on the Collapsed Spine EVPN VXLAN architecture, focusing on route validation, EDA validation, and network interface status. It includes command examples and validation steps for network administrators.

Route Validation: The document outlines the process for validating routes within the default network-instance and IP VRFs. It provides a command example for displaying routes in the default network-instance, highlighting key attributes such as route type, owner, and next-hop interfaces.

EDA Validation: EDA (Edge Data Analytics) validation is performed using Kubernetes CLI commands. The document emphasizes the importance of specifying namespaces when accessing resources. It provides examples of node status validation, including DHCP, gNMI port, and node readiness.

Onboarding and Synchronization: After initial deployment, EDA deploys an NPP pod per node to continue the onboarding process. The document specifies expected states such as ONBOARDED being "true" and NODE being "Synced" with the correct SR Linux version.

Node and Interface Status: Detailed descriptions of target nodes and topological nodes are provided, including metadata, specifications, and status. The document includes examples of interface status across the fabric, indicating operational states and speeds.

VLAN Status: The document concludes with a summary of VLAN statuses, showing operational states and last change timestamps for various VLANs within the network.

Overview: This document provides a comprehensive overview of the Collapsed Spine EVPN VXLAN architecture, focusing on network validation, automation, and orchestration using digital twins and Containerlab.

1. Specifications and Status:

Various VLANs and bridge domains are listed with their operational states, such as 'up' or 'degraded'.
IRB interfaces are detailed with MTU settings and operational states, all marked as 'up'.

2. IRB Interface Details:

IRB interfaces are configured with specific ARP timeout settings, IP addresses, and proxy settings.
Operational states and last change timestamps are provided for each interface.

3. VRF Description:

Details of VRF configurations, including VNI, EVI, and operational states, are provided.

4. Transaction Results:

EDA transactions are listed with their results, indicating successful operations over a period of 8 days.

5. Automation and Orchestration:

Digital twins are used for continuous validation and learning, with deployment scripts available for setting up a collapsed spine EVPN VXLAN fabric.
Deployment and destruction scripts are provided for managing the digital twin environment.

6. Reference Designs:

Differences between Nokia Validated Designs (NVDs) and reference designs are explained.
Reference designs are available for exploration and testing, with repositories accessible on GitHub.

Conclusion: The document emphasizes the importance of validated designs and digital twins in ensuring efficient and redundant network operations, providing tools and scripts for deployment and management.

Catalog excerpts

Open the catalog to page 1

Collapsed Spine EVPN VXLAN Nokia is committed to diversity and inclusion. We are continuously reviewing our customer documentation and consulting with standards bodies to ensure that terminology is inclusive and aligned with the industry. Our future customer documentation will be updated accordingly. This document includes Nokia proprietary and confidential information, which may not be distributed or disclosed to any third parties without the prior written consent of Nokia. This document is intended for use by Nokia’s customers (“You”/”Your”) in connection with a product purchased or licensed...

Open the catalog to page 2

Collapsed Spine EVPN VXLAN 2 Reference architecture overview 6 4.1 Underlay with IPv6 link-local addressing for point-to-point interfaces 6.3.7 Onboarding nodes in EDA with using a TopoNode Custom Resource .. 48 Use subject to Terms available at: www.nokia.com/terms

Open the catalog to page 3

Use subject to Terms available at: www.nokia.com/terms

Open the catalog to page 4

Collapsed Spine EVPN VXLAN Figure 7 Packet flow for Layer 2 tagged and untagged traffic using local bias Figure 8 Packet flow for Layer 2 tagged and untagged traffic exiting via a remote VTEP when local member interface of Ethernet segment is Figure 9 Packet flow between tagged or untagged interfaces across spines 14 Figure 13 Traffic flow for source behind ToR communicating with a destination behind single-active Ethernet segment when packet is hashed to Figure 14 Traffic flow for source behind ToR communicating with a destination behind single-active Ethernet segment when packet is hashed to...

Open the catalog to page 5

Collapsed Spine EVPN VXLAN Executive summary Nokia Validated Designs (NVDs) is a workstream dedicated to producing validated recommendations to the consumer about Nokia’s portfolio across market segments. NVDs are accomplished with extensive requirement analysis from a multitude of customers along with deep research of the technology development in the industry segment to form the solution’s design. After the design has been compiled, it goes through an intense array of hardware, software, traffic, and failure tests to form the validated design. The resultant design and collateral provide the...

Open the catalog to page 6

Collapsed Spine EVPN VXLAN This section describes the various components involved in this validated design and the design and technology choices that were made. In general, a collapsed spine design provides the following advantages: • reduction of the required number of devices and capacity of the devices in data centers where the scale-out and size of the data center are known to be limited and the main considerations become cost and power utilization • re-use of legacy Layer 2 switches (even Layer 2 switches of other vendors) as top of rack (ToR) switches in more modern data center designs...

Open the catalog to page 7

Collapsed Spine EVPN VXLAN Layer 2 untagged interfaces two-way ES-based Link Aggregation Group (LAG) in all-active multihoming mode two-way ES-based LAG in single-active multihoming mode Layer 2 tagged interfaces Layer 3 point-to-point interfaces (exported as EVPN Type-5 routes into the fabric) High-level operational workflow Figure 2 shows a high-level operational workflow for the NVD-based fabric deployment and lifecycle management. The intent-based approach, combined with the prescriptive nature of the validated design and the flexibility of Nokia’s Event Driven Automation (EDA), makes the...

Open the catalog to page 8

Collapsed Spine EVPN VXLAN • Because EDA as a platform does not need to be reinstalled for new patches or apps, it provides a high degree of flexibility and customizability for modern DC fabric needs. Figure 3 shows a high-level design of the fabric. The topology is a collapsed spine fabric with BGP EVPN as the control plane and VXLAN as the data plane encapsulation method with point-to-point layer 3 links between the two spines. These point-to-point interfaces are configured with IPv6-link local addressing (as shown in Figure 3), with each spine advertising its IPv4 loopback address with an...

Open the catalog to page 9

Collapsed Spine EVPN VXLAN Figure 4 IPv4 systemO addresses advertised with an IPv6 next-hop Integrated routing and bridging (IRB) interfaces are configured on the spines using a distributed anycast gateway model. These IRB interfaces act as the default gateway for all servers in the fabric. Servers can connect either directly to the spines or to the ToR switches, depending on customer requirement. For routing between VNIs, this design uses an asymmetric routing model (as described in RFC 9135), along with symmetric routing using EVPN Type-5 routes for certain subnets (servers connected via Layer...

Open the catalog to page 10

Table 1 Platform positioning Note: Alternate platforms can be positioned in the roles shown above based on cost, hardware, and performance requirements. July 2025 Issue 1 Use subject to Terms available at: www.nokia.com/terms

Open the catalog to page 11

Collapsed Spine EVPN VXLAN Nokia data center portfolio Traffic patterns In this section, we describe common traffic patterns that are validated in the collapsed spine EVPN VXLAN NVD. These traffic patterns include forwarding across Layer 2 tagged and untagged interfaces (singlehomed and multihomed), Layer 3 interfaces, two-way all-active Ethernet segment LAG, and two-way single-active Ethernet segment LAG. Figure 7 and Figure 8 show traffic ingress on a single-homed interface and egress out of an Ethernet segment member interface (either local or remote). In Figure 7, when there is an interface...

Open the catalog to page 12

VLAN 10-tagged VLAN 30 - tagged VLAN 100-L3 VLAN 10 - untagged VLAN 20 - tagged VLAN 100-L3 _ VLAN 10 - untagged VLAN 40 - tagged Figure 7 Packet flow for Layer 2 tagged and untagged traffic using local bias forwarding spine2 VLAN 10-tagged VLAN 30 - tagged VLAN 100-L3 £> spinel VLAN 10 - untagged VLAN 20 - tagged VLAN 100-L3 _ VLAN 10 - untagged VLAN 40 - tagged Figure 8 Packet flow for Layer 2 tagged and untagged traffic exiting via a remote VTEP when local member interface of Ethernet segment is down on ingress VTEP July 2025 Issue 1 Use subject to Terms available at: www.nokia.com/terms

Open the catalog to page 13

All Nokia catalogs and technical brochures

way collapsed spine EVPN/VXLAN
35 Pages
Nokia Validated Design
72 Pages