Offering Ethernet Services in Provider Networks Bijan Raahemi

Offering Ethernet Services in Provider Networks Bijan Raahemi Research and Innovation Center Alcatel Canada

Outline Introduction Metro Ethernet Network Solutions Infrastructure Ethernet Services Provider Bridged Networks GMPLS-Enabled Metro Networks MPLS-Enabled Metro Networks VPLS, Providing Ethernet Connectivity over Provider Networks VPLS Model Scaling VPLS Ethernet-Services-in-Provider-Networks 14 April 2004 — 2 —

Why Talking about ? Why Are We Talking about Services? Past Focus Infrastructure Today’s Focus Revenue Generating Services Infrastructure: Becoming commodity products (with Equipment vendors trying to minimize costs and maximize reliability) Services: This is the area where companies (whether they're carriers or vendors) can really differentiate themselves – Providing Service over (commodity) infrastructure. It is not a matter of transmitting bits and bytes any more! Why Are We Talking about Ethernet? More than 90% of desktop computers terminates on Ethernet networks It is a standard and low cost technology Proposals in MEF, IETF, and ITU to extend Ethernet in Metro 10G WAN PHY extends Ethernet into Wide Area Networks (Standard completed in 2002) Ethernet-Services-in-Provider-Networks 14 April 2004 — 3 —

Ethernet Services Market Revenue (USD Millions) U.S Ethernet Services Forecast 4,000 EIA EPL 3,000 2,500 2,000 1,500 1,000 500 2002 VPLS 3,500 2003 2004 2005 2006 2007 Source: Yankee Group August 2003 Ethernet Private Line (EPL) Point-to-Point CAGR 58.6% Ethernet Internet Access (EIA) CAGR 102.4% Ethernet Multipoint (VPLS) CAGR 134.2% Ethernet-Services-in-Provider-Networks 14 April 2004 — 4 —

Infrastructure Ethernet Over Dark Fiber Connecting Ethernet switches via dark fibers Easy and cheap Supports Ethernet Virtual Private Line, and Ethernet Virtual Private LAN service Drawbacks: – No OAM capability – Cannot easily accommodate the dedicated switching bandwidth to individual subscribers (for Ethernet private lines) Ethernet Over SONET/SDH Ethernet frames are encapsulated into GFP frames, Mapped into a SONET channel using virtual concatenation, LCAS can be used to keep a connection running at a reduced rate if members of the virtual concatenation group fail, or add more members if the customer requests additional bandwidth. Ethernet-Services-in-Provider-Networks 14 April 2004 — 5 —

Infrastructure (cont’d) Ethernet over ATM The IEEE 802.1Q (priority information) is mapped to the right ATM virtual circuit and service class. Allows a network operator to support Ethernet services with the end-to-end QOS and resiliency associated with ATM SLAs. It also provides flexibility in service topology, from point-to-point, to multipoint, with optional levels of oversubscription. Drawback: Costly (for Greenfield deployment) Resilient Packet Ring (RPR), IEEE 802.17 Supports a significant degree of bandwidth efficiency on rings through the implementation of bandwidth sharing, spatial reuse, and statistical multiplexing. Supports sub 50ms ring-based resiliency on packet switched network architectures. Can run over SONET/SDH or native Ethernet transport networks Ethernet Over IP/MPLS (we will talk more about this later in this presentation) Pseudowire VPLS Ethernet-Services-in-Provider-Networks 14 April 2004 — 6 —

What’s Going on . in Industry? Most carriers have multiple Ethernet services delivery technologies. Most boxes in the network support them. Ethernet services can be supported by many network infrastructures: ATM , IP/MPLS, SONET/SDH, WDM SONET/SDH equipment providers support Ethernet services using a mix of GFP, VCAT, LCAS, and RPR, leveraging the installed base of transmission equipment and its proven OAM and protection. Multi-service ATM switch vendors add Ethernet services blades to their products that can offer Ethernet-over-ATM Transparent LAN Service (TLS) with strict guaranteed QOS. Router vendors support a range of Ethernet VPNs over an IP backbone. A lot is happening, and much of it could have a significant influence on the telecom services. look at the recent issue of the IEEE Communication Magazine :”Ethernet WAN Transport” (March 2004) LightReading Webinar Sponsored by Cisco Systems, Fujitsu, Metrobility Optical Systems, and RiverstoneNetworks, April 15, 2004: “Metro Ethernet Services, What Customers Want? Ethernet-Services-in-Provider-Networks 14 April 2004 — 7 —

What’s Going on . in Standards? Familiar Language Enterprises are used to Frame Relay services: committed information rate (CIR), Excess information rate (EIR), etc. – SLA (Service Level Agreement). – Service Providers want to be able to talk to their customers in a familiar language. Interworking with other Layer 2 services, e.g., Frame Relay or ATM. Different infrastructures are suited to different Ethernet service types: The ITU-T is adapting Ethernet to SDH and MPLS transport networks. – The IETF is emulating Ethernet links and LANs over a packet switched network. – The IEEE is enhancing Ethernet capabilities such that services can be offered over pure Ethernet networks. – The MEF is defining the service attributes and service parameters that enable a consistent set of features associated with various Ethernet services. – Standardizations are working on multiple combinations of protocols. Ethernet-Services-in-Provider-Networks 14 April 2004 — 8 —

Ethernet Service Components The MEF’s basic model of Ethernet services is based on three key components: The customer equipment (CE), either a switch (IEEE 802.1Q bridge) or a router. An Ethernet User Network Interface (UNI), based on a standard IEEE 802.3 Ethernet PHY and MAC, from 10 Mbit/s to 10 Gbit/s. The Metro Ethernet Network (MEN), which may use different transport and service delivery technologies, such as SONET/SDH, WDM, RPR, MAC-in-MAC, Q-in-Q (VLAN stacking), or MPLS. Metro Ethernet Forum (MEF) http://www.metroethernetforum.org The MEN should be scalable and flexible enough to support a wide range of services in cost-effectIve manner (comparable to TDM, Frame Relay, or ATM) Ethernet-Services-in-Provider-Networks 14 April 2004 — 9 —

Ethernet Service Types Ethernet Line (E-Line) Ethernet LAN (E-LAN) point-to-point connectivity E-line services can be used to create Ethernet private line services, Ethernetbased Internet access services, and point-to-point Ethernet VPNs. multipoint-to-multipoint (any-to-any) connectivity. E-LAN Services can be used to create multipoint Ethernet VPNs, and Ethernet Transparent LAN services. UNI Customer Equipment UNI PE Metro Ethernet Network PE Customer UNI Equipment PE EVC3 UNI Customer Equipment PE EVC1 Customer Equipment Point-to-Point EVC Carrier Networks PE EVC2 Ethernet Learning Bridge Metro Ethernet Forum (MEF): http://www.metroethernetforum.org Ethernet-Services-in-Provider-Networks 14 April 2004 — 10 — UNI Customer Equipment

Ethernet Services Examples Ethernet Private Line to ASP for storage application [from Metro Ethernet Forum] EtherRelay service using E-Line Service type Enterprise B, Headquarters UNI-A1 CE UNI-A1 Metro Ethernet Network Metro Ethernet Network UNI-B1 CE Application Service Provider EVC-3 Enterprise A, Branch Site 2 EVC-1 EVC-2 CE UNI-A3 CE EVC-1 Enterprise A, Headquarters UNI-A2 Enterprise A, Headquarters CE UNIASP UNI-A2 CE CE Protected Access Line Enterprise A, Branch Site 1 Enterprise A, Branch Site 1 Intranet/Extranet application using E-Line service type Ethernet-Services-in-Provider-Networks 14 April 2004 LAN Extension application using E-LAN service type — 11 —

Outline Introduction Metro Ethernet Network Solutions Infrastructure Ethernet Services Provider Bridged Networks GMPLS-Enabled Metro Networks MPLS-Enabled Metro Networks VPLS, Providing Ethernet Connectivity over Provider Networks VPLS Model Scaling VPLS Ethernet-Services-in-Provider-Networks 14 April 2004 — 12 —

Solution 1: Provider Bridged Network Frame Format MAC DA MAC SA VLAn ID (12 bits) .1p (3 bits) CFI (1 bit) Eth Type 0x8100 .1Q PVLAN tag VLAN Tag .1Q Eth Type Orig. EthTyp Data Metro Ethernet Network CES CES UNI-A PE-A PE-B CE-A UNI-B CE-B CES PE-C CE : Customer Equipment PE : Provider Edge Bridge/Router UNI: User to Network Interface CES: Core Ethernet Switch/Bridge Spanning Tree UNI-C CE-C Ethernet-Services-in-Provider-Networks 14 April 2004 — 13 — CRC

Ethernet is Evolving .Carrier Class Ethernet Challenges with a Pure Ethernet Solution? Ethernet was originally developed for the LAN applications (shared medium), providing best effort access. It uses flat addressing (scalability). It lacks traffic engineering (load balancing) It lacks rich OAM (Operation, Administration, Maintenance) features. Its spanning tree protocol converges slowly. It lacks Quality of Service (QoS). Solution in Progress Scalability: IEEE 802.1ad work on Q-in-Q Fast Convergence of Spanning Tree: IEEE 802.1w work on Rapid Spanning Tree OAM is being standardized (IEEE Ethernet in the First Mile) to monitor link operation, and improve fault isolation MEF is specifying architecture, protocols and management for metro Ethernet. This will result in specifications for EVC protection, QOS, an Ethernet NNI, and OAM&P for end-to-end management of Ethernet services, regardless of the underlying transport technology. Class of Service: Various solutions are being proposed for this shortcoming. Ethernet classes of service (COS) can be identified via IEEE 802.1Q, user priority bits (802.1p), MPLS EXP bits, or DiffServ Codepoints, IEEE 802.3ad (Link Aggregation): Faster connections between switches managed as a single connection, load balancing among the individual links within a logical connection For the service providers, a final value is the ability to add or subtract bandwidth to a connection in whatever combination of bandwidths (10 Mbit/s, 100 Mbit/s, 1 Gbit/s) is available on that switch. Ethernet-Services-in-Provider-Networks 14 April 2004 — 14 —

Ethernet Bridging Technology is Evolving But is it Enough? 1997 Enterprise Bridging VLANs 802.1D Spanning Tree Per-port Ping RMON statistics and Traceroute 2000 2004 Metro-Oriented Extensions Bridging Technology Futures VLANs VLANs 802.1D Spanning Tree Per-port Ping RMON statistics and Traceroute 802.3ad link aggregation 802.1P/DiffServ 802.1w 802.1s RSTP MIST stacking Ethernet Ring Spanning Tree Bridging is trying to adapt . VLAN Ethernet-Services-in-Provider-Networks 14 April 2004 802.1D Spanning Tree Per-port RMON statistics Ping and Traceroute 802.3ad link aggregation 802.1P/DiffServ 802.1w RSTP 802.1s MIST VLAN stacking Ring Spanning Tree EFM 802.1ad provider bridges — 15 —

Solution 2: GMPLS-Enabled Metro Network Just a potential solution. Not all the details are worked out yet. How it works Core nodes are simple Ethernet switches (Electrical counterpart of Optical switch) No need to run a spanning tree protocol. A new label (such as MAC address) is added to the packet at Ingress. A forwarding table is maintained at each core switch. Packets are forwarded in the MEN based on the outer label. This network solution is capable of providing many features such as traffic engineering, Quality of service, path protection, fault recovery, and OAM functionalities. Challenges A complete standard solution not available yet. Perceived as a costly solution. Ethernet-Services-in-Provider-Networks P-MAC DA P-MAC SA MAC DA MAC SA .1Q Eth Type VLAN Tag Orig. EthTyp CRC Data Frame Format GMPLS Control Plane CES CES UNI-A PE-A PE-B CE-A CE-B CES PE-C CE : Customer Equipment PE : Provider Edge Bridge/Router UNI: User to Network Interface CES: Core Ethernet Switch/Bridge UNI-C CE-C 14 April 2004 UNI-B — 16 —

Solution 3: MPLS-Enabled Metro Network Tunnel Label VC Label MAC DA MAC SA VLAN Tag Eth Type Data CRC Frame Format (Martini Encap) Metro Ethernet Network LSR UNI-A PE-A CE-A LSP1 LSP2 UNI-B CE-B LSP3 PE-C CE : Customer Equipment PE : Provider Edge UNI: User to Network Interface LSP: Label Switched Path LSR: Label Switch Router Ethernet-Services-in-Provider-Networks PE-B UNI-C CE-C 14 April 2004 — 17 —

The Advantages of an MPLS-Enabled Metro Ethernet: Deterministic, Scalable, Resilient Networks and Services Support for tens of 1000s of subscribers Robust, manageable control plane protocols End-to-end OA&M tools Enables inter-metro services Traffic Engineering for efficient use of network resources Fast failover for services Per-service QoS Sophisticated SLAs Sophisticated billing models Leverages MPLS switching No Spanning Tree Protocol No VLANs or Q-tag management Ethernet-Services-in-Provider-Networks 14 April 2004 — 18 —

Outline Introduction Metro Ethernet Network Solutions Infrastructure Ethernet Services Provider Bridged Networks GMPLS-Enabled Metro Networks MPLS-Enabled Metro Networks VPLS, Providing Ethernet Connectivity over Provider Networks VPLS Model Scaling VPLS Ethernet-Services-in-Provider-Networks 14 April 2004 — 19 —

VPLS VPLS is a network service that offers layer 2 multipoint connectivity between edge devices in a single bridge domain. VPLS transforms the packet network into a switched LAN. To the enterprise, the VPLS service makes the service provider network operate as a single VLAN, with a unique SLA, protection attributes, availability attributes and MAC address learning and forwarding for scalable multipoint configurations. All customer sites using VPLS appear to be on the same LAN, regardless of their locations. Customer edge devices appear to each other as connected via single logical learning bridge with fully meshed ports. Ethernet-Services-in-Provider-Networks 14 April 2004 — 20 —

VPLS Model Site A CE CE may be a router, a switch, or a host. PE is a router which includes a "bridge" module. At minimum, the "bridge" module must perform MAC addresses learning, and aging. Attachment Circuit may be a Frame Relay DLCI, an ATM VPI/VCI, an Ethernet port, a VLAN, a PPP connection, an MPLS LSP, etc. A Pseudo Wire is a connection between two PE devices. MTUs are typically located in large buildings, serving different customers. In the IETF PPVPN terminology, an MTU is called a Layer2 PE (L2PE). Site B PE Service Provider Network PE P CE P Attachment Circuit Site C PE Layer 2 Link LSP Tunnel CE Pseudo Wire MTU/L2PE VPLS A CE : PE : P : MTU: Site D Customer Edge Device Provider Edge Device Provider Router Multi Tenant Unit Ethernet-Services-in-Provider-Networks CE 14 April 2004 — 21 —

VPLS Building Blocks Network Elements the transport layer, as well as switches and routers. Tunneling encapsulation of a data packet into payload of another data packet. Signaling (also called auto-configuration); the mechanism by which tunnels are established, and routing information are exchanged. VPLS edge device discovery (also called Auto-discovery function); the process by which one PE router learns which other PE routers are participating in the VPLS. MAC address learning the PE learns the source MAC addresses of the devices in the network and keeps them in a separate forwarding information base (FIB) for every VPLS. Flooding this function is used by the routers to learn unknown MAC addresses. Loop avoidance Ethernet-Services-in-Provider-Networks 14 April 2004 — 22 —

VPLS Options: Lasserre-V.Kompella (LDP) vs. K.Kompella (BGP) The Lassere-V.Kompella draft uses LDP for pseudowire signaling; a point-to-point signaling already used in Draft Martini. LDP is point-to-point signaling protocol. It establishes the connections one by one. The Lassere-V.Kompella draft does not define an auto-discovery method (so there is a need for extension of LDP, or to do it manually, or to develop proprietary solutions) The Lassere-V.Kompella draft is supported by many vendors including Alcatel. The K.Kompella draft uses BGP protocol for both signaling and auto-discovery. BGP is a broadcast signaling protocol. Every time it sets up a new connection, it uses route reflector to broadcast messages to every PE router. Since BGP is already being used for signaling and discovery in Layer 3 VPN, it can be re-used for VPLS as well. However, since BGP is a broadcast protocol, it may not be bandwidth efficient. The K.Kompella draft is mainly supported by Juniper. Ethernet-Services-in-Provider-Networks 14 April 2004 — 23 —

Scaling VPLS If all MTU devices are full-blown VPLS PE devices, then as the number of customer grows, there will be a significant increase in: - The number of tunnels (every PE needs to maintain one tunnel to every peer PE) The number of Pseudo Wires (every PE needs to maintain one PW per VPLS per every peer PE) The total number of MAC addresses per PE The number of signaling connections PE PE PE PE PE Ethernet-Services-in-Provider-Networks 14 April 2004 — 24 —

Hierarchical VPLS (Hub-and-Spoke) Hierarchical VPLS distributes the VPLS functions between PE devices and MTUs. It reduces the replication requirement (data plane) and the number of signaling connections (control plane). However, it does not reduce the number of MAC addresses that need to be maintained. PE still does the Ethernet bridging. MTU Spoke VC Can be MPLS (draft Martini) or VLAN Tag (Q-in-Q) MTU PE PE MTU PE MTU MTU Hub Ethernet-Services-in-Provider-Networks 14 April 2004 — 25 —

H-VPLS Reduces Replication Hierarchical VPLS reduces the replication requirement when the full mesh of pseudo wires grows. Basic VPLS Hierarchal VPLS MTU PE MTU PE PE PE PE MTU PE MTU MTU PE Ethernet-Services-in-Provider-Networks PE 14 April 2004 — 26 —

H-VPLS Facilitates Provisioning & Signaling Hierarchical VPLS facilitates provisioning and signaling when a new site is added to the network. Basic VPLS Hierarchal VPLS New site New site MTU PE MTU PE MTU PE PE PE PE MTU PE MTU MTU PE Ethernet-Services-in-Provider-Networks PE 14 April 2004 — 27 —

De-Coupled VPLS De-coupled VPLS distributes the VPLS functions between PEs and MTUs. De-coupled VPLS reduces the number of MAC addresses to maintain, and the number of signaling connections, but does not limit the number of Pseudo-Wires as the Hierarchal VPLS does. All Ethernet MAC functions (MAC switching, learning, aging, flooding, STP, etc) and Pseudo-Wire termination functions are performed in the MTU, while the auto-discovery and the LSR (MPLS) functions are performed in the PE. Ethernet-Services-in-Provider-Networks 14 April 2004 — 28 —

De-coupled VPLS, how it works? All sites attached to MTU A, B, and C belong to one VPLS (orange). The link between MTU and PE is able to maintain multiple virtual circuits, implemented using MPLS labels or VLAN tags. PE acts as an LSR/LER. It does not implement Ethernet bridging functions. Each MTU needs to establish and maintain one Pseudo-Wire to every other MTU associated with that particular VPLS. De-coupled VPLS MTU B CE CE PE MTU A CE PE MTU C CE PE CE CE Signaling Ethernet-Services-in-Provider-Networks 14 April 2004 Signaling — 29 —

De-coupled VPLS: how it addresses scalability? MTUs (L2PEs) have only one signaling connection. PEs are MPLS routers, and act as forwarding engine. They do not provide Ethernet bridging functions. Number of core signaling connections for PE devices does not depend on the number of L2PE (MTU) devices. It depends only on the number of PEs. However, the number of Pseudo-Wires is highly dependent on the number of L2PEs participating in a particular VPLS. Also, addition of a new L2PE (MTU) into an existing VPLS will trigger the provisioning of a new Pseudo-Wire in every MTU that belongs to that VPLS. When an MTU receives an unknown unicast or a multicast frame from a CE, it needs to broadcast the frame on every Pseudo-Wire to every peer MTU. Ethernet-Services-in-Provider-Networks 14 April 2004 — 30 —

Summary There is a lot going on both in industry and in standardization to define and support Ethernet services in the MAN. Ethernet is evolving into a reliable, scalable, and manageable transport technology (opportunities for innovation and new solutions) Presented different solutions to support Carrier Class Ethernet Services in the MAN. MPLS-Enabled Metro Ethernet is the feasible solution available today. Ethernet-Services-in-Provider-Networks 14 April 2004 — 31 —

Thank You www.alcatel.com Ethernet-Services-in-Provider-Networks 14 April 2004 — 32 —