Scalability | Resiliency | Operations & Management | Synchronization
Challenged with relentlessly competitive marketplace, North American network operators are always on the lookout for technologies that can make them more efficient. Traditional TDM services (T1’s, T3’s, SONET) have provided profitable, high quality services for many years and have set the standards bar for performance. But recent expansions in the demand for bandwidth hungry services are exceeding the capacity of traditional networks. To meet this challenge, network operators have turned to the most ubiquitous, cost-effective network technology available – Ethernet. But with a market accustomed to high-quality TDM services, can Ethernet provide a suitable alternative?
In its original conception, Ethernet was intended to be an Enterprise technology. Simplicity and ease of operation were the initial attraction for network designers. Enterprise operators wanted technologies that were self-configuring, and flexible enough to handle most traffic conditions without the need for complex configurations. Little attention was given to the characteristics that are priorities in the carrier world – robustness, quality of service, manageability.
The evolution of basic Enterprise class services into a platform capable of supporting the massive, complex traffic present in commercial networks has created Carrier Ethernet. Several recent Ethernet enhancements have been critical to meeting commercial network operator needs:
Ethernet uses physical network addresses as a basis for forwarding packets. In a carrier network with a large number of customers, the total number of elements can far exceed even a large Enterprise network. Common Ethernet switches are not designed to handle this address space size. In addition, provisioning conflicts (VLAN’s, etc.) between customer networks can become common. It becomes critical to build an infrastructure that can provide customer segregation of traffic.
Initially, switch designer created various ways to separate customer traffic based on physical port or VLAN Id. Independent VLAN learning (IVL) provided carriers a way to manually provision customer traffic into logically separate address spaces. But this was only a stop gap measure. A more robust, standards based design was needed.
Basic Ethernet standards (802.1Q) support VLAN’s – a rudimentary traffic segregation feature. Expanding on this basis, Carrier Ethernet developers added QinQ – the ability to wrap a customer packet in a carrier header, and give the outside wrapper a different VLAN ID. This multiplied the number of VLAN’s supported in the network from 4095, to over 16 million (4095 customers * 4095 VLAN’s each)
Following the success of the QinQ standard, the Ethernet standards bodies developed 802.1ah, also known as the MAC-in-MAC solution. This provided the ability for the carrier network to isolate the physical addresses in each customer network just as QinQ isolated the logical VLANs. Carriers could now use industry standard switches and wrap incoming packets in their own layer-2 header. This eliminates the need to propagate customer end-point addresses (MAC addresses) throughout their network allowing for much greater scalability and customer traffic isolation.
Carrier Ethernet switches supporting the features mentioned above give today’s network operators the ability to modularize their network traffic, and provide each customer a high quality experience.
One of the major attractions of traditional carrier services has been reliability – carriers touting 5 - 9’s reliability (equating to less than 5 minutes of down time per year) are common. Traditional network infrastructure (circuit based, path-protected) provides predictable, resilient services.
Ethernet infrastructure, by contrast, is dynamic (by design) and yields adaptive, but un-predictable performance. Convergence after a topology change (a broken path or node failure, etc.) could take minutes on large networks using standard spanning tree protocol (STP). Rapid spanning tree improves performance. In some designs, RSTP can restore a network hundreds of milliseconds, but this is still far from the repeatable, sub-50 mSecs resiliency provided in other carrier services.
Recently, efforts are underway to eliminate this reliance on spanning tree (or variants). A combination of the MAC-in-MAC functionality, removal of spanning tree and manual provisioning of forwarding paths, yields a network that utilizes Ethernet as a transport technology, but functions like a circuit-based network. Referred to as PBB-TE or PBT, carriers are exploring this new paradigm across metro networks.
In its original design, Ethernet was intended as a best-effort environment. Limited attention was given to the inclusion of troubleshooting and management tools, simply because the network was easily accessible physically. Carrier networks frequently cover regional geographical areas and require the ability to self-diagnosis and report on problems to minimize truck rolls and decrease mean time to repair (MTTR).
The standards bodies continue to enhance “Carrier Ethernet” with the development of Operation, Accounting and Management (OAM) capabilities within packet network infrastructures. Recently approved standards improve the ability to diagnose physical connectivity issues and report on the network performance (802.3ah, 802.1ag, IETF RFC4329, etc.)
One of the frequently overlooked features provided by traditional TDM/SONET networks is the presence of a clock signal useful for synchronization and precision time keeping. Various applications, network elements, and radio gear rely on the base TDM/SONET carrier signal to remain in “synch” with similar gear in their environment.
Packet networks by contrast, require no synchronization in order to forward packets; the network environment provides no precision timing signal. Higher level application protocols (NTP, etc.) have been developed in order to provide access to a common time source, but these do not provide fine grained or constant timing reference.
Recent efforts within the standards bodies have begun to address this need. Two new standards are under development that would utilize packet network resources, and provide a constant timing signal for use by higher level applications. Precision Time Protocol (PTP, 802.1as, IEEE 1588) relies upon periodic exchanges of time messages to control clock drift, and assure synchronization. Synchronous Ethernet (G.8261), a standard currently under review, aims to use the physical interface (the layer-1 power signature) to provide a synchronous clocking signal.
Although challenges remain to the development of both these standards, Ethernet is quickly evolving to provide carrier service features (synchronization and timing) while maintaining it’s low cost, simple to operate model.
