7/17/2002 -- As the excess capacity in long distance fiber networks pulls more and more carriers into the swamp, development continues on improved means for high-speed local access to data networks. In theory at least, once more end users require higher speed connections the overcapacity in the backbones will be absorbed and growth can continue in that area as well. A couple of new network topologies now under development bear watching in the access and metropolitan area network space: Resilient Packet Ring (RPR) and Passive Optical Networks (PONs.)

Resilient Packet Ring Rationale
As discussed in an earlier article (see "Understanding WANs: A Technical Primer"), the circuit-switched metropolitan and wide area network infrastructure designed for voice traffic is not well suited to carrying packet-based data. The time domain multiplexed SONET protocol allocates fixed time slots, which is inefficient with bursty data; its speeds do not match well with Ethernet; and it has significant overhead associated with encapsulating the IP (Internet protocol) packets. These limitations apply regardless of access technology, whether leased T1, Frame Relay, or ATM over OC-3, since they all get aggregated onto SONET rings at the central office.

For this reason, some carriers have installed native optical Ethernet overlay networks. These, of course, are well matched to the packet-based LANs and make efficient use of the available bandwidth. However, as a transport technology Ethernet has some drawbacks. First, it is not well suited to the ring topology in which most metropolitan fiber is laid. In order to provide an alternate path around the ring in case of a fiber cut or equipment failure Spanning Tree is typically used, giving a restoration time measured in tens of seconds rather than the 50 milliseconds provided by SONET. And second, a ring is effectively a shared media (as with non-switched, repeater-based Ethernet) that needs appropriate media access control mechanisms to fairly allocate bandwidth across the entire network; Ethernet itself cannot provide this.

In order to fix these problems, a new metropolitan network architecture has been proposed for the media access control (MAC) sublayer of the data link layer in the protocol stack. Resilient Packet Ring is being developed under the auspices of IEEE 802.17 to combine the best elements from SONET and switched Ethernet in order to provide an efficient, fair, resilient, and deterministic ring-based network.

A-Ring-A-Ding-Ding
Resilient Packet Ring is fundamentally based on a bidirectional ring topology, with two fibers carrying data around the ring in opposite directions. As with SONET, RPR connects each node to the ring using an Add-Drop Multiplexer (ADM). This MAC-level device performs three functions: Add, inserting access traffic from the connected customer to the ring; Drop, stripping traffic directed from the ring to the customer; or Pass, forwarding transit traffic along the ring. These three basic functions do not require any node-to-node communication, and should be easily and inexpensively implemented.

Resiliency
A bidirectional ring topology has built-in redundancy, as there are two possible paths between any two nodes on the ring. RPR supports two mechanisms for recovering from a (single) fiber cut in under 50 milliseconds. "Wrapping" involves having the two nodes on either side of a fiber cut recognize that one link is down and redirecting traffic from the inbound to outbound fiber on its other side to isolate the down segment. "Steering" has the original sending node recognize that the initial direction in which it sent the packet no longer reaches the destination node, and it then resends the packet in the other direction. Note that neither of these approaches can compensate for two fiber cuts between different nodes on the ring as a mesh topology may be able to.

Broadcast/Multicast Traffic
Ethernet is good at providing broadcast or multicast traffic among nodes (sometimes too good, resulting in inadvertent broadcast storms bringing down networks.) Being logically structured as point-to-point, SONET, does not handle this well as each packet must be replicated for every destination node. With RPR, a single packet may be both received by each ADM and then forwarded on to the next, making broadcast or multicast traffic as efficient as Ethernet.

Bandwidth Allocation
As a shared medium, an Ethernet ring of switches is subject to bandwidth hogging by downstream nodes. This is because that while the switches may be physically connected on a ring topology where all are equal, Spanning Tree effectively disconnects one link so that the logical topology is linear. Each switch may limit the bandwidth on its immediate links, but there is no mechanism to look at the bandwidth utilization of the network as a whole. This means that the nodes further downstream from, say, the node connected to an Internet backbone, have their bandwidth limited more and more the farther away they are from the access node since each upstream node carries the traffic from all downstream nodes as well as its own. SONET, on the other hand, allocates point-to-point circuits that have a fixed amount of bandwidth, but has no provision for sharing unused capacity on one circuit with another. In a packet based network, dynamically changing traffic requires real-time adjustment of bandwidth on a global basis in order to optimize the total network. RPR provides a feedback mechanism whereby each node monitors what is going on in its vicinity and broadcasts that data to all nodes so that each can adjust itself to the current state of the network utilization. Individual flows of data between nodes can be engineered to have minimum and maximum available rates, making it easy to provision quality of service to each flow.

RPR Applications
These characteristics of a Resilient Packet Ring are tailored to allow a metropolitan service provider to sell data transport services to multiple buildings along the path of a dark fiber ring. Each ADM along the ring can provide one or more Ethernet or DS1/DS3 drops with defined bandwidth characteristics for last mile uplink access, while the ring itself has sub-50 millisecond protection. Since global fairness policies can be enforced across the entire network, RPR thus combines the packet-switching bandwidth efficiency of Ethernet with the guaranteed service levels of SONET connections.

Passive Optical Networks
As part of the push to bring cost-effective broadband fiber-based access to a wider range of subscribers than can afford DS-3 and OC-3 connections, passive optical networks (PONs) began to be considered in the early ‘90s. The basis of PONs is that all active electronics are housed either at the central office or at the subscriber, with only passive devices in the field. This has the potential to be a more reliable, lower-maintenance architecture than placing active devices such as switches and repeaters in environmentally unfriendly conditions.

The Physical Architecture
Naturally, the folks coming up with PON designs had to invent some new acronyms to describe the elements in the system. The interface at the central office is known as an Optical Line Termination, or OLT. On the user side is an Optical Network Unit, ONU, if only one subscriber is directly connected, or an Optical Network Termination, ONT, if it interfaces to a number of subscribers via another physical layer such as DSL.

A singlemode fiber runs from the OLT at the central office out into the field for a distance of up to 20 km. There, passive optical splitters physically divide the fiber into a tree of up into 32 to 64 fibers for connection to the individual ONU/ONTs. This is exactly analogous to the way cable TV uses splitters to divide power over coax connections to individual subscribers. If you split one fiber into two this way, each would have half the original signal strength, expressed as –3 dB. In practice, for 32 subscribers, the signal strength of each connection is roughly 17 dB less than the main trunk; this power budget is what provides the distance limitation on the system as a whole (see "Fiber Optics 101").

PONs can be designed with either one or two fibers for the main connection to the OLT. In a two-fiber system, 1310 nm is generally used for both downstream and upstream traffic, since the two directions are on different fibers. More common is the single fiber architecture using very basic WDM, where the downstream traffic (from the OLT/central office) is at 1490 or 1550 nm and the upstream traffic (to the OLT/central office) is at 1310 nm. In either case, it is important to note that downstream traffic is broadcast to all ONU/ONT nodes on the fiber simultaneously, while some method of contention must be worked out for the subscriber nodes to share the upsteam fiber. As an aside, a new enhancement is putting the complete multichannel analog video distribution of a hybrid-fiber-coax cable TV system on a separate downstream wavelength, but this is totally independent of the data services we are discussing here.

Once APON a Time
Much of the initial work on PONs was done by the Full Service Access Network (FSAN) group and standardized by the International Telecommunications Union in 1995, and because they are telco folks they naturally designed the higher network layers around ATM (rather than Ethernet and IP.) This is how the acronym APON comes about. APON downstream speeds are defined at 155 Mbit/sec or 622 Mbit/sec, with 155 Mbit/second upstream, all of which is shared by the users on that fiber. ATM, remember, uses fixed length 53-byte cells that are addressed using a Virtual Path Identifier (VPI) and Virtual Channel Identifier (VCI), and that is how downstream cells get to their proper destination. Upstream traffic uses a somewhat complicated time division multiplexing contention scheme, managed by the OLT to provide the requisite amount of bandwidth to each ONU. Quality of service may be managed on APONs through these grants of bandwidth. Because all data in each direction is visible to each node on the fiber tree, security is absolutely mandatory, though that specified in the original standard is very weak.

Ethernet Everywhere!
Since the APON standard was developed, the belief in a universal ATM architecture from desktop to backbone has run aground on the cost-effectiveness of Ethernet in the LAN. And chopping up Ethernet/IP frames into ATM cells at an access point is less efficient that using native IP across the network. So some bright sparks in the IEEE Ethernet in the First Mile working group started talking about using a PON physical layer with Ethernet frames; hence the acronym EPON. A PON fiber tree can be looked on as a large shared media system, just like original coax-based Ethernet. For downstream traffic, it also works the same, with the OLT head-end broadcasting frames that can be picked up by the appropriate destination node via MAC address. Upstream, however, is a problem since the distances (and hence transit times) across the network preclude use of Ethernet’s old CSMA/CD contention protocol. Some sort of TDM mechanism will need to be employed for the upstream traffic; different versions are under development. As with an APON, security is an issue as it is on any shared media network; 802.1q virtual LANs can provide a start but its maximum of 4096 VLANs could be a limitation. All of these issues still need to be sorted out and standardized, even while proprietary versions are shipping. But the big advantages of EPON versus APON are the efficiencies of end-to-end Ethernet/IP framing, and the ability to scale with ordinary Ethernet from 1 Gbit/sec to 10 Gbit/sec (see Cranking it up Again) and beyond.

Back to the Real World
Both Resilient Packet Ring and Passive Optical Networks provide technically elegant solutions to genuine problems faced in the metropolitan and access networks today, and keep many smart people busy on committees. But in the networking world, there is always more than one way to solve any given problem, and the best technical solution does not always win out. While both of these emerging technologies claim to be able to benefit from the high-volume Ethernet marketplace, they are not Ethernet and will require different silicon which at least to this writer seems to negate that claim. Next-generation SONET is being incrementally developed, adding many of the packet-friendly features promised by RPR. And since the CLECs and overlay network providers are either bust or shaky, the primary MAN providers are shaping up to be the baby Bells and other ILECs who are very SONET-oriented and less likely to adopt totally new technologies. And from where is the capital going to come to pull fiber to all those businesses and residences that could benefit from PONs? Even if it did materialize, alternative architectures using standard Ethernet fiber switches in environmentally hardened configurations may or may not be preferable and would not require learning new acronyms; after all, cable TV providers have had active equipment in the field for years in hybrid-fiber-coax systems.

Whether or not these two technologies gain widespread acceptance in the marketplace, the fundamental issues and ways in which they solve them will remain. At a minimum, at least this article has given some insight into those problems and solutions, while it may also give you a peek into the Next Big Thing in optical networking.