Cisco - Deploying Optical Infrastructure 605

605 1143_06F9_x

1

© 1999, Cisco Systems, Inc.

Deploying Optical IP Infrastructures Session 605

605 1143_06F9_x

2

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

1

Agenda

• Introduction • Transmission Alternatives • IP Network Architecture • Summary

605 1143_06F9_x

3

© 1999, Cisco Systems, Inc.

Applications Driving IP Traffic Growth • E-commerce

Rel. Bit Volume

• E-mail

250

• Information search/Access

200 150

Voice

• Conferencing/ multimedia

100

• Video/imaging

50

“From 2000 on, 80% of service provider profits will be derived from IP-based services.”

Data (IP)

1997

1998

1999

2000

2001

Traffic Projections for Voice and Data

— CIMI Corp. 605 1143_06F9_x

4

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

2

Internet Bandwidth Drivers • Internet hosts growing exponentially (16 million in Jan ’97) • Web users expected to reach 160 million by 2000 • TCP-WWW now accounts for 75% of Internet traffic

150 100 50 0 1996

1997

1998

1999

2000

Source: IDC Data Sizes (kbytes) 10000 1000

• Traffic type changing rapidly from text to image to video 605 1143_06F9_x

World Wide Web Users (Millions)

200

100 10 0 E-mail

File

Image

Hi Res Image

Movie

5

© 1999, Cisco Systems, Inc.

Projected IP Backbone Bandwidth Requirements 5000 4500

2 x OC48

POP City-Pair Bandwidth Requirements

4000 3500

Mbps

3000 2500

OC48

2000 1500 1000

NA-Tier 1 ISP

2 x OC12 OC12

NA-Tier 2 ISP

500 0

Jan ’96 605 1143_06F9_x

Jan ’97

Jan ’98

Jan ’99

6

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

3

Current Network Not Optimized for Packet-Based Services

• Most service provider infrastructures are based on circuit switched technologies • These infrastructures are optimized for n x DS0 for implementing voice, leased line services

605 1143_06F9_x

7

© 1999, Cisco Systems, Inc.

Circuit Switched Infrastructures

• Circuit switched infrastructures were used first to offer voice and leased line services and early IP services • ATM was used as a more efficient circuit switched infrastructure and for Frame Relay and IP transport

605 1143_06F9_x

Voice

Leased Line

IP

Frame Relay

Digital Cross Connects

ATM

ATM

SONET/SDH Optics

8

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

4

U.S. Network Capacity Is Exploding By More Than 8,000% 100 90 80 70 60 50 40 30 20 10

Level 3 Frontier Qwest GTE (Qwest fiber) IXC Williams Sprint MCI WorldCom AT&T 1.2

Total Bandwidth: 99.8

30.0

21.7

7.0 3.5 7.7 7.4

terabits/second

terabits/second

9.0 6.8 6.6

0 1996

21.8

terabits/second

1999

2001

Fortune Magazine, 3/15/99 605 1143_06F9_x

9

© 1999, Cisco Systems, Inc.

Need to Build Service-Optimized Networks Legacy TDM TDM TDM Voice, Voice, Leased Leased Line Line

Multiservice Optical Internetworks

Optical Internet

Multiservice Multiservice Frame Frame Relay, Relay, ATM ATM

IP IP

SONET/SDH SONET/SDH

Optical

Choosing the right infrastructure is a function of: Services to be offered Infrastructure Available 605 1143_06F9_x

10

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

5

Agenda • Introduction • Transmission Alternatives Dark Fiber SONET/SDH DWDM

• IP Network Architecture • Summary 605 1143_06F9_x

11

© 1999, Cisco Systems, Inc.

Fiber Networks

Electrical

Fiber Optic Transmission System (FOTS)

Optical

Fiber Optic Transmission System (FOTS)

Electrical

• A basic fiber optic system consists of A transmitting device, which generates the light signal An optical fiber cable, which carries the light A receiver, which accepts the light signal transmitted

• Single time-division multiplexed information stream 2.5 Gbps (OC-48/STM-16) is current state of the art 10 Gbps (OC-192/STM-64) is next generation 605 1143_06F9_x

12

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

6

Using Dark Fiber • Considerations when deploying IP infrastructures over dark fiber Fiber Plant - capacity & topology Power Budgets, optics reach Signal Loss Optical Attenuation (dB/km) Dispersion - Chromatic & Modal 605 1143_06F9_x

13

© 1999, Cisco Systems, Inc.

Using Dark Fiber • Effective alternative if fiber capacity is not constrained • Typical case for networks that have a limited geographic coverage • Lighting up fiber with routers provides lowest cost/bit infrastructure • Network design must address restoration 605 1143_06F9_x

14

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

7

Using SONET/SDH SONET ADM

SONET Facility

SONET Facility

Matrix (Synchronous) Pass Through Drop

Add

Blue Facility Dropped 605 1143_06F9_x

Orange Facility Added 15

© 1999, Cisco Systems, Inc.

SONET Overhead Layers Path Line

Line Section

Section

PTE (ADM, DSLAM,…

Path Termination

Service (DS1, DS3…) Mapping Demapping

605 1143_06F9_x

Section

REG

Section

REG

PTE (ADM, DSLAM,…

ADM or DCS Section Termination

Line Termination

Section Termination

PTE = Path Terminating Element MUX = Terminal Multiplexer REG = Regenerator ADM = Add/Drop Multiplexer DCS = Digital Cross-Connect System

Path Termination

Service (DS1, DS3…) Mapping Demapping

16

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

8

SONET Ring Configurations

2F BLSR

UPSR

• Unidirectional Path Switched Ring (UPSR)

• Bi-directional Switched Ring (BLSR)

• Deployed in MANs for access and aggregation

• Deployed in WANs

• All traffic homing to central node 605 1143_06F9_x

• Neighbor-to-neighbor traffic 17

© 1999, Cisco Systems, Inc.

UPSR Protection Simplicity at the Expense of Capacity

• 2 fiber ring topology • Head end bridge, tail end switch (1+1) 605 1143_06F9_x

18

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

9

BLSR Protection Supports Both Span and Ring Switching

• 4 Fiber ring topology • Supports both span and ring switching • Requires signaling between ADMs 605 1143_06F9_x

19

© 1999, Cisco Systems, Inc.

APS between Routers and ADMs

• APS is used to extend SONET protection to tributaries • All traffic goes to working router, protect router is idle 605 1143_06F9_x

20

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

10

Limitations of SONET Protection

Facility-based Protection

• SONET only protects the transmission infrastructure • SONET protects all traffic equally 605 1143_06F9_x

21

© 1999, Cisco Systems, Inc.

Using SONET/SDH • Accepted transport architecture in most service provider networks, except some ‘Greenfield Carriers’ • Used primarily to transport Circuit Switched traffic and some packet based traffic • Provides performance monitoring and self healing (50msec switchover) but at the expense of bandwidth efficiency (BLSR) • Limited availability of 622Mbps and 2.5Gbps tributary interfaces in not readily available or economical 605 1143_06F9_x

22

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

11

Dense Wave Division Multiplexing Provides Fiber Gain • • • •

16 channel x 2.5Gbps = 40Gbps 24 channel x 2.5Gbps = 60Gbps 40 channel x 2.5Gbps = 100Gbps 80 channel x 2.5Gbps = 200Gbps

• 4 channel x 10Gbps = 40Gbps • 16 channel x 10Gbps = 160Gbps • 128 channel x 10Gbps = 1280Gbps • Multiplexed wavelengths can be amplified as one composite signal using Erbium Doped Fiber Amplifiers (EDFAs) • Fiber non-linearities such as attenuation and dispersion impose limits on speed and distance 605 1143_06F9_x

23

© 1999, Cisco Systems, Inc.

Using DWDM • Used in several service provider networks • Used to provide bandwidth gain (example - 40 channels of 2.5 Gbps on a single fiber instead of a single channel) • High cost for systems can easily be justified in areas where additional fiber deployment may be required (typically in long-haul networks). For example, a link between Cheyenne and Omaha: DWDM equipment costs $17 million Laying new fiber costs $190 million

• Network Design must address restoration 605 1143_06F9_x

24

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

12

Summary of Transmission Alternatives • Use of dark fiber makes sense if there are no capacity constraints. This is typical for limited geographic areas. • SONET/SDH is widely deployed today and accepted for transporting circuit based traffic due to the self-healing capabilities • DWDM makes sense in long-haul networks where additional fiber deployment is extremely expensive 605 1143_06F9_x

25

© 1999, Cisco Systems, Inc.

Agenda • Introduction • Transmission Alternatives • IP Network Architecture Core Aggregation Access

• Summary 605 1143_06F9_x

26

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

13

Network Architecture Components

Access

Distribution

Intra-PoP

Core

Intra-PoP

Distribution

Intra-PoP

Access

Intra-PoP

• In an Optical IP network there are multiple environments with different characteristics • Each environment requires an optimized solution based on network design criteria 605 1143_06F9_x

27

© 1999, Cisco Systems, Inc.

IP Transport Architecture Alternatives

IP

Value Added Services

TDM-like Services Traffic Engineering

FR/ATM Protection

SONET/SDH OPTICAL

605 1143_06F9_x

Raw Bandwidth

28

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

14

IP Network Core

• There are multiple alternatives for building a core infrastructure for IP Frame Relay/ATM Packet Over SONET/SDH Dynamic Packet Transport

• Either one of these can utilize any transmission alternatives 605 1143_06F9_x

29

© 1999, Cisco Systems, Inc.

FR/ATM Core

FR/ATM

Network Design FR/ATM Switches connect to transmission equipment Routers connect to ATM switches via UNI interfaces FR/ATM connections provide appropriate CoS between routers

Design Considerations Network Capacity & Scale - 622Mbps ATM interfaces on routers Layer 3 Network design - full peering requires n(n-1)/2 connections Restoration could be achieved in Physical layer (diverse fiber routes or SONET/SDH) Logical layer (routing in FR/ATM switches) 605 1143_06F9_x

30

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

15

Packet Over SONET/SDH Core Network Design Point-to-point POS connections over dark fiber, SONET/SDH, DWDM directly attached to routers Optical Regenerators may be required to extend reach beyond 80km per span Use IP Class of Service techniques (ACL, CAR, WRED, DRR)

Design Considerations Network Capacity - up to 2.5Gbps interfaces available today Traffic Distribution - hub/spoke Restoration could be achieved in Physical layer (diverse fiber routes or SONET/SDH) Network layer (load-share over multiple paths)

605 1143_06F9_x

31

© 1999, Cisco Systems, Inc.

Packet over SONET/SDH (PoS)

• Point-to-Point Protocol, IETF RFC 1661

IP

Datagrams

• PPP in HDLC- Like Framing, IETF RFC 1662

PPP

Protocol encapsulation Link Initialization

• PPP over SONET/SDH, IETF RFC 1619

HDLC Framing

PPP Packet Delineation Error Control

SONET/SDH

605 1143_06F9_x

Byte Delineation

32

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

16

POS Performance

Total OH % per packet Min. packet (20 bytes)

35.78

Ave. packet (354 bytes)

2.76

Max. packet (4352 bytes)

0.94

Average Packet

• Total overhead is the sum of byte-stuffing and header/trailer overhead OH% = 0.78 + 7/N

• Total overhead for average packet is < 3% 605 1143_06F9_x

33

© 1999, Cisco Systems, Inc.

POS Enables Flexible Connectivity Dark Fiber

POS Point-to-Point Connectivity ADM ADM

~ ~ ~

ADM ADM

ADM ADM

WDM

~ ~ ~

ADM ADM

SONET/SDH Ring or Linear Point to Point

605 1143_06F9_x

• Runs over dark fiber, SONET, or WDM • Enables transport “mix and match” • Provides efficient evolution path for incumbents • Provides optimized transport for greenfield builds 34

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

17

Dynamic Packet Transport Core Network Design Packet based Ring Optical Regens may be required to extend reach

Working BFP

Design Considerations Network Capacity - 622Mbps interfaces Traffic Distribution - distributed sources/sinks of traffic Restoration provided by Intelligent Protection Switching in the Spatial Reuse Protocol (50msec switchover time for a 16 node ring)

605 1143_06F9_x

DPT Ring

35

© 1999, Cisco Systems, Inc.

Spatial Reuse Protocol • New Layer 2 MAC technology SRP

Path Section Concatenated plus Line OverPayload Overhead head

Spatial Reuse Protocol Uses SONET/SDH framing Bandwidth efficient Fairness (SRP-fa) Scalable Fast protection switching and service restoration Multicasting and priority

MAC MAC IP IP Packet Packet MAC MAC IP IP Packet Packet

GSR GSR 75XX 75XX

75XX 75XX

DPT-Based LAN/MAN/WAN 75XX 75XX

• Enables DPT functionality 605 1143_06F9_x

…

75XX 75XX

GSR GSR 75XX 75XX

36

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

18

Spatial Reuse

• Destination stripping GSR GSR

• Bandwidth consumed only on traversed segment • Multiple nodes transmit concurrently

Cisco Cisco 75XX 75XX

Cisco Cisco 75XX 75XX

• Dynamic, per-packet spatial reuse

DPT Ring

GSR GSR Cisco Cisco 75XX 75XX

• Control via SRP-fa instead of token passing

605 1143_06F9_x

Cisco Cisco 75XX 75XX

Cisco Cisco 75XX 75XX

37

© 1999, Cisco Systems, Inc.

Intelligent Protection Switching Fiber Cut

• Like SONET/SDH, DPT provides Proactive performance monitor and self-healing via ring wrapping Fast 50-ms restoration Protection switching hierarchy

GSR GSR

Cisco Cisco 75XX 75XX Cisco Cisco 75XX 75XX

• Unlike SONET/SDH, DPT provides signaling via explicit control messages Multilayer awareness and elastic cooperation differentiated handling by priority enhanced pass-through mode Fast IP service restoration on large rings No dedicated protection bandwidth and intelligent rehoming after wrap Minimal configuration and provisioning 605 1143_06F9_x

GSR GSR Cisco Cisco 75XX 75XX

Cisco Cisco 75XX 75XX

Detects Alarms and Events and Wraps Ring ~50 ms

38

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

19

DPT Enables Transport Flexibility and Evolution Dark Fiber

DPT Ring SONET SONET ADM ADM

~ ~ ~

SONET SONET ADM ADM

SONET SONET ADM ADM

WDM

~ ~ ~

SONET SONET ADM ADM

SONET/SDH Ring or Linear Point to Point

605 1143_06F9_x

• Runs over dark fiber, SONET, or WDM • Enables transport “mix and match” • Provides efficient evolution path for incumbents • Provides optimized transport for greenfield builds 39

© 1999, Cisco Systems, Inc.

Sample Core Architecture

• Majority of IP Core Backbones being deployed today use DWDM or SONET/SDH as the transmission media directly connected to POS interfaces on routers • Majority of IP Core Backbones are operating at 2.5Gbps 605 1143_06F9_x

40

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

20

Typical POS Core Design POP B Working

POP A

P

Protect

P

W

W POP C

POP E 605 1143_06F9_x

POP D 41

© 1999, Cisco Systems, Inc.

4 fiber “BLSR” using Routers Working

IN

OUT P

Protect

W

605 1143_06F9_x

P W

42

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

21

Line Cut, initial routing Working

X IN

OUT P

P

Protect

W

605 1143_06F9_x

W

43

© 1999, Cisco Systems, Inc.

Line Cut, after routing convergence Working

X IN

OUT P

Protect

W

605 1143_06F9_x

P W

44

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

22

Network Architecture Components

Access

Distribution

Intra-PoP

Core

Intra-PoP

Distribution

Intra-PoP

Access

Intra-PoP

• The distribution network connects to multiple access networks • The distribution network must provide the interface breadth, density and termination to connect efficiently to current and future transmission infrastructures • The distribution network must exhibit efficient traffic aggregation 605 1143_06F9_x

45

© 1999, Cisco Systems, Inc.

Aggregation Network

• An IP Aggregation network must be able to collect traffic from various access networks Dedicated Access SONET Multi-service (FR, ATM) DSL 605 1143_06F9_x

46

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

23

Dedicated Access Aggregation

• Aggregation interfaces from n x DS0 to OC12c/STM4c

n x DS0 DS1

• Typical access rates are increasing (256kbps to 45Mbps)

GSR GSR

DS3 OC-3/STM-1 OC-12/STM-4

• OC-3/STM1 and OC12/STM-4 accesses becoming available 605 1143_06F9_x

Cisco Cisco 75XX 75XX

47

© 1999, Cisco Systems, Inc.

SONET/SDH Aggregation OC-3/STM-1

… .

DS-3 DS-3

OC-3/STM-1

DS-3

OC-3/STM-1

OC-12/STM-4 Channelized

OC-12/STM-4 Channelized

OC-12/STM-4 SONET/SDH TDM Ring OC-48/STM-16

Discrete DS3, OC-3c, OC12c or Channelized OC12 interfaces

605 1143_06F9_x

48

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

24

SONET Access Solutions SONET devices like Cisco ISR3303 can be used for local access loops At the PoP, Circuit traffic can be directed to the circuit network and the IP traffic can be aggregated into the IP Edge via appropriate interfaces PABX

Circuit Traffic

IP Traffic ADM/DXC ADM/DXC

SONET Access

TDM TDM

ISR 3303

TDM TDM

TDM TDM

ISR 3303

ISR 3303 8 x T1’s and 8 x 10BaseT’s 605 1143_06F9_x

ISR 3303 49

© 1999, Cisco Systems, Inc.

Multiservice Aggregation Applications

Allows aggregation of IP traffic from multiple remote locations Connect routers via ATM interfaces to multiservice network

IP Core

UNI

UNI

Multi-service Network for Aggregation CPE Router

Edge Router Edge Router 605 1143_06F9_x

Edge Router 50

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

25

Aggregating IP Traffic from DSL Edge

DSLAM OC-3 ATM

DSLAM

IP Traffic coming over DSL links can be aggregated into routers via ATM interfaces (OC-3/STM-1 or OC-12/STM-4) 605 1143_06F9_x

51

© 1999, Cisco Systems, Inc.

Metro Aggregation Requirements • All of the PoPs that aggregate traffic from multiple accesses need to efficiently aggregate the traffic • This aggregation network must be IP Optimized Scalable High Reliability / Availability 605 1143_06F9_x

52

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

26

Dynamic Packet Transport (DPT) Aggregates Traffic • DPT can be used to efficiently and reliably aggregate traffic from multiple PoPs in a metropolitan network

605 1143_06F9_x

Core

DPT Ring DSL Edge

Multiservice Edge Dedicated Access Edge

53

© 1999, Cisco Systems, Inc.

Network Architecture Components

Access

Distribution

Intra-PoP

Intra-PoP

Core

Distribution

Intra-PoP

Access

Intra-PoP

• As the number services and users, an efficient design is needed to efficiently implement and scale the PoP 605 1143_06F9_x

54

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

27

Layer 2 Intra-PoP Architecture Backbone Routers GigE Switch Edge Routers

Backbone Routers

Backbone routers connected to Edge routers or servers using Layer 2 Switches

Backbone routers connected to Edge routers or servers directly using Layer 2 (typically GE or FE) interfaces

Edge Routers 605 1143_06F9_x

55

© 1999, Cisco Systems, Inc.

DPT Offers An Alternative POP Architecture • Dynamic Packet Transport is an efficient alternative for intra-PoP connectivity

GSR GSR Cisco Cisco 75XX 75XX

Cisco Cisco 75XX 75XX

• Benefits Cost-effective (uses less slots) Bandwidth efficient

DPT Ring

Cisco Cisco 75XX 75XX

GSR GSR Cisco Cisco 75XX 75XX

Self healing

Cisco Cisco 75XX 75XX

Configuration ease 605 1143_06F9_x

56

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

28

Agenda

• Introduction • Transmission Alternatives • IP Network Architecture • Summary

605 1143_06F9_x

57

© 1999, Cisco Systems, Inc.

Summary • IP traffic growth and optical infrastructure availability is paving the way for Optical Internetworking • Several transmission alternative available dark fiber, SONET/SDH, DWDM • IP network architecture is comprised of several environments - Core, Aggregation, Access • Optimal Design is a function of the services to be offered and infrastructure available. 605 1143_06F9_x

58

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

29

Optical IP Networks Sample Architecture

TDM TDM

TDM TDM

TDM TDM

TDM TDM

Metro Access

Metro Aggregation

Hybrid TDM, Data Rings using ISR 3303

DPT Rings Dark Fiber using GSR 12000 and 7500

605 1143_06F9_x

Backbone Point to Point PoS over SONET/SDH, DWDM, or Dark Fiber using GSR 12000

Metro Aggregation

Metro Access

DPT Rings Dark Fiber using GSR 12000 and 7500

Hybrid TDM, Data Rings using ISR 3303 59

© 1999, Cisco Systems, Inc.

Case Study Discussion Legacy TDM TDM TDM Voice, Voice, Leased Leased Line Line

Multiservice Optical Internetworks

Optical Internet

Multiservice Multiservice Frame Frame Relay, Relay, ATM ATM

IP IP

SONET/SDH SONET/SDH

Optical

Choosing the right infrastructure is a function of: Services to be offered Infrastructure Available 605 1143_06F9_x

60

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

30

Other Sessions to attend

Session 604 Introduction to Optical Internetworking

Session 606 Advanced Optical Technology Concepts

Session 1202 GSR Product Update

605 1143_06F9_x

61

© 1999, Cisco Systems, Inc.

Please Complete Your Evaluation Form Session 605

605 1143_06F9_x

62

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

31

605 1143_06F9_x

63

© 1999, Cisco Systems, Inc.

Cisco Systems Confidential

32