Network Test :: Benchmarking Services Network Test Methodologies::Results Contact Information About Network Test

Light Reading Lab Test: Internet Core Routers

Scheduled for Publication in December 2000

Test Methodology

 

v. 1.21 Copyright Ó 2000 by Network Test Inc. Vendors are encouraged to comment on this document and any other aspect of test methodology. Network Test reserves the right to change the parameters of this test at any time.

 

By David Newman, Glenn Chagnot, and Jerry Perser

 

Please forward comments to dnewman@networktest.com, glenn.chagnot@spirentcom.com, and jerry.perser@spirentcom.com

 

 

1          Executive Summary

This document describes a series of tests to be conducted on IP routers intended for use in the core of Internet service provider networks. Results of these tests will be published by Light Reading (www.lightreading.com), the online periodical of optical networking.

 

The tests described here include:

 

 

All devices in this test will use POS (packet over Sonet) interfaces operating at OC-48 (2.4 Gbit/s).

 

Section 2 of this document describes the test bed in general terms and introduces the test equipment to be used.

 

Section 3 describes the specific tests to be performed.

 

The URL for this document is http://www.networktest.com/LR_router_00/meth1.2.html

A companion spreadsheet that describes the IP prefixes, prefix length distribution, and packet length distribution is also available via anonymous FTP. The URL is:

ftp://public.networktest.com/LR_router_00/LR_router_prefixes.zip

 

 

2          The Test Bed

This section discusses the test bed topology; offers configuration instructions for participating vendors; and introduces the test equipment to be used.

 


2.1        Test Bed Topology

Figure 1 below shows the basic configuration of equipment to be used in all tests.

 

 

Schematic diagram of Internet core router test bed

Figure 1: The Internet Core Router Test Bed

 

Each vendor must supply four routers, each equipped with six OC-48c POS interfaces. Each router must support BGP4 routing. We define a “router” as a device that supports BGP4 and contains a unique L2 forwarding database and L3 routing table for each 6 POS interfaces.

 

Support for MPLS is preferable but not mandatory. As noted in the invitation letter mailed to prospective participants, Light Reading plans to give separate awards for “best BGP device,” “best MPLS device,” and “best overall device.” Nonparticipation in the MPLS tests does not preclude winning awards in the other events.

 

In addition, vendors must supply any cabling necessary to interconnect their core interfaces. Network Test and/or Spirent Communications will supply cabling to link the tester devices to the routers.

 

2.2        Device Configuration

2.2.1       Layer-1 Considerations

Vendors must configure edge interfaces as follows:

 

Framing: Sonet (not SDH)

Rate: OC-48c

Clocking: Router A is the clock source; all other devices use loop timing from Router A

Path signal label: 0xCF

FCS: 16-bit CRC

 

2.2.2       Layer-2 Considerations

Vendors must configure edge interfaces as follows:

 

Encapsulation: PPP with HDLC framing (RFC 1662; header = FF 03 00 21)

MRU: 1500 bytes

Maximum configurations: 10

Maximum failures: 5

Maximum terminations: 2

Magic number: 2

Restart timer: 3

Retry count: 1

LCP enabled

 

2.2.3       Layer-3 Considerations

This test bed models part of the core network for a single service provider; addressing inter-provider connectivity issues is beyond the scope of this project. Accordingly, 1 autonomous system (AS) will be used in the entire test bed.

 

For all tests except maximum BGP table capacity (section 3.3), vendors should set BGP hold timers to zero (infinite, no keepalives exchanged) to avoid conflicts with offered test traffic.

 

Vendors must configure the edge interfaces with the following IP addresses:

 

Router

Interface

DUT

A

1

217.0.1.2/24

A

2

217.0.2.2/24

A

3

217.0.3.2/24

B

4

217.0.4.2/24

B

5

217.0.5.2/24

B

6

217.0.6.2/24

C

7

217.0.7.2/24

C

8

217.0.8.2/24

C

9

217.0.9.2/24

D

10

217.0.10.2/24

D

11

217.0.11.2/24

D

12

217.0.12.2/24

 

The prefixes and test traffic to be offered are described in a separate document. This document, an Excel spreadsheet, is available for anonymous FTP here:

 

ftp://public.networktest.com/LR_router_00/LR_router_prefixes.zip

 

2.3        Test Equipment

2.3.1       Test Hardware

2.3.1.1       Spirent SmartBits

The principal test instrument for this project is the SmartBits traffic generator/analyzer manufactured by Spirent Communications Inc. (Chatsworth, Calif.). Spirent’s SmartBits 6000 chassis will be equipped with the company’s new POS-6505A SmartModules. The  SmartModule offers line-rate traffic generation and analysis over OC-48c interfaces.

 

Three OC-48c SmartBits interfaces will be attached to each of four routers under test. Thus, a total of 12 OC-48c SmartBits interfaces will be attached to the system under test.

2.3.1.2       Adtech AX-4000

In addition to the SmartBits, an Adtech AX/4000 Broadband Test System equipped with OC-48c interfaces will be used for troubleshooting and protocol capture and decode purposes.

2.3.1.3       Physical Cables

All interfaces will be connected using single-mode fiber-optic cabling and SC (rectangular) connectors.

 

3          Test Procedures

For each routine in this section, this document describes:

 

·        the test objective;

 

3.1        Baseline IP forwarding, latency, and jitter measurements

3.1.1       Objective

To determine baseline packet loss, latency, and jitter for routed IP traffic

3.1.2       Configuration

--Test bed topology shown in section 2.1

--Total BGP prefixes advertised to the SUT: 201,165 (approximately 2.3 times the 88,500 prefixes shown in core routers on 21 August 2000, according to Telstra)

--No overlapping prefixes

--Prefix lengths will range from /13 to /26, with distribution following Mae-East and Mae-West statistics taken from http://www.merit.edu/ipma/routing_table/#prefix_length     

3.1.3       Procedure

Using the Spirent SmartBits to offer incrementally heavier traffic loads, Network Test will determine the maximum forwarding rate each system under test can sustain with zero packet loss.

 

Two iterations will be run: One consisting exclusively of 40-byte IP packets[1], and one with a quad-modal distribution of packet sizes.

 

In the quad-modal distribution, the lengths and distribution of the IP packets are as follows:

 

IP packet length (bytes)

Streams per SmartBits interface

Percentage of total streams[2]

40

37

56.06%

1,500

15

22.73%

576

11

16.67%

52

3

4.55%

 

The packet length distribution is a composite of 22 samples taken from Merit, a consortium of Michigan-based ISPs, between 28 August 2000 and 13 September 2000. The raw data is available from http://moat.nlanr.net/Datacube. The packet lengths shown here are the four highest medians of all IP packet lengths between 20 and 1,500 bytes.

 

3.1.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

 

3.2        Longest-match lookup times

3.2.1       Objective

To determine the impact of route lookups for mixed-length prefixes on device forwarding rates, latency, and jitter

3.2.2       Configuration

--Test bed topology shown in section 2.1

--Total BGP prefixes advertised to the SUT: 251,165 (includes the 201,165 prefixes from the baseline tests (section 3.1), plus 50,000 additional, shorter prefixes to force longest-match lookups

 

--Overlap of 50,000 prefixes, about 20 percent of total prefixes

3.2.3       Procedure

Network Test will offer the same maximum offered load as in the baseline test (section 3.1).

 

Network Test will compare results of this test from results from the baseline test and determine any differences in forwarding rates, latency, and jitter.

3.2.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

 

3.3        Maximum BGP4 table capacity

3.3.1       Objective

This test will determine the maximum number of BGP4 prefixes one router will learn and propagate.

3.3.2       Configuration

--Test bed topology shown in section 2.1, although only routers A and B are used

--Vendors should set BGP hold timers to 30 seconds for this test only

3.3.3       Procedure

The Spirent SmartBits tester will be attached to one interface of routers A and B. The routing tables of the devices under test will be cleared. Then 80,000 /22 prefixes will be advertised by the tester to Router A. The correct learning of all prefixes will be verified on the BGP4 instance running on the SmartBits tester attached to Router B. 

 

The number of prefixes to be injected will begin at 80,000 and increase in increments of 40,000 until the system under test fails to propagate one or more prefixes. The routing table of the devices under test will be cleared between iterations.

3.3.4       Test Metrics

Maximum prefixes learned

 

3.4        Performance during route flapping on stable and unstable paths

3.4.1       Objective

This test will determine the effect, if any, on forwarding rate during route flapping.

3.4.2       Configuration

Test bed topology shown in section 2.1.

 

All devices under test will be configured to run BGP4. The Spirent SmartBits tester will be attached to one interface of routers A and B to generate route flapping and to verify table entries at every flap event.

3.4.3       Procedure

The Spirent SmartBits 6000 will offer 40-byte packets to all POS interfaces in a partially meshed pattern at line rate. Then this procedure will be followed:

 

Step 1. Advertise 201,165 prefixes to Router A. Each prefix has primary, secondary, and tertiary routes.

Step 2. Withdraw 50,291 primary routes from the routing table. (This is 25 percent of all prefixes.)

Step 3. Verify that traffic is routed over 50,291 secondary routes.

Step 4. Verify that traffic is routed on all other available routes.

Step 5. Re-advertise the original 50,291 primary routes after an interval of 30 seconds.

Step 6. Verify that all traffic is routed over the primary paths.

Step 7. Remove and re-advertise 50,291 routes from the routing table continuously in 30-second intervals for a period of 120 seconds.

 

The routes removed in each iteration will be noncontiguous entries in the routing table.

 

3.4.4       Test Metrics

Forwarding rate of stable paths (packets per second)

Forwarding rate of unstable paths (packets per second)

 

3.5        Baseline MPLS forwarding, latency, and jitter

3.5.1       Objective

To determine baseline packet loss, latency, and jitter for label-switched traffic

3.5.2       Configuration

Test bed topology shown in section 2.1.

 

The SmartBits tester will initiate nine label-switched paths (LSPs) at each edge interface – one label apiece for each of the nine possible destination interfaces. The nine labels concatenate all the IP prefixes described in section 3.1.3. The total number of LSPs for this test is 108 (12 edge interfaces, each with 9 LSPs).

3.5.3       Procedure

Using the Spirent SmartBits to offer incrementally heavier traffic loads, Network Test will determine the maximum forwarding rate each system under test can sustain with zero packet loss.

 

This test will be run exclusively with 40-byte IP packets (with packet lengths increased by MPLS labels, of course).

3.5.4       Test Metrics

Maximum forwarding rate with zero loss (percent of media rate)

Average latency (microseconds)

Average jitter (microseconds)

 

3.6        Maximum LSP capacity

3.6.1       Objective

This test will determine the maximum number of label-switched paths one router can maintain.

3.6.2       Configuration

Test bed topology shown in section 2.1, although only routers A and B are used

3.6.3       Procedure

The Spirent SmartBits tester will be attached to one interface of routers A and B. The forwarding tables of the devices under test will be cleared. Then the SmartBits attached to router A, acting as a label edge router (LER), will offer one label to Router A, establishing a label-switched path (LSP) to a destination interface on Router B. The SmartBits will create primary paths using RSVP-TE (resource reservation protocol-traffic engineering). The SmartBits will continue to initiate LSPs at a low rate – approximately 10 per second – until traffic fails to reach the destination interface on Router B.

3.6.4       Test Metrics

Maximum LSPs maintained

 



[1] References to packet lengths in this document cover IP packets – the length from the first byte of the IP header to the last byte of the packet payload before the CRC. Packet lengths do not include link-layer headers, MPLS labels, or CRC fields.

[2] Percentages do not add to 100.00 percent due to rounding.

Network Test Footer. Privacy Policy. Contact us to learn more.