OSPF sequence numbers – why 80 million is smaller than 70 million

So a bit of a specific topic today. Going through Doyle’s Routing TCP/IP Volume 1, I felt my brain melt as he went through explaining sequence numbers in link-state advertisements (in a general sense, not specific to just OSPF). He describes two types of number “spaces” – the range of possible values – to describe how protocols sequence their LSA’s.

Ignoring the historic bits, such as Radia Perlman’s “lollipop space”, which is essentially a combination of cycling numbers with a fixed initialization value (this was part of the first version of OSPF drafts – not relevant for OSPFv2 or anything else), numbering spaces either follow linearly or circular.

In linear spaces, numbers start at x and end at y. The issue with linear space is that you could potentially “run out” of number space. This could cause a link-state protocol to be unable to distinguish between LSA’s that are the most recent from the originating router or just LSA’s being flooded from one router to the next. Link-state protocols, when receiving an LSA with the highest possible sequence number, shut down and age out it’s link-state database (LSDB) to flush all the older LSA’s out. To mitigate this, the designers had to make sure the field for a sequence number was large enough so as to never reasonably hit that highest possible value (y). Both OSPFv2 and IS-IS uses this number space scheme.

Circular number spaces never end – once a maximum value number is reached, it “resets” back to the lower boundary of the space. Since IS-IS and OSPFv2 use linear spaces, this is included for completeness. Perlman’s lollipop scheme used both linear and circular as a combination but these are not included in modern link state protocols.

IS-IS uses a rather simple scheme for it’s number space. A router will originate it’s own directly-connected link states with a sequence number of one (0x00000001), with a maximum sequence number of 4.2 billion (0xFFFFFFFF). This is because the IS-IS field for sequence numbers in its LSP’s (link state packet) uses unsigned 32-bit integers. These values range from 1 – 4294967295 in decimal.

OSPF, on the other hand, uses signed 32-bit integers. While it uses the same scheme for number spaces as IS-IS (linear), the way the values are represented (especially on a router’s database outputs) is…different.


Net Link States (Area 1)

Link ID         ADV Router      Age         Seq#       Checksum      1862        0x80000237 0x00D860      12          0x80000001 0x00E8F5

So…it starts are 80 000 000?

Obviously, the seq. number is represented in hexadecimal format…but why 0x80000001? Doesn’t that translate to 2 billion decimal? The detail to note is the fact that this field is a signed integer. That means the integers actually range from – 2147483648 to + 2147483648. When processing this field in binary, the CPU needs a way of comparing sequence numbers to determine which one is “higher” – in this case, closer to positive +2147483648.

Programming languages such as C/C++ must pay particular attention to integers declared as signed vs unsigned. Some google- and wiki-fu later, the reason we see sequence numbers starting at 0x80000001 (0x80000000 is reserved via the RFC standard) is because the left-most/most significant bit determines whether a number is represented as a positive value or a negative value. When the MSB is set, the integer is a negative value. When the MSB is not set, it is a positive integer.


0x80000001 is 1000 0000 …. 0000 0001 in binary
Since the MSB is set, this is the “first” integer value in a 32-bit signed integer range. It doesn’t make sense to think of these values in decimal values, since this does indeed translate “directly” to 2 billion. These sequence numbers will increment 0x80000002….all the way to 0xFFFFFFFF (-1 in decimal). Incrementing one more time would start the sequence at decimal 0. This is because the MSB must become “unset” for it to represent positive values. The range then continues from 0x00000001 until 0x7FFFFFFE. Again, from the RFC, 0x7FFFFFFF is reserved (actually, an LSA received with this maximum possible sequence number triggers OSPF to flush its LSDB…more nuts and bolts to be expanded on later).


The choice of using signed vs unsigned gets kind of blurred between hardware and software. The use of signed integers simplifies ALU designs for CPUs and most (if not all) programming languages implement signedness in their integer data types…Why the IETF chose to use signed integers as part of the OSPFv2 spec? Who knows…


Anyways, this really bothered me for a couple days. I feel better now that it’s on paper. Any gross errors or omissions, leave it in the comments!


PS: More math-savvy folks will scream at this in regards to two’s complement role here with signed integer binary representation…I just wanted to know and jot down why IOS shows the starting sequence numbers in show ip ospf database as 0x80000001. So there you have it. Further reading for the curious

CCIE or bust! And other going-on’s

The time has come…

I’ve finally made the committed decision to pursue my number for CCIE Routing and Switching. Like most folks in networking, I’ve gotten to the point where I’m feeling quite confident in my skills; solid foundations with just a few cobwebs here and there to knock out (mostly due to re-focusing). This decision has come to me after moving on from my VAR support job, which covered the entire breadth of Cisco but prevented my skills from becoming specialized, to a network engineer doing implementation for a financial org. Since I’m settling in to the new job, I’ve come to realize that all the nitty-gritty routing and switching bits are what interest me the most. Sure, I’ve done a bit of this and a bit of that in other areas (mostly in wireless and data center) but I’m an R&S guy.

Which brings me to my next bit of personal news – I’ve now gone from support to implementation. For those in the NOC’s or VAR’s, I would highly recommend it as a next step after you’ve gotten your feet wet in the trenches of support. It’s nice to learn what happens when things break and how to resolve issues, however, in my humble opinion, in order to have that deep understanding, you have to be there to know *why* something is configured or designed a certain way. Delving into the world of real-world business challenges and requirements, as well as ITIL and change management (ugh, how I loathe thee…a “necessary evil” some may say), I know get to make decisions on how my network looks and how it functions to accomplish a certain goal. Whatever those goals may be, such as a new project or business requirement. For those who are looking to move up in the world of networking, implementation is required experience.

So, while I haven’t been blogging much here (seriously, just so much to learn and write about…some may say too much!), I will be focusing on hitting the books and lab prep. I’m shooting for a Q2 2014 target. Wish me luck!

PS: There are so many good blogs out there with CCIE “notes” – however, I could start banging out tidbits here and there for things that stump me or just bother me…More to come.

Modular Chassis vs Fixed Configuration Switches – Part 2 When/Where

Part 2 of my chassis vs fixed config switch showdown. In this post, I’ll provide some examples of where you might find both form factors deployed and some of the use cases for both. Click here for part 1 of this series.


Fixed Configuration

  • Campus

One of the more obvious places to find fixed configuration switches is in the campus network. This may include a campus of all sizes, from SMB to large enterprise. Here, the best bang for your buck lends itself to fixed-port switches since you typically will see “dumb switches” deployed in the access layer for port density and provide host connectivity to your network. Examples include lower-end Catalyst 2900 series and Catalyst 3600/3700 series switches.

Not only will you find these switches in the access of campus networks. Many smaller joints will use mid-range Catalyst 3750’s (for example) and stack them for distribution and/or core layer functions. Some of the advantages of using a switch stack for your core are distributed control plane across the stack, as well as port density for the rack space. If you’re connecting a bunch of dumb Layer 2 switches across your access, you can easily uplink them to slightly-beefier Layer 3 switches such as 3560’s and 3750’s arranged in a stack configuration. Of course, you are subject to hardware failures since these platforms do not have any redundancy built in to each member switch. However, for smaller organizations that just require the number of ports in a (relatively) small package, these do just fine.

  • Data Center

Fixed-port switches also find themselves in many “top-of-rack” (ToR) deployments. One only has to look as far as the Nexus 5000 Series, Juniper’s EX switching line, Brocade’s VDX line and countless other vendors. These switches typically have the 10 Gigabit density requires to connect large numbers of servers. Depending on your requirements, native Fibre Channel can also be used (in the case of Cisco Nexus 5500’s and Brocade VDX 6700’s) to provide FCoE connectivity down to servers. Rack space and port density again are factors when deploying these fixed configuration switches. In the case of data centers, the FCoE capabilities are also typically found on these platforms where they may not exist in chassis offerings.

  • ISP

One lesser-seen place to find fixed-port switches is in an ISP environment. Metro Ethernet comes to mind here. Providing Layer 2 VPN (L2VPN) services, ISP’s need equipment that can interface with the CPE and provide the connectivity up to the SP core. Cisco’s ME Series finds that place on this list, providing Q-in-Q/PBB as “dumb SP access”.

Modular Chassis

  • Campus

In the campus network, you’ll seen chassis’s in all places. On the one hand, with proper density and power requirements, you can easily deploy something like a Cisco Catalyst 4500E in a wiring closest for host connectivity. On the other hand, you would also see Catalyst 6500E’s in the campus core, providing high-speed switching with chassis-style redundancy to protect against single-device failures. Redundant power and supervisors help mitigate failures of one core device.

  • Data Center

In classical designs, the data center core is where you’d most likely see a chassis-based switch deployed. High-speed switching is the name of the game here, as well as resiliency against failures. Again, the redundant hardware found in chassis’s protect against failures in the data, control and management plane. Cisco Nexus 7000’s and Juniper’s high-end EX8200 platforms

In some designs, you may see a chassis switch deployed in an “End of Row” (EoR) fashion. This follows the design principles of the fixed-config ToR deployments, except here you could deploy a chassis switch to (again) improve redundancy. While definitely not required for all environments, if you couldn’t possibly allow a failure of your first switch that touches the end hosts, the extra redundancy (supervisors, power, switch fabrics, etc.) fit the bill.

  • ISP

Since I don’t work in the field of service providers, I’ll present what I feel are appropriate places to find a chassis-based switch. More than likely you’ll be using chassis-based routers here but I’ll include it for completeness. Cisco 6500/7600, ASR1000/9000’s, Juniper MX, all chassis offerings that you could see in an ISP’s network. This includes (but not limited to) ISP core for high-speed MPLS or IP switching/routing and PE deployments. This is where rich services provided by these offerings live, in order for service providers to be able to offer an array of MPLS-based services. This extends from L3VPNs to L2VPNs as well (either L3TPv3-based, EoMPLS-based or VPLS-based deployments). Being critical/core devices, I would imagine most service providers to require the level of fault-tolerance offered by a chassis-based system, especially with strict SLA’s with customers.


And there you have it. Hopefully, I’ve given you a good overview of the what, where, when, and why of fixed configuration and modular chassis switches. Given the state of things with our vendors, you can typically find any and all form factors provided by each and hopefully will choose the fit platform to fit the requirements of your environment. Cisco, Juniper, Brocade and HP are the first names that I can think of. There are other players as well including Arista, BNT, Dell, Huawei, Alcatel-Lucent, and a slew of others might fit specific markets and environments but may not cover the broad spectrum required for every network. As always, do your research and you’ll find what you need just fine.


Any corrections or feedback, you know what to do! Thanks for reading.

Modular Chassis vs Fixed Configuration Switches – Part 1 The Why

One thing that seems to be brought up a lot in conversation around the office, especially for newer folks entering the networking biz, is the choice of using larger modular chassis-based switches versus the smaller simpler fixed-configuration cousins. In fact, most people (myself included when I got my start) don’t even know that “chassis” are an option for switch platforms. This is completely understandable for the typical college and/or Cisco NetAcad. graduate, since the foundational education is focused almost exclusively on networking theory and basic IOS operations. So when, where and why do you use a chassis-based versus a fixed configuration switch?

PS: I’ve decided to make this a two-part series, due to the verbosity of the information. This article will focus on comparing the two different classes of switches and why you might use over the other. In the second part, I’ll provide some real world use cases for both and where they might be typically deployed.

In this article, I’ll be using the following Cisco platforms* for comparison between the two options:

*Note: I’ll be sticking to Cisco purely for simplicity’s sake. Other vendors such as Juniper, HP and Brocade carry their own lines of switching platforms with similar properties. With a bit of research, you can apply the same logic when evaluating the different platforms for your specific implementation.


Port Density

This is one of the more obvious variables. The Catalyst 6509, a 14RU 9-slot chassis, can have upwards of 384 (336 in dual-supervisor setups) Gigabit copper interfaces. These chassis can also utilize 10Gbps line cards, with those densities in just over 100 10GbE ports per chassis. However, it’ll depend on (especially on 6500’s) what supervisor modules you’re running along with your PFC/CFC/DFC daughtercards that will determine if those densities are a bit lower or higher than those numbers. This is where it’s critical to do your research before placing your orders or moving forward with implementations.

On the flip-side, fixed-configuration switches are just that – fixed chassis with limited modular functionality. Some exceptions apply, such as the Nexus 5500 switches that have slots to add additional modules to. However, generally speaking, WYSIWYG with these classes of switches. If we look at the same rack space of 14RU as with the previous example, that could potential be fourteen 48-port Gigabit Ethernet switches. A slew of Catalyst 3750’s gives you a whooping 672 ports in the same rack space. However, keep in mind that’s fourteen switches, as opposed to a single chassis-based switch. You’ll have to keep this in mind when putting this hardware into your topology (to be discussed below). Unless of course you plan to run your switches in a stack via technology such as Cisco StackWise, which helps reduce the burden of managing so many switches separately. Since Cisco stacks are restricted to a maximum of 9 switches per stack, you’ll be looking to manage at least two different switch stacks to match the 14RU and achieve the most port density in this comparison.

A side note for 10-Gig connectivity. The Nexus 5548UP can use up to 32 10GbE plus 16 with optional expansion module in a 1RU form factor. To compare to the 6509, that’s over 600 10GbE ports in a 14RU space. While it may be unfair to compare a newer Nexus series switch directly to a 6500 chassis, it’s comparison is purely for the discussion of form factor differences. A quick look at the Nexus 7009 chassis has similar 10GbE density as the 6509, which increasing densities with the larger chassis’s.

At the end of the day, fixed-configuration switches (generally speaking) packs more ports in the same amount of rack space as chassis switches.

Interface Selection

Typically, fixed-configuration switches are copper-based FastEthernet and/or GigE. Exceptions again exist, as with the Nexus 5500’s (Unified Ports models) being able to run their interfaces in Ethernet or native Fibre Channel. In your fixed-config’ers, you’ll also typically have higher-speed uplink ports that support higher-speed optics such as SFP/SFP+, GBIC, XENPAK and X2’x. And of course, there are SFP-based flavours that give you all fiber ports if that’s what tickles your fancy.

On your chassis-based switches, you will see that you have a wider choice of interface types. You’ll have your 10GbE line cards, which can be a combination of SFP/SFP+, XENPAK (older), X2, RJ45 and GBIC transceivers. You can also make use of DWDM/CWDM modules and transceivers for your single-mode long-range runs. Also, with 40 Gigabit Ethernet becoming more relevant and deployed, QSFP+ and CFP connectivity is an option as well (if the chassis in question can properly support it). The only restriction on chassis-based switches are what line cards you have to work with.

Due to the nature of fixed switches, it’s natural that a chassis comprised of modular line cards has more interface selection.


Here’s where things become a little more subtle. Again, for simplicity’s sake, I’ll restrict this section to the following models of switches:

  • Catalyst 6509 Chassis w/ Supervisor 720
  • Catalyst 3750-X 48-port Gigabit Switch
  • Nexus 5548UP Switch

Let’s start with the fixed switches. The fixed-configuration Catalyst 3750-X switches are equipped with 160 Gbps switch fabrics which should be ample capacity for line-rate gigabit speeds. Let’s also not forget that fabric throughput is not the only measure of performance. These switches, and most similarly designed fixed-configuration “access” switches, have smaller shared port buffers which can become problematic with very bursty traffic.

On the Nexus 5548UP’s, we see a different story. Being a 10GbE switch with a 960Gbps fabric, naturally we see the 5548 have much higher performance than it’s LAN cousins. Port buffers on the Nexus 5500’s are dedicated per port (640KB), allowing these switches to handle bursts very easily.

The Catalyst 6509, being a modular chassis-based switch, bases its performance on the Supervisor modules in use, as well as the specific line card+daughter card combination. For simplicity’s sake, let’s assume a Supervisor 720 (since the switch fabric is located on the supervisor in these switches, that’s 720Gbps switching capacity across the entire chassis) and X6748-GE-TX 48-port 1Gig ethernet modules. Due to the hardware architecture of these chassis, each slot is constrained to 40 Gbps capacity per slot, so slight over-subscription for 48x1Gbps line cards will occur. Luckily, each port is given a 1.3MB buffer so bursty traffic is handled just fine on these line cards. When using DFC daughter cards, line cards will even handle their own local switching and so won’t be constrained to the 40Gbps-per-slot restriction. This is because packets don’t need to traverse the backplane when switched port-to-port on the same line card. May I reiterate that for these kind of switches, do your homework. The performance of a chassis-based switch depends on more factors due to the combination of supervisor, switch fabric and line cards in use.


One of the most obvious benefits of a chassis-based switch is redundant and highly-available hardware. The Catalyst 6500 is typically deployed with two Supervisor modules. By having two supervisors, you’re protected by failures in the data plane (due to redundant active/standby switch fabrics), the control plane (NSF/SSO) as well as the management plane (IOS is loaded on both supervisors and thus can continue to operate even when an active Sup fails). Chassis switches also utilize redundant power supplies to protect against electrical failures.

On the other side of the coin, you have fixed configuration switches. While some newer switches do utilize redundant power supplies, none of them use separate modules or components for data, control or management plane. They utilize on-board switch fabrics for forwarding as well as a single CPU for running your IOS images and control plane protocols.

Chassis here is the clear winner.


Let me start this section by delving into what I mean by “services”. This includes features that are considered additional to basic switching. This will include support for things such as partial and/or full Layer 3 routing, MPLS, service modules such as firewalls and wireless LAN controllers (WLC) and other extras you may find on these platforms.

I think it’s safe to say that, generally speaking, fixed configuration switches are simple devices. As with the Catalyst line, with software, you can utilize full Layer 3 routing and most normal routing protocols such as OSPF, EIGRP and BGP. Keep in mind, however, that you will often be limited by TCAM capacity for IP routes so forget about full BGP tables and the like. However, they get the job done. One great benefit is support for Power over Ethernet (PoE), which is usually standard on copper-based fixed configuration switches.

The Nexus 5500’s are a bit of an exception. On the one hand, out of the box, they are Layer 2 only devices that can only support (albeit limited) Layer 3 with an expansion module. However, with the Unified Ports, they also support native Fibre Channel as well as modern NX-OS software. I would say that, for specific use cases, the compromise is quite reasonable. Elaborating on that will be in my next post.

The Catalyst 6500 is the champion of services. Being a modular chassis, Cisco developed many modules that were specifically designed with services in mind. This includes the Firewall Services Module (FWSM), Wireless Service Module (WiSM) controllers, SSL VPN module, and ACE/CSS load balancers. While these modules have fallen out of favour due to performance constraints of the chassis itself as well as lack of development interest and feature parity with standalone counterparts, the fact of the matter is that there are still many Cisco customers in the field with these in place. The WiSM, for example, is essentially a WLC4400 on a blade. Being able to convenient integrate that directly into a chassis saves rack space as well as ports by using the backplane directly to communicate with the wired LAN. Other services supported on the 6500 from a software standpoint include Virtual Switching System (VSS) (with use of the VS-SUP720 or SUP2T), full Layer 3 routing with large TCAM, MPLS and VPLS support (with proper line cards) and PoE.

The chassis win in this category due to the modular “swiss army knife” designs.


I’ll just briefly mention the cost comparison between the two configurations. You will typically see a chassis-based switch eclipse a fixed configuration switch in terms of cost, due to the complexity of the hardware design as well as all the modularity with chassis. You’ll always have a long laundry list of parts that will have to be purchased in order to build a chassis-based switch, including the chassis itself, power supplies, supervisors, line cards and daughter cards (if applicable). Fixed configuration switches typically have a much lower cost of entry, with only limited modularity with certain platforms.

And there you have it. Hopefully, this will give you an insight into why you might use one form factor over the other. In my next part, I’ll provide some use-case examples for each and where you may typically deploy one versus the other.

CCIP completed, onto a different brand of Koolaid

Earlier this month, I sat for my Qos 642-642 exam to complete my CCIP certification. Other than a few gripes with out-dated information, the exam went over pretty smoothly and I hammered out a pass. I’ve written previously of my motivations for obtaining the CCIP cert and am glad to have stuck with it. Even though the certification will officially retire in a week or so, a lot of the topics covered will also be on the CCIE R&S version 4.0 blueprint. I doubt I’m finished with BGP, MPLS and QoS so I’m keeping that knowledge tucked away for the time being 😉

Just one last note on CCIP, I would highly recommend Wendell Odom’s Cisco QoS Exam Cert Guide for anyone looking to learn about QoS on Cisco IOS. This is one of the best Cisco Press books I’ve read and continue to reference it for everything IOS QoS.

Now that I’ve a broad brush of Cisco R&S technologies with my CCNP and CCIP, I’ve decided to re-visit my Juniper studies. While we don’t work all that much with Juniper at $DAYJOB, we have Juniper gear in the lab to play with. Recently, I’ve been using EX4200 and EX4500 switches as well as working through Juniper’s free JNCIS-ENT study guide. Coming from a Cisco background and particular having gone through CCNP, I’m finding there’s a good amount of overlap. It’s just learning all the JUNOS hierarchies and “where is that feature” in JUNOS.

Upcoming posts will cover some basic JUNOS switching on EX and interoperating with Cisco Catalyst 3560/3750’s. I’ll also be finishing a lot of my draft posts from earlier this year covering BGP, MPLS and some vendor ranting 😛

Stay tuned.

MPLS VPN Label Basics – The LIB, the LFIB and the RIB(s)

LDP, or Label Distribution Protocol, is used to advertise label bindings to peers in an MPLS network.

The Label Information Base, or LIB, contains all received labels from remote peers and is similar to the IP RIB. Not all labels received from LDP neighbors are used since there will be a best path selected and to be used for forwarding for each prefix. Forwarding decisions are based on the Label Forwarding Information Base, or LFIB, once the best path towards the next-hop LSR is determined. How this is determined is based on the close relationship between the LIB, the LFIB and the IP routing table (RIB).

For clarity, we’ll be talking about non-ATM MPLS forwarding. ATM MPLS uses different LDP discovery, label retention and distribution methods because of ATM’s unique forwarding method and encapsulation(s).

Here’s our simple MPLS topology. We have two PE routers, connecting two customer sites. We also have a route reflector to reduce the number of IBGP connections required between PE routers. This is part of my MPLS lab so the irrelevant routers and configs will be omitted.

PE1 Router ID:
PE2 Router ID:
RR Router ID:

Routing within the MPLS network is provided by basic single-area IS-IS.

So how does MPLS build its Label FIB? First, let’s look at the VRF’s defined for this customer. We’ll be using VRF “Red” on both PE routers:

PE1#show ip vrf
  Name                             Default RD          Interfaces
  Red                              65000:1             Fa1/0
PE2#show ip vrf
  Name                             Default RD          Interfaces
  Red                              65000:1             Fa1/0

For VPNv4 routing between customer sites, MP-BGP is used to distribute label bindings for VRF routes. LDP will distribute label bindings for the Loopback0 BGP next-hop’s. OSPF is used between CE and PE routers.

On PE1, here are all the customer routes connected via Fa1/0

PE1#show ip route vrf Red ospf | in FastEthernet1/0
O IA [110/2] via, 00:23:55, FastEthernet1/0
O [110/101] via, 00:23:55, FastEthernet1/0
O [110/2] via, 00:23:55, FastEthernet1/0

OSPF routes running in VRF Red are redistributed into MP-BGP under “address-family ipv4 vrf Red”.

PE1#show ip bgp vpnv4 rd 65000:1
BGP routing table entry for 65000:1:, version 14
Paths: (1 available, best #1, table Red)
  Advertised to update-groups:
  Local from (
      Origin IGP, metric 2, localpref 100, weight 32768, valid, sourced, best
      Extended Community: RT:65000:1 OSPF DOMAIN ID:0x0005:0x000000010200 
      mpls labels in/out 25/nolabel

Here we can see the MPLS label binding that will be sent to other PE routers. PE routers with a VRF matching the same route targets will import these routes into the VRF of other sites.

In the MPLS LDP Forwarding table, an entry is created for these “local” VRF routes. That is, the routes reachable via the next-hop CE router:

PE1#show mpls forwarding-table vrf Red
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop    
Label  Label or VC   or Tunnel Id      Switched      interface              
25     No Label[V]   0             Fa1/0

This is the label that will be advertised to MP-BGP peers (in this case, reflected to PE2).

PE1 will also have a label binding for its own BGP next-hop IP address, which is the Loopback0 interface under the global routing table:

PE1#show mpls ldp bindings local 32
  lib entry:, rev 4
        local binding:  label: imp-null

This is advertised as an Implicit Null label, to avoid performing two lookups (once in the LFIB and another in the RIB for its connected prefix). Core P routers will have a label binding for this prefix:

CoreP#show mpls ldp bindings local
  lib entry:, rev 14
        local binding:  label: 17

In order for the correct labels to be used for forwarding, two labels will have to be used. The top label will be used to forward packets in the core (P) MPLS network to the BGP next-hop (either the loopback of PE1 or PE2, depending on the packet destination from the CE sites). The bottom label will be used to identify the VRF and outgoing interface to route packets towards the customer router(s).

So, for customer at Site B to reach network at Site A, PE2 will use the following labels:

  • Label 17 for the transport label to PE1, received from MPLS core router(s); identified via RIB lookup in the VRF “Red” to identify next-hop IP address
  • Label 25 for the VPN label, received from PE1 via MP-BGP; identified in the VPNv4 BGP RIB

To verify:

Packet received on Fa1/0 destined for from Site B router(s), performs VRF Red RIB lookup:

PE2#show ip route vrf Red ospf  
Routing Table: Red is variably subnetted, 9 subnets, 3 masks
O IA [110/52] via, 00:46:32

PE2 identifies next-hop IP address, which is the BGP next-hop of PE1. Since it is traversing the MPLS network on the outgoing interface FastEthernet2/0 into the core, it needs to be labeled before transit:

PE2#show ip bgp vpnv4 rd 65000:1
BGP routing table entry for 65000:1:, version 34
Paths: (1 available, best #1, table Red, RIB-failure(17))
  Not advertised to any peer
  Local (metric 20) from (
      Origin IGP, metric 2, localpref 100, valid, internal, best
      Extended Community: RT:65000:1 OSPF DOMAIN ID:0x0005:0x000000010200 
      Originator:, Cluster list:
      mpls labels in/out nolabel/25
PE2#show mpls ldp bindings 
lib entry:, rev 8
        local binding:  label: 17
        remote binding: lsr:, label: 17

Therefore, packets destined for customer Site A will be sent with the labels 17 and 25.

PE2#traceroute vrf Red

Type escape sequence to abort.
Tracing the route to

  1 [MPLS: Labels 17/25 Exp 0] 76 msec 52 msec 72 msec
  2 [MPLS: Label 25 Exp 0] 84 msec 40 msec 40 msec
  3 132 msec *  60 msec

Below I will attempt to illustrate the decision process and relationship between all the entries in an MPLS router to demonstrate these relationships:

  1. An incoming packet from Site B, destined for, is received on PE2’s VRF interface Fa1/0.
  2. IP lookup is performed in the VRF table “Red” and identifies next-hop IP address known via global routing table. This route was redistributed from BGP into OSPF (hence the RIB failure) via PE1 next-hop of
  3. BGP RIB lookup is performed to identify the VPN label. Under the VPNv4 address family, outgoing label is 25, as advertised by PE1
  4. Global RIB lookup is performed for BGP next-hop learned in VRF. Actual IP next hop in the MPLS core is identified ( via outgoing interface FastEthernet2/0.
  5. Outgoing interface is an MPLS-enabled interface. LIB lookup performed to find bound address of the MPLS core next-hop of Based on LDP neighbor that has bound IP address, remote label received from that LDP neighbor is used for transport label to PE1 loopback.
  6. LFIB entry created with Label 17, outgoing interface FastEthernet2/0 with next-hop IP address of into core MPLS network and is routed onto PE1.

Example of “show mpls ldp neighbor” displays bound addresses for core P router(s). LIB entry selected for forwarding in LFIB is based on which LDP neighbor this next-hop IP address in the global RIB is bound to. In this case, only one LDP neighbor exists:

PE2#show mpls ldp neighbor 
    Peer LDP Ident:; Local LDP Ident
        TCP connection: -
        State: Oper; Msgs sent/rcvd: 118/119; Downstream
        Up time: 01:33:30
        LDP discovery sources:
          FastEthernet2/0, Src IP addr:
        Addresses bound to peer LDP Ident:

In an MPLS VPN network, the label bindings received from remote peers (LIB), the label forwarding table (LFIB) and the various IP routing tables (VRF RIB, global RIB, BGP RIB, etc.) all work together in tandem to create the label stack used to forward packets from one VPN site to another. This is the basic forwarding paradigm of Multiprotocol Label Switching and enables service providers to provide L3VPN services to customers along with proper separation of customer routing via the use of VRF’s. References used in this post are Luc De Ghein’s MPLS Fundamentals book from Cisco Press and Cisco documentation, found at http://www.cisco.com/go/mpls.

BGP+MPLS Exam Passed! QoS and other things

Hi All,
I’ve been staying away from the Twitters and blogging to focus down on my BGP+MPLS composite exam. I wrote it this afternoon and passed, w00t! I wanted to give a HUGE thanks to Jarek Rek at his blog hackingcisco.blogspot.com. His labs are great to practice configuring Cisco IP routing and I recommend anyone preparing for CCNP ROUTE, CCIE R&S or anything routing-related to check it out. Thanks again Jarek!

So other than beating my chest, I will be finishing up some outstanding blog posts around my BGP and MPLS studies before moving on to my QOS exam. I’ve also been involved more and more with Juniper at work, along with trying to get up to speed with L2VPN technologies like basic EoMPLS. Metro Ethernet is a whole other rabbit hole that I wish to descend into eventually but at the moment, it’s still a bit of a mystery. It makes keeping up with blogging and goofing off at home challenging since I’m in study mode for CCIP while getting pulled in twenty different directions for real-world job stuff.

I’m currently looking for my next book go to through in prep of my QOS exam. My coworker had recommended Cisco Press’ “End to End QoS Network Design” while most of Learning@Cisco seems to recommend the IP Telephony QOS Exam study guide. That’s still up in the air until I review the exam topics. If anyone has a solid recommendation for 642-642, please let me know in the comments!

Last update, I picked up the newest edition of “TCP/IP Illustrated Volume 1”. Stevens book is often recommended by the experts and is considered the bible of Layers 4 and up. It’s a comprehensive tome and a great reference.

More technical posts coming shortly.

BGP Aggregate Addresses

This month I am studying in preparation for my CCIP BGP+MPLS exam (booked June 27th). I decided to go through with the CCIP certification, despite my annoyance with the new Service Provider track because the BGP, MPLS and QoS topics are covered in length on the CCIE Routing & Switching blueprints. I figure this is a good bridge to fill in the gaps that CCNP R&S leaves out (in particular MPLS and QoS, which isn’t covered at all in any of the CCNP blueprints).

As such, I’ve been able to dive into all the knobs and switches that BGP offers to control routing policy. For those who have gone through the newer CCNP R&S track, the BGP fundamentals are explained and covered enough to get engineers familiarized with its operations. There’s a lot of depth lacking in CCNP and for good reason…BGP can be a career in and of itself. In service provider environments, when you’re pulling half a million IPv4 routes from upstream peers and providing L3VPN services to your customers via MPLS, you need a protocol like BGP that can scale.

Route summarization, when half a million routes are available on the global Internet table, can help keep specific and unnecessary routes from propagating out to upstream providers and thus alleviate memory and CPU required for carrying these thousands of routes. To summarize a set of routes in BGP, you have a few options:

  • Manual static Null0 routes advertised in BGP
  • aggregate-address command

Let’s look at a scenario. This is taken out of a BGP topology I’ve been working on this week to help me gain a better understanding of some of the more advanced BGP topics.


  • for all CE-facing Point-to-Point links
  • for BGP update souce loopbacks
  • for all inter-AS Point-to-Point links
  • allocated to Enterprise A from this ISP
  • allocated to ISP from registry

*Note: this is my best guess of how an ISP would assign addressing in its network. Being an enterprise guy, I’ve yet to be exposed to any service provider network. For those with more experience, any corrections on this please let me know in the comments below 🙂

In this topology, we have one route reflector “RR” with IBGP running between RR and all the PE routers (just PE1 and PE2 in this case).
We want to aggregate all of the ISP’s routes to advertise upstream to Upstream SP at AS 200.

Below is the BGP RIB on our route reflector before any aggregation. These are all the routes advertised by the PE routers as well as any allocations given to customers who require more than a single address.

RR#sh ip bgp
BGP table version is 31, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
* i100.100.200.0/24            0    100      0 65501 i
*>i                   0    100      0 65501 i
*>i100.100.255.0/31            0    100      0 i
*>i100.100.255.8/31            0    100      0 i
AS200#sh ip bgp
BGP table version is 37, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>                          0 400 i
*>                          0 400 i
*>                          0 400 i

As you can see, in a huge service provider network, the BGP RIB would be filled with any public IP addresses used to connect its customers to the outside world, as well as any allocations given by this ISP to its larger customers (such as Enterprise A in this case, which is dual homed at PE1 and PE2). Also included is the BGP RIB of the upstream AS 200 router, who receives these specific prefixes from AS 400.

Now let’s reduce the routing table by aggregating them into a summarized route. First, we’ll start by adding in a static route to the Null0 interface and advertise it in BGP:

! On RR:
conf t
 ip route Null0
router bgp 400
 network mask
RR#sh ip ro static is variably subnetted, 11 subnets, 4 masks
  S is directly connected, Null0
RR#sh ip bgp
BGP table version is 25, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*>                            32768 i
*>i100.100.255.0/31            0    100      0 i
*>i100.100.255.8/31            0    100      0 i

And to verify on the upstream AS:

AS200#sh ip bgp
BGP table version is 31, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*>            0             0 400 i
*>                          0 400 i
*>                          0 400 i
*>                          0 400 i

Since we’re still advertising the more specific routes inside the ISP AS 400, manual filtering will be required on the router reflector. This can be accomplished by a simple prefix list or route-map on RR.

! on RR
ip prefix-list OurAlloc permit 
! Match only our allocated address space
router bgp 400
 neighbor prefix-list OurAlloc out

The problem with this approach is that, while it is fairly simple, does require you to manually filter any more-specific routes on the edge of your AS. Also, if you are serving multihomed customers with their own address allocation (independent of this ISP’s allocation), you will have to take those into account as well in your filtering.

The other way to aggregate a set of routes in BGP is through the aggregate-address command. This command not only creates a Null0 route automatically but also suppresses more-specific routes from the BGP RIB. Using only the aggregate-address summarization, upstream peers will only receive the aggregated route and not the individual more-specific prefixes.

! on RR
router bgp 400
 aggregate-address summary-only
RR#sh ip bgp
BGP table version is 31, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*>                            32768 i
s i100.100.200.0/24            0    100      0 65501 i
s>i                   0    100      0 65501 i
s>i100.100.255.0/31            0    100      0 i
s>i100.100.255.8/31            0    100      0 i

AS200#sh ip bgp
BGP table version is 37, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
          r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight Path
*>            0             0 400 i

As you can see, after using the aggregate-address command on AS 400 RR, only our configured summarized address is advertised out to AS200. You can also see the suppressed routes in RR’s BGP RIB, since we used the “summary-only” parameter in the aggregate-address command. All more-specific routes are suppressed from being advertised to BGP peers thus reducing what used to be many routes to just what’s configured.

Aggregation, combined with proper filtering, should be performed wherever and whenever possible. As of today, CIDR Report indicates over 410,000 routes exist in the global table. With aggregation (as estimated by CIDR Report), as much as half of all the routes in existence today can be aggregated.

CCIP retired, new CCNP Service Provider offers no bridge for current CCNP’s

Last week, Cisco announced the retirement of the Cisco Certified Internetwork Professional, the professional-level Cisco certification for service provider networking:

Retirement of CCIP Certification

Beginning October 29, 2012, Cisco CCIP certification will be retired and Cisco will no longer issue new certifications. Individuals interested in pursuing a professional-level Cisco Service Provider certification are encouraged to obtain the new Cisco CCNP Service Provider certification.


The CCIP certification has been a logical next step for individuals completing their CCNP route/switch certifications. A lot of the topics covered (specifically in ROUTE and the old BSCI) also apply to the service provider routing basics which makes it a logical bridge for many CCNP engineers. Essentials such as BGP, IS-IS (removed from ROUTE) as well as controlling routing information such as route filtering, redistribution, path manipulation, etc, are all covered in length throughout the CCNP and should be very familiar for those coming out of their R&S studies.

With the recent announcement of CCIP’s retirement and with the new SP track, this bridge is no longer possible.

Referring to the new CCNP Service Provider curriculum, the following exams are required for certification under the new track:

  • 642-883 SPROUTE Deploying Cisco Service Provider Network Routing
  • 642-885 SPADVROUTE Deploying Cisco Service Provider Advanced Network Routing
  • 642-887 SPCORE Implementing Cisco Service Provider Next-Generation Core Network Services
  • 642-889 SPEDGE Implementing Cisco Service Provider Next-Generation Egde Network Services

Not to mention also under the new track, a valid CCNA Service Provider is required to be certified for CCNP SP.

For those who have had their CCNP’s for some time, much of the CCNA SP topics in the exam blueprint is pure review. Oh the topics in the CCNA SP blueprint(s) (oh yeah, there’s two $250US exams for CCNA SP) I could only find the following topics that aren’t covered in CCNA/CCNP R&S:

  • Basic IOS-XE & IOS-XR CLI operations and router configurations
  • Transport Technologies such as SONET, SDH, DWDM, ROADM
  • Describe relationship between users, user groups, tasks groups and task IDs in IOS XR
  • Configure Resilient Ethernet Protocol (REP) on Cisco IOS switches
  • Configure QinQ on Cisco IOS switches
  • Carrier-grade NAT (CGN) and NAT64
  • Manage IOS XE and IOS XR software packages

Every other topic that is in both SPNGN1 and SPNGN2 exam blueprints covers CCNA-level routing and switching basics, with a few CCNP-level topics thrown in as well such as GRE tunnels and First Hop Redundancy Protocols, amongst others.

What this means is CCNP engineers must now commit both the time and money to acquire the CCNA SP certification before even attempting a CCNP SP certification. With so much overlap in the R&S certification track, it’s a real wonder why Cisco didn’t think to have some sort of bridge exam to get CCNP’s up to speed in preparation for CCNP SP material.

While it’s not completely unexpected that this track got a significant facelift; after all, covering just QoS, BGP and MPLS leaves out a lot of Cisco’s product-centric features such as promoting their IOS XR routing platforms, it has now put a deadline for those considering CCIP studies. And while Cisco has provided a migration for existing CCIP’s, it still puts prospective certified engineers in a crappy spot. What as now 3 exams (2 if you took the BGP+MPLS composite) are 4 brand new exams with nothing but instructor-led training courses…which are also not cheap.

So for those CCNP network admins who are considering Cisco’s service provider certification tracks, here are the exams (with associated costs) required to reach professional-level certification:

For CCIP (last day to certify October 29, 2012):

  • 642-902 Implementing Cisco IP Routing (ROUTE): $200 USD
  • 642-642 Quality of Service (QOS): $200 USD
  • 642-611 Implementing Cisco MPLS (MPLS): $200 USD Last day to test July 27, 2012
  • 642-661 Configuring BGP on Cisco Routers (BGP): $200 USDLast day to test July 27, 2012

Taking into account that you should have ROUTE or BSCI through your CCNP certification, you are looking at a total of 3 exams at $600 USD total*.

*note: You can also taken composite exam 642-691 BGP+MPLS, also last day to test July 27th this year, making total exams 2 at $400 USD.

Now, if you’re interested in going down Cisco’s new SP track, you’ll be starting at ground-zero with CCNA-Service Provider before moving into CCNP SP:

For CCNA-Service Provider:

  • 640-875 Building Cisco Service Provider Next-Generation Networks, Part 1: $250 USD
  • 640-878 Building Cisco Service Provider Next-Generation Networks, Part 2: $250 USD

For CCNP-Service Provider:

  • 642-883 Deploying Cisco Service Provider Network Routing: $200 USD
  • 642-885 Deploying Cisco Service Provider Advanced Network Routing: $200 USD
  • 642-887 Implementing Cisco Service Provider Next Generation Core Network Services: $200 USD
  • 642-889 Implementing Cisco Service Provider Next Generation Edge Network Services: $200 USD

Cisco is offering a credit towards 642-883 for those who have completed the ROUTE exam (which is part of CCNP). Other than that, with the new SP track, you are tasked with a total of 5 exams, totaling a cost of $1100 USD ($500 for CCNA-SP, $600 for CCNP-SP).

This year will be a rough transition for those looking to enter the service provider track. Actual CCIP certification will only be available until October 29th of this year, with both the BGP and MPLS core exams being retired by the end of June. And for those looking to go down the newly released track, training material will be sparse and not as widely available as those offered by Cisco Press covering BGP and MPLS.

In any case, it’s something that I’m passion about and will be looking forward to seeing developments with these certifications. Since I obtained my CCNP last year, I’ve always been interested in large service provider networks so naturally I’ve gravitated towards this side of Cisco certification. I also hold a lot of respect for the titans of our industry, namely Ivan Pepelnjak, for their deep knowledge of MPLS, BGP and everything routing.

In the meantime, I’ll be looking to crank out some blog posts to compliment my studies currently with MPLS. For those looking to learn more about the protocol, I must recommend Luc De Ghein’s fantastic book MPLS Fundamentals. Further on my reading list is Ivan’s book MPLS & VPN Architectures, Sam Halabi’s Internet Routing Architectures and Randy Zhang & Micah Bartell’s BGP Design and Implementation. If I can fit a certification in between all that information, that’ll be fantastic. However, with Cisco’s current and newly updated SP track, it’s not as high on my agenda. I doubt it’s high on other network pros’ to-do lists either.

UPDATE: After making a similar post on the Cisco Learning Network forum, I received a reply from Rigo the Community Manger, who explained that a valid CCIP certification can be used as a prerequisite towards the new CCNP SP certifications. CCNA Service Provider is not required if you already have a valid CCIP. This makes things a little easier this year while CCIP is still being issued new certifications (until October 29th this year).

CCO Gems: Cisco Aironet Antennas and Accessories Reference Guide

In an attempt to wrap my head around a lot of the Layer 1 details in the Wireless LAN world, I came across the following doc on Cisco’s website under their wireless antennas product info:

Cisco Aironet Antennas and Accessories Reference Guide

One of the most important units of measure to understand when working with RF is the decibel (dB). While this unit is used in many other fields such as sound and audio, this doc presents it in a clear and concise manner as it relates to WLAN’s:

The decibel (dB) scale is a logarithmic scale used to denote the ratio of one power value to another.
For example:
X1`dB = 10 log10 (Power A/Power B)
An increase of 3 dB indicates a doubling (2x) of power. An increase of 6 dB indicates a quadrupling (4x) of power. Conversely, a decrease of 3 dB reduces power by one half, and a decrease of 6 dB results in a one fourth of the power.





0 dB

1 x (same)

0 dB

1 x (same)

1 dB

1.25 x

-1 dB

0.8 x

3 dB

2 x

-3 dB

0.5 x

6 dB

4 x

-6 dB

0.25 x

10 dB

10 x

-10 dB

0.10 x

12 dB

16 x

-12 dB

0.06 x

20 dB

100 x

-20 dB

0.01 x

30 dB

1000 x

-30 dB

0.001 x

40 dB

10,000 x

-40 dB

0.0001 x

If you’re a newbie like me, this makes a lot more sense than some of the other documentation I’ve read on the subject, especially with the table above that clearly shows how dB increases logarithmically.

Now that we know a decibel is a relative value, we need to know what values are being measured and referenced. In the wireless world, those two values are Received Signal Strength Indicator or RSSI, and Signal-To-Noise ratio (SNR).

RSSI is a vendor-specific grading of received signal strength. Because it is vendor-specific and not a standard measure, these values cannot be used to compare between vendors. It is usually measured in decibel milliwatt (dBm), which is basically the ratio of power referenced to 1 milliwatt. For example, 3 dBm is 2mW of power. Again, the Cisco Aironet Antenna reference guide provides a good chart for common dBm values to give you an idea of how dBm scales in reference to wattage.

Table 2. Common mW Values to dBm Values





0 dBm

1 mW

0 dBm

1 mW

1 dBm

1.25 mW

-1 dBm

0.8 mW

3 dBm

2 mW

-3 dBm

0.5 mW

6 dBm

4 mW

-6 dBm

0.25 mW

7 dBm

5 mW

-7 dBm

0.20 mW

10 dBm

10 mW

-10 dBm

0.10 mW

12 dBm

16 mW

-12 dBm

0.06 mW

13 dBm

20 mW

-13 dBm

0.05 mW

15 dBm

32 mW

-15 dBm

0.03 mW

17 dBm

50 mW

-17 dBm

0.02 mw

20 dBm

100 mW

-20 dBm

0.01 mW

30 dBm

1000 mW (1 W)

-30 dBm

0.001 mW

40 dBm

10,000 mW (10 W)

-40 dBm

0.0001 mW

Signal-To-Noise ratio is defined as the power ratio between a signal, such as a WLAN waveform, and the background noise. Again, we’re measuring in decibels since it is the ratio of the RSSI and the surrounding garbage RF noise.

Since I’m coming from a wired world where everything is essentially plug-and-play, I’m working from the ground up at Layer 1 to hone my WLAN skills.
Anyways, the Cisco Aironet document here gives a very succinct overview of various L1 WLAN concepts such as 802.11 modulation techniques, antennas ratings and specs, understanding RF power levels, as well as a slew of other little details. The first 1/3 of the document gives a nice overview of these concepts, while the other 2/3’s list all of Cisco’s Aironet antenna products.

This is one of those gems on Cisco’s site that I’m surely packing the PDF away for future reference (especially for my Cisco wireless studies).