Methods for connecting FEXs

If you’re wondering what an FEX is, check out this article: http://www.cciezone.com/?p=231.

FEXs are connected to the parent switch(es) via fabric interfaces.  These are 10Gbps interfaces which connect the two switches (think of a stacking cable on a 3750-series switch).

There are two methods of configuring fabric interfaces:

  • Static pinning
  • EtherChannels

Although these are somewhat differentiated in some of the documentation, the static pinning looks almost the same as that used for EtherChannels.  The main difference is that static pinning uses physical interfaces, whereas EtherChannel uses a single Port-Channel interface for the fabric interface (there are some minor configuration differences and requirements as shown below).

Take a look at two fabric interface configuration examples.  The first is using an EtherChannel, the second capture is using static pinning.

EtherChannel

The following diagram shows a representation of the connectivity and configuration of using an EtherChannel for the fabric interface.  We’re only using two 10Gbps interfaces in this example, but we could use up to four 10Gbps interfaces (using all of the 10Gbps interfaces on the N2148T).

Nexus 5000/2000 EtherChannel Fabric Interface Example

Nexus 5000/2000 EtherChannel Fabric Interface Example

EtherChannel fabric interface configuration example:

!
fex 100
  pinning max-links 1
  description First_Nexus_2148T
!
interface port-channel100
  switchport mode fex-fabric
  fex associate 100

interface Ethernet1/19
  switchport mode fex-fabric
  fex associate 100
  channel-group 100

interface Ethernet1/20
  switchport mode fex-fabric
  fex associate 100
  channel-group 100
!

Now let’s look at the show fex output:

N5K# show fex 100
FEX: 100 Description: First_Nexus_2148T   state: Online
  FEX version: 4.0(1a)N2(1a) [Switch version: 4.0(1a)N2(1a)]
  Extender Model: N2K-C2148T-1GE, Extender Serial: PRIVATE_SERIAL
  Part No: 73-12009-05
 pinning-mode: static    Max-links: 1
  Fabric port for control traffic: Eth1/19
  Fabric interface state:
    Po100 - Interface Up. State: Configured
    Eth1/19 - Interface Up. State: Active
    Eth1/20 - Interface Up. State: Active
N5K#

Static Pinning

The following diagram shows a representation of the static pinning fabric interface example:

Nexus 5000/2000 Static Pinning Fabric Interface Example

Nexus 5000/2000 Static Pinning Fabric Interface Example

Here’s a sample static pinning fabric interfaces configuration:

!
fex 101
  pinning max-links 2
  description Non_PC_Fex_Config
  type "Nexus 2148T"
!
interface Ethernet1/19
  switchport mode fex-fabric
  fex associate 101

interface Ethernet1/20
  switchport mode fex-fabric
  fex associate 101
!

Now let’s look at the show fex output:

N5K# show fex 101
FEX: 100 Description: First_Nexus_2148T   state: Online
  FEX version: 4.0(1a)N2(1a) [Switch version: 4.0(1a)N2(1a)]
  Extender Model: N2K-C2148T-1GE, Extender Serial: PRIVATE_SERIAL
  Part No: 73-12009-05
 pinning-mode: static    Max-links: 2
  Fabric port for control traffic: Eth1/19
  Fabric interface state:
    Eth1/19 - Interface Up. State: Active
    Eth1/20 - Interface Up. State: Active
N5K#

Summary

Notice that in the top example, we’re using an EtherChannel, yet the physical interfaces in the EtherChannel bundle are shown.  Note also that the pinning-mode for both examples is listed as static.  Whether we use an EtherChannel or physical interfaces (static pinning), the pinning-mode is always static.  This is why I mentioned that they both look virtually identical.

There are some definite requirements for using an EtherChannel for FEXs, such as the FEX parameter pinning max-links must be set to 1.

Conclusion

This is pretty basic article, but it shows some of the differences between the methods of configuring fabric interfaces between a Nexus 5000-series and a Nexus 2148T.  In some future articles we’re going to explore some of the behaviors of NX-OS with FEXs.

Related posts:

  1. The FEX information is cached until it’s re-assigned to a different FEX Continuing our journey into the world of FEXs on the...
  2. What happens when the wrong serial number is entered for the FEX? Continuing on the journey of FEXs on the Nexus platform,...
  3. Experimenting with Static Pinning What happens when more links are associated with an FEX...
  4. Why EtherChannels should be used for FEX interfaces I prefer to use port-channel interfaces for the fabric interfaces...
  5. EtherChannel behavior in NX-OS I’ve had a lot of discussions with clients about the...

Related posts brought to you by Yet Another Related Posts Plugin.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Leave a Reply