A Primer on Some of the Differences Between IOS and NX-OS

There are many similarities between IOS and NX-OS, however there are some significant differences as well. Before delving further into this topic, here’s what I was working with: NX-OS 4.0(1a)N1(1) (Nexus 5000) and IOS 12.4(17). This article gives a brief summary of the some of the differences (and similarities) of NX-OS and IOS.

CIDR Notation

To start with, IOS does not support CIDR notation (with the exception of IP prefix-lists), however NX-OS gladly accepts CIDR notation (in fact this is the preferred notation within NX-OS). Note the following example, how NX-OS allows for either the standard IP-and-wildcard-mask notation or allows for IP/len (CIDR) notation:

switch(config-acl)# permit ip ?
  A.B.C.D      Source network address
  A.B.C.D/LEN  Source network prefix
  any          Any source address
  host         A single source host

switch(config-acl)# permit ip

Compare the above output with the output from the same command within IOS:

r1(config-ext-nacl)#permit ip ?
  A.B.C.D  Source address
  any      Any source host
  host     A single source host

r1(config-ext-nacl)#permit ip

Although the above examples are showing traffic filtering/route filtering, CIDR notation is used pretty much everywhere within NX-OS. For instance, interface IP addresses, can be specified in CIDR notation (the legacy address/mask format is also accepted in NX-OS).

What About IP Prefix-Lists?

On the Nexus 5000, IP prefix-lists are not supported in NX-OS. Based on some CCO docs I’ve seen, IP prefix-lists are supported within NX-OS on the Nexus 7000. This makes sense, as these are used for route filtering, not for filtering transit traffic, and the Nexus 5000 doesn’t have a rich L3 feature-set (none really, except to support remote management). IOS of course supports our beloved IP prefix-lists.

No more standard, extended ACLs…

Yep, that’s right – NX-OS gets rid of the idea of standard and extended (as well as numbered) ACLs. All ACLs are named, as well as all are extended ACLs (in their behavior). Here’s a capture on NX-OS:

switch(config)# ip access-list ?
  WORD  List name

switch(config)# access-list ?
                 ^
% invalid command detected at '^' marker.
switch(config)# access-list

Note that there’s no area for numbers (you could use a number as the name, although it would still act as a named ACL):

switch(config)# ip access-list 7
switch(config-acl)#

Contrast the above behavior with IOS:

r1(config)#ip access-list ?
  extended    Extended Access List
  log-update  Control access list log updates
  logging     Control access list logging
  resequence  Resequence Access List
  standard    Standard Access List

r1(config)#access-list ?
  <1-99>            IP standard access list
  <100-199>         IP extended access list
  <1100-1199>       Extended 48-bit MAC address access list
  <1300-1999>       IP standard access list (expanded range)
  <200-299>         Protocol type-code access list
  <2000-2699>       IP extended access list (expanded range)
  <700-799>         48-bit MAC address access list
  dynamic-extended  Extend the dynamic ACL absolute timer
  rate-limit        Simple rate-limit specific access list

r1(config)#access-list

ACL management is greatly simplified in NX-OS, in that we have ONE type of ACL – named ACLs.

ACL re-sequencing

IOS allows us to re-number (re-sequence) named ACLs, which can be handy. NX-OS also supports this, although it’s performed differently than we’re familiar with on IOS. At first glance, I thought that NX-OS didn’t allow for re-sequencing of ACLs:

switch(config)# ip access-list ?
  WORD  List name

switch(config)# ip access-list 7 ?

switch(config)# ip access-list 7
switch(config-acl)# ?
  <1-4294967295>  Sequence number
  deny            Specify packets to reject
  exit            Exit from command interpreter
  no              Negate a command or set its defaults
  permit          Specify packets to forward
  remark          Access list entry comment
  statistics      Enable statistics for the ACL

switch(config-acl)#

Compare this with IOS:

r1(config)#ip access-list ?
  extended    Extended Access List
  log-update  Control access list log updates
  logging     Control access list logging
  resequence  Resequence Access List
  standard    Standard Access List

r1(config)#

At first it looked like we couldn’t resequence ACLs in NX-OS — oh no! Well, looking into it a bit further, I found the following:

switch(config)# resequence ?
  ip    Configure IP features
  ipv6  Configure IPv6 features
  mac   MAC configuration commands

switch(config)# resequence ip access-list ?
  WORD  List name

switch(config)# resequence ip access-list

We do have feature parity between NX-OS and IOS in re-sequencing ACLs (although the commands differ – no big deal as long as we’re aware of the differences!).

Difference in Wildcard Mask Behavior

For sometime wildcard masks have offered power to those who understand them and plagued those who do not. In IOS, if you wanted to match the 1st and 3rd octets and ensure that the 2nd octet was always odd, you could use the following ACL:

r1(config)#ip access-list extended acl1
r1(config-ext-nacl)#permit ip 10.11.10.0 0.1.0.255 any
r1(config-ext-nacl)#

Let’s try this in NX-OS:

switch(config)# ip access-list acl1
switch(config-acl)# permit ip 10.11.10.0 0.1.0.255 any
ERROR: Given ACL rule is invalid
switch(config-acl)#

Oops! It looks like NX-OS will not allow non-contiguous wildcard masks anymore. But wait, let’s try a contiguous wildcard mask and see what happens:

switch(config-acl)# permit ip 10.11.10.0 0.0.3.255 any
switch(config-acl)#

Okay, suspicion confirmed! I looked at the NX-OS command reference and did not find anything stating that the permit statement only accepts wildcard masks that are contiguous, however the above tinkering seems to indicate that this is the case.

This makes sense when you look at the ACL – notice how NX-OS stored the prefixes:

switch(config-acl)# sh ip access-list acl1

IP access list acl1
        10 permit ip 10.11.10.0/16 any
        20 permit ip 10.11.10.0/22 any
switch(config-acl)#

Note how all of the entries (which were entered using a wildcard mask) were converted to CIDR notation? Okay – things are fitting together as to why non-contiguous masks are not supported (even though I couldn’t find this stated as such in the NX-OS Command Reference – it could be there, but I didn’t see it).

Lack of “do” to Run Exec-Level Commands
In IOS, within config mode, the “do” command must be pre-pended to any exec-level command you want to issue:

r1(config)#do sh ip access-list acl1
Extended IP access list acl1
    10 permit ip 10.10.10.0 0.1.0.255 any
r1(config)#

In NX-OS, this is removed – exec-level commands may be run from within config mode without any special command:

switch(config-acl)# sh ip access-list acl1

IP access list acl1
        10 permit ip 10.11.10.0/16 any
        20 permit ip 10.11.10.0/22 any
switch(config-acl)#

Pretty slick, huh?

Where’s My GigabitEthernet Interfaces?

Good question! NX-OS goes the way of some other vendors in simplifying the interface naming conventions, removing the interface speed from the interface name. There are no longer any FastEthernet, GigabitEthernet or TenGigabitEthernet interfaces on NX-OS. All interfaces are Ethernet interfaces.

It’s still possible to specify 1Gbps or 10Gbps on the interface, just that the interface name itself won’t reflect the change in speed.

switch(config)# sh int brief

--------------------------------------------------------------------------------
Ethernet      VLAN   Type Mode   Status  Reason                   Speed     Port
Interface                                                                   Ch #
--------------------------------------------------------------------------------
Eth1/1        1      eth  access down    SFP validation failed       10G(D) --
Eth1/2        1      eth  access down    SFP not inserted            10G(D) --
Eth1/3        1      eth  access down    SFP validation failed       10G(D) --
Eth1/4        1      eth  access down    SFP not inserted            10G(D) --
Eth1/5        1      eth  access down    SFP validation failed       10G(D) --
Eth1/6        1      eth  access down    SFP not inserted            10G(D) --
Eth1/7        1      eth  access down    SFP validation failed       10G(D) --
Eth1/8        1      eth  access down    SFP not inserted            10G(D) --
Eth1/9        1      eth  access down    SFP not inserted            10G(D) --
Eth1/10       1      eth  access down    SFP not inserted            10G(D) --
Eth1/11       1      eth  access down    SFP not inserted            10G(D) --
Eth1/12       1      eth  access down    SFP not inserted            10G(D) --
Eth1/13       1      eth  access down    SFP not inserted            10G(D) --
Eth1/14       1      eth  access down    SFP not inserted            10G(D) --
Eth1/15       1      eth  access down    SFP not inserted            10G(D) --
Eth1/16       1      eth  access down    SFP not inserted            10G(D) --
Eth1/17       1      eth  access down    SFP not inserted            10G(D) --
Eth1/18       1      eth  access down    SFP not inserted            10G(D) --
Eth1/19       1      eth  access down    SFP not inserted            10G(D) --
Eth1/20       1      eth  access down    SFP not inserted            10G(D) --
Eth2/1        1      eth  access down    SFP not inserted            10G(D) --
Eth2/2        1      eth  access down    SFP not inserted            10G(D) --
Eth2/3        1      eth  access down    SFP not inserted            10G(D) --
Eth2/4        1      eth  access down    SFP not inserted            10G(D) --
Eth2/5        1      eth  access down    SFP not inserted            10G(D) --
Eth2/6        1      eth  access down    SFP not inserted            10G(D) --

--------------------------------------------------------------------------------
Port   VRF          Status IP Address                              Speed    MTU
--------------------------------------------------------------------------------
mgmt0  --           down   10.1.1.53                               --       1500
switch(config)# int e1/1
switch(config-if)# speed 1000
switch(config-if)# sh int brief

--------------------------------------------------------------------------------
Ethernet      VLAN   Type Mode   Status  Reason                   Speed     Port
Interface                                                                   Ch #
--------------------------------------------------------------------------------
Eth1/1        1      eth  access down    Link not connected         1000(D) --
Eth1/2        1      eth  access down    SFP not inserted            10G(D) --
Eth1/3        1      eth  access down    SFP validation failed       10G(D) --
Eth1/4        1      eth  access down    SFP not inserted            10G(D) --
Eth1/5        1      eth  access down    SFP validation failed       10G(D) --
Eth1/6        1      eth  access down    SFP not inserted            10G(D) --
Eth1/7        1      eth  access down    SFP validation failed       10G(D) --
Eth1/8        1      eth  access down    SFP not inserted            10G(D) --
Eth1/9        1      eth  access down    SFP not inserted            10G(D) --
Eth1/10       1      eth  access down    SFP not inserted            10G(D) --
Eth1/11       1      eth  access down    SFP not inserted            10G(D) --
Eth1/12       1      eth  access down    SFP not inserted            10G(D) --
Eth1/13       1      eth  access down    SFP not inserted            10G(D) --
Eth1/14       1      eth  access down    SFP not inserted            10G(D) --
Eth1/15       1      eth  access down    SFP not inserted            10G(D) --
Eth1/16       1      eth  access down    SFP not inserted            10G(D) --
Eth1/17       1      eth  access down    SFP not inserted            10G(D) --
Eth1/18       1      eth  access down    SFP not inserted            10G(D) --
Eth1/19       1      eth  access down    SFP not inserted            10G(D) --
Eth1/20       1      eth  access down    SFP not inserted            10G(D) --
Eth2/1        1      eth  access down    SFP not inserted            10G(D) --
Eth2/2        1      eth  access down    SFP not inserted            10G(D) --
Eth2/3        1      eth  access down    SFP not inserted            10G(D) --
Eth2/4        1      eth  access down    SFP not inserted            10G(D) --
Eth2/5        1      eth  access down    SFP not inserted            10G(D) --
Eth2/6        1      eth  access down    SFP not inserted            10G(D) --

--------------------------------------------------------------------------------
Port   VRF          Status IP Address                              Speed    MTU
--------------------------------------------------------------------------------
mgmt0  --           down   10.1.1.53                               --       1500
switch(config-if)#

Notice how the 1Gbps SFP is no longer failing validation on Eth1/1? This is another possible gotcha – it didn’t auto-sense the SFP speed and adjust the port speed. Looking at Eth1/3, it can be seen that the default port speed is 10Gbps (and that it doesn’t auto-detect/sense the speed):

switch(config-if)# sh run int eth1/3
version 4.0(1a)N1(1)

interface Ethernet1/3

switch(config-if)# sh int e1/3
Ethernet1/3 is down (SFP validation failed)
  Hardware is 1000/10000 Ethernet, address is 000d.ecd0.bbca (bia 000d.ecd0.bbca)
  MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  Port mode is access
  auto-duplex, 10 Gb/s, media type is 10g
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned on
  Rate mode is dedicated
  Switchport monitor is off
  Last clearing of "show interface" counters never
  5 minute input rate 0 bytes/sec, 0 packets/sec
  5 minute output rate 0 bytes/sec, 0 packets/sec
  Rx
    0 input packets 0 unicast packets 0 multicast packets
    0 broadcast packets 0 jumbo packets 0 storm suppression packets
    0 bytes
    0 No buffer 0 runt 0 Overrun
    0 crc 0 Ignored 0 Bad etype drop
    0 Bad proto drop
  Tx
    0 output packets 0 multicast packets
    0 broadcast packets 0 jumbo packets
    0 bytes
    0 output CRC 0 ecc
    0 underrun 0 if down drop     0 output error 0 collision 0 deferred
    0 late collision 0 lost carrier 0 no carrier
    0 babble
    0 Rx pause 0 Tx pause 0 reset

switch(config-if)#

STP PortFast

NX-OS does not accept the typical IOS spanning-tree portfast command that we’re so familiar with. Instead, NX-OS adopts the IEEE 802.1D terminology (finally!), in that you specify the port type and its behavior matches the standards-defined behavior.

To configure a port as an edge port (essentially enable PortFast behavior on the port) in NX-OS:

switch(config-if)# spanning-tree port type edge
Warning: portfast should only be enabled on ports connected to a single
 host. Connecting hubs, concentrators, switches, bridges, etc... to this
 interface  when portfast is enabled, can cause temporary bridging loops.
 Use with CAUTION

%Portfast has been configured on Ethernet1/1 but will only
 have effect when the interface is in a non-trunking mode.
switch(config-if)#

Note that we’re still able to enable PortFast on trunk ports in NX-OS:

switch(config-if)# spanning-tree port type edge ?

  trunk  Consider the interface as edge port (enable portfast) even in trunk
         mode 

switch(config-if)# spanning-tree port type edge

I’m glad to see this change in NX-OS. Cisco forged ahead with so many improvements with PVST+ in IOS when the rest of the industry was stuck in legacy STP (802.1D) only. Many of these improvements were standardized (thanks to Cisco!) in the IEEE 802.1w standard (which has been merged into the IEEE 802.1D standard). It’s nice to see that NX-OS is dropping the Cisco-proprietary terminology and using the standards-based terminology.

Watch Those Features

NX-OS is very modular – much more so than even IOS Modularity (Modular IOS). Many protocols (called features within NX-OS) are run within their own protected memory space. This is really good in that it can help mitigate one protocol/feature from negatively effecting another feature. The downside is that these features are not enabled by default – they have to be manually enabled (very much like a service on other OSs). After they’re enabled (started), this is placed in the configuration, so that they don’t have to be enabled each time the Nexus is power-cycled.

Each feature is enabled/disabled using the feature command. To enable a feature, use the feature , to disable, use the no feature command.

What is considered a feature (and must be manually enabled) on NX-OS? Well, this differs between the platforms (because of the difference in feature-sets). The following shows the features on the Nexus 5000:

switch(config)# feature ?
  fcoe            Enable feature
  interface-vlan  Enable/Disable interface vlan
  lacp            Enable/Disable LACP
  private-vlan    Enable/Disable private-vlan
  tacacs+         Enable/Disable tacacs+
  udld            Enable/Disable UDLD

switch(config)#

The following is taken from a Nexus 7000:

n7k (config)# feature ?
  bgp             Enable/Disable Border Gateway Protocol (BGP)
  cts             Enable/Disable CTS
  dhcp            Enable/Disable DHCP Snooping
  dot1x           Enable/Disable dot1x
  eigrp           Enable/Disable Enhanced Interior Gateway Routing Protocol
                  (EIGRP)
  eou             Enable/Disable eou(l2nac)
  glbp            Enable/Disable Gateway Load Balancing Protocol (GLBP)
  hsrp            Enable/Disable Hot Standby Router Protocol (HSRP)
  interface-vlan  Enable/Disable interface vlan
  isis            Enable/Disable IS-IS Unicast Routing Protocol (IS-IS)
  lacp            Enable/Disable LACP
  msdp            Enable/Disable Multicast Source Discovery Protocol (MSDP)
  netflow         Enable/Disable NetFlow
  ospf            Enable/Disable Open Shortest Path First Protocol (OSPF)
  ospfv3          Enable/Disable Open Shortest Path First Version 3 Protocol
                  (OSPFv3)
  pbr             Enable/Disable Policy Based Routing(PBR)
  pim             Enable/Disable Protocol Independent Multicast (PIM)
  pim6            Enable/Disable Protocol Independent Multicast (PIM) for IPv6
  port-security   Enable/Disable port-security
  private-vlan    Enable/Disable private-vlan
  rip             Enable/Disable Routing Information Protocol (RIP)
  scheduler       Config commands for scheduler
  tacacs+         Enable/Disable tacacs+
  udld            Enable/Disable UDLD
  vrrp            Enable/Disable Virtual Router Redundancy Protocol (VRRP)

n7k(config)# feature

By default all features are disabled (not enabled). How can you tell which features are enabled? Well, I’ve looked through many CCO docs and have seen mention of the show feature command, but have not found it listed in any of the NX-OS command references (for the Nexus 5000 or Nexus 7000).

When I tried it on the Nexus 5000, it didn’t work:

switch# show feature
              ^
% invalid command detected at '^' marker.
switch#

Also, when trying it on the Nexus 7000, it doesn’t work:

n7k# show feature ?
                     ^
% invalid command detected at '^' marker.
n7k#

So although some of the Cisco NX-OS config guides (for the N5k and N7k) indicate that this command exists and can be optionally used, I can’t seem to get it to work. Not finding it in the NX-OS command references (for the N5k and N7k) seem to confirm that this command doesn’t yet exist (hopefully we’ll see it in the future?).

Back to the original question – how do we find out which services are enabled? Take a look at the running config and that will tell you:

switch#
switch# conf t
switch(config)# feature lacp
switch(config)# sh run | i feature
feature lacp
switch(config)#

NX-OS is pretty intelligent. In certain instances, if you try to look at a disabled feature, for example, it will tell you that the feature isn’t enabled:

switch(config)# show fcoe
FCoE/FC feature is not desired.
switch(config)#

Other times, the command tree structure itself will disappear from the CLI:

switch(config)# no feature lacp
switch(config)# sh lacp ?
                    ^
% invalid command detected at '^' marker.
switch(config)# sh lacp
switch(config)#
switch(config)# feature lacp
switch(config)# sh lacp ?
No matches in current mode, matching in (exec) mode
  counters           LACP counters
  interface          Specify a interface
  internal           Show lacp service internal status
  neighbor           LACP interface neighbor
  port-channel       LACP port-channels
  system-identifier  Show system-identifier information

switch(config)# sh lacp

So, sometimes NX-OS will tell you that the feature is not enabled, while other times the protocol/feature just appears to be missing altogether. Either way, ensuring that you’re aware of the features enabled and required for the environment is critical.

Some features do not require enablement before being able to configure certain aspects of the feature. FCoE is a great example – it’s possible to go into an interface and enable FCoE on the interface (in this example, using mode on) even when the fcoe feature is not enabled. Without the feature being enabled, there won’t be any FCoE support provided, but it can be pre-configured.

One last note about features: there are many features that I consider “core” features which must be enabled if required. For instance, if your AAA design incorporates the usage of TACACS+, NX-OS considers this a feature and therefore it must be explicitly enabled. Failure to enable it will cause woes. In this example, TACACS+ isn’t even available within the AAA commands:

switch(config)# aaa group server ?
  radius  RADIUS server group name

switch(config)# aaa group server
switch(config)#
switch(config)# feature tacacs
switch(config)# aaa group server ?
  radius   RADIUS server group name
  tacacs+  TACACS+ server group name

switch(config)# aaa group server

Any golden configs which are ported from IOS-based systems to NX-OS must recognize and incorporate features into the configuration steps, as failure to recognize this will result in a possible loss of available commands (because the feature isn’t enabled).

Conclusion

I like NX-OS, however there are some differences (some of which have been outlined here) that we’ll all need to get used to if we’re to be proficient in NX-OS. This article is a very high-level overview of just a few of the differences. Using NX-OS on the Nexus 7000 yields far greater differences (around routing protocol configuration, etc.).

I suspect that we’ll find that using ACLs as route filters might not be as useful in NX-OS as we can’t use non-contiguous wildcard masks. Then again, this is typically needed in the real world.

Overall, I think the NX-OS changes are good and will make life easier. If you get a chance, give NX-OS a chance!

Related posts:

  1. Nexus (NX-OS) protocol support (and cleanup) Over the years, Cisco has been very instrumental in the...
  2. Missing VLANs are not automatically created in NX-OS In IOS, if we assign a switchport to a non-existent...
  3. Some NX-OS features can’t be manually enabled The whole idea around features in NX-OS has been intriguing...
  4. Introducing the Nexus 2148T (Fabric Extenders) The Nexus 5000-series (5010 and 5020) support the Nexus 2000-series...
  5. Methods for connecting FEXs If you’re wondering what an FEX is, check out this...

Related posts brought to you by Yet Another Related Posts Plugin.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Leave a Reply