A Primer on Some of the Differences Between IOS and NX-OS
There are many similarities between IOS and NX-OS, however there are some significant differences as well. Before delving further into this topic, here’s what I was working with: NX-OS 4.0(1a)N1(1) (Nexus 5000) and IOS 12.4(17). This article gives a brief summary of the some of the differences (and similarities) of NX-OS and IOS.
CIDR Notation
To start with, IOS does not support CIDR notation (with the exception of IP prefix-lists), however NX-OS gladly accepts CIDR notation (in fact this is the preferred notation within NX-OS). Note the following example, how NX-OS allows for either the standard IP-and-wildcard-mask notation or allows for IP/len (CIDR) notation:
switch(config-acl)# permit ip ? A.B.C.D Source network address A.B.C.D/LEN Source network prefix any Any source address host A single source host switch(config-acl)# permit ip
Compare the above output with the output from the same command within IOS:
r1(config-ext-nacl)#permit ip ? A.B.C.D Source address any Any source host host A single source host r1(config-ext-nacl)#permit ip
Although the above examples are showing traffic filtering/route filtering, CIDR notation is used pretty much everywhere within NX-OS. For instance, interface IP addresses, can be specified in CIDR notation (the legacy address/mask format is also accepted in NX-OS).
What About IP Prefix-Lists?
On the Nexus 5000, IP prefix-lists are not supported in NX-OS. Based on some CCO docs I’ve seen, IP prefix-lists are supported within NX-OS on the Nexus 7000. This makes sense, as these are used for route filtering, not for filtering transit traffic, and the Nexus 5000 doesn’t have a rich L3 feature-set (none really, except to support remote management). IOS of course supports our beloved IP prefix-lists.
No more standard, extended ACLs…
Yep, that’s right – NX-OS gets rid of the idea of standard and extended (as well as numbered) ACLs. All ACLs are named, as well as all are extended ACLs (in their behavior). Here’s a capture on NX-OS:
switch(config)# ip access-list ?
WORD List name
switch(config)# access-list ?
^
% invalid command detected at '^' marker.
switch(config)# access-list
Note that there’s no area for numbers (you could use a number as the name, although it would still act as a named ACL):
switch(config)# ip access-list 7 switch(config-acl)#
Contrast the above behavior with IOS:
r1(config)#ip access-list ? extended Extended Access List log-update Control access list log updates logging Control access list logging resequence Resequence Access List standard Standard Access List r1(config)#access-list ? <1-99> IP standard access list <100-199> IP extended access list <1100-1199> Extended 48-bit MAC address access list <1300-1999> IP standard access list (expanded range) <200-299> Protocol type-code access list <2000-2699> IP extended access list (expanded range) <700-799> 48-bit MAC address access list dynamic-extended Extend the dynamic ACL absolute timer rate-limit Simple rate-limit specific access list r1(config)#access-list
ACL management is greatly simplified in NX-OS, in that we have ONE type of ACL – named ACLs.
ACL re-sequencing
IOS allows us to re-number (re-sequence) named ACLs, which can be handy. NX-OS also supports this, although it’s performed differently than we’re familiar with on IOS. At first glance, I thought that NX-OS didn’t allow for re-sequencing of ACLs:
switch(config)# ip access-list ? WORD List name switch(config)# ip access-list 7 ? switch(config)# ip access-list 7 switch(config-acl)# ? <1-4294967295> Sequence number deny Specify packets to reject exit Exit from command interpreter no Negate a command or set its defaults permit Specify packets to forward remark Access list entry comment statistics Enable statistics for the ACL switch(config-acl)#
Compare this with IOS:
r1(config)#ip access-list ? extended Extended Access List log-update Control access list log updates logging Control access list logging resequence Resequence Access List standard Standard Access List r1(config)#
At first it looked like we couldn’t resequence ACLs in NX-OS — oh no! Well, looking into it a bit further, I found the following:
switch(config)# resequence ? ip Configure IP features ipv6 Configure IPv6 features mac MAC configuration commands switch(config)# resequence ip access-list ? WORD List name switch(config)# resequence ip access-list
We do have feature parity between NX-OS and IOS in re-sequencing ACLs (although the commands differ – no big deal as long as we’re aware of the differences!).
Difference in Wildcard Mask Behavior
For sometime wildcard masks have offered power to those who understand them and plagued those who do not. In IOS, if you wanted to match the 1st and 3rd octets and ensure that the 2nd octet was always odd, you could use the following ACL:
r1(config)#ip access-list extended acl1 r1(config-ext-nacl)#permit ip 10.11.10.0 0.1.0.255 any r1(config-ext-nacl)#
Let’s try this in NX-OS:
switch(config)# ip access-list acl1 switch(config-acl)# permit ip 10.11.10.0 0.1.0.255 any ERROR: Given ACL rule is invalid switch(config-acl)#
Oops! It looks like NX-OS will not allow non-contiguous wildcard masks anymore. But wait, let’s try a contiguous wildcard mask and see what happens:
switch(config-acl)# permit ip 10.11.10.0 0.0.3.255 any switch(config-acl)#
Okay, suspicion confirmed! I looked at the NX-OS command reference and did not find anything stating that the permit statement only accepts wildcard masks that are contiguous, however the above tinkering seems to indicate that this is the case.
This makes sense when you look at the ACL – notice how NX-OS stored the prefixes:
switch(config-acl)# sh ip access-list acl1
IP access list acl1
10 permit ip 10.11.10.0/16 any
20 permit ip 10.11.10.0/22 any
switch(config-acl)#
Note how all of the entries (which were entered using a wildcard mask) were converted to CIDR notation? Okay – things are fitting together as to why non-contiguous masks are not supported (even though I couldn’t find this stated as such in the NX-OS Command Reference – it could be there, but I didn’t see it).
Lack of “do” to Run Exec-Level Commands
In IOS, within config mode, the “do” command must be pre-pended to any exec-level command you want to issue:
r1(config)#do sh ip access-list acl1
Extended IP access list acl1
10 permit ip 10.10.10.0 0.1.0.255 any
r1(config)#
In NX-OS, this is removed – exec-level commands may be run from within config mode without any special command:
switch(config-acl)# sh ip access-list acl1
IP access list acl1
10 permit ip 10.11.10.0/16 any
20 permit ip 10.11.10.0/22 any
switch(config-acl)#
Pretty slick, huh?
Where’s My GigabitEthernet Interfaces?
Good question! NX-OS goes the way of some other vendors in simplifying the interface naming conventions, removing the interface speed from the interface name. There are no longer any FastEthernet, GigabitEthernet or TenGigabitEthernet interfaces on NX-OS. All interfaces are Ethernet interfaces.
It’s still possible to specify 1Gbps or 10Gbps on the interface, just that the interface name itself won’t reflect the change in speed.
switch(config)# sh int brief -------------------------------------------------------------------------------- Ethernet VLAN Type Mode Status Reason Speed Port Interface Ch # -------------------------------------------------------------------------------- Eth1/1 1 eth access down SFP validation failed 10G(D) -- Eth1/2 1 eth access down SFP not inserted 10G(D) -- Eth1/3 1 eth access down SFP validation failed 10G(D) -- Eth1/4 1 eth access down SFP not inserted 10G(D) -- Eth1/5 1 eth access down SFP validation failed 10G(D) -- Eth1/6 1 eth access down SFP not inserted 10G(D) -- Eth1/7 1 eth access down SFP validation failed 10G(D) -- Eth1/8 1 eth access down SFP not inserted 10G(D) -- Eth1/9 1 eth access down SFP not inserted 10G(D) -- Eth1/10 1 eth access down SFP not inserted 10G(D) -- Eth1/11 1 eth access down SFP not inserted 10G(D) -- Eth1/12 1 eth access down SFP not inserted 10G(D) -- Eth1/13 1 eth access down SFP not inserted 10G(D) -- Eth1/14 1 eth access down SFP not inserted 10G(D) -- Eth1/15 1 eth access down SFP not inserted 10G(D) -- Eth1/16 1 eth access down SFP not inserted 10G(D) -- Eth1/17 1 eth access down SFP not inserted 10G(D) -- Eth1/18 1 eth access down SFP not inserted 10G(D) -- Eth1/19 1 eth access down SFP not inserted 10G(D) -- Eth1/20 1 eth access down SFP not inserted 10G(D) -- Eth2/1 1 eth access down SFP not inserted 10G(D) -- Eth2/2 1 eth access down SFP not inserted 10G(D) -- Eth2/3 1 eth access down SFP not inserted 10G(D) -- Eth2/4 1 eth access down SFP not inserted 10G(D) -- Eth2/5 1 eth access down SFP not inserted 10G(D) -- Eth2/6 1 eth access down SFP not inserted 10G(D) -- -------------------------------------------------------------------------------- Port VRF Status IP Address Speed MTU -------------------------------------------------------------------------------- mgmt0 -- down 10.1.1.53 -- 1500 switch(config)# int e1/1 switch(config-if)# speed 1000 switch(config-if)# sh int brief -------------------------------------------------------------------------------- Ethernet VLAN Type Mode Status Reason Speed Port Interface Ch # -------------------------------------------------------------------------------- Eth1/1 1 eth access down Link not connected 1000(D) -- Eth1/2 1 eth access down SFP not inserted 10G(D) -- Eth1/3 1 eth access down SFP validation failed 10G(D) -- Eth1/4 1 eth access down SFP not inserted 10G(D) -- Eth1/5 1 eth access down SFP validation failed 10G(D) -- Eth1/6 1 eth access down SFP not inserted 10G(D) -- Eth1/7 1 eth access down SFP validation failed 10G(D) -- Eth1/8 1 eth access down SFP not inserted 10G(D) -- Eth1/9 1 eth access down SFP not inserted 10G(D) -- Eth1/10 1 eth access down SFP not inserted 10G(D) -- Eth1/11 1 eth access down SFP not inserted 10G(D) -- Eth1/12 1 eth access down SFP not inserted 10G(D) -- Eth1/13 1 eth access down SFP not inserted 10G(D) -- Eth1/14 1 eth access down SFP not inserted 10G(D) -- Eth1/15 1 eth access down SFP not inserted 10G(D) -- Eth1/16 1 eth access down SFP not inserted 10G(D) -- Eth1/17 1 eth access down SFP not inserted 10G(D) -- Eth1/18 1 eth access down SFP not inserted 10G(D) -- Eth1/19 1 eth access down SFP not inserted 10G(D) -- Eth1/20 1 eth access down SFP not inserted 10G(D) -- Eth2/1 1 eth access down SFP not inserted 10G(D) -- Eth2/2 1 eth access down SFP not inserted 10G(D) -- Eth2/3 1 eth access down SFP not inserted 10G(D) -- Eth2/4 1 eth access down SFP not inserted 10G(D) -- Eth2/5 1 eth access down SFP not inserted 10G(D) -- Eth2/6 1 eth access down SFP not inserted 10G(D) -- -------------------------------------------------------------------------------- Port VRF Status IP Address Speed MTU -------------------------------------------------------------------------------- mgmt0 -- down 10.1.1.53 -- 1500 switch(config-if)#
Notice how the 1Gbps SFP is no longer failing validation on Eth1/1? This is another possible gotcha – it didn’t auto-sense the SFP speed and adjust the port speed. Looking at Eth1/3, it can be seen that the default port speed is 10Gbps (and that it doesn’t auto-detect/sense the speed):
switch(config-if)# sh run int eth1/3
version 4.0(1a)N1(1)
interface Ethernet1/3
switch(config-if)# sh int e1/3
Ethernet1/3 is down (SFP validation failed)
Hardware is 1000/10000 Ethernet, address is 000d.ecd0.bbca (bia 000d.ecd0.bbca)
MTU 1500 bytes, BW 10000000 Kbit, DLY 10 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA
Port mode is access
auto-duplex, 10 Gb/s, media type is 10g
Input flow-control is off, output flow-control is off
Auto-mdix is turned on
Rate mode is dedicated
Switchport monitor is off
Last clearing of "show interface" counters never
5 minute input rate 0 bytes/sec, 0 packets/sec
5 minute output rate 0 bytes/sec, 0 packets/sec
Rx
0 input packets 0 unicast packets 0 multicast packets
0 broadcast packets 0 jumbo packets 0 storm suppression packets
0 bytes
0 No buffer 0 runt 0 Overrun
0 crc 0 Ignored 0 Bad etype drop
0 Bad proto drop
Tx
0 output packets 0 multicast packets
0 broadcast packets 0 jumbo packets
0 bytes
0 output CRC 0 ecc
0 underrun 0 if down drop 0 output error 0 collision 0 deferred
0 late collision 0 lost carrier 0 no carrier
0 babble
0 Rx pause 0 Tx pause 0 reset
switch(config-if)#
STP PortFast
NX-OS does not accept the typical IOS spanning-tree portfast command that we’re so familiar with. Instead, NX-OS adopts the IEEE 802.1D terminology (finally!), in that you specify the port type and its behavior matches the standards-defined behavior.
To configure a port as an edge port (essentially enable PortFast behavior on the port) in NX-OS:
switch(config-if)# spanning-tree port type edge Warning: portfast should only be enabled on ports connected to a single host. Connecting hubs, concentrators, switches, bridges, etc... to this interface when portfast is enabled, can cause temporary bridging loops. Use with CAUTION %Portfast has been configured on Ethernet1/1 but will only have effect when the interface is in a non-trunking mode. switch(config-if)#
Note that we’re still able to enable PortFast on trunk ports in NX-OS:
switch(config-if)# spanning-tree port type edge ?
trunk Consider the interface as edge port (enable portfast) even in trunk
mode
switch(config-if)# spanning-tree port type edge
I’m glad to see this change in NX-OS. Cisco forged ahead with so many improvements with PVST+ in IOS when the rest of the industry was stuck in legacy STP (802.1D) only. Many of these improvements were standardized (thanks to Cisco!) in the IEEE 802.1w standard (which has been merged into the IEEE 802.1D standard). It’s nice to see that NX-OS is dropping the Cisco-proprietary terminology and using the standards-based terminology.
Watch Those Features
NX-OS is very modular – much more so than even IOS Modularity (Modular IOS). Many protocols (called features within NX-OS) are run within their own protected memory space. This is really good in that it can help mitigate one protocol/feature from negatively effecting another feature. The downside is that these features are not enabled by default – they have to be manually enabled (very much like a service on other OSs). After they’re enabled (started), this is placed in the configuration, so that they don’t have to be enabled each time the Nexus is power-cycled.
Each feature is enabled/disabled using the feature command. To enable a feature, use the feature , to disable, use the no feature command.
What is considered a feature (and must be manually enabled) on NX-OS? Well, this differs between the platforms (because of the difference in feature-sets). The following shows the features on the Nexus 5000:
switch(config)# feature ? fcoe Enable feature interface-vlan Enable/Disable interface vlan lacp Enable/Disable LACP private-vlan Enable/Disable private-vlan tacacs+ Enable/Disable tacacs+ udld Enable/Disable UDLD switch(config)#
The following is taken from a Nexus 7000:
n7k (config)# feature ?
bgp Enable/Disable Border Gateway Protocol (BGP)
cts Enable/Disable CTS
dhcp Enable/Disable DHCP Snooping
dot1x Enable/Disable dot1x
eigrp Enable/Disable Enhanced Interior Gateway Routing Protocol
(EIGRP)
eou Enable/Disable eou(l2nac)
glbp Enable/Disable Gateway Load Balancing Protocol (GLBP)
hsrp Enable/Disable Hot Standby Router Protocol (HSRP)
interface-vlan Enable/Disable interface vlan
isis Enable/Disable IS-IS Unicast Routing Protocol (IS-IS)
lacp Enable/Disable LACP
msdp Enable/Disable Multicast Source Discovery Protocol (MSDP)
netflow Enable/Disable NetFlow
ospf Enable/Disable Open Shortest Path First Protocol (OSPF)
ospfv3 Enable/Disable Open Shortest Path First Version 3 Protocol
(OSPFv3)
pbr Enable/Disable Policy Based Routing(PBR)
pim Enable/Disable Protocol Independent Multicast (PIM)
pim6 Enable/Disable Protocol Independent Multicast (PIM) for IPv6
port-security Enable/Disable port-security
private-vlan Enable/Disable private-vlan
rip Enable/Disable Routing Information Protocol (RIP)
scheduler Config commands for scheduler
tacacs+ Enable/Disable tacacs+
udld Enable/Disable UDLD
vrrp Enable/Disable Virtual Router Redundancy Protocol (VRRP)
n7k(config)# feature
By default all features are disabled (not enabled). How can you tell which features are enabled? Well, I’ve looked through many CCO docs and have seen mention of the show feature command, but have not found it listed in any of the NX-OS command references (for the Nexus 5000 or Nexus 7000).
When I tried it on the Nexus 5000, it didn’t work:
switch# show feature
^
% invalid command detected at '^' marker.
switch#
Also, when trying it on the Nexus 7000, it doesn’t work:
n7k# show feature ?
^
% invalid command detected at '^' marker.
n7k#
So although some of the Cisco NX-OS config guides (for the N5k and N7k) indicate that this command exists and can be optionally used, I can’t seem to get it to work. Not finding it in the NX-OS command references (for the N5k and N7k) seem to confirm that this command doesn’t yet exist (hopefully we’ll see it in the future?).
Back to the original question – how do we find out which services are enabled? Take a look at the running config and that will tell you:
switch# switch# conf t switch(config)# feature lacp switch(config)# sh run | i feature feature lacp switch(config)#
NX-OS is pretty intelligent. In certain instances, if you try to look at a disabled feature, for example, it will tell you that the feature isn’t enabled:
switch(config)# show fcoe FCoE/FC feature is not desired. switch(config)#
Other times, the command tree structure itself will disappear from the CLI:
switch(config)# no feature lacp
switch(config)# sh lacp ?
^
% invalid command detected at '^' marker.
switch(config)# sh lacp
switch(config)#
switch(config)# feature lacp
switch(config)# sh lacp ?
No matches in current mode, matching in (exec) mode
counters LACP counters
interface Specify a interface
internal Show lacp service internal status
neighbor LACP interface neighbor
port-channel LACP port-channels
system-identifier Show system-identifier information
switch(config)# sh lacp
So, sometimes NX-OS will tell you that the feature is not enabled, while other times the protocol/feature just appears to be missing altogether. Either way, ensuring that you’re aware of the features enabled and required for the environment is critical.
Some features do not require enablement before being able to configure certain aspects of the feature. FCoE is a great example – it’s possible to go into an interface and enable FCoE on the interface (in this example, using mode on) even when the fcoe feature is not enabled. Without the feature being enabled, there won’t be any FCoE support provided, but it can be pre-configured.
One last note about features: there are many features that I consider “core” features which must be enabled if required. For instance, if your AAA design incorporates the usage of TACACS+, NX-OS considers this a feature and therefore it must be explicitly enabled. Failure to enable it will cause woes. In this example, TACACS+ isn’t even available within the AAA commands:
switch(config)# aaa group server ? radius RADIUS server group name switch(config)# aaa group server switch(config)# switch(config)# feature tacacs switch(config)# aaa group server ? radius RADIUS server group name tacacs+ TACACS+ server group name switch(config)# aaa group server
Any golden configs which are ported from IOS-based systems to NX-OS must recognize and incorporate features into the configuration steps, as failure to recognize this will result in a possible loss of available commands (because the feature isn’t enabled).
Conclusion
I like NX-OS, however there are some differences (some of which have been outlined here) that we’ll all need to get used to if we’re to be proficient in NX-OS. This article is a very high-level overview of just a few of the differences. Using NX-OS on the Nexus 7000 yields far greater differences (around routing protocol configuration, etc.).
I suspect that we’ll find that using ACLs as route filters might not be as useful in NX-OS as we can’t use non-contiguous wildcard masks. Then again, this is typically needed in the real world.
Overall, I think the NX-OS changes are good and will make life easier. If you get a chance, give NX-OS a chance!
Tags: 5000, 7000, access-list, acl, cidr, cisco, contiguous, differences, do, exec, feature, interface, interfaces, ios, ip address, mask, n5k, n7k, nexus, nexus 5000, nexus 7000, nx-os, portfast, prefix-list, primer, slash notation, spanning-tree, stp, subnet mask, training, tutorial, wildcard
Cisco Nexus, Networking | tim | 
2010-01-17 12:49 pm
Subscribe