Using the Cisco 3650 Managment Port

Configuring some new Cisco 3650s, I wanted to use the management ports rather than setting up management LAN SVIs and so on. This is particularly useful in a DMZ as we know the management port is in a completely different VRF.

Here’s a short summary of the steps taken to get around things not working at first as the traffic wasn’t being source from within the management VRF. IP addresses are only for the purposes of examples.

First off, configure the management interface and default route:

interface GigabitEthernet0/0
 description ** Network Managment Interface **
 vrf forwarding Mgmt-vrf
 ip address 192.168.0.1 255.255.255.0

ip route vrf Mgmt-vrf 0.0.0.0 0.0.0.0 192.168.0.254

Logging

logging source-interface GigabitEthernet0/0 vrf Mgmt-vrf
logging host 192.168.100.2

NTP

ntp server vrf Mgmt-vrf 192.168.100.1

TFTP

ip tftp source-interface GigabitEthernet0/0

AAA needs a modification to work

aaa group server tacacs+ TACACS_GROUP
 server 10.0.0.99
 server 10.0.1.99
 ip vrf forwarding Mgmt-vrf

ip tacacs source-interface GigabitEthernet0/0

SNMP

snmp-server host 10.0.0.102 vrf Mgmt-vrf version 2c YOURSTRING

That covers the essentials!

Spoofing SNMP Traps for testing

Somehow I missed the fact that JunOS allows you to spoof SNMP traps. I discovered this recently and must say it’s very handy, especially when testing new NNMi or other NMS incident configurations. It helpfully populates the varbinds for you with preset values (although you can specify them if desired).

This is done as follows from the JunOS command line:

user@SRX> request snmp spoof-trap ospfNbrStateChange variable-bindings "ospfNbrState = 8"

You can look up varbind names in the appropriate MIB.

A bit easier than the traditional equivalent in shell:

/usr/bin/snmptrap -v 2c -c mycommunity nms.mydomain.com:162 '' .1.3.6.1.2.1.14.16.2.0.2 \
.1.3.6.1.2.1.14.1.1 a 0.0.0.0 \
.1.3.6.1.2.1.14.10.1.1 a "1.2.3.4" \
.1.3.6.1.2.1.14.10.1.2 i "0" \
.1.3.6.1.2.1.14.10.1.3 a "2.3.4.5" \
.1.3.6.1.2.1.14.10.1.6 i "8"

High CPU on Nexus 5K and no SNMP response – snmpd

Strange issue with SNMP not responding today on a nexus 5K. Tried removing and re-adding the SNMP config, removing the acl altogether that we use to control access and still no joy.

Upon checking CPU usage, it seemed quite high. show proc cpu sort output showed that snmpd was quite busy:

PID    Runtime(ms)  Invoked   uSecs  1Sec    Process
-----  -----------  --------  -----  ------  -----------
 4559           59  991518226      0   44.5%  snmpd
 4605          179        87   2065    9.0%  netstack
 1178         2091  1733135010      0    1.0%  kirqd
    1          157  25653477      0    0.0%  init
    2          837   3474116      0    0.0%  migration/0
    3          600  3970856252      0    0.0%  ksoftirqd/0

I was sure I’d dealt with this before and it seems that I was hitting a bug.

Official word is that: There is a memory leak in one of the processes called libcmd that is used by SNMP. Workaround is entering the hidden command:

no snmp-server load-mib dot1dbridgesnmp

The best solution, however, would be to perform a software upgrade to 5.0(3)N2(2) or later where this is fixed.

Nexus 5K SNMP Config

Below are a few examples of setting up SNMP on Cisco Nexus 5Ks. Unfortunately, SNMPv3 is still missing some functionality, such as the ability to restrict access to a defined set of subnets or hosts via an ACL. I won’t go into another rant about how much of a headache SNMPv3 can be with various management systems, I’ll just provide (non-exhaustive) examples:

SNMPv2 with access control

! Create an ACL for allowed sources
ip access-list SNMPMGMT
 permit ip 10.243.100.0/28 192.168.100.1/32

! Create community - default access is ro
snmp-server community MYSTRING group network-operator
snmp-server community MYSTRING use-acl SNMPMGMT
snmp-server host 10.243.21.104 use-vrf management

! Where to send traps to
snmp-server host 10.243.100.4 traps version 2c MYSTRING

! Enable some traps
snmp-server enable traps config ccmCLIRunningConfigChanged
snmp-server enable traps link cisco-xcvr-mon-status-chg
snmp-server enable traps bridge newroot
snmp-server enable traps bridge topologychange

SNMPv3 AuthPriv – SHA/AES – Unable to use an ACL!

! Enable privacy for all SNMP users
snmp-server globalEnforcePriv

! Create User
snmp-server user MYUSER network-operator auth sha MYAUTHKEY priv aes-128 MYPRIVKEY localizedkey

! Where to send traps and which VRF to use
snmp-server host 10.243.100.4 traps version 3 priv MYUSER
snmp-server host 10.243.100.4 use-vrf management

! Enable traps below as per above example.