Adj resolve request: Failed to resolve… [ Cisco 3750X ]

After a migration to a pair of 3750Xs I was getting a bunch of disconcerting ARP errors in the logs. After a bit of digging, this appears to be a known bug. Error message content examples are shown below:

Nov  4 12:41:00: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve Vlan21
Nov  4 12:41:18: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve Vlan28

This was fixed/worked around with:

no ip cef optimize neighbor resolution

Resolve MAC addresses to Port, IP and DNS Name

Resolving MAC address to port, IP and DNS or name service name (or more simply for some, resolve mac to name) is a challenge that every network engineer has come across at some point in their career. It’s easily solved with a bit of thought and logic. Unfortunately the past few products I’ve dealt in the past with for this purpose have either been abandoned or aren’t as multi-vendor as I’d like, so it seems that the only solution is to write your own… bash and expect is sufficient.

If you’re thinking about doing this (and it’s a great learning exercise), you need to get around the following:

– Determining which interfaces are trunks on the switches so you can strip those MAC entries out (CDP works quite well)
– Converting ARP and MAC info into a “clean” format (eg: CatOS and IOS output is a different format)
– Detecting the fields across various pieces of hardware as display output isn’t always consistent for the same commands
– Inconsistent logins/passwords
– Correlating the IP/MAC/Interface information together. This can be done with the UNIX join command and some awk/sed
– What you do with MACs that don’t resolve to an IP address (I include a flag to print these if required)
– Whether the machine you run DNS queries on will be able to resolve the IPs to PTR records
– If using expect, stripping out stray characters (eg \r) that will mess up your greps and other string searches
– Add plenty of debugging so you can quickly tell why something isn’t working properly

I used expect to go and grab the ARP, CDP and MAC information seeing as you can’t get all the required information from SNMP on many devices these days. In my case, this results in the following type of output:

Switch       Interface       VLAN  MAC             IP               DNSName
nycsw12      Fa3/10          100   0060.b0aa.0000    NO_DNS
nycsw12      Fa2/16          99    1060.4b61.0001
nycsw12      Fa2/37          101   1060.4b64.0002
nycsw12      Fa2/42          101   1060.4b68.0003
nycsw12      Fa2/45          98    1060.4b6a.0004
nycsw12      Fa2/32          98    1060.4b6a.0005
nycsw12      Fa3/3           100   2c41.389e.d19f
nycsw13      Fa2/4           100   5c26.0a01.0ac4
nycsw13      Fa2/6           100   6c3b.e531.2ddf

Of course, you can always just use Excel to do a VLOOKUP of your mac-address table output against a sorted table containing all your arp entries, but that’s a bit less automatic.