Adj resolve request: Failed to resolve… [ Cisco 3750X ]

After a migration to a pair of 3750Xs I was getting a bunch of disconcerting ARP errors in the logs. After a bit of digging, this appears to be a known bug. Error message content examples are shown below:

Nov  4 12:41:00: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 172.17.22.57 Vlan21
Nov  4 12:41:18: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 172.17.24.53 Vlan28

This was fixed/worked around with:

no ip cef optimize neighbor resolution

nmap ping scan doesn’t populate arp cache

This drove me absolutely bats**t mental. I couldn’t understand why running an nmap -sP 10.0.0.0/24 (locally attached subnet) as root hit all the available hosts, but didn’t populate the local arp cache when you looked at the output of arp -a.

It seems that the -sP parameter (ping only) doesn’t hit the required level of the OS stack to populate the cache, even when running as root, so the required steps are:

1) Ensure that you can run nmap as root via sudo to allow raw socket capability.
2) Add the –send-ip option to the nmap command. This will ensure that the command uses raw sockets and populates the ARP table.

eg:

nmap -sP 10.0.0.0/24 -n --send-ip

Then you will see that the ARP table is updated with:

/sbin/arp -a

The reason I needed this was because I had to enhance the network mac/ip/dns resolution script to use local ARP entries. Without this option, the hosts are shown as up but the OS arp table never gets updated.

Beware that if you run nmap -sP as an unpriveleged user, it’s almost useless as it targets TCP port 80 (according to the man page) rather than using ICMP echoes.

Apparently, fping has an option to support this (-g), but I rarely see it installed as part of a standard build.