Juniper MX Port Numbering

Unfortunately the juniper port numbering scheme can cause a lot of confusion for people not familiar with the layout. This can get quite confusing when explaining to remote hands on site.

I made a quick reference which I use for some of our MX480s but the principle is the same on other hardware. This is handy for sending to third parties with a port highlighted to ensure they don’t go pulling the wrong cables. :)

juniper-ports

Adding vlans to trunk on Juniper MX – behaviour

Just a small note on how the CLI on an MX behaves when you try to add vlans to an existing trunk as sometimes people are confused about doing this. Obviously it’s not really a problem as nothing happens until commit, but this works in a similar fashion to Cisco “switchport trunk allowed vlan-add xxx” command if you’re using set commands.

Existing config as below:

fe-0/0/3 {
    unit 0 {
        family bridge {
            interface-mode trunk;
            vlan-id-list [ 1 2 3 ];
        }
    }
}

Adding a vlan like this does not overwrite the existing config:

set interfaces fe-0/0/3 unit 0 family bridge vlan-id-list 5

Config is updated as below:

fe-0/0/3 {
    unit 0 {
        family bridge {
            interface-mode trunk;
            vlan-id-list [ 1 2 3 5 ];
        }
    }
}

SPAN Port on Juniper MX Series

Unfortunately creating a SPAN port on a Juniper MX isn’t as easy as on Cisco kit or even, say an SRX. You need to jump through a few hoops creating a forwarding-options config, a firewall filter and also a bit of a kludge with the SPAN interface by creating a static ARP entry to force traffic out.

Here are some example for IOS/NX-OS and so you can see the difference.

Cisco IOS

monitor session 1 source interface Gi0/13 both
monitor session 1 destination interface Gi0/24

Cisco NX-OS

monitor session 1
  source interface Eth10/34
  destination interface Ethernet10/35
  no shut

interface Eth10/35
 switchport
 switchport monitor

Nice and simple. But not on JunOS. :(

All credit and thanks to this post which made it very easy to understand:
http://pingpros.blogspot.nl/2012/12/multiple-ports-port-mirror-on-juniper.html

Source ports in this example are ge-5/2/7 for the port to be mirrored, and xe-4/3/0 for the port that connects to the wireshark or other monitoring device.

1) Setup the port forwarding option.

set forwarding-options port-mirroring input rate 1
set forwarding-options port-mirroring input run-length 1
set forwarding-options port-mirroring family inet output interface xe-4/3/0.0 next-hop 1.1.1.2
set forwarding-options port-mirroring family inet output no-filter-check

2) Create a firewall filter which will mirror the port traffic. I presume term 2 is required so it still allows traffic through as well as port-mirroring.

set firewall family inet filter port-mirror term 1 then port-mirror
set firewall family inet filter port-mirror term 1 then accept
set firewall family inet filter port-mirror term 2 then accept

3) Apply the firewall filter to the port or ports that you want to mirror.

set interfaces ge-5/2/7 unit 0 family inet filter input port-mirror
set interfaces ge-5/2/7 unit 0 family inet filter output port-mirror

4) Configure the SPAN interface with an IP that doesn’t conflict with anything you’re already using within your network and add a dummy arp entry for the next-hop address so traffic is forced out of the interface. Remember to remove any other configuration on this interface beforehand if re-using say, an access port. The MAC address is fictional.

set interfaces xe-4/3/0 unit 0 family inet address 1.1.1.1/30 arp 1.1.1.2 mac 00:11:22:33:44:55

Note that you can add the same config to an existing irb interface to SPAN an irb. This is less painful than trying to do pure L2 span when it’s applicable.

set interfaces irb unit 900 family inet filter input port-mirror
set interfaces irb unit 900 family inet filter output port-mirror

Job done.

UPDATE: It seems Juniper has added “analyzer” functionality in more recent code. I’ll investigate this at a later date.

Juniper MX480 PSU Gripes

This has happened three times in a row to me. It seems that Juniper MX480 PSUs are not very graceful when they blow. I now think it’s unwise to ask anyone to try “re-seating the PSU and power cable” once a PSU has failed because this has a tendency to make the PDU trip, knocking out everything on that power strip. Ouch! Better to just RMA the thing.

Not so much of a problem when all kit in the rack has dual PSUs, but it’s worth bearing in mind in case there is any single PSU kit that isn’t on a static transfer switch in there! Lucky the environment I deal with is doesn’t involve the latter. :)

Faulty PSU shows as:

PEM 0:
  State:     Present
  AC input:  Out of range (1 feed expected, 1 feed connected)
  Capacity:  2050 W (maximum 2050 W)
  DC output: 0 W (zone 0, 0 A at 0 V, 0% of capacity)

Effective Interface Monitoring with iSPI for Metrics

While iSPI for Metrics is useful for graphing, arguably the more critical use for it is in generating alerts when things are not working as expected.

I found today that Juniper does not include “FCS Lan Errors” in the ifInErrors counters which made finding an issue take a bit longer than usual. This is what I’m talking about:

Interface: ge-1/1/0, Enabled, Link is Up
Encapsulation: Ethernet, Speed: 1000mbps
Traffic statistics:                                           Current delta
  Input bytes:            42120054608900 (6704304 bps)           [16457880]
  Output bytes:           33903229061248 (11640616 bps)          [38429868]
  Input packets:             60739200100 (3676 pps)                 [82119]
  Output packets:            51621757514 (5316 pps)                [125843]
Error statistics:
  Input errors:                        0                                [0]
  Input drops:                         0                                [0]
  Input framing errors:          1435325                            [12231]
  Policed discards:                    0                                [0]
  L3 incompletes:                      0                                [0]

For effective monitoring of critical WAN interfaces, you can create an interface group using for example, a filter by interface name (or ifAlias – description if you have a naming standard), which then is set up as a monitoring policy with the thresholds you want. Alternatively you can use a Node group based policy bearing in mind you’ll be alerted for all interfaces on those nodes in the group.

Example:
thresh

You want to enable Interface Fault Polling and Interface Performance Polling to make this work, and may also want to consider lowering the default 5 minute poll period for critical infrastructure.

I tend to set the trigger count at 1, and the rearm count higher (at least 2) so that the incident doesn’t just disappear before someone’s had a chance to have a look at it when the error condition clears.

As a side note, you’ll also want to set generic thresholds… Eg: For “Routers” at least the following:

router-thresh