This document explains the behaviour of ACX7K routers where if PBR is implemented with next-ip, trace-route prints duplicate hops in the output.

Lab logs:

Config:

set interfaces et-0/0/0 unit 0 family inet filter input pbr-cgnat1

set firewall filter pbr-cgnat1 interface-specific

set firewall filter pbr-cgnat1 term 1 from source-address 172.16.1.1/32

set firewall filter pbr-cgnat1 term 1 then next-ip 20.20.20.2/32

--> With PBR:

labroot@jtac-mx104-r2005-re0> traceroute 30.30.30.2 source 172.16.1.1 

traceroute to 30.30.30.2 (30.30.30.2) from 172.16.1.1, 30 hops max, 40 byte packets

 1 20.20.20.2 (20.20.20.2) 1.544 ms 0.613 ms 0.529 ms

 2 20.20.20.2 (20.20.20.2) 0.646 ms 0.542 ms 0.495 ms

 3 30.30.30.2 (30.30.30.2) 0.885 ms 0.991 ms 0.698 ms

--> Without PBR:

labroot@jtac-mx104-r2005-re0# run traceroute 30.30.30.2 source 172.16.1.1   

traceroute to 30.30.30.2 (30.30.30.2) from 172.16.1.1, 30 hops max, 40 byte packets

 1 172.16.1.2 (172.16.1.2) 1.537 ms 0.994 ms 1.062 ms

 2 20.20.20.2 (20.20.20.2) 0.760 ms 0.767 ms 0.841 ms

 3 30.30.30.2 (30.30.30.2) 0.820 ms 0.856 ms 0.702 ms

Considering below example, explaining the root cause here:

R1 <-->ACX7K<-->R3<-->R4

Trace-route from R1 to R3 and implemented PBR on ACX7k.

Root Cause:

PBR in ACX7K is implemented in TCAM filters, where NH/FEC is set as an action.

When TTL=1 is sent from R1, PBR-TCAM sends the packet to the FEC which sends the packet to R3.

The PBR-TCAM is higher priority and it overrides the CONTROL-TCAM filters which match the TTL=0/1 and send to host for ICMP-TTL exception.

So, the packet goes to R3 with TTL=0 and R3 sends back the ICMP-TTL-EXCEED.

When TTL=2 is sent from R1, the packet goes to R3 with TTL=1 and R3 sends back the ICMP-TTL-EXCEED.

That's why we see duplicate entries in traceroute.

Looks the MX has TTL=0/1 check before the PBR forwarding. So, the ICMP is generated even if the packet is supposed to be forwarded by PBR.

Behaviour is ACX7K is due to parallel nature of TCAM lookups and hence it is expected behaviour in ACX

Workaround/Solution:

This behaviour is expected due to BCM limitation and hence below solution can be used.

For a given packet, if the TTL=0/1 whether to forward using PBR or to have ICMP generation can be decided by customer itself.

Say, if traceroute is what needed and we don't expect the TTL=0/1 packets to be forwarded using PBR then below term can be configured.

jtac-acx7100# show firewall

filter pbr-cgnat1 {

  interface-specific;

  term dontfwd-ttl-0/1 {

    from {

      ttl [ 0 1 ];

    }

    then accept;

  }

  term 1 {

    from {

      source-address {

        172.16.1.1/32;

      }

    }

    then {

      next-ip 20.20.20.2/32;

    }

  }

}

After this, traceroute works.

jtac-mx104# run traceroute 30.30.30.2 source 172.16.1.1

traceroute to 30.30.30.2 (30.30.30.2) from 172.16.1.1, 30 hops max, 40 byte packets

 1 172.16.1.2 (172.16.1.2) 1.445 ms 1.084 ms 0.842 ms

 2 20.20.20.2 (20.20.20.2) 0.579 ms 0.528 ms 0.482 ms

 3 30.30.30.2 (30.30.30.2) 0.913 ms 0.814 ms 0.679 ms

We can't have TTL=0/1 packets to both have ICMP exception packet gen, as well as PBR forward.