Description

In Some CFLOWD records Nhop addr and Output Interface are set to 0.0.0.0 and 0 respectively.

Symptoms

There are legitimate cases where the Nhop addr and Output Interface are set to zero in CFLOWD records, such as multicast traffic, or traffic sent to the Routing Engine. However, on a T1600 Juniper routers with Type 4 Enhanced Scaled Flexible PIC Concentrator (T1600-FPC4-ES), this can be observed for unicast traffic that has been forwarded by the router. For such traffic, null Nhop addr and null Output Interface are illegal values.
To illustrate this, a T1600 router was connected to a traffic generator. Below is the sampling configuration that is used in this example: 

talal@sting-re0> show configuration forwarding-options 
sampling {
    input {
        family inet {
            rate 10000;
        }
    }
    output {
        cflowd 10.11.12.13 {
            port 9999;
            version 5;
            local-dump;
        }
    }
}
local-dump is configured so that the CFLOWD records are locally written to the log on the router before being sent to the CFLOWD server.
Sampling is configured on two interfaces: 
xe-0/1/2 {
    unit 0 {
        family inet {
            sampling {
                input;
            }
            address 192.1.1.1/24;
        }
    }
}

xe-0/1/3 {
    unit 0 {
        family inet {
            sampling {
                input;
            }
            address 192.2.1.1/24;
        }
    }
}
In this example, two traffic streams are entering the router, one in xe-0/1/2, and the other in xe-0/1/3.
A quick look at the CFLOWD records locally on the router illustrates the problem: 
Mar 16 10:25:59 v5 flow entry
Mar 16 10:25:59    Src addr: 250.0.0.100
Mar 16 10:25:59    Dst addr: 200.0.0.81
Mar 16 10:25:59    Nhop addr: 0.0.0.0
Mar 16 10:25:59    Input interface: 215
Mar 16 10:25:59    Output interface: 0
Mar 16 10:25:59    Pkts in flow: 1
Mar 16 10:25:59    Bytes in flow: 46
Mar 16 10:25:59    Start time of flow: 204811294
Mar 16 10:25:59    End time of flow: 204811294
Mar 16 10:25:59    Src port: 0
Mar 16 10:25:59    Dst port: 0
Mar 16 10:25:59    TCP flags: 0x0
Mar 16 10:25:59    IP proto num: 61
Mar 16 10:25:59    TOS: 0x0
Mar 16 10:25:59    Src AS: 0
Mar 16 10:25:59    Dst AS: 0
Mar 16 10:25:59    Src netmask len: 0
Mar 16 10:25:59    Dst netmask len: 24
In this example, all the CFLOWD records have Nhop addr and Output Interface set to zero. However, on production routers, the problem can be perceived as random, i.e. for the same source and destination couple of addresses, only a portion of the CFLOWD records will have Nhop addr and Output Interface set to zero.

Solution

This is due to a known issue that is reported in PR/431567

On some JUNOS releases, the Nhop addr and Output Interface in CFLOWD records are systematically set to 0.0.0.0 and 0 respectively.  The problem is triggered when, for a given flow, the input and output interfaces are both located on the same bottom Packet Forwarding Engine (PFE 1).  Even though this is systematic, in a real production network the behavior for a specific couple of source and destination IP addresses may be hard to predict because of:

  • Load balancing, if any
  • The input interface may not be the same for all packets
This means that, for the same source and destination IP addresses, you will sometimes see the output interface set to 0, and sometimes not, depending on whether the input interface is on the same bottom PIC as what the output interface should be.

This issue is resolved in the following JUNOS releases: 9.2R4, 9.3R3, 9.4R3, 9.5R2 and 9.6R1

Related Information