Description

A PTX is seeing an issue the the flow records that are sampled and sent to the flow server.

 

The router is configured for sampling:

 

set chassis fpc 0 sampling-instance sample-ins

set services flow-monitoring version-ipfix template mpls-ipv4 flow-active-timeout 60

set services flow-monitoring version-ipfix template mpls-ipv4 flow-inactive-timeout 15

set services flow-monitoring version-ipfix template mpls-ipv4 template-refresh-rate packets 30

set services flow-monitoring version-ipfix template mpls-ipv4 template-refresh-rate seconds 60

set services flow-monitoring version-ipfix template mpls-ipv4 option-refresh-rate packets 30

set services flow-monitoring version-ipfix template mpls-ipv4 option-refresh-rate seconds 30

set services flow-monitoring version-ipfix template mpls-ipv4 mpls-template

set services flow-monitoring version-ipfix template ipv4 flow-active-timeout 60

set services flow-monitoring version-ipfix template ipv4 flow-inactive-timeout 15

set services flow-monitoring version-ipfix template ipv4 nexthop-learning enable

set services flow-monitoring version-ipfix template ipv4 template-refresh-rate packets 30

set services flow-monitoring version-ipfix template ipv4 template-refresh-rate seconds 60

set services flow-monitoring version-ipfix template ipv4 option-refresh-rate packets 30

set services flow-monitoring version-ipfix template ipv4 option-refresh-rate seconds 30

set services flow-monitoring version-ipfix template ipv4 ipv4-template

set services flow-monitoring version-ipfix template ipv6 flow-active-timeout 60

set services flow-monitoring version-ipfix template ipv6 flow-inactive-timeout 15

set services flow-monitoring version-ipfix template ipv6 nexthop-learning enable

set services flow-monitoring version-ipfix template ipv6 template-refresh-rate packets 30

set services flow-monitoring version-ipfix template ipv6 template-refresh-rate seconds 60

set services flow-monitoring version-ipfix template ipv6 option-refresh-rate packets 30

set services flow-monitoring version-ipfix template ipv6 option-refresh-rate seconds 30

set services flow-monitoring version-ipfix template ipv6 ipv6-template

 

set firewall family inet filter ipv4-ipfix term ipv4-accept then sample

set firewall family inet filter ipv4-ipfix term ipv4-accept then accept

 

set forwarding-options sampling instance sample-ins input rate 100

set forwarding-options sampling instance sample-ins family inet output flow-server 10.99.250.4 port 9995

set forwarding-options sampling instance sample-ins family inet output flow-server 10.99.250.4 autonomous-system-type origin

set forwarding-options sampling instance sample-ins family inet output flow-server 10.99.250.4 version-ipfix template ipv4

set forwarding-options sampling instance sample-ins family inet output inline-jflow source-address 2.2.2.2

 

On irb.1010:

 

set interfaces irb unit 1010 arp-resp unrestricted

set interfaces irb unit 1010 family inet filter input ipv4-ipfix

set interfaces irb unit 1010 family inet address 100.1.1.100/24

 

Which lives in l3-vrf:

 

set routing-instances l3-vrf instance-type vrf

set routing-instances l3-vrf routing-options autonomous-system 65001

 

set routing-instances l3-vrf protocols bgp group e1 type internal

set routing-instances l3-vrf protocols bgp group e1 neighbor 100.0.0.2

set routing-instances l3-vrf protocols evpn ip-prefix-routes advertise gateway-address

set routing-instances l3-vrf protocols evpn ip-prefix-routes gateway-interface irb.1010

set routing-instances l3-vrf interface et-0/0/1.100

set routing-instances l3-vrf interface irb.1010

set routing-instances l3-vrf route-distinguisher 2.2.2.2:2222

set routing-instances l3-vrf vrf-target target:11111:2222

set routing-instances l3-vrf vrf-table-label

 

Traffic comes into et-0/0/1.1010:

 

set interfaces et-0/0/4:1 unit 1010 encapsulation vlan-bridge

set interfaces et-0/0/4:1 unit 1010 vlan-id 1010

 

With a source of 100.1.1.101 and a dest of 123.0.0.123:

 

set routing-instances l2-vrf instance-type mac-vrf

set routing-instances l2-vrf protocols evpn encapsulation vxlan

set routing-instances l2-vrf protocols evpn default-gateway no-gateway-community

set routing-instances l2-vrf protocols evpn duplicate-mac-detection auto-recovery-time 9

set routing-instances l2-vrf protocols evpn extended-vni-list all

set routing-instances l2-vrf protocols evpn multicast-mode ingress-replication

set routing-instances l2-vrf vtep-source-interface lo0.0

set routing-instances l2-vrf service-type vlan-aware

set routing-instances l2-vrf route-distinguisher 2.2.2.2:1010

 

It passes thru irb.1010 and gets sampled.

 

At the server, the IPFIX data will mostly be correct, but the OutputInt is not populated:

 

Frame 1: 794 bytes on wire (6352 bits), 794 bytes captured (6352 bits)

Ethernet II, Src: d0:81:c5:43:73:76 (d0:81:c5:43:73:76), Dst: 00:10:94:00:00:03 (00:10:94:00:00:03)

Internet Protocol Version 4, Src: 2.2.2.2 (2.2.2.2), Dst: 10.99.250.4 (10.99.250.4)

User Datagram Protocol, Src Port: 33092 (33092), Dst Port: 9995 (9995)

Cisco NetFlow/IPFIX

  Version: 10

  Length: 748

  Timestamp: Aug 2, 2024 11:02:27.000000000 Eastern Daylight Time

  FlowSequence: 1187821

  Observation Domain Id: 8388608

  Set 1

    FlowSet Id: (Data) (256)

    FlowSet Length: 732

    Flow 1

      SrcAddr: 100.1.1.101 (100.1.1.101)

      DstAddr: 123.0.0.123 (123.0.0.123)

      IP ToS: 0x00

      Protocol: 253

      SrcPort: 0

      DstPort: 0

      ICMP Type: 0

      InputInt: 627

      SrcAS: 0

      DstAS: 0

      BGPNextHop: 0.0.0.0 (0.0.0.0)

      OutputInt: 0

      Octets: 1002

      Packets: 1

      [Duration: 0.000000000 seconds]

      NextHop: 0.0.0.0 (0.0.0.0)

      SrcMask: 32

      DstMask: 8

      TCP Flags: 0x00

      IPVersion: 04

      [Duration: -1722367204.821000000 seconds]

      [Duration: 0.000000000 seconds]

      Enterprise Private entry: ((null)) Type 368: Value (hex bytes): 00 00 00 01

      Enterprise Private entry: (Juniper Networks, Inc.) Type 32767: Value (hex bytes): 62 65

      Enterprise Private entry: (Juniper Networks, Inc.) Type 32766: Value (hex bytes): 00

 

The route the traffic is resolving to is a BGP learned default:

 

l3-vrf.inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

 

0.0.0.0/0     *[BGP/170] 00:26:41, localpref 100

           AS path: I, validation-state: unverified

          > to 100.0.0.2 via et-0/0/1.100

 

Oddly, if that route is changed to a static route:

 

set routing-instances l3-vrf routing-options static route 123.0.0.0/8 next-hop 100.0.0.2

 

labroot@e05-40> show route table l3-vrf.inet.0 123.0.0.123 

 

l3-vrf.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

 

123.0.0.0/8  *[Static/5] 1d 03:38:34

     > to 100.0.0.2 via et-0/0/1.100

 

The OutputInt is correctly set: 

 

Cisco NetFlow/IPFIX

..

 Set 1

  FlowSet Id: (Data) (256)

  FlowSet Length: 1369

  Flow 1

   SrcAddr: 100.1.1.101 (100.1.1.101)

   DstAddr: 123.0.0.123 (123.0.0.123)

   IP ToS: 0x00

   Protocol: 253

   SrcPort: 0

   DstPort: 0

   ICMP Type: 0

   InputInt: 627

   SrcAS: 0

   DstAS: 0

   BGPNextHop: 0.0.0.0 (0.0.0.0)

   OutputInt: 657

 

Interface indexes:

 

labroot@e05-40> show interfaces irb.1010 | match index 

 Logical interface irb.1010 (Index 1051) (SNMP ifIndex 627)

 

labroot@e05-40> show interfaces et-0/0/4:1.1010 | match index 

 Logical interface et-0/0/4:1.1010 (Index 1055) (SNMP ifIndex 676)

 

labroot@e05-40# run show interfaces et-0/0/1.100 | match index 

 Logical interface et-0/0/1.100 (Index 1040) (SNMP ifIndex 657)

 

Solution

Problem is tracked via PR128032:

https://prsearch.juniper.net/PR1828032

 

Modification History

2024-08-02 : Article Created