Description

The message means TTP packets sent from FPC is dropped by JTD before sending to RE.
 

Symptoms

The JTD (Juniper Tunnel Driver) module in EVO system is used to handle hostbound traffic. This module located between PFEs and Linux Kernel. It translates the packets between various applications (native Linux kernel) and FPC (TTP encapsulation packets).

The log message “kernel: [ttp_proto_recv:1536] packet dropped” shows that JTD is dropping TTP packets getting from FPC.

This KB shows how to capture dropping packets on JTD, capture ttp packets from RE and FPC and provide an example analysis.
 

Solution

To capture dropping packets on JTD
Command:(Required root access)
[vrf:none] root@node-re0:~# ttpcapture.py -i jtdrop
 
Example:
root@node-re0> start shell
[vrf:none] labroot@node-re0:~$ su
Password:
 sh-4.3# ttpcapture.py -i jtdrop
 .......... Start capturing ..........
===================================================================
Time:                   22:53:22.379333
Drop direction:         Ingress
Drop reason:            Invalid TTP header
Payload:
00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 202: (tos 0x0, ttl 64, id 1033, offset 0, flags [DF], proto unknown (84), length 188, bad cksum 888c (->35d4)!)
    128.0.0.16 > 128.0.0.1:  ip-proto-84 168
        0x0000:  4500 00bc 0409 4000 4054 888c 8000 0010
        0x0010:  8000 0001 0103 0603 0000 3395 0094 0000
        0x0020:  0000 0000 0000 8010 0021 0002 0001 00ab
        0x0030:  0004 0000 0000 00c1 0004 0000 0454 0004
        0x0040:  0004 5500 0000 3333 0000 0001 2416 9d5e
        0x0050:  bcdd 86dd 6e00 0000 0040 3aff fe80 0000
        0x0060:  0000 0000 2616 9dff fe5e bcdd ff02 0000
        0x0070:  0000 0000 0000 0000 0000 0001 8600 25cd
        0x0080:  4000 0708 0000 0000 0000 0000 0101 2416
        0x0090:  9d5e bcdd 0501 0000 0000 23f2 0304 7fc0
        0x00a0:  0027 8d00 0009 3a80 0000 0000 2600 6ce5
        0x00b0:  0000 0a21 0000 0000 0000 0016

Analysis:
  • This packet is dropped at JTD with Invalid TTP header.
  • This packet is from FPC0(128.0.0.16) to RE(master- 128.0.0.1)
  • It is ICMP6 packet.

To capture ttp packets on FPC
Command:(Required root access)
root@node-re0> start shell user root
Password:
[vrf:none] root@node-re0:~# chvrf iri ssh fpc0
root@node-re0-fpc0:~# parcel_dump.py -p0 -T

Example:
root@node-re0-fpc0:~# parcel_dump.py -p0 -T
 Finding the cached IFL/IFD database (/var/tmp/IFDB.json).....
 IFL/IFD database found. Use the existing one.....
 Start capturing.....
 ===================================================================
 Time:           19:39:48.712348
 type:           RX_L2
 priority:       3
 protocol:       IPV6
 queue:          3
 ifl/index:      .local..0/0
 len:            148
 nh_index:       0
 hints:          0x00008010 / select_fc ttp_tlv
 TLV type L2_MEDIA - 0x0001/1
 TLV type VRF_ID - 0x00000000/0
 TLV type RX_L2_IFD - 0x00000428/1064
 TLV type DDOS_POLICER - 0x55000000/1426063360
 Payload:
 00:50:56:89:01:4d > 33:33:ff:a8:ff:1d, ethertype 802.1Q (0x8100), length 118: vlan 856, p 0, ethertype IPv6, (hlim 255, next-header ICMPv6 (58) payload length: 60) 2001:730:8000:ffff::c0a8:ff1e > ff02::1:ffa8:ff1d: [bad icmp6 cksum 0x1ddf -> 0x69e7!] ICMP6, neighbor solicitation, length 60, who has 2001:730:8000:ffff::c0a8:ff1d
           source link-address option (1), length 8 (1): 00:50:56:89:01:4d [|icmp6]
         0x0000:  0358 86dd 6000 0000 003c 3aff 2001 0730
         0x0010:  8000 ffff 0000 0000 c0a8 ff1e ff02 0000
         0x0020:  0000 0000 0000 0001 ffa8 ff1d 8700 1ddf
         0x0030:  0000 0000 2001 0730 8000 ffff 0000 0000
         0x0040:  c0a8 ff1d 0101 0050 5689 014d b18b 87f5
         0x0050:  41b4 fffc 4978 6961 0000 0000 1011 1213
         0x0060:  2ed9 4138 0000 297c

Analysis:
  • PFE sends a normal ICMP6 packet toward RE.
  • No issue found.

To capture ttp packets on RE in pcap
Command:(Required root access)
[vrf:none] root@node-re0:~# tcpdump -i iri proto 84 -w /var/tmp/ttp_packets.pcap

Result from PCAP:
Frame 698: 202 bytes on wire (1616 bits), 202 bytes captured (1616 bits)
Ethernet II, Src: 02:00:80:00:00:10 (02:00:80:00:00:10), Dst: 02:00:80:00:00:04 (02:00:80:00:00:04)
Internet Protocol Version 4, Src: 128.0.0.16, Dst: 128.0.0.1
Trivial Tunneling protocol
    type: rx_l2 (1)
    priority: medium (3)
    protocol: IPV6 (6)
    queue: 3
    ifl_index: 0x00003395
    length: 148
    nh_destmask: 0x0000
    nh_index: 0x00000000
    hint: 0x00008010
TLVs
    type: 33
    rsvd: 0
    len (4 bytes unit): 2
    value: 0001
    type: 171
    rsvd: 0
    len (4 bytes unit): 4
    value: 00000000
    type: 193
    rsvd: 0
    len (4 bytes unit): 4
    value: 00000454
    type: 4
    rsvd: 0
    len (4 bytes unit): 4
    value: 55000000
Ethernet II, Src: Cisco_5e:bc:dd (24:16:9d:5e:bc:dd), Dst: IPv6mcast_01 (33:33:00:00:00:01)
    Destination: IPv6mcast_01 (33:33:00:00:00:01)
    Source: Cisco_5e:bc:dd (24:16:9d:5e:bc:dd)
    Type: IPv6 (0x86dd)
Internet Protocol Version 6, Src: fe80::2616:9dff:fe5e:bcdd, Dst: ff02::1
    0110 .... = Version: 6
    .... 1110 0000 .... .... .... .... .... = Traffic Class: 0xe0 (DSCP: CS7, ECN: Not-ECT)
    .... 0000 0000 0000 0000 0000 = Flow Label: 0x00000
    Payload Length: 64
    Next Header: ICMPv6 (58)
    Hop Limit: 255
    Source Address: fe80::2616:9dff:fe5e:bcdd
    Destination Address: ff02::1
    [Source SLAAC MAC: Cisco_5e:bc:dd (24:16:9d:5e:bc:dd)]
Internet Control Message Protocol v6
 
Analysis:
  • RE get the protocol IPv6 with rx_l2 instead of rx_l3. So JTD dropped it becuase protocol IPv6 can't be sent as l2 inject from PFE towards RE.
  • The possible cause might be from misconfiguration that the issue box doesn’t have vlan configuration on receiving interface while another side have vlan-tag.

Modification History

2023-05-25: Initial publication