A PTX10001 is seeing odd behavior with respect to ICMP packet processing at the RE. Pings to Google DNS show loss and errored frames:
user@ptx> ping 8.8.8.8 count 1000 rapid
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
.!EC!EC!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!E.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!E.^C
--- 8.8.8.8 ping statistics ---
608 packets transmitted, 595 received, 2% packet loss, time 6709ms
rtt min/avg/max/mdev = 0.590/0.682/1.216/0.053 ms, ipg/ewma 11.053/0.639 ms
Same for 1.1.1.1:
user@ptx> ping 1.1.1.1 count 1000 rapid
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
.!E.!E.!E.!E.!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!E.!EC!E.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC!EC.!E.!EC!EC!EC!EC^C
--- 1.1.1.1 ping statistics ---
322 packets transmitted, 317 received, 1% packet loss, time 2589ms
rtt min/avg/max/mdev = 0.848/0.941/1.189/0.054 ms, ipg/ewma 8.065/0.951 ms
Route to 1.1.1.1 is via ae14.501:
user@ptx> show route 1.1.1.1
inet.0: 973186 destinations, 5022164 routes (973125 active, 354 holddown, 926 hidden)
+ = Active Route, - = Last Active, * = Both
1.1.1.0/24 *[BGP/170] 3w6d 06:09:35, localpref 100
AS path: 13335 I, validation-state: unverified
> to 10.70.203.1 via ae14.501
Same behavior pinging peer address:
user@ptx> ping 10.70.203.1 rapid count 1000
PING 10.70.203.1 (10.70.203.1) 56(84) bytes of data.
.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!EC!E.!E.!EC!E.!E.!E.!EC!EC!E.!E.!E.!E.!E.!E.!EC!EC!E.!E.!EC!E.!E.!E.!E.!EC!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!E.!EC!E.!E.!E.!E.!E
--- 10.70.203.1 ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 1848ms
rtt min/avg/max/mdev = 1.243/1.790/46.939/2.520 ms, ipg/ewma 1.850/1.482 ms
Or, the local address on the interface:
ae14 {
description "raccordo bai1-Cabase (et-0/0/17) - (vlan terminated on transm. eq. 8700) IN ACTIV.";
flexible-vlan-tagging;
mtu 9192;
encapsulation flexible-ethernet-services;
aggregated-ether-options {
lacp {
active;
}
unit 501 {
vlan-id 501;
family inet {
rpf-check {
mode loose;
filter {
input-list [ Sample-IPv4 Border-IPv4 Accept-IPv4 ];
address 10.70.203.0/31;
user@ptx> ping 10.70.203.0 rapid count 1000
PING 10.70.203.0 (10.70.203.0) 56(84) bytes of data.
.!EC!EC!E
--- 10.70.203.0 ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 110ms
rtt min/avg/max/mdev = 0.082/0.090/0.234/0.016 ms, ipg/ewma 0.110/0.091 ms
Note that system statistics for icmp show a large number of checksum errors:
user@ptx> show system statistics icmp
icmp:
0 drops due to rate limit
2086 calls to icmp_error
0 errors not generated because old message was icmp
Output Histogram
1013161551 echo reply
2333 destination unreachable
39576 echo
2379835 time exceeded
0 parameter problem
0 source quench
0 routing redirect
0 time stamp
285 time stamp reply
0 address mask request
0 address mask reply
0 router advertisement
0 router solicitation
0 extended echo
0 extended echo reply
0 messages with bad code fields
0 messages less than the minimum length
67613593 messages with bad checksum
0 messages with bad source address
0 messages with bad length
0 echo drops with broadcast or multicast destination address
0 timestamp drops with broadcast or multicast destination address
Input Histogram
250635 echo reply
3002614 destination unreachable
1014625073 echo
That is increasing:
user@ptx> show system statistics icmp | match checksum
67615349 messages with bad checksum
{master}
67615387 messages with bad checksum
67615423 messages with bad checksum
ICMPv6 does not have this issue:
user@ptx> ping 2001:41a8:5000:2::1 rapid count 10000
PING 2001:41a8:5000:2::1(2001:41a8:5000:2::1) 56 data bytes
--- 2001:41a8:5000:2::1 ping statistics ---
10000 packets transmitted, 10000 received, 0% packet loss, time 1256ms
rtt min/avg/max/mdev = 0.079/0.102/0.546/0.023 ms, ipg/ewma 0.125/0.100 ms
And, there are no issues with checksums for TCP or other protocols.
After taking a packet capture for the TTP packets that the RE is receiving, the problem was found. The router is connected to the Internet and it is receiving a flood of spoofed source packets over ICMP (DA changed below from the routable address):
29 0.000712 2.23.164.121 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
38 0.000441 2.23.164.251 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
45 0.007643 23.64.58.144 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
68 0.001612 2.23.164.150 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
116 0.000806 95.101.24.14 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
117 0.009212 23.64.58.199 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
120 0.016331 2.23.164.231 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
121 0.000546 2.23.164.249 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
127 0.001385 2.23.164.215 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
198 0.000614 2.23.164.168 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
205 0.004322 2.23.164.171 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
206 0.000563 2.23.164.172 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
283 0.002495 2.23.164.169 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
284 0.009471 2.23.164.242 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
397 0.003370 23.64.58.203 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
469 0.001941 2.23.164.164 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
470 0.002446 2.23.164.200 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
481 0.005383 2.23.164.132 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
518 0.001130 95.101.24.2 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
528 0.008067 2.23.164.172 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
549 0.000026 2.23.164.168 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
602 0.000798 2.23.164.217 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
612 0.001200 2.23.164.169 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
615 0.000741 2.23.164.215 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
621 0.009581 23.64.58.205 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
622 0.002839 2.23.164.239 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
700 0.000392 95.101.24.10 10.70.203.48 ICMP 130 Unknown ICMP (obsolete or malformed?)
The lo0 filter in this case allows open ICMP with a policer:
term ICMP-reply-any {
from {
protocol icmp;
icmp-type [ echo-request echo-reply ];
then {
policer 100k;
count ICMP-Echo-Reply-any;
accept;
So, there is no restriction to ICMP packets hitting the RE. The ping utility on the RE opens up a socket for ICMP when the ping is initiated and once that socket is open, the noise is received causing the errored output.
If the above term is deactivated, the checksum errors stop and the pings are now successful with no checksum errors reported.