Customers may experience random silent FPC reboots after configuring the disable-fpc action under the DDoS protection settings. If DDoS protection is disabled at the FPC level for any protocols, and there is a sudden spike in traffic, the FPC might undergo a silent reset due to an overload of control packets.

In particular, we have observed scenarios where disabling DDoS protection for TTL-related protocols, combined with a surge in MPLS traffic with TTL1 on any FPC at the incoming fabric side, can overwhelm the packet I/O system. This situation is caused by an IXGBE reset (Tx timeout event), leading to a silent FPC reboot.

Excess traffic for any protocol could overwhelm PacketIO if DDOS is disabled at FPC level.

The syncer down event and unreachable node issue occurred following this incident. This can affect traffic and may also cause flapping in other protocols.

1. IXGBE reset (TX timeout) leading to FPC reset.

2. The issue is reported only on the FPC that is receiving an excess of control packets.

The following Kernel dump can be reported under journalctl:

kernel: ------------[ cut here ]------------
kernel: WARNING: CPU: 3 PID: 17170 at dev_watchdog+0x27f/0x290
kernel: Modules linked in: uio_pci_hostif 8021q macvlan intel_rapl_msr intel_rapl_common ebtable_filter sb_edac x86_pkg_temp_thermal intel_powerclamp kvm_intel kvm irqbypass m25p80 spi_nor leds_supercon_ledx jnx_mastership_supercon leds_supercon_led2 gpio_supercon_hotswap thermal uio_pdrv_genirq uio arptable_filter arp_tables pcc_cpufreq sam_core mtdoops i2c_sam i2c_dev gpio_sam gpio_ptxpmb_cpld jnx_cbios_debugfs ofpart jnx_pmb_nvram mtd jnx_f1_bcpld
kernel: CPU: 3 PID: 17170 Comm: fp-if-pool Kdump: loaded Not tainted 5.2.60-yocto-standard-g7274013 #1
kernel: Hardware name: Juniper Networks Inc. 0D06/0D07, BIOS CBDE_P_F1P1_00.28.01 12/07/2020
kernel: RIP: 0010:dev_watchdog+0x27f/0x290
kernel: Code: 72 87 48 ff eb 88 4c 89 ff c6 05 61 8e e0 00 01 e8 66 80 fb ff 44 89 e9 4c 89 fe 48 c7 c7 28 d7 ec a7 48 89 c2 e8 5d 24 56 ff <0f> 0b e9 66 ff ff ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00
kernel: RSP: 0000:ffffaa4d833bce18 EFLAGS: 00010286
kernel: RAX: 0000000000000000 RBX: ffff8918d6064ec0 RCX: 0000000000000000
kernel: RDX: 0000000000000007 RSI: 0000000000000096 RDI: 00000000ffffffff
kernel: RBP: ffffaa4d833bce48 R08: 0000000000000001 R09: 00000000000005c1
kernel: R10: 00000000000025a8 R11: 0000000000000004 R12: ffff8918d615c440
kernel: R13: 0000000000000007 R14: ffff8918d6064f40 R15: ffff8918d615c000
kernel: FS:  00007f76b89fe700(0000) GS:ffff8918df580000(0000) knlGS:0000000000000000
kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: CR2: 00007f0e47e0d788 CR3: 0000000785352004 CR4: 00000000003606e0
kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
kernel: Call Trace:
kernel:  <IRQ>
kernel:  ? qdisc_put_unlocked+0x40/0x40
kernel:  ? qdisc_put_unlocked+0x40/0x40
kernel:  call_timer_fn+0x37/0x180
kernel:  ? qdisc_put_unlocked+0x40/0x40
kernel:  expire_timers+0xb3/0x130
kernel:  run_timer_softirq+0x92/0x170
kernel:  ? enqueue_hrtimer+0x42/0xb0
kernel:  ? sched_clock_cpu+0x1a/0xe0
kernel:  __do_softirq+0x10d/0x366
kernel:  ? __hrtimer_get_next_event+0x4d/0x80
kernel:  irq_exit+0xac/0xd0
kernel:  smp_apic_timer_interrupt+0x9a/0x1a0
kernel:  apic_timer_interrupt+0xf/0x20
kernel:  </IRQ>
kernel: RIP: 0033:0x7f7726535030
kernel: Code: 05 48 3d 00 f0 ff ff 77 01 c3 48 8b 15 49 be 0a 00 f7 d8 64 89 02 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 <55> 48 8b 05 b8 1c 0b 00 48 c1 c8 11 64 48 33 04 25 30 00 00 00 48

We have been scenarios where the FPC reboot was triggered by an excess of MPLS traffic with TTL 1 being directed through the FPCs that lacked DDoS Protection TTL safeguards. Juniper strongly recommends re-enabling the DDoS Protection Default parameters and do not disable DDos at the FPC level to prevent this type of traffic from overloading the internal communication path between the FPC and the RE. 


Please avoid configuring something similar under the DDOS-protection stanza:

    ddos-protection {
        protocols {
             ttl {
                 aggregate {
                     bandwidth 10000;
                     disable-fpc;   >>>>>>>>>>>>>>>>>>>>
                 }
             }
         }
     }