This article explains the following syslog messages:
'NF Chip RODR packet drop'
'RODR offset overflow count incremented'
Syslog Messages:
Jan 24 23:38:34 : %PFE-3: fpc5 Cmerror Op Set: NF Chip: NFCHIP(0) RODR packet dropJan 24 23:38:34 : %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet dropJan 24 23:38:34 : %PFE-3: fpc5 NFAB(5/1): RODR offset overflow count incremented (1)Jun 24 23:38:34 alarmd[1644]: %DAEMON-5: Alarm set: FPC color=RED, class=CHASSIS, reason=FPC 5 Major Errors Jun 24 23:38:34 craftd[1645]: %DAEMON-5: Major alarm set, FPC 5 Major Errors
user@host > show chassis hardware <snip> FPC 5 REV 07 710-013560 xxxxx E2-FPC Type 3
Check Alarm:
user@host> show chassis alarms no-forwarding1 alarms currently activeAlarm time Class Description2020-01-24 23:38:34 UTC Major FPC 5 Major Errors
Check if FPC is online:
user@host> show chassis fpc detailSlot 5 information: State Online Temperature 53 degrees C / 127 degrees F Total CPU DRAM 1024 MB Total SRAM 72 MB Total SDRAM 1280 MB
If FPC is offline, you can directly go to restarting the FPC to bring it back online. Detailed logs:
Jan 20 09:12:01 %PFE-3: fpc5 NF Chip: NFCHIP(1) RODR packet drop <-- Make note of NFCHIP number. Here, it is NFCHIP 1 Jan 20 09:13:16 %PFE-3: fpc5 NFAB(5/1): RODR offset overflow count incremented (1)Jan 20 09:21:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 09:34:14 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 09:36:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 09:49:14 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 09:51:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 10:04:17 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 10:06:23 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 10:19:13 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 10:21:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop – clear
Jan 20 09:12:01 %PFE-3: fpc5 NF Chip: NFCHIP(1) RODR packet drop <--
Jan 20 09:13:16 %PFE-3: fpc5 NFAB(5/1): RODR offset overflow count incremented (1)Jan 20 09:21:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 09:34:14 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 09:36:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 09:49:14 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 09:51:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 10:04:17 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 10:06:23 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop - clear Jan 20 10:19:13 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop Jan 20 10:21:20 %PFE-3: fpc5 NF Chip:NFCHIP(1) RODR packet drop – clear
Typically, when an FPC receives packets in the form of cells, it reassembles it. So at the time of reassembly when the FPC is congested and when it is unable to receive the cells further, it will drop the packet without reordering (RODR) it, then we see the logs as above.
Check RODR statistics on NCHIP 1 for FPC 5:
user@host> request pfe execute command "show nchip 1 rodr" target fpc5 NFAB(1/0) Nin RODR statistics: (110713827)
Statistics Total Rate/sec Peak Rate Err Sec cell_timeout 307 0 64 5 ra_overflow 1293 0 23 627 summary_pkt_drop 1324 0 6 1079 err_cell 0 0 0 0 offset_overflow 7 0 1 7
The above command helps us understand the rate and error rate at which the occurrence happened.
In most cases, alarm clears on its own and the errors stop appearing. If the error does not clear or FPC is offline, please restart the FPC.
FPC Restart:
user@host> request chassis fpc restart slot slot-number
Post restart, if the error still remains or FPC still remains offline, contact JTAC for further investigation.Collect the following logs for JTAC analysis:
show pfe statistics traffic fpc x <--- Replace x with affected slot show chassis sibs show chassis fabric fpcs show chassis fabric sibs show chassis fabric topology request pfe execute command "show hsl2 statistics crc " target fpcX <-- Replace x with affacted FPC. Run 3-4 times at 5 mins interval