This article describes how to troubleshoot interface-down scenarios.
The customer reported that one of their interfaces is still down even after replacing cables and optics.Verification:user@router> show interfaces et-0/0/7 extensive Mar 05 18:08:57Physical interface: et-0/0/7, Enabled, Physical link is Down Interface index: 153, SNMP ifIndex: 515, Generation: 156 Description: 007:remote-device:HundredGigE0/6/0/25:CircutID-123456789:Bundle-Ether22: Link-level type: Ethernet, MTU: 9192, Speed: 100Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled Device flags : Present Running Down ASIC-Error Interface flags: Hardware-Down SNMP-Traps Internal: 0x4000 Link flags : None CoS queues : 8 supported, 8 maximum usable queues Hold-times : Up 0 ms, Down 0 ms Damping : half-life: 0 sec, max-suppress: 0 sec, reuse: 0, suppress: 0, state: unsuppressed Current address: b4:8a:5f:b3:39:1b, Hardware address: b4:8a:5f:b3:30:2b Last flapped : 2023-07-05 18:55:43 UTC (34w5d 23:13 ago)<snip>Check the light level:
The interface is not transmitting light on all lanes:FPC0(router vty)# show qsfp 8 diagnostics Module temperature: 29.79 C Supply voltage: 3.2163 V Lane 0 Tx bias: 0.000 mA Tx power: 0.000 mW / - Inf dBm Rx power: 0.862 mW / -0.64 dBm TX Laser: Disabled Lane 1 Tx bias: 0.000 mA Tx power: 0.000 mW / - Inf dBm Rx power: 0.854 mW / -0.68 dBm TX Laser: Disabled Lane 2 Tx bias: 0.000 mA Tx power: 0.000 mW / - Inf dBm Rx power: 0.913 mW / -0.40 dBm TX Laser: Disabled Lane 3 Tx bias: 0.000 mA Tx power: 0.000 mW / - Inf dBm Rx power: 0.909 mW / -0.41 dBm TX Laser: DisabledFPC0(router vty)#Checking the FPC errors we can see that the interface stayed down because the PFE it is mapped to is disabled:You can usually see the FPC getting disabled in the log messages but if the logs are overwritten you can use the below commands to check:user@router> show chassis fpc errors 0 FPC Scope Category Level Occurred Cleared Threshold Action-Taken Action 0 board functional Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE memory Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE io Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE storage Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE switch Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE processing Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE internal Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE pfe functional Minor 0 0 10 0 LOG| Major 1 1 1 3 GET STATE|CM ALARM| Fatal 1 1 1 2 DISABLE PFE memory Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE io Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE storage Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE switch Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE processing Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE internal Minor 0 0 10 0 LOG| Major 0 0 1 0 GET STATE|CM ALARM| Fatal 0 0 1 0 DISABLE PFE Pfe-State: pfe-0 -ENABLED | pfe-1 -DISABLED | pfe-2 -ENABLED | pfe-3 -ENABLED | pfe-4 -ENABLED | pfe-5 -ENABLED |user@router>user@router> show chassis pic fpc-slot 0 pic-slot 0 FPC slot 0, PIC slot 0 information: Type 30x100GE/30x40GE/96x10GE MACSEC State Online PIC version 2.17 Uptime 533 days, 10 hours, 47 minutes, 23 seconds PIC port information: Fiber Xcvr vendor Wave- Xcvr JNPR Port Cable type type Xcvr vendor part number length Firmware Rev 0 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 1 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 2 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 3 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 4 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 5 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 6 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 7 100G AOC 7M MM WAVESPLITTER WS-Q28-AOCLC072 850 nm 0.0 8 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 9 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 10 100GBASE LR4 SM SOURCE PHOTONICS SPQCELRCDFB 1310 nm 0.0 11 100GBASE LR4 SM SOURCE PHOTONICS SPQCELRCDFB 1310 nm 0.0 12 100GBASE LR4 SM O-NET 1140200002 1302 nm 0.0 13 100GE DWDM2 SM INPHI CORP IN-Q2AY2-44-M1 1542.14 nm 5 14 100GBASE LR4 SM SOURCE PHOTONICS SPQCELRCDFB 1310 nm 0.0 15 100GBASE LR4 SM SOURCE PHOTONICS SPQCELRCDFB 1310 nm 0.0 16 100GE DWDM2 SM INPHI CORP IN-Q2AY2-38-M1 1546.92 nm 5 17 100GE DWDM2 SM INPHI CORP IN-Q2AY2-39-M1 1546.12 nm 5 18 100GE DWDM2 SM INPHI CORP IN-Q2AY2-41-M1 1544.53 nm 5 19 100GBASE LR4 SM CIG TRQ5E22ENF-LF000 1310 nm 0.0 20 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 21 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 22 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 23 100G AOC 7M MM Ligent Photonics DQF8503-4C07 850 nm 0.0 24 100GBASE LR4 SM CIG TRQ5E22ENF-LF000 1310 nm 0.0 25 100GE DWDM2 SM INPHI CORP IN-Q2AY2-38-M1 1546.92 nm 5 28 100GBASE LR4 SM CIG TRQ5E22ENF-LF000 1310 nm 0.0 Port speed information: Port PFE Capable Port Speeds 0 0 4x10GE, 40GE, 100GE 1 1 4x10GE, 40GE, 100GE 2 0 4x10GE, 40GE, 100GE 3 1 4x10GE, 40GE, 100GE 4 0 4x10GE, 40GE, 100GE 5 1 4x10GE, 40GE, 100GE 6 0 40GE, 100GE 7 1 40GE, 100GE >>>>>>>>>>>>>>>>>>>> Port 7 is mapped to PFE 1 8 0 4x10GE, 40GE, 100GE 9 1 4x10GE, 40GE, 100GE 10 2 4x10GE, 40GE, 100GE 11 3 4x10GE, 40GE, 100GE 12 2 4x10GE, 40GE, 100GE 13 3 4x10GE, 40GE, 100GE 14 2 4x10GE, 40GE, 100GE 15 3 4x10GE, 40GE, 100GE 16 2 40GE, 100GE 17 3 40GE, 100GE 18 2 4x10GE, 40GE, 100GE 19 3 4x10GE, 40GE, 100GE 20 4 4x10GE, 40GE, 100GE 21 5 4x10GE, 40GE, 100GE 22 4 4x10GE, 40GE, 100GE 23 5 4x10GE, 40GE, 100GE 24 4 4x10GE, 40GE, 100GE 25 5 4x10GE, 40GE, 100GE 26 4 40GE, 100GE 27 5 40GE, 100GE 28 4 4x10GE, 40GE, 100GE 29 5 4x10GE, 40GE, 100GE user@router>