Description

The following messages may be seen on MPC10E 100G interfaces after multiple fast flaps 
[Apr 18 20:54:42.769 LOG: Info] pfe_ifd_link_updown: IFD et-0/1/2 - pfe link DOWN processing is complete
[Apr 18 20:54:43.463 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:54:43.463 LOG: Info] ifd_process_flaps_common: IFD et-0/1/2 - pfe link UP processing is complete 
[Apr 18 20:54:43.486 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:54:43.486 LOG: Info] smic_mac_process_linkstate: Enabling MAC Rx for ifd et-0/1/2
[Apr 18 20:54:43.703 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:54:43.703 LOG: Info] ifp et-0/1/2 ifd_mdown: 2438441547 ms
[Apr 18 20:54:43.703 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:54:43.704 LOG: Info] pfe_ifd_link_updown: IFD et-0/1/2 - pfe link DOWN processing is complete
[Apr 18 20:54:44.463 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:54:44.463 LOG: Info] ifd_process_flaps_common: IFD et-0/1/2 - pfe link UP processing is complete 
[Apr 18 20:54:44.485 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:54:44.486 LOG: Info] smic_mac_process_linkstate: Enabling MAC Rx for ifd et-0/1/2
[Apr 18 20:54:44.486 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:54:44.486 LOG: Info] ifp et-0/1/2 ifd_mdown: 2438442330 ms
[Apr 18 20:54:44.496 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:54:44.497 LOG: Info] pfe_ifd_link_updown: IFD et-0/1/2 - pfe link DOWN processing is complete
[Apr 18 20:54:59.488 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:54:59.488 LOG: Info] ifd_process_flaps_common: IFD et-0/1/2 - pfe link UP processing is complete 
[Apr 18 20:54:59.513 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:00.268 LOG: Info] ifp et-0/1/2 ifd_mdown: 2438458110 ms
[Apr 18 20:55:00.488 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:55:00.489 LOG: Info] pfe_ifd_link_updown: IFD et-0/1/2 - pfe link DOWN processing is complete
[Apr 18 20:55:53.563 LOG: Debug] pfeman_aftman_is_wan_ifd_ready: ifd et-0/1/2 in ready state in AFT->PLATFORMD shared mem.
[Apr 18 20:55:53.563 LOG: Info] ifd_process_flaps_common: IFD et-0/1/2 - pfe link UP processing is complete 
[Apr 18 20:55:53.568 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.572 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.573 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.574 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.574 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.574 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.576 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.576 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.576 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.589 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.589 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.589 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.593 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.593 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.593 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.595 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.595 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.595 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.596 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.596 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.596 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.598 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.598 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.598 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.601 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.601 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.601 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.610 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.610 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.610 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.615 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.615 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.615 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.620 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.620 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
[Apr 18 20:55:53.620 LOG: Info] MQSS(2): QUADPORT4: Detected Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.624 LOG: Info] MQSS(2): QUADPORT4: Cleared Ethernet MAC Local Fault Delta Event for Port 0 (et-0/1/2)
[Apr 18 20:55:53.624 LOG: Info] smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete

 

Symptoms

The interface in the logs may go down and may no recover itself after the flap and log  "Physical link et-7/2/0 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete"  keep showing. 

Solution

This is a software bug introduced after 20.4 versions and later fixed in the PR1735539.
 The logs can fill up quickly as they are logged every one second. To stop these log message you can try the following:
  • Flap the interface, but this may or not fix the issue
set interfaces et-0/1/2 disable commit then rollback 
  • Add a syslog filter remove these messages

KB9382 [juniper.net] https://supportportal.juniper.net/s/article/Junos-Platform-Example-How-to-prevent-certain-syslog-messages-from-being-written-to-the-log-file


To recover the port, change the port speed to any of these 400/10/40 and then back to 100G. Please note this W/A will restart the PIC on which the affected interface is hosted.
set chassis fpc 0 pic 1 port 2 speed 10g commit then rollback 

This will help to destroy the IFD (physical interface et-x/y/z) and recreate it and previous down event will get processed correctly.

Here is an example for port et-0/1/2 and et-1/2/4 log messages for reference  

Apr 10 07:30:01  JTAC-MX960-RE0 fpc0 smic_filter_link_state_zt: Physical link et-0/1/2 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 

set chassis fpc 0 pic 0 port 0 number-of-sub-ports 4
set chassis fpc 0 pic 0 port 0 speed 10g
set chassis fpc 0 pic 0 port 1 speed 100g
set chassis fpc 0 pic 0 port 2 speed 100g
set chassis fpc 0 pic 0 port 3 number-of-sub-ports 4
set chassis fpc 0 pic 0 port 3 speed 10g
set chassis fpc 0 pic 0 port 4 speed 100g
set chassis fpc 0 pic 1 port 0 number-of-sub-ports 4
set chassis fpc 0 pic 1 port 0 speed 10g
set chassis fpc 0 pic 1 port 1 speed 100g
set chassis fpc 0 pic 1 port 2 speed 100g<<<< Change the speed to 10g commit, then change it back to 100g
set chassis fpc 0 pic 1 port 3 speed 100g
set chassis fpc 0 pic 1 port 4 speed 100g
set chassis fpc 0 pic 2 port 0 number-of-sub-ports 4
set chassis fpc 0 pic 2 port 0 speed 10g
set chassis fpc 0 pic 2 port 1 speed 100g
set chassis fpc 0 pic 2 port 2 speed 100g
set chassis fpc 0 pic 2 port 3 speed 100g
set chassis fpc 0 pic 2 port 4 speed 100g

Apr 10 07:30:01  JTAC-MX960-RE0 fpc1 smic_filter_link_state_zt: Physical link et-1/2/4 is now UP but the previous link down processing is still in progress Will defer interface UP until link down is complete 
set chassis fpc 1 pic 0 port 0 number-of-sub-ports 4
set chassis fpc 1 pic 0 port 0 speed 10g
set chassis fpc 1 pic 0 port 1 speed 100g
set chassis fpc 1 pic 0 port 2 speed 100g
set chassis fpc 1 pic 0 port 3 speed 100g
set chassis fpc 1 pic 0 port 4 speed 100g
set chassis fpc 1 pic 1 tunnel-services bandwidth 10g
set chassis fpc 1 pic 1 port 0 speed 100g
set chassis fpc 1 pic 1 port 1 speed 100g
set chassis fpc 1 pic 1 port 2 speed 100g
set chassis fpc 1 pic 1 port 3 speed 100g
set chassis fpc 1 pic 1 port 4 speed 100g
set chassis fpc 1 pic 2 port 0 number-of-sub-ports 4
set chassis fpc 1 pic 2 port 0 speed 10g
set chassis fpc 1 pic 2 port 1 speed 100g
set chassis fpc 1 pic 2 port 2 speed 100g
set chassis fpc 1 pic 2 port 3 speed 100g
set chassis fpc 1 pic 2 port 4 speed 100g<<<< Change the speed to 10g commit, then change it back to 100g

Bouncing the PIC on which the affected interface was tested and is a valid W/A for this issue. This will also destroy the IFD and recreate it :

 request chassis pic fpc-slot <fpc-slot-number> pic-slot <pic-slot-number> offline  
 request chassis pic fpc-slot <fpc-slot-number> pic-slot <pic-slot-number> online  

 

Modification History

05-31-2023 Initial Draft