Description

After Upgrades to EX4400-MP | EX4100-MP | EX4400-T | EX4300-MP | EX4400-P standalone or VC devices to  22.4R3-S3; we may observe that after restart some of the interfaces may not come up. 

Symptoms

 

On the device; if you check the Junos CLI shows interfaces as down:

 

router> show interfaces terse | match down

mge-1/0/30              up    down
mge-1/0/30.0            up    down eth-switch
mge-1/0/31              up    down
mge-1/0/31.0            up    down eth-switch

 

On the device; if you check the BCM interface will be seen in STP block state:

 

router> start shell
router%
router% cprod -A fpc1 -c 'set dcbcm bcmshell "ps"'

HW (unit 0)
                  ena/        speed/ link auto    STP                                            lrn  inter   max   cut   loop        
           port  link  Lns   duplex scan neg?   state   pause  discrd ops   face frame  thru?  back   encap
      ge30( 28)  up     1    1G  FD   SW  No     Block          None   FA  SGMII  1518                IEEE      ---> Interface seen as in "STP BLOCK" State
      ge31( 27)  up     1  100M  FD   SW  No     Block  TX RX   None   FA  SGMII  1518                IEEE

 

Solution

Root Cause of the problem:

  • Interface UP/DOWN events are queued and delivered to PFEMAN through a socket that is also used for transmitting interface statistics.
  • In the existing design, EX/QFX platforms export per-port queue statistics for 12 queues using 6 IPC messages per port. With this fix, the queue statistics response sent to the PFEMAN socket has been optimized and consolidated into 2 IPC messages per port, significantly reducing the message volume.
  • The issue occurs when the IPC pipe becomes congested. Under such conditions, IPC write operations can fail, preventing interface state updates from being propagated from the kernel to the PFE. As a result, the PFE may not receive the latest link-state notifications.
  • This can lead to a condition where the link state of random ports on backup or line cards is reported as DOWN at the IFD level, even though the physical link remains operational and UP at the hardware level.
  • By reducing the number of IPC messages required for queue statistics reporting, this fix lowers socket congestion and minimizes the likelihood of interface state update failures.

 

Fix Releases (per the latest PR):

 

junos:23.4R2-S8 junos:24.2R2-S4 junos:24.2X4 junos:24.4R2-S3 junos:25.2R2-S1 junos:25.4R1-S1 junos:25.4R2 junos:26.1R1 junos:26.2R1

Modification History

2025-11-10 : Article Created

2026-08-02: Updated Solution and Internal sections.