The "CMD FSM state error" message reports that the Forwarding State Machine of the reorder queue received an unexpected value.

This is a Troubleshooting Article for a PFE ASIC Syslog Event.
To view other documented syslog events related to XMCHIP, XLCHIP, MQCHIP, LUCHIP, EACHIP, and PECHIP, see KB31893 - Master Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] .

.

When a "CMD FSM state error" event occurs, a message similar to the following is reported:
Note : Either prior or post CMD FSM state error, you will also see a "CMD reorder ID error" error message.

Nov 30 14:15:19.464 atlas-re0 : %PFE-3: fpc8 MQSS(0): DRD: RORD1: CMD reorder ID error - Command 13, Reorder ID 0, QID 30720
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 Cmerror: Draining ASIC error message queue
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 cmerror_process_queue: module = MQSS(0)
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 Cmerror: processing the task op_type 1 for level 1 level_count 0 occur_count 0 clear_count 0 level_threshold 1 level_action 0x4 item errid 2229195 item_threshold 1 item_count 0 item_sub_err_state 0 sub_item errid 0 sub_item_state 0 item_times
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 Cmerror: Level 1 count increment 1 occur_count 1 clear_count 0
Nov 30 14:15:19.464 atlas-re0 : %PFE-6: fpc8 Error: /fpc/8/pfe/0/cm/0/MQSS(0)/0/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cb), severity: major, module: MQSS(0), type: Description for MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 Cmerror: Level 1 count 1 (occur_count 1 clear_count 0)crossed threshold 1 action 0x4
Nov 30 14:15:19.464 atlas-re0 : %PFE-7: fpc8 cmerror_take_action_helper: performing action 4 for level 1 err_id /fpc/8/pfe/0/cm/0/MQSS(0)/0/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cb) module id 24
Nov 30 14:15:19.464 atlas-re0 : %PFE-3: fpc8 Cmerror Op Set: MQSS(0): MQSS(0): DRD: RORD1: CMD FSM state error - Command 13, Reorder ID 0, QID 0, FSM State 5

Indications:

• If there are no other error messages and further events reported, there is no operational impact

• If there is operational impact, the following additional events may be reported:

  • KB32168 - Syslog message: HOST LOOPBACK WEDGE DETECTED [juniper.net]

• Alarm will be raised

root@LAB_RE0> show chassis fabric fpcs no-forwarding

Fabric management FPC state:

FPC 4
...
  PFE #4
             : Fabric Disabled
  PFE #5

Apr 17 03:19:04  LAB_RE0 fpc4 MQSS(4): DRD: RORD1: CMD reorder ID error - Command 13, Reorder ID 1656, QID 5570 
Apr 17 03:19:04  LAB_RE0 fpc4 CMError: /fpc/4/pfe/0/cm/0/MQSS(4)/4/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cc), scope: pfe, category: functional, severity: major, module: MQSS(4), type: DRD_RORD_ENG_INT: CMD FSM State Error, oc_category: default 
Apr 17 03:19:04  LAB_RE0 fpc4 Performing action get-state for error /fpc/4/pfe/0/cm/0/MQSS(4)/4/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cc) in module: MQSS(4) with scope: pfe category: functional level: major, oc_category: default 
Apr 17 03:19:04  LAB_RE0 inetd[17152]: Number of tftp connections at max limit (1)
Apr 17 03:19:05  LAB_RE0 fpc4 MQSS(4): DRD: Error: Fabric reorder ID timeout error - Valid 1, Reorder ID 1657 
Apr 17 03:19:05  LAB_RE0 fpc4 cmtfpc_mqss_drd_reorder_id_timeout_callback: Injecting recovery parcel PFE/CC/DRD/RID=4/0/1/1657, Data = d16790200000000000000000000000000000 
Apr 17 03:19:05  LAB_RE0 fpc4 MQSS(4): DRD: RORD1: CMD reorder ID error - Command 13, Reorder ID 2084, QID 346 
Apr 17 03:19:06  LAB_RE0 fpc4 MQSS(4): DRD: Error: Fabric reorder ID timeout error - Valid 1, Reorder ID 1925 
Apr 17 03:19:06  LAB_RE0 fpc4 cmtfpc_mqss_drd_reorder_id_timeout_callback: Injecting recovery parcel PFE/CC/DRD/RID=4/0/1/1925, Data = d17850200000000000000000000000000000 
Apr 17 03:19:43  LAB_RE0 fpc4 Performing action cmalarm for error /fpc/4/pfe/0/cm/0/MQSS(4)/4/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cc) in module: MQSS(4) with scope: pfe category: functional level: major, oc_category: default 
Apr 17 03:19:43  LAB_RE0 fpc4 Performing action disable-pfe for error /fpc/4/pfe/0/cm/0/MQSS(4)/4/MQSS_CMERROR_DRD_RORD_ENG_INT_REG_CMD_FSM_STATE_ERR (0x2203cc) in module: MQSS(4) with scope: pfe category: functional level: major, oc_category: default 
Apr 17 03:19:43  LAB_RE0 fpc4 PFE 4: 'PFE Disable' action performed. Bringing down ifd et-4/4/0 459 

This indicates the lookup microcode within LUSS is sending an incorrect command parcel to the MQSS Center Chip.

{DIDYOUKNOWSERVICENOWTOKEN.EN_US}

Perform these steps to determine the cause and resolve the problem (if any).  Continue through each step until the problem is resolved.

1. Collect the show command output.

   {SYSLOGSERVICENOWTOKEN.EN_US}

   Capture the output to a file (in case you have to open a technical support case). To do this, configure each SSH client/terminal emulator to log your session.

   show log messages
show log chassisd
start shell network pfe <fpc#>
show nvram
show syslog messages
exit

2. Analyze the show command output.

   In the 'show log messages', review the events that occurred at or just before the appearance of the "CMD FSM state error" message. Frequently these events help identify the cause.

• No RMA is required

• Reboot FPC using below command to recover PFE:
  request chassis fpc slow <fpc-slot> offline|online

• Contact your technical support representative if this event is seen repeatedly

This article is indexed in  KB31893 - Primary Index of Articles for Troubleshooting PFE ASIC Syslog Events [juniper.net] ; tag EACHIPTSG

Note: KB Team - All changes to this article must be approved by the AI-Scripts Review team ( [email protected] ) before re-publishing.