Description

The article introduce an i2c timedout err log " tcbc i2c accelerator error:  Write failed for (PEM.*) group .* device .* bytes 1 err 60 "

Symptoms

The i2c write timedout error might happened immediately after a new PSM insertion.

Mar  8 11:28:50 2024  Router chassisd[23505]: %DAEMON-5-CHASSISD_SNMP_TRAP7: SNMP trap generated: FRU insertion (jnxFruContentsIndex 22, jnxFruL1Index 10, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName PSM 9, jnxFruType 18, jnxFruSlot 9)
Mar  8 11:28:54 2024  Router kernel: %KERN-3: tcbc i2c accelerator error: Group 0x30 device 0x52 cmd timedout 16108 usecs
Mar  8 11:28:54 2024  Router kernel: %KERN-3: tcbc i2c accelerator error:  Write failed for (PEM9) group 0x30 device 0x52 bytes 1 err 60
Mar  8 11:28:54 2024  Router kernel: %KERN-3: tcbc i2c accelerator error: Group 0x30 device 0x51 cmd timedout 17117 usecs
Mar  8 11:28:54 2024  Router kernel: %KERN-3: tcbc i2c accelerator error:  Write failed for (PEM9) group 0x30 device 0x51 bytes 1 err 60

<< log chassisd
Mar  8 11:28:54  PSM 9 i2c slave not ok, status 0x8 failed 12 times
Mar  8 11:28:54  PSM 9 i2c slave not ok, status 0x8 failed 1 times

Solution

As the i2c access has retry mechanism , hence different action is recommended based on the frequency of i2c error:

  1. For one time i2c error, which just indicates a transient i2c issue between RE and corresponding PSM , so keep monitoring without an immediate action.
  2. For persist i2c error on an online PSM , it is usually recommended to reseat the PSM first, and replace it if same issue reoccur.
  3. For persist i2c error on a new added PSM , please also check the history of the PSM slot , the previous defect PSM (if there is) might left a error state on the slot , so RE reboot/switchover should be considered before replacing the new added PSM.

Modification History

2024-03-08 : Article Created