Physical interface remains stable, only MACSEC sessions get terminated after running for sometime over 10G interface. 4x25G SFP28 extension module is being used. MACSEC trace logs & message logs doesn't have conclusive alerts logged during issue time. xxxxxx> show security macsec connections Jan 13 11:12:59 Interface name: xe-0/2/0 CA name: MACSEC-DCI Cipher suite: GCM-AES-128 Encryption: on Key server offset: 0 Include SCI: yes Replay protect: off Replay window: 0 Outbound secure channels SC Id: xx:xx:xx:xx:xx:aa/1 Outgoing packet number: 85069793 Secure associations AN: 1 Status: inuse Create time: 00:03:32 Inbound secure channels SC Id: xx:xx:xx:xx:xx:bb/1 Secure associations AN: 1 Status: inuse Create time: 00:03:32 Interface name: xe-0/2/1 CA name: MACSEC-DCI Cipher suite: GCM-AES-128 Encryption: on Key server offset: 0 Include SCI: yes Replay protect: off Replay window: 0 Outbound secure channels SC Id: xx:xx:xx:xx:xx:cc/1 Outgoing packet number: 0 Secure associations AN: 0 Status: inuse Create time: 1d 01:13:25 AN: 1 Status: inuse Create time: 17:01:12 AN: 2 Status: inuse Create time: 08:49:05 AN: 3 Status: inuse Create time: 00:36:55 Inbound secure channels SC Id: xx:xx:xx:xx:xx:cc/1 Secure associations AN: 0 Status: inuse Create time: 1d 01:13:25 AN: 1 Status: inuse Create time: 17:01:12 AN: 2 Status: inuse Create time: 08:49:05 AN: 3 Status: inuse Create time: 00:36:55 MacSec configuration on on the 4x25G SFP28 port extension module: security { macsec { traceoptions { file MACSEC-TRACE size 10m files 5; flag normal; } connectivity-association MACSEC-DCI { security-mode static-cak; mka { sak-rekey-interval 28800; key-server-priority 100; } include-sci; pre-shared-key { ckn 8ABCC459BB10567AB150C248E8CAEB5981D8AEE7F25CF2B4C5AD76CD547B1187; } exclude-protocol lldp; exclude-protocol lacp; } interfaces { xe-0/2/0 { connectivity-association MACSEC-DCI; } xe-0/2/1 { connectivity-association MACSEC-DCI; } } } }
MACsec is configured over direct connections and configured with AES-128 cipher suite. There were few errors seen during the rekey (automatic PN expiry and SA switching) PFE_MACSEC_FUNC_FAIL: BCM MACsec Error: API bcm_plp_europa_secy_sa_chain(&cfye_pa, active_cfye_sa_handle, &cfye_sa_handle, &cfye_sa_params) at tvp_drv_bcm_macsec_tx_sa_create:917 -> 2 Post this - the MACsec programming didnt happen on the h/w and traffic was dropped. sak-rekey-interval 28800; This additional sak-rekey triggered through control plane might interfere with the auto SA rekey happening from h/w.
Recomendation: 1) XPN cipher suite with sak-rekey configuration retained - preferred 2) Non XPN cipher suite with sak-rekey configuration removed Use XPN cipher suite,This will avoid auto rekey with less traffic,sak-rekey-interval works fine with XPN cipher suite. Configured XPN cipher resolved this issue.
set security macsec connectivity-association MACSEC-XX cipher-suite gcm-aes-xpn-128