Description

MPC11 MACSec issues

Symptoms

MX2010-with MPC11 configured for AE and MACSEC. AE0 has 4 legs and 2 of the legs are not going into CD state but stay in a default/detached state. Removing MACSEC or configuring MACSEC to excluded LACP for encryptions brings up all legs into a CD state. From looking at the data, seems like port et-7/2/1 is not sending encrypted packets at all while the MX304 et-0/1/5 is.   Everything points to the MPC11 not TX encrypted packets.

 

 

Topology:

MX2010            AE0            MX304

       -et-0/2/1---100g----et-0/0/5

       -et-2/2/1---100g----et-0/0/13

       -et-7/2/1---100g----et-0/1/5 ****default/attached

       -et-0/5/1---100g----et-0/1/13****default/attached

 

Configuration:

 

  ae0 {

    apply-groups INTERFACES_BACKBONE;

    description x

    aggregated-ether-options {

      lacp {

        active;

      }

    }

    unit 0 {

      family inet {

        address 64.4.244.188/31;

      }

      family inet6 {

        address 2620:110:f00f:ffff::64.4.244.188/127;

      }

    }

security {

  authentication-key-chains {

    key-chain macsec-keys {

      key 1 {

        key-name x;

        start-time x

 

  macsec {

    connectivity-association macsec-data {

      cipher-suite gcm-aes-xpn-256;

      security-mode static-cak;

      mka {

        should-secure;

      }

      pre-shared-key-chain macsec-keys;

    }

    interfaces {

 

 interfaces {

      et-0/1/1 {

        connectivity-association macsec-data;

      }

      et-0/2/1 {

        connectivity-association macsec-data;

      }

      et-0/5/1 {

        connectivity-association macsec-data;

      }

      et-0/6/1 {

        connectivity-association macsec-data;

      }

      et-2/1/1 {

        connectivity-association macsec-data;

      }

      et-2/2/1 {

        connectivity-association macsec-data;

      }

      et-2/3/1 {

        connectivity-association macsec-data;

      }

      et-2/6/1 {

        connectivity-association macsec-data;

      }

      et-7/2/1 {

        connectivity-association macsec-data;

      }

      et-7/3/1 {

        connectivity-association macsec-data;

      }

    }

  }

-----------------------------------------------------------------------------

AE0 is directly connected to a MX304. The MX2010 MPC11 seems not to be sending any encrypted packets.  

 

Example:

 

MX2020: Focus is on et-7/2/1 but same issue seen on et-0/5/1.

ruvora@JNRP-re0> show lacp interfaces ae0 

Aggregated interface: ae0

  LACP state:    Role  Exp  Def Dist Col Syn Aggr Timeout Activity

   et-2/2/1    Actor  No  No  Yes Yes Yes  Yes   Fast  Active

   et-2/2/1   Partner  No  No  Yes Yes Yes  Yes   Fast  Active

   et-0/2/1    Actor  No  No  Yes Yes Yes  Yes   Fast  Active

   et-0/2/1   Partner  No  No  Yes Yes Yes  Yes   Fast  Active

   et-0/5/1    Actor  No  Yes  No  No  No  Yes   Fast  Active

   et-0/5/1   Partner  No  Yes  No  No  No  Yes   Fast  Passive

   et-7/2/1    Actor  No  Yes  No  No  No  Yes   Fast  Active

   et-7/2/1   Partner  No  Yes  No  No  No  Yes   Fast  Passive

  LACP protocol:    Receive State Transmit State     Mux State 

   et-2/2/1         Current  Fast periodic Collecting distributing

   et-0/2/1         Current  Fast periodic Collecting distributing

   et-0/5/1        Defaulted  Fast periodic      Detached

   et-7/2/1        Defaulted  Fast periodic      Detached

 

ruvora@JNRP-re0> show security macsec connections interface et-7/2/1 

    CA name: macsec-data  

    Cipher suite: GCM-AES-XPN-256 Encryption: on

    Key server offset: 0    Include SCI: no

    Replay protect: off     Replay window: 0

     Outbound secure channels

      SC Id: B4:F9:5D:C4:C0:97/1

      Outgoing packet number: 1

      Secure associations

      AN: 1 Status: inuse Create time: 1d 06:52:09

     Inbound secure channels

      SC Id: B4:F9:5D:8A:45:12/1

      Secure associations

      AN: 1 Status: inuse Create time: 1d 06:52:09

 

uvora@JNRP-re0> show security macsec statistics interface et-7/2/1 

  Secure Channel transmitted

    Encrypted packets: 0******NOT SENDING

    Encrypted bytes:  0

    Protected packets: 0

    Protected bytes:  0

  Secure Association transmitted

    Encrypted packets: 0

    Protected packets: 0

  Secure Channel received

    Accepted packets: 0

    Validated bytes:  0

    Decrypted bytes:  0

  Secure Association received

    Accepted packets: 0

    Validated bytes:  0

    Decrypted bytes:  0

 

ruvora@JNRP-re0> show configuration | display set | match et-7/2/1 

set security macsec interfaces et-7/2/1 connectivity-association macsec-data

set interfaces et-7/2/1 description x

set interfaces et-7/2/1 gigether-options 802.3ad ae0

 

 

 

 

ruvora@JNRP-re0> start shell pfe network fpc7 

root@JNRP-re0-fpc7:pfe> show interfaces ae0 

 Name: ae0      Index: 128  IflCount: 1   Type: 29     Weight: 1   

 

 CfgState:   Up  OverallState: Up        GlobalSlot: 255 PfeInst: 255

 Local:    Yes  IsAggregate: Yes       MTU: 9192    PfeId:  0

 ChassisId:  0   ChassisLocal: Yes       LocalSlot: 255 

 LinkState:  Up  Macsec:    Disabled     Pic: 255    PicPort: 0

 VlanEnabled: False

 ChannelCount: 0                  IfdSpeed:   200000000000

 

 Flags:      0x0000000000008000

 SpecificFlags:  0x0000000000000000

 MacAddress:   b6:f9:5d:c4:bb:02

 InitTime:    Tue Jun 4 23:55:18 2024

 StateChangeTime: Mon Jun 17 03:59:05 2024

 GE Flags:

     lpbk: False  flowCtrl: False  lacp:   False

     lldp: False  padFrame: False  restrictQ: False

 

 No Tpids

 

 OutputStream: 65535

 StreamNumber: 65535

 StreamToken: None

 StreamFlags: 0x00

 IIFTable: Null

  

 Redundancy States:

 [0] selectorToken:14390 listToken:14391 unilistToken:14392

  selMode:2 type:17 aggType:4 selType:2 selSubtype:0 unilistMode:5

  targeting:No adaptive:No randomMode:No rotateHash:No local-bias:No

  Steering (chassisId:0 pfeId:10 nhToken:2351 fwdToken:14409)

  Use Count:3 Operational Weight:1

  Default links:

   [0] et-2/2/1 token:14409 hits:0

   [1] et-2/2/1 token:14409 hits:0

   [2] et-2/2/1 token:14409 hits:0

   [3] et-2/2/1 token:14409 hits:0

   [4] et-2/2/1 token:14409 hits:0

   [5] et-2/2/1 token:14409 hits:0

   [6] et-2/2/1 token:14409 hits:0

   [7] et-2/2/1 token:14409 hits:0

  

  IfdName       IfdIndex LinkIndex Weight  OperState SelectorToken

  et-2/2/1      190    64    1     Up    2941

  et-7/2/1      218    64    1     Down   14388

  et-0/2/1      263    64    1     Up    8950

  et-0/5/1      272    64    1     Down   14336

 

 802.1BR Extended Port:  EC-ID: 0    Satellite-Id: 0

 

 

 Aggregate member list

 IfdIndex   Name         Weight  Activated State  ActiveToken

 190     et-2/2/1       1     Yes    Up   2942

 218     et-7/2/1       1     No     Down  14389

 263     et-0/2/1       1     Yes    Up   8951

 272     et-0/5/1       1     No     Down  14337

 

oot@JNRP-re0-fpc7:pfe> show interfaces et-7/2/1 

 Name: et-7/2/1   Index: 218  IflCount: 1   Type: 172    Weight: 1   

 

 CfgState:   Up  OverallState: Up        GlobalSlot: 7  PfeInst: 2

 Local:    Yes  IsAggregate: No        MTU: 9192    PfeId:  30

 ChassisId:  0   ChassisLocal: Yes       LocalSlot: 7  

 LinkState:  Up  Macsec:    Enabled     Pic: 2     PicPort: 11

 VlanEnabled: False

 ChannelCount: 1                  IfdSpeed:   100000000000

 

 Flags:      0x0000000000008000

 SpecificFlags:  0x0000000000100000

 MacAddress:   b6:f9:5d:c4:bb:02

 InitTime:    Tue Jun 4 23:55:41 2024

 StateChangeTime: Tue Jun 18 11:10:38 2024

 GE Flags:

     lpbk: False  flowCtrl: False  lacp:   True

     lldp: True   padFrame: False  restrictQ: False

 

  tpid[0] = 0x8100

 

  

 

 OutputStream: 1100

 StreamNumber: 696

 StreamToken: 3067

 StreamFlags: 0xa5 (LACP | ESMC | LLDP | DOT1X)

 FeatureList:

 Container token: 3065

 #6 IIFLkup tokens:

 Mask : 0x1

 [ IIFLkup:3068 ]

  

 IIFTableToken: 3068

 VlanTblSize: 0

 

 802.1BR Extended Port:  EC-ID: 1    Satellite-Id: 0

 

oot@JNRP-re0-fpc7:pfe> show macsec_drv list 

 

 

 

 ASIC_INDEX | PORT_GROUP | KEY_INDEX |     KEY_NAME

     0 |     0 |    210 |     et-7/0/0

     0 |     4 |    211 |     et-7/0/1

     1 |     0 |    216 |     et-7/1/4

     1 |     1 |    215 |     et-7/1/3

     1 |     2 |    213 |     et-7/1/1

     1 |     3 |    212 |     et-7/1/0

     1 |     4 |    214 |     et-7/1/2

     2 |     0 |    221 |     et-7/2/4

     2 |     1 |    220 |     et-7/2/3

     2 |     2 |    218 |     et-7/2/1

 

 

root@JNRP-re0-fpc7:pfe> show macsec_drv 218 dump_ds 

 

 

MACSEC_DRV_DUMP_PORT [0x7fd23d7dc800]:

     MD port lookup key 218 lookup name[ et-7/2/1 ]: index 0, AN 0, SA idx 0 SC idx 0, parent_ptr[0x7fd23d75beb0]

  TX SC

     mem0: protframe 1, xpnmode 1 ptextoff 0 ptextsz 0 sci -5406186778055475199 stagoff 0 saindex1Valid 1 index1 128

     mem1: saindex0Valid 1 index0 0 npn 1

     mem2: keytype 1 ssci 1

  RX SC

     mem0: ptextoff 0 ptextsz 0 sci -5406187029235892223 sa index0 0 index1 128 index2 256 index3 384

     mem1: keytype 1 xpnmode 1 ssci 0

TX SA

RX SA

 

MACSEC_DRV_DUMP_PG [0x7fd23d75beb0]:

    debug_flag 0 index 2 port_count 10 vport_count 32 mdpg_fake_kats_err 0 fcs_bytes_enable 0

    rx_reg_addr 0x60210000 tx_reg_addr 0x60248000 parent_ptr 0x7fd1097de2c0 child_ptr 0x7fd23d7dc800 vchild_ptr 0x7fd1099c6540

 

MACSEC_DRV_DUMP_DRV [0x7fd1097de2c0]:

    debug flag 1 lookup key 2 pg count 5 child_ptr 0x7fd23d75be00

 

root@JNRP-re0-fpc7:pfe> show macsec_drv 218 rx_stats 

 

 

 in_pkts_notusing_sa :  in_pkts_unused_sa :      in_pkts_ok :   in_pkts_invalid :  in_pkts_not_valid

=======================================================================================================

          0 :          0 :          0 :          0 :          0 - sa[ 0]

          0 :          0 :          0 :          0 :          0 - sa[ 1]

          0 :          0 :          0 :          0 :          0 - sa[ 2]

          0 :          0 :          0 :          0 :          0 - sa[ 3]

 

=======================================================================================================

    in_pkts_unchk : 0

   in_pkts_delayed : 0

    in_pkts_late : 0

    stat_untagged : 0

     stat_notag : 0

     stat_badtag : 0

  stat_unknown_sci : 0

     stat_no_sci : 0

    InPktsOverrun : 0

  octets_validated : 0

  octets_decrypted : 0

 link_drop_pkt_count : 0

  ingress_pkt_count : 0

  egress_pkt_count : 0

 runt_pkt_drop_count : 0

 

root@JNRP-re0-fpc7:pfe> show macsec_drv 218 tx_stats    

 

 

 out_pkts_protected :  out_pkts_encrypted

====================================================

          0 :          0 - sa[ 0]

          0 :          0 - sa[ 1]

 

====================================================

     stat_unctrl : 0

  untagged Packets : 0

   OutPktsTooLong : 0

      stat_err : 0

  octets_protected : 0

  octets_encrypted : 0

 link_drop_pkt_count : 0

  ingress_pkt_count : 0

  egress_pkt_count : 0

egress_pkt_drop_count : 0

 

 

ruvora@JNRP-re0# run show lacp statistics interfaces ae0 

Jun 18 12:40:21

Aggregated interface: ae0

  LACP Statistics:    LACP Rx   LACP Tx  Unknown Rx  Illegal Rx 

   et-2/2/1        2475    2475      0      0

   et-0/2/1        2475    2475      0      0

   et-7/2/1          3     66      0      0

   et-0/5/1          0    2475      0      0

 

{master}[edit]

ruvora@JNRP-re0# run show lacp statistics interfaces ae0   

Jun 18 12:40:24

Aggregated interface: ae0

  LACP Statistics:    LACP Rx   LACP Tx  Unknown Rx  Illegal Rx 

   et-2/2/1        2478    2478      0      0

   et-0/2/1        2478    2479      0      0

   et-7/2/1          3     69      0      0

   et-0/5/1          0    2479      0      0

 

{master}[edit]

ruvora@JNRP-re0# run show lacp statistics interfaces ae0   

Jun 18 12:40:25

Aggregated interface: ae0

  LACP Statistics:    LACP Rx   LACP Tx  Unknown Rx  Illegal Rx 

   et-2/2/1        2480    2480      0      0

   et-0/2/1        2480    2480      0      0

   et-7/2/1          3     71      0      0

   et-0/5/1          0    2480      0      0

 

{master}[edit]

ruvora@JNRP-re0# run show lacp statistics interfaces ae0   

Jun 18 12:40:27

Aggregated interface: ae0

  LACP Statistics:    LACP Rx   LACP Tx  Unknown Rx  Illegal Rx 

   et-2/2/1        2481    2481      0      0

   et-0/2/1        2481    2482      0      0

   et-7/2/1          3     72      0      0

   et-0/5/1          0    2482      0      0

 

Solution

https://gnats.juniper.net/web/default/1811300#external_tab

 

On Junos EVO MX2008/MX2010/MX2020 platforms with MX2K-MPC11E line cards, and MACSec (IEEE 802.1AE standard) configured on line card ports. When line card comes online for first time, it is seen that ports are not being mapped correctly (port group value mismatch between picd and security) resulting in MACSec not working on some ports.

 

Modification History

2024-09-29 : Article Created