Description

After upgrading ​​QFX5100 Virtual Chassis (2 qfx5100-96s and 1 qfx5100-24q)​ to 17.3R3-S4 Flex, some packet loss is observed due to CRC erros on the vcp port. 

Symptoms

[master:0]
root@QFX1> show virtual-chassis    

Preprovisioned Virtual Chassis
Virtual Chassis ID: 2ad6.1038.9a52
Virtual Chassis Mode: Enabled
                                                Mstr           Mixed Route Neighbor List
Member ID  Status   Serial No    Model          prio  Role      Mode  Mode ID  Interface
0 (FPC 0)  Prsnt    VB3715290014 qfx5100-96s-8q 129   Master*      N  VC   2  vcp-255/0/101
                                                                           1  vcp-255/0/96
1 (FPC 1)  Prsnt    VB3715320008 qfx5100-96s-8q 129   Backup       N  VC   0  vcp-255/0/96
                                                                           2  vcp-255/0/99
2 (FPC 2)  Prsnt    TB3716340189 qfx5100-24q-2p   0   Linecard     N  VC   0  vcp-255/1/0
                                                                           1  vcp-255/1/1​
[master:0]
root@QFX1> show virtual-chassis vc-port statistics vcp-255/1/0 extensive    
fpc0:
--------------------------------------------------------------------------
error: Statistics for vcp-255/1/0 not found

localre:
--------------------------------------------------------------------------
error: Statistics for vcp-255/1/0 not found

fpc2:
--------------------------------------------------------------------------

                           RX                     TX

Port: vcp-255/1/0     
Total octets:              620122064928           28196900785348
Total packets:             4428342022             201403702446 
Unicast packets:           4428341215             201403474057 
Broadcast packets:         807                    224069       
Multicast packets:         0                      4320         
MAC control frames:        0                      0            
CRC alignment errors:      195 <-- CRC errors  
Oversize packets:          0            
Undersize packets:         0            
Jabber packets:            0            
Fragments received:        0            
Ifout errors:              0            
64        octets frames:   0            
65-127    octets frames:   2279946      
128-255   octets frames:   205829233849 
256-511   octets frames:   128641       
512-1023  octets frames:   128641       
1024-1518 octets frames:   244854       
Rate packets per second:   844040                 1            
Rate bytes per second:     118166347              106  


[master:0]
root@QFX1> show version | match "Model|^Junos:" 
Model: qfx5100e-96s-8q
Junos: 17.3R3-S4.2 flex
Model: qfx5100e-96s-8q
Junos: 17.3R3-S4.2 flex
Model: qfx5100e-24q-2p
Junos: 17.3R3-S4.2 flex

Solution

The vcp port on the qfx5100-24q is not properly initializing.

Please refer to PR1444845 - CRC errors might be seen on QFX5100-VC for the list of fixed releases.

Reconfigure VCP connection on 24Q setup to initialize properly and CRC will no longer seen.

Here are the steps to do so:

  1. Login to the virtual chassis and connect to member 2 (the qfx5100-24q):

    [master:0]
    root@QFX1> show virtual-chassis    
    
    Preprovisioned Virtual Chassis
    Virtual Chassis ID: 2ad6.1038.9a52
    Virtual Chassis Mode: Enabled
                                                    Mstr           Mixed Route Neighbor List
    Member ID  Status   Serial No    Model          prio  Role      Mode  Mode ID  Interface
    0 (FPC 0)  Prsnt    VB3715290014 qfx5100-96s-8q 129   Master*      N  VC   2  vcp-255/0/101
                                                                               1  vcp-255/0/96
    1 (FPC 1)  Prsnt    VB3715320008 qfx5100-96s-8q 129   Backup       N  VC   0  vcp-255/0/96
                                                                               2  vcp-255/0/99
    2 (FPC 2)  Prsnt    TB3716340189 qfx5100-24q-2p   0   Linecard     N  VC   0  vcp-255/1/0
                                                                               1  vcp-255/1/1​
    
    [master:0]
    root@QFX1> request session member 2                           
    Last login: Wed Jan  8 22:09:54 from master
    
    --- JUNOS 19.1I-20191212.0.2159 Kernel 64-bit  JNPR-11.0-20191126.201716__ci_f
    At least one package installed on this device has limited support.
    Run 'file show /etc/notices/unsupported.txt' for details.
    root@QFX1:LC:2% cli
    warning: This chassis is operating in a non-master role as part of a virtual-chassis (VC) system.
    warning: Use of interactive commands should be limited to debugging and VC Port operations.
    warning: Full CLI access is provided by the Virtual Chassis Master (VC-M) chassis.
    warning: The VC-M can be identified through the show virtual-chassis status command executed at this console.
    warning: Please logout and log into the VC-M to use CLI.
    
  2. Delete one of the virtual chassis ports:

    {linecard:2}
    root@QFX1> request virtual-chassis vc-port delete pic-slot 1 port 0 
    Port deletion initiated, use cmd show virtual-chassis vc-port to verify
    
  3. Verify the port is deleted:

    {linecard:2}
    root@QFX1> show virtual-chassis           
    
    Preprovisioned Virtual Chassis
    Virtual Chassis ID: bd1c.c712.d06f
    Virtual Chassis Mode: Enabled
                                                    Mstr           Mixed Route Neighbor List
    Member ID  Status   Serial No    Model          prio  Role      Mode  Mode ID  Interface
    0 (FPC 0)  Prsnt    VB3119310003 qfx5100-96s-8q 129   Master       N  VC   1  vcp-255/0/98
                                                                               1  vcp-255/0/97
                                                                               1  vcp-255/0/99
    1 (FPC 1)  Prsnt    VB3119300023 qfx5100-96s-8q 129   Backup       N  VC   0  vcp-255/0/98
                                                                               2  vcp-255/0/101
                                                                               0  vcp-255/0/97
                                                                               0  vcp-255/0/99
    2 (FPC 2)  Prsnt    VG3716010077 qfx5100-24q-2p   0   Linecard*    N  VC   1  vcp-255/1/1
    
  4. Convert the port back to vcp:

    {linecard:2}
    root@QFX1> request virtual-chassis vc-port set pic-slot 1 port 0    
    Port conversion initiated,  use show virtual-chassis vc-port to verify
  5. Verify the port is converted back to vcp:

    {linecard:2}
    root@QFX1> show virtual-chassis                                     
    
    Preprovisioned Virtual Chassis
    Virtual Chassis ID: bd1c.c712.d06f
    Virtual Chassis Mode: Enabled
                                                    Mstr           Mixed Route Neighbor List
    Member ID  Status   Serial No    Model          prio  Role      Mode  Mode ID  Interface
    0 (FPC 0)  Prsnt    VB3119310003 qfx5100-96s-8q 129   Master       N  VC   1  vcp-255/0/98
                                                                               2  vcp-255/0/101
                                                                               1  vcp-255/0/97
                                                                               1  vcp-255/0/99
    1 (FPC 1)  Prsnt    VB3119300023 qfx5100-96s-8q 129   Backup       N  VC   0  vcp-255/0/98
                                                                               2  vcp-255/0/101
                                                                               0  vcp-255/0/97
                                                                               0  vcp-255/0/99
    2 (FPC 2)  Prsnt    VG3716010077 qfx5100-24q-2p   0   Linecard*    N  VC   0  vcp-255/1/0
                                                                               1  vcp-255/1/1
  6. Confirm CRC is no longer seen:

    {linecard:2}
    root@QFX1> show virtual-chassis vc-port statistics vcp-255/1/0 extensive 
    fpc0:
    --------------------------------------------------------------------------
    error: Statistics for vcp-255/1/0 not found
    fpc1:
    --------------------------------------------------------------------------
    error: Statistics for vcp-255/1/0 not found
    localre:
    --------------------------------------------------------------------------
                               RX                     TX
    Port: vcp-255/1/0     
    Total octets:              135644                 200081       
    Total packets:             728                    722          
    Unicast packets:           728                    715          
    Broadcast packets:         0                      7            
    Multicast packets:         0                      0            
    MAC control frames:        0                      0            
    CRC alignment errors:      0            
    Oversize packets:          0            
    Undersize packets:         0            
    Jabber packets:            0            
    Fragments received:        0            
    Ifout errors:              0            
    64        octets frames:   0            
    65-127    octets frames:   1246         
    128-255   octets frames:   2            
    256-511   octets frames:   23           
    512-1023  octets frames:   23           
    1024-1518 octets frames:   101          
    Rate packets per second:   50                     55           
    Rate bytes per second:     5104                   7846 
    

Related Information