An ACX7024 acting as a boundary router running Interprovider Option B:
https://www.juniper.net/documentation/us/en/software/junos/vpn-l3/topics/topic-map/l3-vpns-interprovider.html
Is seeing an issue with MPLS EXP bit marking for Class of Service. The ACX is an ASBR running Option B in the following topology:
Host1 --- [PE1 (vrf1)] --- [ASBR] --- [ACX] --- [PE2 (vrf1)] --- Host2
The ACX runs EBGP and peers with the ASBR:
set protocols bgp group AS577-EBGPB type external
set protocols bgp group AS577-EBGPB family inet-vpn unicast
set protocols bgp group AS577-EBGPB family inet6-vpn unicast
set protocols bgp group AS577-EBGPB peer-as 64577
set protocols bgp group AS577-EBGPB neighbor 10.77.3.0
As well as an IBGP session to PE2 where the VRF resides:
set protocols bgp group iBGP type internal
set protocols bgp group iBGP local-address 10.70.219.80
set protocols bgp group iBGP family inet-vpn unicast
set protocols bgp group iBGP family inet6-vpn unicast
set protocols bgp group iBGP peer-as 65500
set protocols bgp group iBGP neighbor 10.70.219.82
set protocols bgp group iBGP vpn-apply-export
The VRF label and corresponding routes tied to the instance on PE1 is advertised to the ASBR:
labroot@f22-38> show route advertising-protocol bgp 10.77.3.0 detail
bgp.l3vpn.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
* 167.70.219.82:10:120.0.0.0/24 (1 entry, 1 announced)
BGP group AS577-EBGPB type External
Route Distinguisher: 167.70.219.82:10
VPN Label: 18
Nexthop: Self
Flags: Nexthop Change
AS path: [65500] I
Communities: target:65001:200
* 167.70.219.82:10:200.0.0.220/32 (1 entry, 1 announced)
The interface connected to the ASBR on the ACX is ae5:
set interfaces ae5 flexible-vlan-tagging
set interfaces ae5 encapsulation flexible-ethernet-services
set interfaces ae5 aggregated-ether-options lacp active
set interfaces ae5 aggregated-ether-options lacp periodic fast
set interfaces ae5 unit 1 vlan-id 1
set interfaces ae5 unit 1 family inet address 10.77.3.1/31
set interfaces ae5 unit 1 family iso
set interfaces ae5 unit 1 family mpls maximum-labels 8
set interfaces lo0 unit 0 family inet address 10.70.219.80/32
With the peering to PE2 over et-0/0/4:
set interfaces et-0/0/4 flexible-vlan-tagging
set interfaces et-0/0/4 encapsulation flexible-ethernet-services
set interfaces et-0/0/4 unit 2 vlan-id 2
set interfaces et-0/0/4 unit 2 family inet address 10.0.0.1/24
set interfaces et-0/0/4 unit 2 family iso
set interfaces et-0/0/4 unit 2 family mpls maximum-labels 8
CoS has ae5 set for classifiers/rewrite:
set class-of-service interfaces ae5 unit 1 classifiers exp COREEXP
set class-of-service interfaces ae5 unit 1 classifiers inet-precedence IPPREC
set class-of-service interfaces ae5 unit 1 rewrite-rules exp EXP-REWRITE
set class-of-service classifiers exp COREEXP forwarding-class AF loss-priority high code-points EXP3
set class-of-service classifiers exp COREEXP forwarding-class BE loss-priority high code-points EXP0
set class-of-service classifiers exp COREEXP forwarding-class BE loss-priority high code-points EXP1
set class-of-service classifiers exp COREEXP forwarding-class BE loss-priority high code-points EXP2
set class-of-service classifiers exp COREEXP forwarding-class BE loss-priority high code-points EXP4
set class-of-service classifiers exp COREEXP forwarding-class BE loss-priority high code-points EXP6
set class-of-service classifiers exp COREEXP forwarding-class EF loss-priority low code-points EXP5
set class-of-service classifiers exp COREEXP forwarding-class NC loss-priority low code-points EXP7
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class AF loss-priority high code-point EXP3
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class AF loss-priority low code-point EXP3
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class BE loss-priority high code-point EXP0
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class BE loss-priority low code-point EXP0
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class EF loss-priority high code-point EXP5
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class EF loss-priority low code-point EXP5
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class NC loss-priority high code-point EXP7
set class-of-service rewrite-rules exp EXP-REWRITE forwarding-class NC loss-priority low code-point EXP7
With the same on et-0/0/4:
set class-of-service interfaces et-0/0/4 unit 2 classifiers exp COREEXP
set class-of-service interfaces et-0/0/4 unit 2 classifiers inet-precedence IPPREC
set class-of-service interfaces et-0/0/4 unit 2 rewrite-rules exp EXP-REWRITE
The ACX has a swap action for label 18 to label 16 out et-0/0/4.2:
labroot@f22-38> show route label 18
mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
18 *[VPN/170] 1w0d 17:07:23, metric2 10, from 167.70.219.82
> to 2.0.0.2 via et-0/0/4.2, Swap 16
Traffic is sent to that destination. When checked, it is going out over forwarding-class AF:
labroot@f22-38> show interfaces queue et-0/0/4 egress forwarding-class AF
Physical interface: et-0/0/4, up, Physical link is Up
Interface index: 1248, SNMP ifIndex: 513
Description: jtac-MX240-r004_xe-1/3/0
Forwarding classes: 8 supported, 4 in use
Egress queues: 8 supported, 4 in use
Queue: 2, Forwarding classes: AF
Queued:
Packets : 6661823121 10005 pps
Bytes : 453003972228 5443056 bps
Transmitted:
Tail-dropped packets : 0 0 pps
Tail-dropped bytes : 0 0 bps
RED-dropped packets : 0 0 pps
RED-dropped bytes : 0 0 bps
So, it is expected that they are marked with EXP3. But, with a packet capture, the EXP is instead at a 5:
Frame 1: 64 bytes on wire (512 bits)
Ethernet II
802.1Q Virtual LAN
MultiProtocol Label Switching Header
0000 0000 0000 0001 0000 .... .... .... = MPLS Label: 16 (0x00010)
.... .... .... .... .... 101. .... .... = MPLS Experimental Bits: 5
.... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1
.... .... .... .... .... .... 0011 1101 = MPLS TTL: 61
Internet Protocol Version 4
Problem is tracked and fixed via PR185525:
https://prsearch.juniper.net/PR1855256