Several hundred VDIs were unable to complete provisioning and reach the Available state.
Investigation identified DHCP traffic being rate-limited by the Juniper Control Plane Policing (COPP) / DDOS protection policy, resulting in DHCP packet drops and failed VDI provisioning.
A large number of Virtual Desktop Infrastructure (VDI) instances failed to complete provisioning and did not reach the Available state.
The issue was traced to Juniper COPP/DDOS protection policies dropping DHCP traffic after the DHCPv4/v6 aggregate traffic rate exceeded the configured threshold.
The environment experienced a mass VDI reprovisioning event after desktops were reformatted and reconfigured, generating DHCP traffic bursts that exceeded the configured control-plane policing limits.
As a result, DHCP requests were dropped, preventing VDIs from obtaining IP addresses and completing the provisioning process.
-An APSTRA alarm is received related to COPP (Control Plane Policing Probe)
For COPP the Limit was hit and this stop the VDIs from being provisioned
The Control Plane Policing (COPP) probe validates output on all managed switches in the fabric and protects the control plane CPU from overload caused by host-path punted packets.This probe raises per-protocol violations, with each violation indicating a dropped packet
https://www.juniper.net/documentation/us/en/software/apstra6.1/apstra-user-guide/topics/concept/predefined-probe-control-plane.html
-Going into ASPTRA the COPP violations we see this graphic for DHCPv4v6-aggregate
-Apstra generates COPP Violations alarms.
-DHCPv4v6-aggregate violations observed in Apstra Analytics.
-L3NHOP-AGGREGATE violations may also be present.
-DHCP packet drops reported by DDOS protection statistics.
-Large provisioning events result in intermittent or widespread VDI deployment failures.
End User Symptoms
VDIs remain stuck in Provisioning or Customizing state.
VDIs fail to reach the "Available" state.
Users are unable to access virtual desktop resources.
Common VDI lifecycle states are:
Increase the DHCPv4/v6 aggregate DDOS bandwidth threshold.
Previous Configuration:
Bandwidth: 1000 pps
Burst Size: 200 packets
Updated Configuration:
Bandwidth: 2000 pps
The DHCPv4/v6 aggregate COPP/DDOS protection threshold was configured at 1000 packets per second (pps).
During a large-scale VDI reprovisioning event, DHCP traffic peaked at 1450 pps, exceeding the configured limit. The control plane protection mechanism began dropping DHCP packets, preventing VDIs from obtaining IP addresses and causing provisioning failures.
On this following example the arrival rate exceeds the 1000 pps bandwith limit:
Use the following command on the affected Juniper switch to verify whether DHCP traffic is being rate-limited by DDOS/COPP protection: "show ddos-protection protocols dhcpv4 aggregate"
DHCPv4/v6 Aggregate DDOS Policy:
Bandwidth: 1000 ppsBurst Size: 200 packetsRecovery Time: 300 seconds
Maximum observed DHCP arrival rate:
1450 pps
-Since the Max arrival rate (1389 pps) exceeded the configured Bandwidth (1000 pps), DHCP packets were dropped, causing VDI provisioning failures.
The threshold was increased to 2000 pps, resolving the issue.
#configure#set system ddos-protection protocols dhcpv4v6 aggregate bandwidth 2000#commit
-L3NHOP-AGGREGATE violations were also present in Apstra; however, these were determined to be a downstream effect of the DHCPv4v6-aggregate COPP violations. Based on the observed traffic patterns and successful resolution after increasing the DHCPv4/v6 aggregate threshold, DHCP rate limiting was identified as the primary cause of the VDI provisioning failures.
Recommendations
-Continuously monitor DHCP packet-per-second (PPS) rates.
-Establish a baseline of normal DHCP traffic within the environment.
-Review DDOS/COPP thresholds regularly as the VDI environment grows.
-Monitor future mass provisioning events for traffic spikes.
-Be aware that if the configured threshold is exceeded:
Traffic will be dropped. Recovery occurs after 300 seconds. Additional violations will trigger another 300-second recovery period.
*Since the network fabric is managed through Juniper Apstra, any modifications to COPP/DDOS protection policies should be performed through Apstra rather than directly on the switches.