HPE Networking Apstra software product version 6.2.0 is available to licensed, registered HPE customers from the Juniper Apstra software download site.
Apstra 6.2.0 is extending the fabric policy API to add a new knob `junos_forwarding_table_host_profile`
This is to handle recommended configuration for Unified Forwarding Table profiles as per https://www.juniper.net/documentation/us/en/software/junos/multicast-l2/topics/topic-map/layer-2-forwarding-tables.html#id_lmd_smg_w5b__d125e1836
For new blueprints, this is `enabled` by default.For existing blueprints following upgrade, this is disabled. A customer would have to plan a maintenance window to apply this configuration. Changing this configuration is disruptive, and may cause all routes to be removed and re-added.
If this is set to enabled, configuration will render the following on QFX5130 and QFX5700 only:
system { packet-forwarding-options { forwarding-profile { host-profile; } } }
The BGP Monitoring probe and its dashboard are now auto-enabled by default for every blueprint; no manual instantiation needed. An anomaly is raised when a BGP session's flap count increases and stays true for at least 2 minutes within a 5 minute rolling window; it remains visible while the condition holds and clears automatically once stability returns. The dashboard shows Flap Count and Live Anomalies by session type (BGP Unicast, BGP EVPN).
When creating a VXLAN-type Virtual Network, the auto-assigned Virtual MAC Address will begin with 02:90:69 (Juniper OUI with LAA bit set) if you do not specify a preferred value.
Apstra Supports Juniper Model ACX7024X in Device Profile
Apstra Supports Arista Models 7280DR3-24-F & 7280TR3-40C6-F in Device Profile and Supports Arista 7800R3-48CQ-LC in Line Card Profile
Apstra supports Juniper Model QFX5250-64OE-L in Device Profile
Apstra now Supports Juniper QFX5140-24CD8O model in Device Profile.
Apstra now supports the Juniper ACX7020 model in Device Profiles
New Device Profile is available for the Arista DCS-7010TX-48-DC-F
You can now use Device Profile EX4100 for 24P/T/MP and 48P/T/MP variants with Apstra. This model is limited to Standalone Access-switch only, no support of Access-Pair, nor leaf role.
Logical Device List - Applicable Device Profiles ColumnYou can now see which device profiles are associated with each Logical Device directly from the Logical Device list. The new *Applicable Device Profiles* column displays all linked profiles inline, and is fully searchable - making it easier to find and select the right Logical Devices based on real device names without having to drill into individual records.
You can now view port configuration constraints directly within Device Profiles. We've added helpful annotations at the device, port, and transformation levels to clearly show which ports share physical resources and how configuring one port's speed or breakout mode may affect neighboring ports - giving you complete visibility before making configuration changes.
You can now use the 24.4 last supported software version (LSV) on the QFX10K.
The Uncommitted tab now displays errors organized by priority level, with colour-coded indicators designed to improve user experience and error resolution efficiency.
Devices can now be onboarded to Apstra in telemetry-only mode and added to a Freeform Blueprint, enabling Apstra to collect device telemetry but not manage the switch configuration. This enables you to gain valuable Day-2 insights on the health of your fabric while keeping your current fabric management workflows.
In this release, AOS lets you quickly rollback devices to their previous NOS version with a single click, eliminating the need to manage images or repeat upgrade workflows. The feature works for both individual devices and upgrade groups, automatically uses the correct rollback image, validates eligibility, and tracks the entire operation with clear status and audit logs, which makes recovery fast, simple, and reliable.
A new button called "Getting Started" has been added to the blueprints home page, below the "Create Blueprint" button, to open a pop-up that shows the getting started workflow.
You can now configure L2 Multitenancy support in the DC Reference Design, enabling Service Providers and Hosting Providers to deliver isolated services to multiple tenants on shared infrastructure. Each tenant can access the full 4K VLAN range with complete isolation, eliminating manual VLAN tracking and translation workarounds.
Supported Use Cases
Single-Tagged (VLAN-Bundle) - Standard single-tagged traffic handling across the fabric.
Dual-Tagged (Q-in-Q) - Inner C-VLAN and outer S-VLAN tagging on both ingress and egress.
Asymmetric Tagging - Dual-tagged on ingress, single-tagged on egress.
VLAN Translation - Translate VLAN IDs between ingress and egress leaf devices.
VLAN Translation with Overlap - VLAN translation supporting overlapping ranges across tenants.
Selective IRB Placement (CRB-Like) - Select specific leaf devices for IRB placement to support L3 Gateway functionality.
Introducing the concept of Switching Zones (Rendering a MAC-VRF):
Configure multiple MAC-VRFs per blueprint, supporting both VLAN-Aware and VLAN-Bundle EVPN service types with isolated forwarding per tenant.
User-defined Switching Zones cannot be extended through Integrated DCI. Only Virtual Networks present in the default Switching-Zone (i.e. "evpn-1" MAC-VRF can be extended through integrated DCI).
Encapsulate Inner-VLAN-ID - Enable Q-in-Q encapsulation on Virtual Networks.
Override VLAN-ID - Set explicit VLAN IDs within Connectivity Template primitives.
Prerequisites for the above capabilities:
MAC-VRF must be enabled on the blueprint. Customers using "Default Switch Instance" should migrate to MAC-VRF before leveraging SP features.
Qualified platforms:
QFX5120
QFX5130
You can now create an interface policy for Link Auto-negotiation. The capability is available for SONiC only.
You now have a "Clone Multiple times" button in the Rack-designer allowing you to specify the number of desired clones as well as they visual layout in a matrix.
Apstra now supports the "Periodic Rate" analytical processor, giving you the ability to measure how quickly a value changes over time, not just by how much. By accounting for the time elapsed between data points, it delivers a precise rate of change that helps you better monitor trends, detect anomalies, and make informed decisions based on the speed of change in your network metrics.
The optical transceivers probe now by default excludes any non-provisioned ports from the scope of monitoring, saving you from false positive anomaly related to transceivers inserted in unused ports. This can be modified by editing the predefined probe parameters and unchecking the "Raise anomaly only on provisionned ports" if you want to include every ports in the monitoring scope.This probe is also now auto-enabled by default.
A new enhancement to the auto-enabled MAC Monitor Probe now allows analysis of the historical behaviour of all MAC addresses across the blueprint. This helps you better understand past network states and troubleshoot issues more effectively, in addition to the existing Real-Time view.With this update, two new views are introduced:1. Time-Series View - Allows you to analyze MAC address behavior over a selected time range (up to 30 days). It shows how MAC states change over time, helping you identify patterns such as intermittent issues, state transitions, or recurring problems. You can drill down into a specific MAC to see detailed state transitions with exact timestamps.2. Snapshot View - Provides a point-in-time view of all MAC addresses at a specific timestamp in the past. This helps you quickly understand the network state at that moment and correlate issues with events, such as identifying when MACs were missing or recovered.
Together, these views enable both trend-based analysis (Time-Series) and point-in-time debugging (Snapshot), making it easier to detect, investigate, and resolve MAC-related issues in the network.Anomaly detection for MAC state transitions (Expected → Missing) needs to be enabled manually via Raise Anomaly checkbox option.
New analytics dashboard for interface FEC Statistics powered by an auto-enabled IBA probe "FEC Statistics". The probe is supported on the following EVO-based models (QFX5140, QFX523|40|50) and collects the following metrics:
Pre-FEC BER
FEC Uncorrected Errors
FEC Histogram (Bin0 -> Bin15 for RS-544, stored as time-based rates)All data is persisted in MetricDB with 30-day retention and surfaced through a dedicated FEC dashboard with real-time and historical views.
This release supports upgrade paths from previous Apstra 6.0.X and 6.1.X releases.
You must use VM-to-VM upgrades from Apstra 6.0.X, but In-Place upgrades are available for Apstra 6.1.X to 6.2.0. Refer to the Apstra Installation and Upgrade Guide for more information on upgrade procedures.
You can now deploy the Apstra VM on HPE Morpheus as the hypervisor.
The Offbox-unit capability, previously available as a Tech Preview in version 6.1.0, is now Generally Available. Once enabled, you can access the new Offbox units interface to gain comprehensive visibility into your Offbox units, including how units are associated with systems. This page will also notify you when distribution is suboptimal, using two threshold levels: Information and Warning. When a Warning alert appears, it is recommended to schedule a controller restart to achieve better distribution.You also can filter and search systems through various criteria such as Device Hostname, Device management IP, Container Name, Offbox Unit Name.
A new option has been added to the Platform → Password Complexity Parameters for repeating patterns. The default option is enabled for preventing repeating patterns of 3 characters. This will prevent users from using passwords like "Aa1!Aa1!Aa1!Aa1!" that meet other password complexity requirements but are simple 3-character repeating phrases.
Interfaces contain now both the name and the description in the interfaces dashboard. Since many users put information to identify interfaces in the interface description field
A telemetry dashboard has been added that shows the health of the flow collector (CPU, RAM, disk usage) as well as how many packets and flows have been processed. This data can also be used to set up alerts like a high watermark alert when disk usage exceeds 90%.
When configuring flow, using the flow config (manage-config) script, the license will automatically be verified upon adding the Apstra server IP and credentials.
The following updates have been made for switch operating systems qualified for the Apstra 6.1.0 release.
Juniper Networks:Junos Evolved for AIDC:23.4x100-D4525.2x100-D20
Junos (All roles):23.4R2-S725.2R2-S1
Junos Evolved for IP-Forwarder role (Spines in EVPN or any role in an IP-Fabric):23.4R2-S7-EVO25.2X100-D20
Junos Evolved for EVPN leaf roles:23.4R2-S7-EVO25.2X100-D20
Junos Interconnect Gateway Leaf:23.4R2-S725.2R2-S1
Junos Evolved Interconnect Gateway Leaf:23.4R2-S7-EVO25.2X100-D20
Junos Evolved for ACX and PTX23.4R2-S7-EVO25.4R1
Cisco Systems:10.2(6)10.3(4a)10.4(6)M
Arista Networks:4.30.3M4.33.4M
Dell EMC & Edgecore:Enterprise SONiC 4.4.2Enterprise SONiC Edge Standard 4.4.2Enterprise SONiC 4.5.1Enterprise SONiC Edge Standard 4.5.1
The Virtual Network APIs have been improved with several updates and schema revisions. Certain endpoints have been deprecated in favour or using alternative endpoints. please consult the API Changelog for comprehensive information about all these changes.
Device Telemetry data is now displayed in the Active Topology view for a selected device, rather than being launched to the Managed Devices page. This enables you to see the device telemetry in the context of the fabric blueprint.
In both the Staged and Active tabs, the right-side menu for Selection + Build/Anomalies is shown only in the Topology sub-tab. All other tabs (Nodes, Links, Interfaces, Racks, Pods) do not display the right-side menu that enables full-screen tables.
You now can Holistically define Security Policies across IPv6 Virtual networks.
You can now drain systems with improved precision and reliability. We've introduced dedicated drain policies for underlay (IPv4 and IPv6) and overlay (EVPN) sessions, preventing traffic black-holing scenarios - especially when a leaf acts as a DCI gateway with static routing.
A new banner message at the top of the Devices → ZTP Status → Devices and Devices → ZTP Status → Services pages to provide a link to the dedicated ZTP UI page for configuration.
You can now turn on streaming for any part of an IBA probe without changing its predefined status. You do this through the predefined probe menu for each probe, which now has a "Streaming Stages" parameter allowing you to choose any any stage.
We have significantly improved the reliability and performance of EVPN IBA probes for Juniper-based deployments through the following updates:
Modernized Collector Architecture: The new Collector implementations leverage gRPC/gNMI technology, delivering faster performance and better scalability compared to the legacy CLI-based polling collectors used in earlier Apstra versions.
Version-Specific Collection Defaults
Junos 23.4 defaults to gNMI periodic collection with a 120-second collection interval
Junos 25.2 and later defaults to gNMI on-change collection, which streams updates in real time as changes occur on the device
Flexible Collection Type Selection: The system now automatically selects the optimal collection method when set to any (the default setting). This intelligent selection considers the target Junos version and chooses the most appropriate collection type. You can also manually specify a collection type through the Probe edit settings UI. Available options include:
"any" (default) - automatic selection based on Junos version
"gRPCPeriodic" - scheduled interval-based collection
"gRPCOnChange" - real-time change streaming
"polling" - legacy CLI-based collection (still available but not recommended)
Explicitly selecting on-change collection for Junos 23.4 devices is not supported and will produce inaccurate telemetry data, as this version does not support on-change gNMI for EVPN routes. This improvement adds extra configuration ( set protocols bgp enable-oc-attr-set ) into the rendered configuration.
set protocols bgp enable-oc-attr-set
You can now enforce TLS certificate validation across all device download operations - including system-agent downloads, device keeper probes, and NOS image upgrades - protecting your infrastructure against man-in-the-middle (MITM) attacks.To enable strict validation, set pki_verification_mode = strict in aos.conf. When left at the default (disabled), existing behavior is preserved for backward compatibility.
You can now use Windows Server 2022/2025 hypervisor versions to run Apstra.
You can use VMware ESXi version 9 as the hypervisor to run Apstra.
Junos on-box agent support has been expanded from EVO to include non-EVO platforms. This enhancement improves scalability by eliminating the requirement for large worker node clusters to host off-box agents during deployment.This functionality is available on the following Junos platforms:
x86-based CPU systems, including the QFX5120 series and EX4400 series (ARM-based CPUs such as the EX4100 are not supported). Virtual form factors such as vEX are also supported.
Junos version 25.2R2-S1.Similar to other on-box agents, this agent operates within the Management VRF mgmt_junos.
ZTP over IPv6 is now available for Junos Evolved only. The Apstra ZTP server (specifically, the internal HTTP or TFTP that serves the ztp.py bootstrap script) must be in the same subnet as the device(s) being ZTP'ed. Attempting to ZTP a Junos device over IPv6 will not work with the 25.2 version that is qualified against Apstra 6.2.0.
The Apstra UI has been updated to reflect the HPE brand identity including the login screen, menu bar icon, browser favicon, and CLI login message.
Implementation a standardized approach for representing tagged objects in the Graph database by utilizing a "Tag" Graph node with a relationship to the corresponding tagged object.Within the Apstra Datacenter reference design, tags are supported by the following objects:
System
Interface
Link
Security Policy
Virtual Network
Endpoint
Routing Zone
Connectivity TemplateUse the predefined query catalog which has been enhanced with examples covering each use case.
The Flow dashboards, and login screen now have HPE logos and branding.
Flow collector to periodically check Apstra controller for license key
The Blueprint wide search has been expanded to provide more comprehensive searching and now supports searching tags across all Blueprint items, so the specific "Find by tags" button is no longer needed, as it has less functionality than the Blueprint search.
Tech Previews give you the ability to test functionality and provide feedback during the development process of innovations that are not final production features. The goal of a Tech Preview is for the feature to gain wider exposure and potential full support in a future release. Customers are encouraged to provide feedback and functionality suggestions for a Technology Preview feature before it becomes fully supported.
Tech Previews may not be functionally complete, may have functional alterations in future releases, or may get dropped under changing markets or unexpected conditions, at Juniper’s sole discretion. Juniper recommends that you use Tech Preview features in non-production environments only.
Juniper considers feedback to add and improve future iterations of the general availability of the innovations. Your feedback does not assert any intellectual property claim, and Juniper may implement your feedback without violating your or any other party's rights.
These features are "as is" and voluntary use. Support Services will attempt to resolve any issues that customers experience when using these features and create bug reports on behalf of support cases. However, Juniper may not provide comprehensive support services to Tech Preview features. Certain features may have reduced or modified security, accessibility, availability, and reliability standards relative to General Availability software. Tech Preview is not supported under existing service agreements, SLAs, or support service.
For additional details, please contact Juniper Support or your local account team.
You can now use a second collection method for your custom collector, based on gNMI Subscription. This feature is available as a Tech Preview in version 6.2.0.A new gNMI node has been introduced, operating on the same principle as the CLI node. As part of this, the previous XML transform has been renamed to CLI to better distinguish it from the newly added gNMI option.
Apstra's BuilderAgent may crash if the generic system's interface name (if_name), identified by LLDP from the generic system, contains trailing space after the user creates generic systems based on the gathered LLDP data. This is because the automatically generated interface's description for the generic system contains if_name information with trailing space, which causes a validation error in the BuilderAgent.
The below signature can be seen in the BuildAgent.err log file.File "/usr/lib/python3.10/dist-packages/aos/grappa/table/cpp_graph.py", line 909, in set_noderaise ValidationError(errors)lollipop.errors.ValidationError: Invalid data: {'description': 'Invalid description. Only ASCII characters with codes 32 - 126, except "?", "<", ">" ,\'"\' and are allowed. The description should not start or end with a space.'}
The built-in device profile Juniper_ACX7100-32C uses a model selector that only matches 'ACX7100-32C' exactly. This causes the device profile to fail matching against other SKU variants of the ACX7100-32C product line, such as the 'ACX7100-32C-K'. As a result, devices with variant model names are not automatically associated with the correct device profile during onboarding
When multiple show-tech jobs are triggered simultaneously, the /var/log partition can reach high utilization, causing the controller to enter read-only mode. This may result in incomplete job status updates, leaving several jobs stuck in in-progress or pending states. In a corner case, this condition can also lead to repeated crashes of SystemAgentManager, preventing automatic recovery even after disk space is reclaimed. Avoid triggering bulk show-tech collection on a large number of devices. Ensure sufficient disk space is available before running show-tech.
In a future release, a fix is planned to handle empty or incomplete job status data to prevent SystemAgentManager crashes, along with checks to ensure sufficient disk space before initiating bulk show-tech operations.
In rare situations where several dozens of BGP peerings to external generic devices have been setup on an NXOS edge leaf switch, the upgrade configuration may fail to apply within the allotted 120 seconds with a timeout. The upgrade configuration is modifying all the external BGP neighbors to always use the next-hop-self command, to properly support RFC-5549.
Future Apstra versions will address this problem by applying the upgrade configuration piecemeal, ensuring that the smaller parts will take way less than 120 seconds each to apply.
When attempting to convert leaf switches from ESI to MLAG within the same blueprint, the operation fails because Apstra does not allow mixing ESI and MLAG redundancy models at the rack level. This restriction is enforced starting in Apstra 4.2.0 and is expected behavior. During the operation, users may see the following error in the UI or REST API Explorer:
"Combining MLAG and ESI leaf pairs not supported"
Currently, converting ESI racks to MLAG racks or vice versa requires replacing all racks within a single FE operation using the REST API. However, due to limitation, this conversion can cause BuilderAgent to fail when links are present between external generic system and leaf switches. In this occurs, please revert the changes and follow the steps outlined in workaround section.
Apstra intentionally prevents mixed ESI and MLAG redundancy configurations at the rack level. A future release is planned to address the current limitation in the rack modification operation when external generic systems are connected.
The Device Reboots Required widget shows only the total devices which are checked needing reboot if shared-tunnel vxlan routing-options is not configured. However this widget is incorrectly showing the actual amount of devices which need actual reboot. This is because it is incorrectly "spotlight_mode" set to true for this widget
The predefined widget now correct has "spotlight_mode" set to False
When performing an Edit Rack (Change Rack) operation on a blueprint that was originally created in Apstra version earlier than 6.0.0, single-attached generic systems may be incorrectly reassigned to the wrong leaf node within an ESI or MLAG leaf pair. This occurs because the rack modification logic uses alphabetical label ordering to map generic systems to leaf nodes, rather than the rg_position value stored in the system node. If the alphabetical order of leaf node labels does not match their rg_position ordering, single-attached servers are mapped to the incorrect leaf, resulting in unintended configuration changes on the affected leaf devices - including removal or modification of server-facing interfaces.
In environments with a large number of devices using freeform design, the Apstra UI exhibited slow and laggy performance during topology editing operations. Specifically, managing LAG configurations on leaf devices caused significant delays before the configuration window appeared, often triggering browser unresponsiveness alerts
In an EVPN Interconnect Group where multiple Remote Gateways share associations with the same Local Gateways, removing a Local Gateway association from one Remote Gateway may cause the Interconnect Route Distinguisher values for that Local Gateway to appear as empty or missing in the UI for an unmodified Remote Gateway.This behavior is caused by a cache handling defect in the Web Experience API, where the cache incorrectly clears the RD values for a Local Gateway across all Remote Gateway associations when only a single association is modified. The non-web experience API endpoint (/api/blueprints/{blueprint_id}/evpn_interconnect_groups) is not affected. The underlying EVPN Interconnect configuration, device configuration, and deployment remain correct and are not impacted.
In Apstra 6.2.0, the Web Experience API cache handling has been corrected so that removing a Local Gateway association from one Remote Gateway does not incorrectly affect the cached information for other Remote Gateway associations.
In releases prior to 6.2.0, users were able to use tag-only matchers (has_all(), has_any(), has_none()) on non-tag properties in graph queries when those properties appeared in subsequent nodes of a query path (i.e., after the first node). These graph queries could be used in Predefined Graph Queries (Global Catalog), Freeform Resource Groups, and IBA probes. In all cases, these queries were not functioning correctly. They caused internal errors and crashes of controller components such as ScotchAgent, ProbeAgent, and BlueprintTaskManagerAgent.
Despite their incorrect behavior, users were able to save these graph queries in the Predefined Graph Queries catalog and in IBA probe configurations. After upgrading to 6.2.0, such graph queries are no longer causing internal errors or crashes. However, they will not execute. When run, they are rejected with a validation error indicating that the tag matcher is not supported for the specified attribute.
This applies to any saved graph query where has_all(), has_any(), or has_none() was applied to a non-tag property in a multi-node query path. Users who have such queries saved before upgrading should expect them to be rejected upon execution in 6.2.0.
Remove a graph query or redact a graph query by removing a usage of tag matchers from non-tag properties.
When a configlet action (import/delete) occurs in a blueprint with a large number of configlets, it can fail with a timeout, and the BlueprintDiffProducerAgent process can crash due to a heartbeat timeout. The problem occurs when the blueprint has a configlet with an incorrect Jinja expression via configlet import/delete actions. Failure of rendering configuration with incorrect Jina expressions can cause all configlets to be re-evaluated for all eligible devices, potentially resulting in much longer configlet processing.
Upgrading to Apstra 6.2.0 removes the redundant "unused" port role wherever it appears next to a real role on a logical device port group or an interface map interface. A port whose only role is "unused" is left unchanged.
Apstra assigns interface names by grouping the ports of an interface map by their speed and their set of roles, and then allocating names from those groups in order. Removing the redundant role can change that grouping in two ways. Two port groups of the same speed can become identical and merge into one, for example ["access", "generic", "unused"] and ["access", "generic"]. Two port groups of the same speed can also change their relative order, for example ["leaf", "unused"] and ["leaf", "superspine"], because "unused" no longer participates in the ordering.
In both cases Apstra recalculates which physical port each link is assigned to for every system using that logical device. After the upgrade the blueprint reports uncommitted changes that move the affected links to different interface names. The upgrade itself completes without errors and no anomalies are raised, so the change is only visible in the blueprint diff.
No predefined logical device or interface map in the global catalog is affected, so this applies only to logical devices created or edited by the user. Interface names that were assigned explicitly are preserved and are not recalculated.
Review the uncommitted changes in each blueprint after upgrading and before committing.
If interface names have been reassigned and you want to keep the previous assignment, assign the required interface name explicitly on each affected link. Explicitly assigned names are preserved and are not recalculated.
To remove the condition permanently, edit the logical device so that ports of the same speed sharing the same roles are modelled as a single port group holding the union of those roles, rather than as two port groups that differ only by the "unused" role. Changing the logical device requires the corresponding interface map to be regenerated and reassigned.
When executing a Junos rollback via the System Agent, the device is being rolled back unconditionally and without any involvement of the Apstra device agent. As a result the device configuration after the rollback might be different (For e.g., the Junos version statement), which would cause a config deviation anomaly. This behavior is expected when using Junos rollback feature.
When rack types are exported, modified, and then re-imported into a blueprint, generic systems in the rack may be unexpectedly renamed, resulting in some servers losing their original user-defined labels even though only rack parameters were changed.
The Last Modified field of the interface telemetry service, which uses gRPC periodic mode, is inadvertently updated according to the interface telemetry service's default interval even if no data has been collected from the device. This problem is noticed when gRPC periodic mode is used for Interface telemetry service. The Last Modified field should be updated if a status change is observed, and the Last Fetched field should be updated if the device reports telemetry data.
The Last Fetched and Last Modified fields in the Interface Telemetry Service using gRPC periodic mode are correctly updated with the right information.
The Platform -> Product Usage page does not take into account the quantity field in each license. It simply counts the number of licenses. Product Usage erroneously displays insufficient license warnings even though a license with multiple quantities is applied correctly since it is seen as a single license. Apstra operation would remain unaffected even if there were product usage warnings.
In ESI-based EVPN deployments, the MAC Monitor probe may incorrectly report MAC addresses as missing even though they are present on the device. This can cause Virtual Networks Containing Systems With Missing MAC Addresses anomalies to remain active even after the underlying network issue has been resolved or maintenance has been completed.
The missing MAC counting logic has been corrected in Apstra 6.2.0, preventing stale Missing MAC counts and persistent anomalies.
The MAC Monitor probe has been updated to provide clearer visibility into remotely learned MAC addresses. Starting with Apstra 6.2.0, remote MAC addresses are represented with the interface as remote and the Next Hop Type as metaRemote in the MAC Address Table. This provides a consistent representation of remote MAC addresses regardless of the underlying remote interface.
On ACX7100-32C devices, configuring a port with 10GE speed (non-channelized) could result in the port failing to link up, accompanied by "Invalid Port Speed Configuration" and "Optics does not support configured speed" alarms. This occurred because the built-in ACX7100-32C device profile did not automatically generate the required "unused" configuration for the adjacent port within the same port group(for example 0 and 1 are in the same group for 10GE)
Starting in Apstra 6.1.0, the root partition is read-only, and the NTP service override configuration (override.conf) is stored in a non-standard path (/var/root/usr/local/lib/systemd/system/ntp.service.d/) instead of under /etc/systemd/. Because this path is outside the standard systemd configuration directory, systemd does not automatically reload it during boot. As a result, the NTP service starts without the override configuration applied.
Junos EVO 22.4 and earlier versions (including service releases) are not qualified for use with Apstra 6.2.0 anymore. The Apstra 6.2.0 Onbox Agent connects to a newer gRPC/gNMI communication endpoint on the host device that is not available on these older Junos EVO releases. As a result, the agent will fail to establish a telemetry subscription, which may trigger "Check gRPC Server Reset Count" anomalies in the Device Telemetry Health probe.
If telemetry collection from an older Junos EVO version is required, add use_old_grpc_unix_socket = 1 under the [system_agent] section of the controller's aos.conf file to fall back to the legacy communication method. This setting must be applied and followed by a controller restart before any Apstra 6.2.0 system agent is installed. If agents have already been installed, uninstall all existing agents first, then apply the setting, restart the controller, and reinstall the agents.
This workaround is strictly temporary and may cause compatibility issues with newer Junos versions. Customers are advised to upgrade affected devices to a qualified Junos release.
After VM to VM upgrade, the agent page job section shows the Job history from the previous Apstra VM with a view icon to download/view the file. Actually, agent job files are not copied in VM to VM upgrade, so the files do not exist in the current AOS VM. The Apstra UI is reporting the error, but the UI is not good.
Each blueprint keeps the latest 100 blueprint-related tasks' information, such as API call with request data. When a blueprint accumulates the latest 100 of tasks with significant request data sizes, the ScotchAgent task query consumes excessive CPU resources. The task query, which executes regularly per blueprint, performs a deep copy of all task data, taking 12-14 seconds per query. This causes ScotchAgent to sustain over 90-100% CPU utilization, degrading the overall Apstra UI responsiveness.
When Blueprint->Stage->Catalog->Tags was selected in the Freeform Blueprint, it doesn't show any tags in the UI. The issue is caused by not linking correctly to tagged objects for AGGREATE LINK in the UI APPLIED TO column.
Interface telemetry uses gRPC periodic mode to collect interface status data from the JUNOS and EVO devices. Apstra (the gRPC client) needs the delivery of all interface status information every round (the default period is 120 + delta seconds). False alerts for interface anomalies results from the devices occasionally sending incomplete information (just for a partial set of interfaces) inside a round, and the interface status for absent interfaces would be represented as missing status.
The UI cannot display SDT collector query results containing special IEEE 754 floating-point values, such as Infinity, -Infinity, or NaN. This is a known limitation of the SDT/IBA APIs. The backend may return these special IEEE 754 floating-point values, but they cannot be represented by the JSON standard. As a result, the UI displays a *not valid JSON* error instead of the expected query results.
Special IEEE 754 floating-point values are uncommon in normal telemetry workloads and are typically produced by exceptional numeric conditions, such as arithmetic overflow or division by zero. This limitation affects only the presentation of query results in the UI and does not necessarily indicate a failure in the underlying SDT or IBA data processing.
The use of special floating-point values is discouraged because they are not consistently supported throughout the IBA data pipeline. Even if the UI could display these values, they may not be processed consistently by all pipeline components. To ensure predictable behavior, telemetry data sources should avoid generating these values where possible.
The SDT UI has been enhanced to display a clear error message when pipeline test results contain special IEEE 754 floating-point values that cannot be represented by the JSON standard. Previously, the UI displayed only a generic error indication without clearly identifying the cause.
When navigating to Staged > Physical > Racks, the UI may display a "Not enough 10G ports on leaf" error instead of the expected rack statistics. This occurs when a rack type was created (via API) with a generic system group label identical to a leaf switch group label. During upgrade to 5.1.0 or later, the upgrade plugin could not correctly reconcile the peer_switch reference in rack_type_json for such racks, resulting in mismatched data between the rack type JSON and the staged graph.
Validation logic for a security zone assumes the description includes valid ASCII characters. If the description includes violating characters, the upgrade from 5.1.0/6.0.0 to 6.1.X would fail with a error related with security zone.
Validation Logic for security zone's description 1. Description should not start or end with space 2. Description should include only printable ASCII characters (ranging from 33 to 126), excluding double quotation (") character, less than (<) character, and backslash (\) character.
The SAML authentication workflow has been enhanced to improve the security of the authentication token exchange. Instead of passing the authentication token as part of the browser URL during the final stage of the SAML login process, Apstra now uses a short-lived, secure, HTTP-only cookie to transfer the token. Because secure cookies are used, the Apstra Web UI must be accessed over HTTPS when using SAML authentication. SAML login over HTTP is no longer supported.
The SAML login workflow has been updated to exchange the authentication token using a short-lived, HTTP-only, Secure, SameSite=Strict cookie instead of including the token in the browser URL. This change reduces the exposure of authentication tokens while maintaining the existing SAML authentication experience.
On HPE EX4100 devices, PIC1 ports default to Virtual Chassis Port (VCP) mode. When these ports are used as network uplinks to a leaf device, Apstra renders the interface configurations successfully; however, the interfaces are not created on the device because VCP mode prevents them from operating as standard network ports. This causes Apstra to report anomalies indicating missing interfaces. PIC2 ports are unaffected and operate normally as network ports.
Before installing the system agent for EX4100 device into Apstra, please convert the PIC1 VCP ports to network-port mode by following the below steps1. Before onboarding the EX4100 device into Apstra, please convert the PIC1 VCP ports to network-port mode at first
request virtual-chassis mode network-port
2. Reboot the device to apply the change
3. Confirm the ports are removed from the VCP port list:
show virtual-chassis vc-port
4. Proceed with onboarding the device into Apstra by installing the system agent.
Junos 25.2 introduced a backward-incompatible change where the speed 1g operand is no longer accepted in copper interface stanzas on EX4400 series switches (including EX4400-24T, EX4400-48T, and EX4400-48MP). This causes configuration deployment failures for blueprints that include these devices when running Junos 25.2. The issue does not affect Junos 23.4 or earlier releases, where speed 1g continues to be accepted.
Apstra 6.2.0 addresses this by updating all EX-series device profiles with copper interfaces in PIC 0 to remove the explicit speed 1g setting. Without the speed command, copper interfaces default to 1g auto-negotiation, which produces the same operational behavior across both Junos 23.4 and 25.2. Upgrade plugins are provided to apply the fix to both Catalog and Blueprint device profiles and interface maps.
For customers on earlier Apstra versions who encounter this issue after upgrading Junos to 25.2: avoid upgrading Junos to 25.2 on EX4400 copper-interface switches until upgrading to Apstra 6.2.0, which includes corrected device profiles and upgrade plugins that automatically apply the fix to existing blueprints.
The Apstra offbox agent may fail to connect to and manage Juniper ACX7100-48L devices running Junos EVO 22.x releases. On affected devices with BIOS ROM firmware version 16.02 or later, the data interfaces are not properly initialized under older Junos EVO images, causing the show chassis mac-addresses command to return empty output. The offbox agent requires chassis MAC address information during device onboarding; when this information is unavailable, the agent aborts with an error and continuously restarts, preventing device management.This issue affects only ACX7100-48L devices that have been updated to a newer BIOS firmware version. Devices with BIOS ROM firmware version 13.01 are not affected.
Upgrade affected ACX7100-48L devices to Junos EVO 23.4R2-S7.3 or later. This version resolves the BIOS firmware compatibility issue and restores proper interface initialization and chassis MAC address reporting on devices with newer BIOS firmware.
Cisco NXOS rollback is used throughout Apstra to restore an NXOS device to a previously recorded pristine state. Certain versions of NXOS automatically insert an ipv6 link-local address line under the mgmt0 interface. When attempting to rollback from a running configuration (which contains this auto-added line) to a pristine configuration that does not contain it, NXOS will re-add the line immediately after removal. This causes the rollback verification to detect a difference and emit a failure. This can occur during a full config apply or a pristine restore operation.
Manually edit the pristine configuration of the affected device to include the ipv6 link-local line that NXOS automatically generates. Since NXOS will add this line regardless, including it in the pristine configuration eliminates the difference detected during rollback verification, allowing the operation to succeed.
Selective IRB mod, which allows selectively disabling ipv4_mode or ipv6_mode on assigned to leaf node for a virtual network with enabled IPv4 connectivity or IPv6 Connectivity), is only recognized and supported for configuration rendering on Junos platforms. Configuring selective IRB on non-Junos platforms can result in incomplete or unexpected configuration rendering on network devices, leading to undefined forwarding behavior or unresolvable ARP/ND issues. Upgrading plugin for 6.2.0 would block any invalid selective IRB configuration for non-Junos platforms. If upgrading to 6.2.0 fails with selective IRB configuration for non-Junos platforms, please fix selective IRB configuration and try to upgrade again.
Split horizon packet drops may be incorrectly reported in the transmit (TX) discard counters on interfaces of SONiC 4.5.1 devices. The elevated TX discard counts are primarily observed on leaf-facing-spine interfaces. Investigation confirmed that these are legitimate split horizon drops --no actual packet loss occurs-- but the counters were erroneously incrementing, which could trigger persistent TX discard anomalies in device traffic probes.
--
Future versions of SONiC will no longer increment TX discard counters for legitimate split horizon drops.
ZTP can fail on HPE EX4100 devices if the ZTP bootstrap script (ztp.py) cannot communicate with the device over SSH. This happens due to a mismatch in the key exchange algorithm between the device and ZTP VM. You can fix this issue using the following workaround.
To run ZTP for an EX4100 device successfully, you must enable a special setting in the Apstra ZTP server. Edit '/containers_data/tftp/settings.sh' manually and change 'USE_SIGNED_ZTP_TARBALL=no' to 'USE_SIGNED_ZTP_TARBALL=yes' This tells the Apstra ZTP to use a new bootstrap mechanism using a properly signed ztp.py script to bootstrap the device. The new signed method of bootstrapping is available from Apstra ZTP 6.2.0. Changing the setting applies to all Juniper devices that will use Apstra ZTP for bootstrapping .
When doing a full config apply, the SONiC configuration renderer in Apstra 6.2.0 is adding the BREAKOUT_PORTS table to the rendered tables. When running SONiC 4.5.1 with breakout ports configured, the show interface transceiver sonic-cli command may return a transaction error due to the absence of the BREAKOUT_PORTS table. Note: This release note applies only to the case of a full config apply operation (done by a direct config_db.json rendering). It does not apply to interface breakouts that have been configured in a day-2 incremental change (done via restconf).
If you are upgrading from an earlier version of Apstra, the CONFIG_DB database is not changed. If the user decides that the BREAKOUT_PORTS table should be added, then the user must manually do a full config apply operation. For additional clarification, please contact the HPE Apstra Support Team.
The state_check processor outputs a state column for each series. However, users may not be able to filter series in the output stage using the per series state column when querying stage data. Users attempting to use a filter such as properties.state = 'false, true' may see no results, even if matching data is present. This is a known limitation in how filtering works on series-level data in the state_check processors output. The filtering behavior may not align with user expectations when attempting to use conditions on the state column. Engineering evaluating potential improvements to allow filtering on state column in state processor for a future release.
Decreasing the value of the "Time Window" property of the "Time in State" processor, either directly or via predefined probe parameters, may not have an immediate effect but rather after the expiration of a previously configured time window or an update of the input stage.
Disable and enable back the affected probe
In affected configurations, Apstra does not render IPv6 Router Advertisement (RA) prefix settings on IRB interfaces. DHCPv6 clients may then learn only a /128 host route instead of the expected subnet prefix routes, resulting in loss of IPv6 connectivity to both local subnet and remote IPv6 destinations.
As a workaround, configure the IPv6 prefix under the Router Advertisement configuration using a configlet.
set protocols router-advertisement interface irb. prefix /Ex: set protocols router-advertisement interface irb.101 prefix 101:1:1:1::/64
This causes Router Advertisements to include the Prefix Information Option (PIO), allowing clients to learn the on-link subnet and install the appropriate IPv6 subnet route.
During a VM-to-VM upgrade, data from the apstra-edge service is extracted into the apstra_edge Docker volume on the target controller. If the Docker volume has not been pre-created prior to extraction, Docker initializes the directory (/var/lib/docker/volumes/apstra_edge/_data) under root ownership (root:root). Because the apstra-edge container process executes under an unprivileged service account (aos_edge, UID/GID 10001), the container encounters permission denied errors when attempting to access or write state data upon startup.
Please, update the ownership of the apstra_edge Docker volume directory to user and group ID 10001 (aos_edge) in the target controller (upgraded controller)
sudo chown -R 10001:10001 /var/lib/docker/volumes/apstra_edge/_data
{nogotmat}
When modifying the maximum disk usage setting in a blueprint, the following behavior may be observed:
1. If the maximum disk usage is set to a value lower than the current disk usage, the blueprint commit fails. 2. If the maximum disk usage is increased to a value greater than the current disk usage and committed, the commit may still fail.
In the above scenario, the blueprint commit version does not advance, and the updated maximum disk usage setting is not honored.
After increasing the maximum disk usage to a valid value, make an additional, unrelated change in the blueprint and commit again. This subsequent commit succeeds and correctly applies the updated maximum disk usage setting.
While configlet is being applied to the SONiC device, if the device agent restarts after being disconnected from the controller, the agent executes any remaining changes and collects the running configuration as golden configuration to monitor for configuration anomalies. Because the process of applying configlet changes is still running independently of the agent, it introduces changes into the running configuration even when the golden configuration is collected by the agent. The following changes from the process cause configuration anomalies in the SONiC device.
After reviewing the running configuration on the SONiC device, if all the changes from the configlet are correctly applied, the customer can safely accept changes to avoid further configuration anomalies.
In Apstra blueprints configured with Data Center Interconnect (DCI) and Routing Zone Footprint Optimization enabled (default is enabled), deleting a Routing Zone (Security Zone) can trigger an improper evaluation of the footprint activation state across other Routing Zones. When a Routing Zone is removed, internal evaluation rules incorrectly alter the cached activation state of unmodified routing zones. Consequently, border leaf nodes carrying the DCI interconnect may unassign or remove loopback IP addresses belonging to unrelated VRFs/Routing Zones, potentially causing unexpected traffic disruption or configuration diffs.
Disable Routing Zone Footprint Optimization in the blueprint fabric settings to prevent automated footprint recalculations from affecting inactive or unmodified Routing Zones.
1. In the Apstra GUI, navigate to Staged > Fabric Settings.2. Locate Routing Zone Footprint Optimization and set it to Disabled.3. Review any IP resource allocation warnings or build errors in the staged changes and resolve allocations before committing the blueprint.
Task Error will show: "virtual_network": "IPv4 in VRF \"default\" is not supported with Routing Zone disable_ipv4=\ "True\"" with rail-id's in the payload.
Provisioning of VLANS is currently not possible when the default RZ is configured with disable_ipv4=True (IPv6 only). When changes to Apstra allowed the creation of RZ without IPv4 (IPv6 only), the API endpoint, which creates VLANs, incorrectly assumed IPv4 is always present.
After a device is redeployed, upon using gNMI Periodic or gNMI OnChange collection mode, EVPN VXLAN Type-3 (evpn_vxlan_type3) and Type-5 (evpn_vxlan_type5) route validation probes may report sustained telemetry anomalies because required extended-community information is not immediately available on the device. The issue is more pronounced in environments with a high route scale where EVPN route convergence takes longer post-redeployment, causing telemetry monitoring to flag route validation discrepancies.
Disable and re-enable the EVPN Type-3/5 route validation probes from the UI [ Blueprint > Analytics > Probes ]. Re-enabling the probes forces the telemetry collectors to initiate a fresh gNMI re-subscription with the device, after which the false anomalies will clear within a few minutes.
When VXLAN parameters (such as VNI) are modified in a deployed blueprint, the EVPN Type-5 route probe may incorrectly report route states for the affected Security Zone. This manifests in two ways:
Routes that are present on the device and previously shown as "Expected" are reclassified as "Unexpected" in the EVPN Table output stage.
Routes that are genuinely missing from the device may not be detected by the Missing Routes output stage, resulting in missing-route anomalies not being raised.
The issue occurs because the probe's internal state management does not correctly reconcile shared Security Zone route expectations when individual VXLANs within that zone are updated. The issue is not specific to the telemetry collection method (gRPC or CLI-based polling) and affects any operation that modifies VXLAN properties within a Security Zone.
Restart the Apstra controller service after making VXLAN changes in a deployed blueprint. Upon restart, the EVPN Type-5 probe recomputes all expected route state from scratch, restoring correct route classification in the EVPN Table and Missing Routes output stages.
Float literals expressed in scientific notation (e.g., 1.234e-05, 4.263E-06) are not recognized within the custom telemetry (aka SDT: schema-driven telemetry collector) collector's expressions. Attempting to use such literals, whether in condition filters, function arguments, or comparison statements, triggers an expression parsing error:
expression parsing error: Syntax error, mismatched input 'E' expecting <EOF>
This prevents the collector from operating when the expression contains any float value in scientific notation format.
Two options are available:
1. Convert to fixed-point notation. Replace scientific notation literals with their equivalent decimal form. For example, use 0.00001234 instead of 1.234e-05.2. Wrap in float() with a string argument. Pass the scientific notation value as a string to the float() function. For example, use float('1.234e-05') instead of the bare literal 1.234e-05.
Float literals in a scientific notation, such as 1.234e+02 are not allowed for int() function within the custom telemetry (aka SDT: schema-driven telemetry collector) expressions. Use of such literals will trigger expression parsing error "cannot convert value '1.234e+02' to Int with base 10, invalid trailing characters" and prevent the collector from operation.
Convert float literals from scientific notation to fixed-point notation before passing them to int().
Example: int('1.234e+02') -> int('123.4')
Alternatively, wrap the scientific-notation string with float() before applying int():
Example: int(float('1.234e+02'))
When using gNMI-based telemetry collectors (Periodic mode) for EVPN routes, offbox agents including offbox unit (combined default 16 agents showing much higher memory usage), may exhibit significantly elevated memory consumption (4x - 7x increase) compared to CLI-based polling. The issue is most pronounced in large-scale deployments with multiple devices collecting EVPN route data simultaneously. The root cause is internal object destruction queue pressure leading to memory fragmentation during gNMI telemetry processing.
Switch affected telemetry collectors from gNMI to CLI polling mode. After changing the collection method, restart the offbox agent to release the excess memory. CLI-based polling maintains stable memory usage (~500 MB RSS) under the same scale conditions. Please contact the HPE Apstra support team for further assistance
When viewing IBA probe stage data with a time-series aggregation function (e.g., average with a 1-hour aggregation interval), the system may return two data samples for the same aggregation period. This occurs when the schema of the probe's historical data has changed - for example, due to a probe configuration update or a modification of related data in the graph database. At the aggregation time boundary where the schema change occurred, MetricDB produces two partial result sets (one from the old schema and one from the new schema), resulting in duplicate timestamps with incomplete aggregated values displayed in the UI tooltip and time-series overlay.
No user action is required. The UI mitigates this inconsistency by retaining only the second (most recent) sample when two samples share the same aggregation timestamp. Users may observe slightly shifted data in the tooltip or chart overlay for the specific time period in which the schema change occurred; however, data outside of that boundary is unaffected.
On Junos devices, Apstra automatically configures multihop { ttl 1; } under the l3rtr BGP peer group used for Generic Systems. When BGP TTL is set to 0 in a Connectivity Template (intended for directly connected peers), no neighbor-level multihop stanza is generated, but the neighbor continues to inherit multihop ttl 1 from the l3rtr group.
In Junos, the presence of the BGP multihop hierarchy causes recursive next-hop resolution to select only a single path. As a result, VXLAN traffic may use only one next hop across uplinks (such as DCI/WAN links on Border Leaf switches) even when multiple underlay ECMP paths are available.
If BGP multihop is not required for directly connected Generic System peers, use an Apstra Configlet to remove the group-level multihop configuration from the l3rtr BGP group. Ensure that any required TTL is configured at the neighbor level, as removing the group-level configuration may cause BGP sessions to flap if the required TTL is not otherwise configured.
Alternatively, if multihop must be retained, configure Junos multipath-resolve for the VXLAN routing table to preserve multiple next hops during recursive route resolution. Please contact HPE Apstra Support for assistance.
In 6.1.1, Apstra starts tracking host fingerprint keys for the managed devices. When an Apstra controller upgrade is performed, no learned host key is present for each managed device. Because the host fingerprint key is absent, each agent's processes cannot begin until the SSH connection has been successfully established. The customer should run the update host key job for offbox agents in the managed device after upgrading the controller.
Please run the update host key job for offbox agents in the managed device after Apstra Controller Upgrade
In Apstra AOS 6.2.0, when querying the MAC Monitor Probe (or MAC Table Time-Travel/History view) over a historical time window defined from start time T1 to end time T2, the underlying metricdb QueryManager initializes pagination filters based exclusively on the active series snapshot captured at T1 time. If new MAC address series are learned or discovered dynamically after T1 (within the [T1, T2] interval), those newly discovered series are excluded from the initial pagination index filter. As a result, MAC addresses discovered after T1 fail to appear in the historical MAC log and timeseries views for that query range.
1. Adjust Query Start Time: To view history for MAC addresses learned during an operational window, set the query start time to a timestamp prior to the initial learning event of the target MAC addresses.2. Narrow Time Ranges / Refresh Snapshot: Run historical queries using shorter time windows or refresh the query to force a new baseline snapshot containing the newly discovered MAC series.
NOS rollback on Junos and EVO using Apstra Controller currently supports the rollback feature on devices with a single RE. Dual RE is not supported in this release.
When the SSH host key of an offbox Junos OS device is rotated, the CHECK job correctly detects the mismatch and fails with an SSH host key verification error. Even if there is a key mismatch, the Show-Tech job status for offbox devices will appear as Success. This is because the controller is able to reach itself and talk to the offbox container, even though it cannot reach the real device. In this case, the Show-Tech job status will appear as Success, but the real device logs and commands won't be in the show-tech tarball. The recommendation is to run a CHECK job on the device first. If it fails due to an SSH host key mismatch, run Update Host Key for the affected device before attempting Show-Tech collection again.
Run the Update Host Key job for the affected devices so that the SSH host keys are refreshed and agent health returns to normal.
1) Navigate to Devices > Managed Devices. 2) Select the affected device(s). 3) Execute the Update Host Keys job from the bulk actions or individual device menu.
Once the host key is updated and the device status returns to normal, re-run the Show-Tech collection.
When the Junos EVPN Next-hop and Interface count maximums parameter in the staged->Fabric settings->Fabric-policy is enabled, Apstra introduced modifying the default hardware settings for VXLAN routing's resource (next-hop and interfaces) for QFX5110, QFX5120, EX4650, and EX4400 devices in the rendered configuration () starting with version 4.2.0. Whenever configuration changes in VXLAN routing's resource, JUNOS triggers PFE automatic restarts to reflect new changes with service impact. The typical scenarios would be when the device becomes deployed, undeployed, or the device is in NOS upgrade. To prevent unnecessary PFE restarts in those scenarios, the configuration for VXLAN routing's resource needs to be included in the pristine configuration.
If the Junos EVPN Next-hop and Interface count maximums parameter in the staged->Fabric settings->Fabric-policy is enabled, add the below configuration into the device's pristine configuration.QFX5120 and EX4650 VXLAN routing's resource
forwarding-options { vxlan-routing { next-hop 45056; interface-num 8192; overlay-ecmp; } }
QFX5110 VXLAN routing's resource
forwarding-options { vxlan-routing { next-hop 32768; interface-num 8192; overlay-ecmp; } }
EX4400 VXLAN routing's resource (add overlay-ecmp if Junos EX-Series Overlay ECMP is also enabled)
forwarding-options { vxlan-routing { next-hop 16384; interface-num 6144; overlay-ecmp; } }
Customers may encounter the following Server-side Validation Error in the Web UI when the pristine configuration contains multiple system stanzas which is not a expected behvavior:
"Cannot parse config: system already parsed."
According to ScotchInventoryAgent logs, POST requests to update the pristine configuration failed with a 422 Unprocessable Entity error, indicating a validation issue:
2025-02-17 23:31:18,730 680:INFO:aos.scotch.libs.scotch_flask:request: POST /api/systems/AN10555621/pristine-config HTTP/1.0 34074 bytes 2025-02-17 23:31:18,737 680:INFO:aos.scotch.libs.scotch_flask:response: 422 55 bytes 0.007347 seconds
Background of the issue:
1. In Apstra 4.2.x, gRPC was introduced to support Telemetry Streaming, and as a result, having two system blocks in the pristine configuration was expected in that release. 2. Starting from Apstra 5.0.0, enhancements were made to automatically merge multiple system stanzas in the pristine configuration during the NOS upgrade process. 3. If a customer chooses to remain on their current NOS version for an extended period, multiple system stanzas can exist in the pristine configuration without causing issues.
If a customer chooses to remain on their current NOS version for an extended period and needs to forcefully update the pristine configuration, they should manually merge the system stanzas within the pristine configuration using the UI and then perform a Force Update.
For further assistance, please contact Juniper Apstra Support.
PTX10002-36QDD configured with EVPN/vxlan on Junos 23.4R2-S5-EVO have ipv6 neighbors fail
Upgrade to version 24.4R1-EVO or later, where the issue is resolved.
Note: Junos 24.4R1-EVO and later are not part of the Apstra 6.1 qualified NOS list (Apstra 6.2.0 supports Junos EVO 25.2R2-S1-EVO as the qualified version)
A known limitation exists in the XML Transform node of the SDT (Schema Driven Telemetry) collector within IBA. When a single XML Transform node is configured with XPath accessors that extract data from multiple distinct nested and repeated XML element groups (i.e., sibling elements that each contain their own repeated child elements), the parser may incorrectly merge rows from adjacent groups into a single output row.
This occurs because the XML-to-table conversion handles only one group of nested and repeated elements reliably. When multiple such groups are present (representing independent 1-to-many relationships), the parser performs a row extension that combines values from the first occurrence of one repeated group with values from its sibling group, resulting in data that appears "smushed" together. Some fields in the affected rows may contain values that belong to a different logical record.
When extracting data from multiple nested and repeated XML element sections via XPath accessors, create a separate XML Transform node for each such section to properly represent each group as an independent table. Then use a Join node to combine the outputs and model the relationships across the separate XML Transform nodes. Note that the Join node accepts only two inputs; therefore, if more than two XML Transform nodes are required, chain additional Join nodes accordingly.
In the IBA (Intent-Based Analytics) Real-Time View, the State field filter for probe only functions correctly with the Equal (=) operator. Applying the Not Equal (!=), Regex (~), In, or Not In operators to the State field returns no results or incorrect results. The UI presents all operator options in the dropdown, but only the Equal operator is supported for enum-type fields in Real-Time View due to backend processing constraints. This limitation does not affect the time-series view, where all operators function as expected.
Use the Equal (=) operator when filtering on the State field in Real-Time View. If advanced filtering (Not Equal, Regex, In, Not In) is required, use the Time-Series View where all operators are fully supported.
When editing an interface policy that includes the Link Auto-Negotiation intent, toggling the intent off via the UI does not correctly disable the setting. After updating the policy, the interface policy table continues to display the previously configured value (e.g., "Enabled"), and the change is not reflected in the Logical Diff. This occurs because the UI does not explicitly clear the field value when the intent is turned off, causing the backend to retain the last configured state. Other policy intents such as 802.1X are not affected by this issue.
Use the API to explicitly set link_auto_negotiation to null in the PATCH payload (API URL: https://<controller-ip>/api/blueprints/<blueprint_id>/interface-policies/policy/<policy-id>) when disabling the intent. Avoid toggling the Link Auto-Negotiation intent off through the UI until the fix is applied. Alternatively, do not set a policy element and then disable its intent, as disabling the intent through the UI makes it invisible rather than resetting it.
https://<controller-ip>/api/blueprints/<blueprint_id>/interface-policies/policy/<policy-id>
When the virtual infra manager is removed from the Apstra controller, Apstra should have cleared any data related to the virtual infra manager. Because it's not cleared, when the same virtual infra manager is added back to Apstra later, old data is still used together with the new collected data from the virtual infra manager's collector. In some scenarios, when old, uncleaned data has an error condition, it can trigger continuous error even if newly collected data doesn't have an error condition.
If the virtual infrastructure manager requires re-onboarding (removing and then adding back) from Apstra, the user must take the actions listed below.1. Remove virtual infra manager from Apstra Controller (External Systems/Virtual Infra Managers).2. Restart the AOS service.3. Add the virtual infra manager back to to the Apstra
When importing Virtual Networks (VNs) using a CSV file into a Blueprint, the import operation may fail with an error similar to Invalid header names: "bound_to_<system_node_label>". This issue occurs because the CSV header validation strictly allows only alphanumeric characters and select symbols (a-z, A-Z, 0-9, (, ), [, ], _, -). If a system node label (such as a leaf or leaf pair name) includes characters outside this set (for example, periods .), the bound_to_ column header in the exported CSV is rejected during re-import.
Invalid header names: "bound_to_<system_node_label>"
(a-z, A-Z, 0-9, (, ), [, ], _, -)
If you need to perform bulk VN modifications using CSV export/import, use the following steps:1. Temporarily rename the label of the system node in the blueprint2. Export and modify the VN CSV (apply bulk changes into the file)3. Import the VN CSV4. Restore Original Node Label5. Review and Commit
On a ZTP server configured for IPv6-only operation, the ZTP service status may incorrectly report a previously configured IPv4 address instead of the current IPv6 address.
This can occur when a stale ZTP_SERVER_IP value remains from a previous IPv4 configuration. As a result, the stale IPv4 address may be displayed for ZTP services even though the ZTP server is configured with only an IPv6 address.
Remove the stale ZTP_SERVER_IP entry from /etc/apstra_ztp/extra.env, while retaining the ZTP_SERVER_IP_V6 entry, and restart the ZTP services: systemctl restart apstra-ztp
After the restart, the ZTP service status reports the configured IPv6 address.
A case has been observed where no telemetry data is sent from a 25.2R2-S1.3 Junos onbox device to an EVPN collector. The collector observes that no data is received during initial sync, which triggers a resubscription to the device. This resubscription cycle repeats up to 4 times (the maximum retry count), but even after all retries, no data is received by the specific collector/leaf pair. The gRPC In Sync Count can be observed in the Analytics > Services > EVPN_VXLAN_TYPE page. This issue is not platform-specific and may occur on any device running the affected Junos version.
Running `restart na-grpc-server` on the device should resolve the issue as a solution until NOS addresses the issue.
HPE EX4100 devices running Junos 22.x releases are susceptible to an OS corruption issue that can cause the device to boot into shell mode instead of Junos CLI after a reboot. This corruption condition persists even if the device is later upgraded to a 23.x or 24.x release, as it originates from the initial onboarding state. Currently, Apstra does not validate or warn the user when onboarding EX4100 devices running affected NOS versions, which may lead to unexpected device failures following reboot operations.
Ensure EX4100 devices are running Junos 23.4R2-S7.7 or 24.4R2-S4.10 (or later) before onboarding into Apstra. Do not onboard EX4100 devices running any Junos 22.x release.
When deploying HPE QFX5130 switches within an Apstra L2 Multi-tenancy (EVPN-VXLAN) environment, packets encapsulated within VXLAN tunnels may have their inner VLAN tags improperly stripped upon egressing switch interfaces configured with mac-vrf service-type vlan-bundle. Consequently, egress frames arrive at destination hosts without the required VLAN tag, resulting in ARP resolution failures, packet loss, and forwarding breakdown within the affected MAC VRF domain
The NXOS rollback feature on the N9K-C93600CD-GX device has significant limitations when the devices' interfaces are broken out.Ports 1-24 in the model are organized into four-port groups: (1, 2, 3, 4), (5, 6, 7, 8), (9, 10, 11, 12), (13, 14, 15, 16), (17, 18, 19, 20), and (21, 22, 23, 24). When port 1 is broken out as 4x10G or 4x25G, port 3 is automatically broken out in the same mode, and vice versa. When any port in the quadruple is split into 2x50G, all four ports are automatically split in the same mode. Similarly, ports 26-28 are organized in pairs of two, i.e. (25, 26) and (27, 28). Both ports in the pair must operate in the same breakout mode.
In most cases where a breakout (or more than one) exists, rollback fails to generate a working rollback patch. The reason for this is that the breakouts cannot be reversed if the remaining broken-out interfaces in the same port group have not been shutdown first. For example, to negate the breakout of port 1, the broken-out interfaces of port 3 must be shutdown, and vice versa. It appears that the rollback logic shuts down the interfaces associated with the port whose breakout is being reverted (port 1 in the previous example), but fails to shut down other broken-out ports in the same port group (port 3).
The safer way for the N9K-C93600CD-GX to be used with AOS is for the customer to avoid using breakouts altogether on the device.No issue with rollback when ports 29-36 have been broken out has been observed. Breakouts on these ports can be rolled back In the case that the last interface of a port-group is the only one used and broken out, would the nxos rollback feature (and rollback to pristine) be successful. However this is highly discouragedIn any other case the only way to reverting to pristine would be to manually shudtown all broken down interfaces before reverting to pristine (or using the rollback to a pristine config)
In some cases, a change in the configuration of an NXOS device (for example, removing a VXLAN VNI from a VN) will take a few seconds to be applied to the running configuration. The Apstra Device Agent collects the running configuration right after a config deployment and may collect a configuration that still shows the state before the change. Ultimately the change makes its way to the running config on the NXOS device, but not before the Apstra Device Agent has collected the so-called golden config. Minute later Apstra Device Agent will re-collect the running config and find the difference with the golden configuration collected a minute ago. config deviation anomaly will be raised.
The user can inspect the difference(s) between the initially collected golden configuration and the current configuration. If the differences indeed reflect the intended change and indicate a mere delay between two consecutive collections of running configuration, the user can just acknowledge the deviation, which will suppress the anomaly.
Configuration deployment may fail on an EOS 4.24 device due to the command `maximum-paths 512` not being accepted under the `router bgp` stanza. In later versions (e.g. 4.30), the same number of maximum-paths is accepted. This has been observed on an Arista DCS-7280CR3MK-32P4 device.
User can clone the specific EOS model predefined device profile to a custom device profile, edit the ecmp-limit (e.g. to 256 from the initial 512) under the hardware-capabilities in that custom device profile, then use that custom device profile on the device exhibiting the problem. Please contact HPE Apstra Support Team if further assistance is needed.
On HPE QFX5130 platform, DHCP relay functionality for both IPv4 and IPv6 may fail to forward DHCP server responses back to clients when selective IRB-enabled Virtual Networks (VNs) coexist with ERB-mode Virtual Networks within the same blueprint reference design.Although client DHCP request messages (such as SOLICIT) are successfully relayed to the DHCP server and the server responds (such as with ADVERTISE messages), the relay agent fails to deliver the server responses back to the requesting clients. Consequently, DHCP clients are unable to complete the IP address assignment process.
To avoid DHCP relay issues on QFX5130 switches, do not deploy a mixed configuration of selective IRB (CRB) and ERB Virtual Networks within the same blueprint. Instead, maintain a consistent Virtual Network gateway model across the entire blueprint reference design
The FEC Histogram data (Symbol Error Per Code Word bins) is not available on QFX5240 platforms running Junos EVO 25.2X100-D20. As a result, the Apstra IBA probe "FEC Statistics," which relies on FEC Histogram metrics to monitor proactive link quality on high-speed interfaces, may report incomplete or inaccurate data for QFX5240 devices
In Apstra-managed EVPN-VXLAN fabrics configured with Centralized Routed Bridging (CRB) / Selective IRB on QFX5130 switches running Junos EVO (e.g., Junos EVO 25.2X100-D20.6-EVO), endpoints connected to non-Border Leaf (pure Layer 2) switches experience ARP and Neighbor Discovery (ND) resolution failures when attempting to resolve the Virtual Gateway (VGW) IP address configured on Border Leaf switches.
The currently available workaround is to configure a permanent static ARP (IPv4) or static Neighbor Discovery (IPv6) entry on the affected connected hosts/servers for the Virtual Gateway IP address, mapping it directly to the Virtual Gateway MAC address
On Junos EVO platforms (confirmed on QFX5130), adding or deleting a Virtual Network(VN) in a user-defined MAC-VRF causes the physical interface to flap, resulting in a brief traffic disruption. This occurs because the configuration change requires transitioning the interface to flexible-vlan-tagging with encapsulation flexible-ethernet-services and creating or removing a sub-interface, which triggers a link flap on EVO plaform. This issue does not occur when the VN is assigned to the default evpn-1 MAC-VRF, as the VN is added as a VLAN member to the existing unit 0 without modifying the interface encapsulation or creating a new sub-interface. This issue is also not observed on non-EVO platforms such as the QFX5120.
Plan VN additions or removals involving user-defined MAC-VRFs during a maintenance window to minimize traffic impact. Where possible, assign VNs to the default evpn-1 MAC-VRF to avoid the interface encapsulation change that triggers the flap.
Apstra uses ACL for security policy. The default TCAM profile (profile-two) only supports matching up to /64 bits for IPv6 source addresses in firewall filters. As a result, IPv6 ACL rules with source or destination address prefix lengths more specific than /64 (e.g., /128) do not match traffic as expected, and traffic that should be denied is permitted. This behavior is specific to Junos EVO. Junos devices handle IPv6 ACLs correctly regardless of prefix length.
Apstra now raises a warning when an IPv6-enabled policy is configured with a source or destination prefix length longer than /64 on a blueprint containing Junos EVO leaf devices, advising the user to apply the appropriate TCAM profile.
Configure the device to use TCAM profile-one, which supports full-length IPv6 source address matching. Apply the following configuration on affected Junos EVO devices:set system packet-forwarding-options firewall-profile profile-one
set system packet-forwarding-options firewall-profile profile-one
Note: Changing the TCAM profile may require a PFE restart. This configuration can be recommended in the pristine configuration instead of the configlet to prevent unexpected PFE restart.
When a Virtual Gateway MAC (VMAC) is modified on an IRB interface via Apstra and the configuration is committed, Junos and EVO re-initialize the affected IRB interface. This causes a momentary interface flap that disrupts any BGP sessions sourced from that IRB, including IPv4 and IPv6 peering sessions with Generic Systems.
Schedule VMAC changes during a maintenance window when temporary BGP session disruption is acceptable. No configuration-level workaround is currently available.
On Juniper PTX Series packet transport routers and virtual EVO (vEVO) platforms running Junos EVO, Enterprise-Style (EP) and Service Provider-Style (SP) interface configurations cannot coexist simultaneously on the same physical interface (IFD) with flexible-ethernet-services enabled. In Apstra, an interface configured with VNs in the default MAC-VRF initially utilizes Enterprise-Style configuration (family ethernet-switching). When a user attaches the first VN belonging to a user-defined MAC-VRF to that interface, Apstra automatically transforms the interface configuration from EP-style to SP-style (unit encapsulation vlan-bridge with flexible-ethernet-services and flexible-vlan-tagging). Similarly, deleting the last user-defined MAC-VRF VN converts the interface configuration back from SP-style to EP-style. Because Junos EVO re-initializes interface logical units (IFLs) during this configuration model transition, active traffic on pre-existing VNs attached to that interface experiences a brief, transient hit during commit execution.
When performing a NOS upgrade or downgrade operation from the Apstra Managed Devices UI on HPE ACX7100 series switches (ACX7100-32C and ACX7100-48L) running Junos EVO 22.x releases, the task may remain stuck indefinitely in the In-progress state even after the NOS image installation successfully completes on the switch. On devices with BIOS ROM firmware version 16.02 or later, a firmware compatibility issue under older Junos EVO images prevents data interfaces from initializing, causing the show chassis mac-addresses command to return empty output. As a result, the Apstra device agent fails to retrieve required chassis MAC information upon startup, repeatedly crashes, and cannot complete post-upgrade configuration deployment to report a successful status.This issue affects only ACX7100 series switches with BIOS ROM firmware version 16.02 or later running Junos EVO 22.x images.
Upgrade affected HPE ACX7100 series devices to Junos EVO 23.4R2-S7.3-EVO or later. This release resolves the BIOS firmware compatibility issue and restores proper data interface initialization, chassis MAC address reporting, and Apstra device agent communication
If the Arista EOS device's internal flash drive does not have sufficient capacity to support the new EOS image, the Apstra ZTP process might not be able to proceed. The internal flash has to be large enough to hold two specific images, the running EOS image and the new image that will be installed.
It may be necessary for the customer to manually delete older images from the internal flash drive before proceeding with the ZTP process.
In multi-tenant EVPN deployments utilizing Q-in-Q VLAN stacking and overlapping VLAN ranges, applying VLAN translation or modifying existing VLAN override mappings on QFX5130 switches running Junos EVO can cause traffic drops at both ingress and egress. When multiple VLAN-related configuration changes are committed simultaneously, or when existing interface VLAN mappings are updated on active interfaces, the Packet Forwarding Engine (PFE) on QFX5130 platforms may fail to properly program access interfaces into the corresponding Virtual Switch Instance / bridge domain (VFI). As a result, ingress membership checks fail, and the translation VNI becomes unmapped, disrupting Layer 2 connectivity across the VXLAN fabric.
Avoid committing multiple VLAN-related configuration updates in a single commit operation. Apply and commit VLAN translation and override rules sequentially
In Apstra L3 Clos reference designs utilizing CRB with Juniper QFX5130 border leaf switches operating as gateway nodes (IRB enabled), an issue occurs with DHCP relay agent packet processing after virtual network under selective IRB mode configuration changes.When a Virtual Network (VN) with selective IRB mode and enabled DHCP Service is deleted across the fabric and subsequently recreated, the DHCP relay agent on the QFX5130 switch fails to append Option-82 (Relay Agent Information Option) to client DHCP request messages prior to forwarding them to the external DHCP server. Because network DHCP servers depend on Option-82 data to enforce scope rules and address assignment policies, DHCP requests missing relay information are ignored or dropped by the server without returning DHCP Offer packets. As a result, client hosts connected to the recreated virtual network fail to acquire or renew IP addresses.
2026-09-18: Added AOS-63682,AOS-63699. Updated RFE-2949
2026-09-15: Added AOS-52788,AOS-58620,AOS-63415
2026-09-08: Initial Publishing