Description

In a spoke site, while removing a LAN department through CSO, a job failed with the error below: 

VLAN-ID must be specified on tagged ethernet interfaces\\error: configuration check-out failed\\error is propagated from DCS

Symptoms

From CSO UI, Job Log showed the following failure:

add_department_to_site_execute_6a9610bd94144141abca56b6d53b48b4  add_department_     failed         [email protected]      tssm add department to site           7:55:23 AM       2020 7:55:49 AM
 
Deploy LAN Segment Details
Name csp.tssm_add_department_to_site
User [email protected]
State failed
 

Import Logs [add_department_to_site_execute_6a9610bd94144141abca56b6d53b48b4] Hierarchical View:

Task: add-department
11 de Sep. de 2020, 16:55:27Stage 2 config start, task_id = add-department ...
11 de Sep. de 2020, 16:55:28Start stage-2-config stage 1: resolve profile add-department success
11 de Sep. de 2020, 16:55:29Start stage-2-config stage 2: create abstract configs success: {'JDM': [[u'default-domain', u'testsite', u'Vtest-site', u'JDM/jdm-add-department-config']], 'JUNOS': [], 'JCP': [[u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config']], 'GWR': [[u'default-domain', u'testsite', u'Vtest-site', u'GWR/gwr-add-department-config']]}
11 de Sep. de 2020, 16:55:29Configs to be deployed include {'JDM': [[u'default-domain', u'testsite', u'Vtest-site', u'JDM/jdm-add-department-config']], 'JCP': [[u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config']], 'GWR': [[u'default-domain', u'testsite', u'Vtest-site', u'GWR/gwr-add-department-config']]}
11 de Sep. de 2020, 16:55:29About to publish and deploy JDM configs [[u'default-domain', u'testsite', u'Vtest-site', u'JDM/jdm-add-department-config']]
11 de Sep. de 2020, 16:55:36Stage 2 config for JDM: [[u'default-domain', u'testsite', u'Vtest-site', u'JDM/jdm-add-department-config']] deployed success
11 de Sep. de 2020, 16:55:36About to publish and deploy JCP configs [[u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config']]
Task: 15
11 de Sep. de 2020, 16:55:37Deploy Config Stage 1: publish config [u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config'] for component JCP SUCCESS
11 de Sep. de 2020, 16:55:37Deploy Config Stage 2: deploy config [u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config'] for component JCP SUCCESS, request_id: f35411d0-746c-49be-ba7c-8feb5338a2d3
11 de Sep. de 2020, 16:55:49Deploy Config Stage 3: JCP received notification: {"status": "SUCCESS", "requestid": "f35411d0-746c-49be-ba7c-8feb5338a2d3", "description": null, "hapi_remote_host": "csp.csp-dms-cms-inv-central-core-2111937828-pk5h8", "hapi_request_id": "UI_a162e717-6cd7-3d5c-3b2e-6049491cc357,2vs6,juMt,lqe7", "details": {"6cf37050-cd33-4c9d-a96f-2c0e53f3c558": {"status": "SUCCESS", "abstract_config_uuid": "6cf37050-cd33-4c9d-a96f-2c0e53f3c558", "description": null, "abstract_config_name": ["default-domain", "testsite", "Vtest-site", "JCP/jcp-add-department-config"], "config_change_id_list": ["dc1f8ffc-05db-49fd-b2d8-985e77dc4987"], "device_component_name": "JCP", "device_uuid": "0f9b3ab8-71d4-4ed4-bf29-63556f0fdbc6"}}, "hapi_job_id": "1f611c33-4a15-45a0-8aa9-b322d7376f30"}
11 de Sep. de 2020, 16:55:49Deploy Config Stage 3: deploy config SUCCESS
11 de Sep. de 2020, 16:55:49Task complete
11 de Sep. de 2020, 16:55:49Stage 2 config for JCP: [[u'default-domain', u'testsite', u'Vtest-site', u'JCP/jcp-add-department-config']] deployed success
11 de Sep. de 2020, 16:55:49About to publish and deploy GWR configs [[u'default-domain', u'testsite', u'Vtest-site', u'GWR/gwr-add-department-config']]
Task: 16
11 de Sep. de 2020, 16:55:50Deploy Config Stage 1: publish config [u'default-domain', u'testsite', u'Vtest-site', u'GWR/gwr-add-department-config'] for component GWR SUCCESS
11 de Sep. de 2020, 16:55:50Deploy Config Stage 2: deploy config [u'default-domain', u'testsite', u'Vtest-site', u'GWR/gwr-add-department-config'] for component GWR SUCCESS, request_id: ad24679a-386e-4f2f-ad42-86991e9eaa11
 

Using Kibana, searching with the request ID of the Job [request ID can be obtained from the monitor Job page of CSO UI], the configuration was found to be failing : 

[edit interfaces ge-0/0/4]\\n 'unit 1'\\n VLAN-ID must be specified on tagged ethernet interfaces\\nerror: configuration check-out failed\\n\\n[edit]\\ncsp@NFX250#  \", \"error_diag\": \"This error is propagated from DCS. It occurs during device command execution. \"}}", "hapi_user_client_ip": "10.107.8.102", "hapi_request_id": "UI_fc445eb8-82f2-bd0a-2f14-aeec3b5a2ff4,KZbh,MxI0,kgYi", "details": {"f726aea8-7380-49af-b5c1-95681f1531cc":  {"status": "FAILURE", "abstract_config_uuid": "f726aea8-7380-49af-b5c1-95681f1531cc", "description": "{\"error_data\": {\"status_code\": \"500\", \"error_tag\": \"Command Execution Error\", \"error_message\": \"requestid: UI_fc445eb8-82f2-bd0a-2f14-aeec3b5a2ff4,KZbh,MxI0,kgYi,sSRM.1ddac240-f497-11ea-9ac7-0242ac105e17  deviceid: 26f72283-9f5d-456a-88aa-52f4737dcfe2 commit comment \\\"CSO_REQ_ID:[email protected]_HS#

Solution

The interface where the job failed was manually disabled directly instead of disabling it through groups configuration.

Troubleshooting:
  1. The error log showed the reason for failure was 'interfaces ge-0/0/4, unit , VLAN-ID must be specified on tagged ethernet interfaces'
  2. Logged into the spoke site where we tried to add the LAN segment
  3. Checked the interface configuration for the interface specified in the log.
  4. It was discovered that interface ge-0/0/4, unit 1 was manually disabled directly instead of disabling through groups configuration.

    Example:

    set interfaces ge-0/0/4 unit 1 disable
Solution:
  1. Removed the manual configuration under the respective interface.
  2. Re-tried the job again for the LAN department removal. It was successful without any issues.
  3. In CSO, the configuration pushed to spoke sites are through Junos groups configuration.

    Example expected configuration:

    set groups dept-configuration interfaces ge-0/0/4 unit 1 disable

If changes are made to any of the configuration outside the group, CSO will not be able to detect it. Thus, when trying to remove the configuration through CSO, it will fail.