Description

This article provides a work-around for correcting the error condition.

Symptoms

  • VPN Publish operation fails for a device.

  • The following error is displayed during the publish attempt:
    “Another publish, unpublish, preview or update job is in progress for this device. Re-try after sometime.”

Solution

Affected Environment details:
Space: 21.3R1 with V8 hotpatch
SD: 21.3R1 with V11 hotpatch


A stale or invalid database entry (NULL vpn_id) exists in the IPSecVPNDeviceEntity table, causing Security Director to incorrectly detect an ongoing job.

Step 1: 
Verify the below error in SD.log While publishing the VPN policy

2024-02-07 18:37:58,432 ERROR [net.juniper.jnap.sm.provisioning.policy.deploy.task.ZoneCreationStrategy] (JobThread2064387) Exception in getting device for moID net.juniper.jnap.sm.om.jpa.SecurityDeviceEntity:3933834
2024-02-07 18:37:58,448 WARN [net.juniper.jnap.sm.provisioning.jobManager.SDProvisioningJobManagerBean] (JobThread2064387) Unknown persistence exception encountered: -> cause: class javax.persistence.PersistenceException-> cause: class org.hibernate.exception.GenericJDBCException-> cause: class java.sql.SQLException
2024-02-07 18:37:58,448 ERROR [net.juniper.jnap.sm.provisioning.common.ExceptionHandlingUtils] (JobThread2064387) Error encountered while calculating/updating configuration for device:eu-gb-ld8-cm-fwe01: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not prepare statement
    at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154) [hibernate-core-5.3.20.Final.jar:5.3.20.Final]
Caused by: org.hibernate.exception.GenericJDBCException: could not prepare statement
Caused by: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_MARKED_ROLLBACK

2024-02-07 18:37:58,459 ERROR [net.juniper.jnap.sm.provisioning.common.ExceptionHandlingUtils] (JobThread2064387) DB failure during conversion. Another conversion may be in progress
2024-02-07 18:37:58,475 ERROR [net.juniper.jnap.sm.provisioning.common.ExceptionHandlingUtils] (JobThread2064387) Rolling back transaction due to exception.

 

Step 2: Create a fresh database backup as a rollback plan before making any changes. Follow the standard Junos Space database backup procedure.
https://www.juniper.net/documentation/us/en/software/junos-space23.1/junos-space-workspaces/topics/task/junos-space-database-backing-up.html

 

Step 3: To login under SD database, run the below command

# mysql -u jboss -p$(grep mysql.jboss /etc/sysconfig/JunosSpace/pwd | awk -F= '{print $2}') sm_db

Verify if any provisioning jobs are still in progress:

mysql> select * from SDProvisioningJobResult where status=2;
Empty set (0.01 sec)

Identify invalid VPN entries:

Sample:
mysql> select id,vpn_id from IPSecVPNDeviceEntity; +---------+---------+ | id | vpn_id | +---------+---------+ | 3998253 | NULL |


Step 4: After clearing the invalid/null entry, retry the VPN publish operation.

mysql> delete from IPSecVPNDeviceEntity where id=3998253;
ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`sm_db`.`IPSecVPNEndPointEntity`, CONSTRAINT `FK_hxvi422vdu1yk5fiwfvh8ycpb` FOREIGN KEY (`device_id`) REFERENCES `IPSecVPNDeviceEntity` (`id`))

mysql> select * from IPSecVPNEndPointEntity where device_id=3998253;
+---------+----------+---------+----------------+-----------+-------+-----------+---------------+-----------------+----------------+---------------------+--------------+---------+---------------+---------------+--------------+------------+--------------------------------+--------------------+-----------+------------+-----------------------+--------+
| id   | domainId | version | externalIfName | ikeGwName | ikeId | ikeIdType | ikePolicyName | ipsecPolicyName | lastModifiedAt | maxTransmissionUnit | presharedKey | proxyId | remoteProxyId | tunnelAddress | tunnelIfName | tunnelZone | vpnNameInDevice        | advpnConnParams_id | device_id | profile_id | remotePeerEndPoint_id | vpn_id |
+---------+----------+---------+----------------+-----------+-------+-----------+---------------+-----------------+----------------+---------------------+--------------+---------+---------------+---------------+--------------+------------+--------------------------------+--------------------+-----------+------------+-----------------------+--------+
| 3998254 |  NULL |   0 | NULL     | NULL   | NULL |   NULL | NULL     | NULL      |     NULL |         -1 | NULL    | NULL  | NULL     | NULL     | NULL    | NULL   | 5_32_160_129_ARAL-OPAL-VPN-LD8 |       NULL | 3998253 |   NULL |         NULL | NULL |
+---------+----------+---------+----------------+-----------+-------+-----------+---------------+-----------------+----------------+---------------------+--------------+---------+---------------+---------------+--------------+------------+--------------------------------+--------------------+-----------+------------+-----------------------+--------+
1 row in set (0.00 sec)

mysql> delete from IPSecVPNEndPointEntity where device_id=3998253;
Query OK, 1 row affected (0.00 sec)

mysql> delete from IPSecVPNDeviceEntity where id=3998253;
Query OK, 1 row affected (0.01 sec)

Post clearing the null entry we should be able to perform VPN publish operation for the affected device.


Note: If the issue persists after applying this workaround, restore the database from the backup taken earlier and engage JTACfor further assistance. Refer to the Junos Space documentation for database restore procedures.

https://www.juniper.net/documentation/us/en/software/junos-space23.1/junos-space-workspaces/topics/task/junos-space-database-restoring.html

Modification History

2024-02-08 : Article Published