Description

This article explains that commit may fail on MX Series routers with the "error: could not copy to juniper.save+" and steps to mitigate the same.

Symptoms

Below logs are observed on customer device-
 

MX# commit comment "comment"
re0:
configuration check succeeds
re1:
error: could not copy to juniper.save+
re0:
error: remote commit-configuration failed on re1

Below logs/messages can also be seen-

Jul 25 02:01:23.076 2019  MX kernel: %KERN-3: jsr_sdrl_setup_primary: Socket already has replication setup (state 0x1)
Jul 25 02:01:23.076 2019  MX kernel: %KERN-3: jsr_sdrl_pri_alloc: failed 17

Jul 25 02:08:02.857 2019  MX kernel: %KERN-3: pid 63755 (repd), uid 0 inumber 1767478 on /.mount: filesystem full
Jul 25 02:08:04.475 2019  MX kernel: %KERN-3: pid 63755 (repd), uid 0 inumber 1767462 on /.mount: filesystem full
Jul 25 02:08:05.689 2019  MX kernel: %KERN-3: pid 67126 (rcp), uid 0 inumber 2327533 on /.mount: filesystem full
Jul 25 02:08:08.935 2019  MX kernel: %KERN-3: pid 63755 (repd), uid 0 inumber 1767468 on /.mount: filesystem full
Jul 25 02:08:12.445 2019  MX kernel: %KERN-3: pid 63755 (repd), uid 0 inumber 1767462 on /.mount: filesystem full
Jul 25 02:08:12.605 2019  MX kernel: %KERN-3: pid 40716 (jdhcpd), uid 0 inumber 1845980 on /.mount: filesystem full


Because the filesystem is full, there will also be kernel synchronization issues between the two Routing Engines:
 

lab@MX> show task replication
        Stateful Replication: Enabled
        RE mode: Master
 
    Protocol                Synchronization Status
    BGP                     NotStarted
Jul 25 02:06:10.155 2019  MX ksyncd[4801]: %DAEMON-3-KSYNCD: resync error, issu_state[0], type ROUTE subtype 18 : Cannot allocate memory
Jul 25 02:06:10.155 2019  MX ksyncd[4801]: %DAEMON-3-KSYNCD: resync error, issu_state[0], type ROUTE subtype 18 : Cannot allocate memory

Solution

In this case, the backup Routing Engine has run out of memory due to which commit is failing.

 

MX>show system storage

Filesystem              Size       Used      Avail  Capacity   Mounted on

/dev/md0.uzip            21M        21M         0B      100%  /
devfs                   1.0K       1.0K         0B      100%  /dev
/dev/gpt/junos           19G        19G      -1.5G      109%  /.mount >>>>>

To prevent commit from failing, perform the following:
 

1.List the files by using the following command:
lab@MX> request system storage cleanup dry-run

2.Identify any unwanted files and delete them by using the following command:

lab@MX> file delete /<path>/file-name

 

When some disk space is freed, commit should be successful

 

Modification History

2024-03-20 : Initial Draft