This article explain more about "USER-5-GSTATD_AVERAGE_DURATION_THRESHOLD_CROSSED: [vtbd1] average write duration of 1200.35 ms crossed threshold of 1000.00 ms" seen just after GRES/NSR activation.
After performing Graceful switchover the below logs start appearing in the device.
USER-5-GSTATD_AVERAGE_DURATION_THRESHOLD_CROSSED: [vtbd1] average write duration of 1200.35 ms crossed threshold of 1000.00 ms
GSTATD is expected to start everytime the GRES and NSR is configured and the process is initiated so in your case also its working as expected. However those threshold time crossed logs are appeared for very small interval of time which was due to kernel busy and if any disk failure or RE bug it would have certainly showed many other issues alongside which is not the case.
The device (mx304) is vmhost device.
vtbd1 is a virtual-block-device mapped through qemu from Linux host.
1. there is no tool like smartctl (which can get statistics from physical disk) that can get detail like that from virtio-block-disk.
2. If, your device really disk problem, usually it will have more "subsequent-effect" like slow operation, rpd-slip, more-and-more THRESHOLD_CROSSED message.
(i.e. it's likely it was at some system transient state that it was "slow"... led to IO delay... and hence, only one incident had been logged.
I check your /var/log/messages.*.gz, it does not look like "it" has issue like that: <experienced long period of IO issue>).
3. From vmhost... the most we can get to (you may consult platform in this regards, we are from FreeBSD-JUNOS vm layer)
If I get it correctly from vhclient -s (its Linux host)
junos drive: (from ps at Linux)
/usr/bin/qemu-system-x86_64 ..... -drive file=/junos/jinstall-current.img,if=none,id=drive-virtio-disk0
root@jtac-mx304-r2019-re0-node:/junos# df .
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/jvg_P-jlvmjunos 32781020 5623636 25462460 19% /junos
root@jtac-mx304-r2019-re0-node:/junos# df /junos
root@jtac-mx304-r2019-re0-node:/junos# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 186.3G 0 disk
|-sda1 8:1 0 511M 0 part
|-sda2 8:2 0 512M 0 part
|-sda3 8:3 0 4G 0 part /.old
|-sda4 8:4 0 4G 0 part
|-sda5 8:5 0 177.2G 0 part <<<<<<<<<<<<<<<<<<<
| |-jvg_P-jlvmrootrw 252:4 0 8G 0 lvm /.old/tdev/.union/.s
| |-jvg_P-jlvmjunos 252:5 0 32G 0 lvm /tmp/junos-lHRCs <<<<<<<<<<<<<<<<<<<
| |-jvg_P-jlvmvm 252:6 0 137.2G 0 lvm /vm
| `-jvg_P-jlvmspare 252:7 0 8M 0 lvm /spare
|-sdb5 8:21 0 177.2G 0 part
| |-jvg_S-jlvmrootrw 252:0 0 8G 0 lvm
| |-jvg_S-jlvmjunos 252:1 0 32G 0 lvm <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
| |-jvg_S-jlvmvm 252:2 0 137.2G 0 lvm
| `-jvg_S-jlvmspare 252:3 0 8M 0 lvm
==> One can check (from Linux)
smartctl -a sda5 / sdb5
the results show nothing special, they look good.
(However, the key is point-2 above...)
This isn't a problem. gstatd is simply reporting that it's taking longer than it should to write to disk. This can indicate any (or more) of the following:
* System is overly busy.
* Disk is bad. Check other logs to verify this.
In short, gstatd is working per design.