We have a pair of SSRs where Node 2 does not show any statistics when both nodes are only. When I disabled the 128T service on Node 1 during a carrier issue, then Node 2 started showing statistics. When Node 1 was reactivated, Node 2 stop showing statistics.
Reviewing the logs showed error - 401 Authorization Required
nginx.service.log
Jan 17 15:26:10 XX nginx[22961]: xx nginx: 127.0.0.1 - "POST /api/v1/metrics/current HTTP/1.1" "request-id=Y2E1OGFkND" 401 172 rt=0.001 ut=- "-" "mars/5.6.12 (25745)"
mars.service.log
Jan 17 20:22:27 XX mars[21220]: ERROR Failed to retrieve peer metrics: failed to retrieve peer metrics: got status code 401: <html>
Jan 17 20:22:09 XX mars[21220]: {"request-id": "f88664b2", "ua": "128t-web-server(22901)", "user": "admin"}
Jan 17 20:22:09 XX mars[21220]: </html>
Jan 17 20:22:09 XX mars[21220]: </body>
Jan 17 20:22:09 XX mars[21220]: <hr><center>nginx</center>
Jan 17 20:22:09 XX mars[21220]: <center><h1>401 Authorization Required</h1></center>
Jan 17 20:22:09 XX mars[21220]: <body>
Jan 17 20:22:09 XX mars[21220]: <head><title>401 Authorization Required</title></head>
Jan 17 20:22:09 XX mars[21220]: ERROR Failed to retrieve peer metrics: failed to retrieve peer metrics: got status code 401: <html>
Jan 17 20:21:05 XX mars[21220]: {"request-id": "ebf58f02", "ua": "128t-web-server(22901)", "user": "admin"}
Jan 17 20:21:05 XX mars[21220]: </html>
Jan 17 20:21:05 XX mars[21220]: </body>
Jan 17 20:21:05 XX mars[21220]: <hr><center>nginx</center>
Jan 17 20:21:05 XX mars[21220]: <center><h1>401 Authorization Required</h1></center>
Jan 17 20:21:05 XX mars[21220]: <body>
Jan 17 20:21:05 XX mars[21220]: <head><title>401 Authorization Required</title></head>
Jan 17 20:21:05 XX mars[21220]: ERROR Failed to retrieve peer metrics: failed to retrieve peer metrics: got status code 401: <html>
Jan 17 09:10:05 XX mars[21220]: INFO Current time 2025-01-17 09:10:05.033047468 +0000 UTC m=+25725249.247517152 is after next sample time 2025-01-17 09:10:00 +0000 UTC. Recomputing next sample time.
Jan 17 09:10:05 XX mars[21220]: INFO No processes produced valid responses
Jan 17 09:10:05 XX mars[21220]: INFO Unable to retrieve from 'highway': failed to retrieve metrics from 'highway': Post "http://localhost/api/v1/metrics": context deadline exceeded
Jan 16 23:41:55 XX mars[21220]: INFO Current time 2025-01-16 23:41:55.015631955 +0000 UTC m=+25691159.230101638 is after next sample time 2025-01-16 23:41:50 +0000 UTC. Recomputing next sample time.
Jan 16 23:41:55 XX mars[21220]: INFO No processes produced valid responses
Node 2 is not showing any stats when both nodes are active
Note - Please perform systemctl restart 128TWeb during off business hours:
systemctl restart 128TWeb