This article explains the disk adjustment and log retention policy in Security Director Insight

Users find difficulties in understanding the SDI Disk adjustment and log retention Policy

Please find below both storage-based and time-based policies that SDI utilizes in order to make sure that SDI has enough storage to remain functional.
 

A) Storage-based Policy:
SDI runs a frequent check every hour to ensure that logs stored in Elasticsearch has not gone past 80% of the allocated storage (disk space) capacity. In the case that logs stored in Elasticsearch have gone past the 80% storage threshold, then SDI will delete the oldest data first. Currently storage-based policy is not configurable either on SD or SDI CLI.

B) Time-based Policy: 
SDI runs a daily check to delete old indices past a certain threshold.

Please find the thresholds below:

• SDI running as an Insights Node:

  Events: 183 days
  Incidents: 183 days

• SDI running as a Log Collector

  Junoslogs: 365 days

Please Note: These thresholds are currently not configurable and are defined only in SDI. In the case where SDI is installed with less than 1.2 TB, then the storage-based policy would start taking effect sooner depending on the syslog/stream log traffic.
 

Note : If you face any issues, please contact JUNIPER JTAC Support for assistance.