This article summarizes the how coalescing affects your licensing.
Understanding coalescing and licensing.
Events are counted against the license before they are coalesced. Every event that comes into the pipeline, coalesced or not, is counted against the license.
Events run in the following hierarchy through the pipeline:
sources ---> event parsing ---> custom rules engine --> mpc (syslog/jdbc/checkpoint) (dsms) (cre) (magistrate, offenses)
Events are not coalesced until they hit the “event parsing” or dsm part of the pipeline. However, the EPS license in qradar is applied between the “sources” queue and the “parsing” queue. Total coalescing values can be seen in the qradar.log every minute:
qradar.log
[root@csd6 ~]# grep StatFilter /var/log/qradar.log | tail -20 Aug 27 11:02:31 127.0.0.1 [ecs] [[type=com.eventgnosis.system.ThreadedEventProcessor][parent=csd6.q1labs.inc:ecs0/EC/Processor2]] com.q1labs.semsources.filters.StatFilter: [INFO] [NOT:0000006000][172.16.77.106/- -] [-/- -]Events per second: 1s:1,1 (peak 9,228) (compression: 0%) 5s:0,1 (peak 3,48) (compression: 40%) 10s:1,2 (peak 2,25) (compression: 38%) 30s:1,1 (peak 2,22) (compression: 30%) 60s:1,2 (peak 2,21) (compression: 29%) Aug 27 11:03:32 127.0.0.1 [ecs] [[type=com.eventgnosis.system.ThreadedEventProcessor][parent=csd6.q1labs.inc:ecs0/EC/Processor2]] com.q1labs.semsources.filters.StatFilter: [INFO] [NOT:0000006000][172.16.77.106/- -] [-/- -]Events per second: 1s:2,2 (peak 9,228) (compression: 0%) 5s:1,1 (peak 3,48) (compression: 14%) 10s:1,2 (peak 2,25) (compression: 38%) 30s:1,1 (peak 2,22) (compression: 29%) 60s:1,1 (peak 2,21) (compression: 28%) {*}Events per second: 1s: 1,1 (peak 9,228 )* :
The StatFilter values are calculated as follows:
60s = number of events over the past minute /60 30s = number of events over the past 30 seconds /30 10s = number of events over the past 10 seconds /10 5s = number of events over the past 5 seconds / 5 1s = number of events in the last second
2019-09-19: Minor, non-technical edit.