This article provides a checklist of items to check for when Kiwi Syslog Server is not keeping up with syslog message processing and is behind by several minutes or even hours.
The typical causes are either an Action that is causing a bottleneck or Kiwi Syslog Server cannot process the syslog messages through the Rule engine at the speed that they are arriving.
- Check for the following bottlenecks:
- Log to File Action configured to log to a network path: Writing current log information to a Network Path will seriously slow down the amount of syslog messages Kiwi Syslog Server can handle per second (~100-300 messages/sec depending on networks). It is best to configure a Log to File Action to log to a local path and then configure an Archive Schedule to move older logs to a network path at regular intervals.
- Forwarder to another host with Spoofing enabled: As Spoofing requires that the syslog packet be deconstructed and rebuilt, it can be resource intensive. As such, it can only handle a certain amount of syslog messages per second (~100 messages/sec).
- Check the volume of incoming syslog messages against the amount of Rules, Filters and Actions you have: Generally Kiwi Syslog Server can comfortably handle 2 million messages per hour (~600-800 messages/sec) with the default Rules. The amount of syslog messages that can be processed per second will inevitably decrease when additional Rules, Filters and Actions are added. To verify this:
- Open the Kiwi Syslog Server Manager.
- Click Manage > Debug options > Get Diagnostic information.
- Scroll down to the Message Buffer Information section and note the following values:
- Message Count
- Message Count Max
Note: Generally if Message Count Max is greater than Message Count and it continues to grow, it is an indication that the syslog messages cannot be processed as fast as they are arriving. To address this, split up the load to a second Kiwi Syslog Server installation.