Troubleshooting: Kiwi Syslog Server Service Fails To Start

11/22/2019 8:34 PM

11/22/2019 8:34 PM

After running the Kiwi Syslog Server executable while performing a fresh install, migration or upgrade,  the service does not start.
In most cases, the error says "Windows could not start the Kiwi Syslog Server service on local computer. Error 1067: The process terminated unexpectedly".

 

Kiwi Syslog Server

Most likely a result of the local system security settings

Please check the local EMET (Enhanced Mitigation Experience Toolkit) configurations and add some exceptions.

a)  For Windows Server settings Data Execution Prevention (DEP) and Structured Exception Handling Overwrite Protection (SEHOP).

• Data Execution Prevention (DEP) 

  If the client has Group Policy Object (GPO) implementation:
  Please try to enable System-Wide DEP as Application Opt-Out and then provide an exception to DEP(data execution prevention) mitigation for both CatTools\Syslogd_Service.exe and CatTools\Syslogd_Manager.exe:
  
  To disable DEP:
    1. Open the Start screen, right click the Computer tile and click Properties in the bar below.
    2. In the System Settings, click Advanced system settings in the left pane, go to the Advanced tab and click Settings in the Performance section. Next in the Performance Options window, go to the Data Execution Prevention tab and select Turn on DEP for essential Windows programs and services only.
    3. Finally click OK in both windows to save the configuration change and restart your computer to apply it.
   

  Some "gotchas":

    1) If System DEP is already configured, simply removing it from the GPO will not necessarily remove it from your system.
    2) You must first run gpupdate /force on your server, and then, from a command line within the EMET installation folder, run the command emet_conf -refresh in order to update EMET itself.

  
  For non-GPO implementation, you can do all this via the EMET GUI:
  Navigate to the EMET GUI, manually add CatTools_Service.exe and CatTools_Manager.exe, and disable DEP for these features: 
  
    1) Open EMET GUI
    2) Disable DEP
    3) Press Apps
    4) Remove all strings that contain Syslog
    5) Reboot computer

• Structured Exception Handler Overwrite Protection (SEHOP)
  
  To disable SEHOP
  1. Open the Start screen, check enter regedit and press Enter to start the Registry Editor.
  2. In the Registry Editor navigate to HKEY_LOCAL_MACHINESYSTEM\CurrentControlSet\Control\Session Manager\Kernel, double click the DisableExceptionChainValidation value to open the editor and change its value to 1.
  3. Click OK to save the value change and restart your computer to apply the configuration change.
   

b) System Registry:

The problem may occur due to the "license" registry keys effects. So, we should check the System registry and remove appropriate entities:
  1) Open application "Registry Editor" ‪C:\Windows\regedit.exe;
  2) Using "Registry Editor" find and remove all keys which contain data:
               "SolarWinds.Licensing.KiwiCatTools.COMWrapper.DLL";
               "SolarWinds.Licensing.KiwiSyslog.COMWrapper.dll"; 

c) Workaround:
Please check if it is possible to install and start CatTools\Syslog as an "Application"

Advancing to development:

Before you do so, collect the following information as it would be helpful in trying to find the root cause
1) Can the customer install Syslog on a "clean" server without security restrictions? 
2) Ask the customer provide logs from process monitor
    - Download it  https://docs.microsoft.com/en-us/sysinternals/downloads/procmon
    - Run it
    - Install Syslog    
    - Collect logs from process monitor
        run cmd, execute "wmic service where started=true get  name, startname" and provide the output.

3) Obtain a list of installed programs https://www.howtogeek.com/165293/how-to-get-a-list-of-software-installed-on-your-pc-with-a-single-command/
4) Obtains a list of installed updates https://blogs.technet.microsoft.com/mu/2016/11/23/how-to-list-updates-that-have-been-installed-on-your-windows-server-2016-machine/

Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment.  You elect to use third party content at your own risk, and you will be solely responsible for the incorporation of the same, if any.