Security Compliance

Understanding SEM Agent time stamps

When looking at Node states in either the Manage > Nodes screen or the Node Health widget, it is possible that you see a Last Connected date in the past for some of your Agent Nodes.

First published date

11/14/2018 8:47 PM

Last published date

3/28/2019 8:07 PM

Overview

This article applies to Security Event Manager (formerly Log & Event Manager).

When looking at Node states in either the Manage > Nodes screen or the Node Health widget, it is possible that you see a Last Connected date in the past for some of your Agent Nodes.

Product section

Security Event Manager

Resolution

The important thing is that there are a number of “times” that describe the SEM Agent and the last time it functioned. In the SEM Flash console, double-click a node, and you see this information.

Last Event: This is the last time an event came from this node. In this example, that was four seconds ago.
Install Date: This is when the Agent was first installed/came on-line to communicate with the SEM appliance.
Last Connected: This is the last time the SEM Agent went through a complete handshake with the Log and Event Manager appliance. In the example, this is more than 4 seconds ago.

In the SEM HTML5 console, you can find this information on agents in the Nodes tab, or by clicking a node in the Node Health widget in the SEM Dashboard.
If you open the SPOP log on the Agent node (the default path in Windows is C:\Windows\SysWOW64\ContegoSPOP\spoplog.txt) and look through the lines, you should see lines like these:

(Sun Sep 27 12:01:08 MDT 2015) II:INFO [AgentVersionHeaderProvider] {main:1} SolarWinds Log and Event Agent (Release: 6.2.0[release] build: 6.2.0.992.565529)
(Sun Sep 27 12:01:08 MDT 2015) II:INFO [SpopModule] {main:1} Java version: 1.7.0_80 Java home: C:\Windows\SysWOW64\ContegoSPOP\jre1.7.0_80
(Sun Sep 27 12:01:09 MDT 2015) WW:WARNING [LEMSlf4jConfigurationManager] {main:1} Can't load logging prefs classpath*:/debug-default.conf
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [SpopModule] {main:1} Event memory was set to: 1000
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [SpopModule] {main:1} Events per queue was set to: 100000
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [SpopModule] {SPOP:8} Starting TriGeo Agent (Release 6.2.0) build [release]
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [SpopModule] {SPOP:8} build server version string: 6.2.0.992.565529
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [InDepthConfigProps] {SPOP:8} nDepth enabled via default because InDepthEnable not present
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [InDepthConfigProps] {SPOP:8} indepth.conf not found at C:\Windows\SysWOW64\ContegoSPOP\indepth.conf

(Sun Sep 27 12:01:09 MDT 2015) WW:WARNING [RawDataClient] {SPOP:8} Status Inactive
(Sun Sep 27 12:01:09 MDT 2015) II:INFO [UpdateClient] {SPOP:8} OS signature: Windows Server 2008 R2;6.1;x86
================ TRUNCATED FOR BREVITY ================
(Sun Sep 27 12:01:13 MDT 2015) II:INFO [ComModuleSpop] {ComModuleSpop:21} Opening installed connection to parent.
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioComNetworkParent] {ComModuleSpop:21} Creating Nio Center for Client.
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioCenterOnClient] {ComModuleSpop:21} Initializing Nio Center.
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioComNetworkParent] {ComModuleSpop:21} Starting Nio Center for Client thread.
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioSelectorOnClient] {NioComNetworkParent:22} Force flush after every nio write: false
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioSelectorOnClient] {NioComNetworkParent:22} Send socket buffer size: 65536
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioSelectorOnClient] {NioComNetworkParent:22} Receive socket buffer size: 8192
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioCenterOnClient] {NioComNetworkParent:22} Nio Center successfully initialized.
(Sun Sep 27 12:01:15 MDT 2015) II:INFO [NioCenterOnClient] {NioComNetworkParent:22} Nio Center Connecting.
(Sun Sep 27 12:01:16 MDT 2015) II:INFO [NioSelectorOnClient] {NioComNetworkParent:22} Successful binding to host: 0.0.0.0/0.0.0.0:37893 , on port: 37893

(Sun Sep 27 12:01:19 MDT 2015) II:INFO [NioSelectorOnClient] {NioComNetworkParent:22} Beginning SSLHandshake.

(Sun Sep 27 12:01:21 MDT 2015) II:INFO [NioComNetworkParent] {ComModuleSpop:21} Nio initialization successful.

(Sun Sep 27 12:01:21 MDT 2015) II:INFO [NioSelectorOnClient] {ComModuleSpop:21} Retrieved hostname machine-hostname-netbios from the network socket on agent node local host

(Sun Sep 27 12:01:21 MDT 2015) II:INFO [NioSelectorOnClient] {ComModuleSpop:21} Retrieved IP address 192.168.10.127 from the network socket on agent node local host

(Sun Sep 27 12:01:23 MDT 2015) WW:WARNING [NioComNetworkParent] {ComModuleSpop:21} Manager connection complete.
(Sun Sep 27 12:01:23 MDT 2015) II:INFO [NioSelectorOnClient] {ComModuleSpop:21} Retrieved hostname machine-hostname-netbios from the network socket on agent node local host
(Sun Sep 27 12:01:23 MDT 2015) II:INFO [NioSelectorOnClient] {ComModuleSpop:21} Retrieved IP address 192.168.10.127 from the network socket on agent node local host
(Sun Sep 27 12:01:23 MDT 2015) WW:WARNING [NioComNetworkParent] {ComModuleSpop:21} Opened connection to manager at 192.168.10.15
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [SpopModule] {SPOP:8} Initializing Spring context
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [SpopModule] {Initialize Secret Center:11} Initializing the secret center
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [SpopModule] {Initialize RCC Server:12} Initializing the Connector Message Center
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [SpopModule] {Initialize RCC Server:12} Initializing the Connector API Command Center
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [ApplicationContexLoader] {SPOP:8} Active profiles: [agent, strongEncryption]
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [ApplicationContexLoader] {SPOP:8} Resource: file [C:\Windows\SysWOW64\ContegoSPOP\spop.conf]
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [ApplicationContexLoader] {SPOP:8} Configurations: [class com.trigeo.core.AgentSpringConfiguration]
(Sun Sep 27 12:01:24 MDT 2015) II:INFO [CommInfoAgent] {ComModuleSpop:21} Dispatching parent connect signal
(Sun Sep 27 12:01:24 MDT 2015) WW:WARNING [LogManagementRegistryClient] {ComModuleSpop:21} parentConnected

If you look at the bolded text above, you will see that the time in the SEM GUI matches the time that the Nio initialization was successful and the SSL Handshake was completed. That was the last time the Agent “connected” even if the Agent has sent events since that time. This time is within seconds of the Agent confirming the parent (the SEM appliance) is connected correctly.

’
The Last Connected time is usually only updated if:

The Agent machine reboots.
The Agent service on the Agent machine is stopped and restarted.
The SEM manager is rebooted.
The SEM manager service is restarted.

Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment. You elect to use third-party content at your own risk, and you will be solely responsible for the incorporation of the same if any.