Missing Audit Events from Message Center in Orion Platform

4/18/2020 10:07 PM

1/12/2023 2:42 PM

Utilize this to help solve issues related to missing Audit events in the Orion web console. This is a rare issue that will take preliminary investigation before determining the root cause and therefore applying the correct solution. 

Orion Platform

Verify that the audit events are logged in the database. You can query the database table directly that stores this information. 

-- Scripts are not supported under any SolarWinds support program or service.
-- Scripts are provided AS IS without warranty of any kind. SolarWinds further
-- disclaims all warranties including, without limitation, any implied warranties
-- of merchantability or of fitness for a particular purpose. The risk arising
-- out of the use or performance of the scripts and documentation stays with you.
-- In no event shall SolarWinds or anyone else involved in the creation,
-- production, or delivery of the scripts be liable for any damages whatsoever
-- (including, without limitation, damages for loss of business profits, business
-- interruption, loss of business information, or other pecuniary loss) arising
-- out of the use of or inability to use the scripts or documentation.
SELECT TOP 10000 * FROM [AuditingEvents] ORDER BY TimeLoggedUtc DESC

Verify the following:

• You're using the audit event search and not System Event search; Alerts & Activity > Message Center > Audit Events
• If the database contains recent auditing events, you have narrowed down the issue to lie within Information Service. This is not able to populate the web interface with the recent database changes. If there are no recent audit events logged at the source, the issue lies within Orion Business layer. If the business layer is not fully starting, check if CPU is high and verify the recommended specs are applied to the affected server. It is recommended to check the appropriate log files. 

 

Orion service restart is highly recommended and fixes most issues with event logging [Start, Stop, or Restart Orion Services with the Orion Service Manager].


If the issue persists through service restart; Orion Core is responsible for logging entries. Perform a core repair then confirm the audit log is working by performing a few events that are normally logged such as enabling HA, logging out and in, unmanaging and remanaging a node, etc.

You can follow the Check Point List to start the troubleshoot on missing events on Message Center Tab.

Check Points:

a) Validate the issue present or resolved 
Open Orion Web Console > Alerts & Activity > Message Center > Filter Checkmark as enable for  SHOW AUDIT EVENTS and you could Uncheck "Activity alerts" and  "event messages?
And extend Time period to Last 12 months, to track the last time receiving audit events
And as second test, change period for TODAY, to duplicate issue or validate functionality once issue was resolved.

b) Enable Audit Trails
    Orion Web Console > Settings > All Settings > Web Console Settings > 
     AUDITING SETTINGS
     Enable Audit Trails   Checkmark for Enable 


b) Validate no issues with Orion Services (Orion Core Services, RabbitMQ)
   b1)  Core Services -Orion Core is responsible for the auditing log. validate if errors on logs for this service, and only  if you found errors, complete an Orion Core Services Repair. 

   Settings> My Orion Deployment > Deployment Heath  > Validate issues with Services or Communication Services

   b2) Validate Rabbit MQ, no errors.
Grab the Logs for RabbitMQ , or review for RabbitMQ under Recommendations.log and complete analysis and troubleshoot. 
Once you found the errors: review Success Center articles, and search for specific issues with RabbitMQ and/on addition validate the general troubleshoot for rabbitMQ, on the following article

Article: Troubleshooting RabbitMQ https://support.solarwinds.com/SuccessCenter/s/article/Rabbit-MQ?language=en_US

Part of the validation will be the Port test 5671 and if system has HA (5671, 4369 and 25672)