Menu Failure due to "NotificationItemsGrouped" showing appearing in the web console for SolarWinds Platform 2025.1 and below

After logging into the web console, you may notice a "Menu failure: We are having difficulty communicating with the Orion server. Some of the backend services are not working properly" message, yet the menu works as expected.

 

ERROR SolarWinds.InformationService.DataProviders.SqlQueryRelation - w3wp| /LM/W3SVC/2/ROOT/api2-3-133638967742426585| SolarWinds.Orion.Web.Platform.Data.SwisRepository.Query()|  (null)      An SqlException occurred.
Message: Execution Timeout Expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Query:
SET DATEFIRST 7;
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SELECT [T1].[NotificationID] AS C1, [T1].[TypeID] AS C2, [T1].[Timestamp] AS C3, [T1].[ImageLink] AS C4, [T1].[DisplayAs] AS C5, [T1].[CustomDismissButtonText] AS C6, [T1].[HideDismissButton] AS C7, [T1].[AcknowledgeByItemId] AS C8, [T1].[DetailsPageLink] AS C9, [T1].[DetailsPageLinkCaption] AS C10, [T1].[Message] AS C11
FROM dbo.NotificationItemsGrouped AS T1
ORDER BY [C3] DESC

This issue occurs because the existing SQL query (related to NotificationItemsGrouped) attempts to group similar/same notifications from the NotificationItems table, resulting in a timeout message.

In our testing scenarios, the problem could arise when the NotificationItems table has over 50K entries due to multiple or duplicate plugin errors over time. The recommendation is to truncate the NotificationItems table and then identify the cause of multiple messages in it.

A fix for this issue was implemented in 2025.1.1 to help avoid timeout scenarios. Consider upgrading your environment. If you cannot upgrade, use the workaround below.  

As a workaround, please truncate the NotificationItems table using the query below. Data bloat and query performance are at the heart of this issue, and often once it has gotten to this point, targeted deletes are unfeasible. As such, truncating the NotificationItems table is the best way to stabilize the deployment. 

Be sure to execute the below script all at once and in its entirety. Due to several constraints on the NotificationsItem table, they need to be dropped before truncation and readded afterward to maintain database consistency. 

-- Scripts are not supported under any SolarWinds support program or service.
-- Scripts are provided AS IS without warranty of any kind. SolarWinds further
-- disclaims all warranties including, without limitation, any implied warranties
-- of merchantability or of fitness for a particular purpose. The risk arising
-- out of the use or performance of the scripts and documentation stays with you.
-- In no event shall SolarWinds or anyone else involved in the creation,
-- production, or delivery of the scripts be liable for any damages whatsoever
-- (including, without limitation, damages for loss of business profits, business
-- interruption, loss of business information, or other pecuniary loss) arising
-- out of the use of or inability to use the scripts or documentation.

--   ***** Drop Constraints *****
ALTER TABLE [dbo].[NotificationItems] DROP CONSTRAINT [FK_NotificationItems_Type]
GO
ALTER TABLE [dbo].[NotificationBlogs] DROP CONSTRAINT [FK_NotificationBlogs_BlogID]
GO
 
ALTER TABLE [dbo].[NotificationMaintenanceRenewals] DROP CONSTRAINT [FK_NotificationMaintenanceRenewals_RenewalID]
GO
 
--   ***** Truncate Table *****
 
TRUNCATE TABLE NotificationItems
 
--   ***** Re-Add Constraints *****
 
ALTER TABLE [dbo].[NotificationItems]  WITH CHECK ADD CONSTRAINT [FK_NotificationItems_Type] FOREIGN KEY([NotificationTypeID])
REFERENCES [dbo].[NotificationItemTypes] ([TypeID])
GO
 
ALTER TABLE [dbo].[NotificationItems] CHECK CONSTRAINT [FK_NotificationItems_Type]
GO
 
ALTER TABLE [dbo].[NotificationBlogs]  WITH CHECK ADD CONSTRAINT [FK_NotificationBlogs_BlogID] FOREIGN KEY([BlogID])
REFERENCES [dbo].[NotificationItems] ([NotificationID])
GO
 
ALTER TABLE [dbo].[NotificationBlogs] CHECK CONSTRAINT [FK_NotificationBlogs_BlogID]
GO
 
ALTER TABLE [dbo].[NotificationMaintenanceRenewals]  WITH CHECK ADD CONSTRAINT [FK_NotificationMaintenanceRenewals_RenewalID] FOREIGN KEY([RenewalID])
REFERENCES [dbo].[NotificationItems] ([NotificationID])
GO
 
ALTER TABLE [dbo].[NotificationMaintenanceRenewals] CHECK CONSTRAINT [FK_NotificationMaintenanceRenewals_RenewalID]
GO

Note: Make sure to perform a database backup before any manual modifications.