Applications Systems

Server & Application Monitor (SAM) and the Apache Log4j Vulnerabilities

In December 2021, multiple CVEs were released for third-party vulnerabilities detected in Apache Log4j software that is utilized widely across the software industry. This third-party component is used in very limited instances within a small subsection of SolarWinds products. This article describes how the following security bulletins impact SolarWinds SAM: CVE-2021-44228, CVE-2021-45046, CVE-2021-4104, CVE-2021-45105, and CVE-2021-44832.

First published date

12/13/2021 12:27 AM

Last published date

2/22/2022 8:50 AM

Overview

In December 2021, the following security bulletins were released for third-party vulnerabilities detected in Apache Log4j:

CVE-2021-44228
CVE-2021-45046
CVE-2021-4104 (Does not affect any other SolarWinds or N-able products)
CVE-2021-45105
CVE-2021-44832 (Affects only the DPA module)

Apache Log4j is a popular Java logging library from Apache Software that is incorporated into a wide range of enterprise software. Vulnerabilities indicated in these CVEs affect numerous software companies. This third-party component is used in very limited instances within a small subsection of SolarWinds products.

CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105 impact SolarWinds SAM and SolarWinds Database Performance Analyzer (DPA), while CVE-2021-44832 only affects DPA. These CVEs do not affect any other SolarWinds or N-able (formerly SolarWinds MSP) products. CVE-2021-4104 does not affect any other SolarWinds or N-able products.

This article describes how SAM is affected if using SAM 2020.2.6. SAM 2020.2.6 Hotfix 4 was released on February 22, 2022, and includes Log4j 2.17.1. See SAM 2020.2.6 Hotfix 4 Release Notes for details.
After applying Hotfix 4, follow steps in this article to apply the latest approved version of Log4j.

Since December 28, 2021, the latest version tested for compatibility with SAM is Log4j 2.17.1. As Apache continues to update Log4j, SolarWinds will test compatibility of future versions and update this article accordingly.

Additional resources include:

Product section

Server Application Monitor

Cause

Apache Log4j Critical Vulnerability (CVE-2021-44228)

Resolution

Affected versions

Important! Perform these steps only if running one of the following versions of SAM:

SAM 2020.2.6
SAM 2020.2.6 with Hotfix 1 applied
SAM 2020.2.6 with Hotfix 2 applied
SAM 2020.2.6 with Hotfix 3 applied

Earlier versions of SAM are based on an older version of Log4j not affected by CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105.

To check your Log4j version, navigate to <Installation directory>\SolarWinds\Orion\APM\jmxbridge and examine log4j jar files. The default path is C:\Program Files (x86)\SolarWinds\Orion\APM\jmxbridge.

Apply a hotfix

SAM 2020.2.6 Hotfix 4 is now available. To apply a Hotfix, download the Hotfix for your SAM version from the Customer Portal, and then follow the installation instructions in the associated release notes.

SAM 2020.2.6 Hotfix 4 installs version 2.17.1 of the affected files. To install the latest version of Apache Log4j, you can manually replace the files as described in the following section. You can also choose the manual option if you are running SAM 2020.2.6, or if you are unable to download and apply a hotfix.

Manually replace the affected files

Complete these steps to manually upgrade Log4j on a Windows system.

Note: SolarWinds DPA is also affected by this issue. However, the files that need to be replaced are not the same for SAM and DPA. If you have both products, please replace only the files listed below for SAM. To update DPA, see Database Performance Analyzer (DPA) and the Apache Log4j Vulnerability (CVE-2021-44228 and CVE-2021-45046).

Important: If using SolarWinds High Availability, disable HA pools before proceeding.

To manually upgrade Log4j, perform these steps on all SAM polling engines, including any HA servers:

To get the latest version of Log4j:
- Navigate to Download Apache Log4j 2.
- Download the latest zip file. As of December 28, 2021, the latest version approved for use with SAM is Log4j 2.17.1.
- Extract files to a temporary folder.
Use the Orion Service Manager or Windows Services Manager to stop the SolarWinds JMX Bridge service.
Navigate to the following folder: <Installation directory>\SolarWinds\Orion\APM\jmxbridge\.
Create backups of existing files by copying them to the Windows Desktop:
1. Copy the Java Service Launcher file, jsl64.ini, from the \jsl folder to the Desktop.
2. Copy the following Log4j Java Archive (.jar) files from the \lib folder to the Desktop, where XX represents the version obtained from Download Apache Log4j 2:

log4j-1.2-api-2.XX.0.jar
log4j-api-2.XX.0.jar
log4j-core-2.XX.0.jar

Delete the three original .jar files in the \lib folder.
If a file is blocked, right-click to select Properties > Unblock.
(Recommended) Compare filenames in the \lib folder with filenames included in jsl64.ini. Depending on the environment, upgrading may require some finetuning in the .ini file.
Copy the new versions of the three .jar files from the extracted ZIP file to the \lib folder.
Navigate to <Installation directory>\SolarWinds\Orion\APM\jmxbridge\jsl.
Launch a text editor, such as Notepad, as an Administrator to edit the jsl64.ini file.

Keep the first 3 lines, where APM_HOME =YOURPATH is specific to your SAM installation.

[defines]
APM_HOME =YOURPATH
PATH= %PATH%

Replace the remaining sections of jsl64.ini with the following code that references Log4j 2.17.1.

Important: If using a different version of Log4j, update 2.17.1 in the following code with that version. For example, edit version numbers in %APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1 and %APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1 in the WildFly sections.

[service]
appname = SolarWinds JMX Bridge
servicename = SWJMXBridgeSvc
displayname = SolarWinds JMX Bridge
servicedescription = SolarWinds JMX Bridge Service
stringbuffer = 16000
starttype=auto
loadordergroup=someorder
useconsolehandler=false
stopclass=com/solarwinds/jmxbridge/Service
stopmethod=stop
stopsignature=()V


[java]
jrepath=%APM_HOME%\..\openjdk\16
params = 5
;JDK 16 breaking change illegal-access=deny by default
param00 = --illegal-access=permit
param01 = -cp
param03 = -Denv.allusersprofile=%ALLUSERSPROFILE%
param04 = com.solarwinds.jmxbridge.Service


;WildFly XX.YY template
;1)download and copy jboss-cli-client in version XX.YY folder into directory '\Orion\APM\jmxbridge\lib\WildFly-{XX.YY}' -> example 'WildFly-18.0.1'
;2)uncomment below param01 and comment previously used
;3)replace in below path {XX.YY} with version of the wildfly client -> example 'WildFly-XX.YY' to 'WildFly-18.0.1'
;param02 = %APM_HOME%\jmxbridge\SolarWinds.JMX.Bridge.jar;%APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1;%APM_HOME%\jmxbridge\lib\wlclient.jar;%APM_HOME%\jmxbridge\lib\wljmxclient.jar;%APM_HOME%\jmxbridge\lib\javax.xml.soap-api.jar;%APM_HOME%\jmxbridge\lib\jaxb-api.jar;%APM_HOME%\jmxbridge\lib\jaxws-api.jar;%APM_HOME%\jmxbridge\lib\jsr181-api.jar.jar;%APM_HOME%\jmxbridge\lib\gmbal-api-only.jar;%APM_HOME%\jmxbridge\lib\ha-api.jar;%APM_HOME%\jmxbridge\lib\javax.activation-api.jar;%APM_HOME%\jmxbridge\lib\axb-core.jar;%APM_HOME%\jmxbridge\lib\jaxb-impl.jar;%APM_HOME%\jmxbridge\lib\jaxws-rt.jar;%APM_HOME%\jmxbridge\lib\management-api.jar;%APM_HOME%\jmxbridge\lib\mimepull.jar;%APM_HOME%\jmxbridge\lib\policy.jar;%APM_HOME%\jmxbridge\lib\saaj-impl.jar;%APM_HOME%\jmxbridge\lib\stax-ex.jar;%APM_HOME%\jmxbridge\lib\streambuffer.jar;%APM_HOME%\jmxbridge\lib\javax.annotation-api.jar;%APM_HOME%\jmxbridge\lib\WildFly-XX.YY\jboss-cli-client.jar


;WildFly 8.2
;param02 = %APM_HOME%\jmxbridge\SolarWinds.JMX.Bridge.jar;%APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1.jar;%APM_HOME%\jmxbridge\lib\wlclient.jar;%APM_HOME%\jmxbridge\lib\wljmxclient.jar;%APM_HOME%\jmxbridge\lib\javax.xml.soap-api.jar;%APM_HOME%\jmxbridge\lib\jaxb-api.jar;%APM_HOME%\jmxbridge\lib\jaxws-api.jar;%APM_HOME%\jmxbridge\lib\jsr181-api.jar.jar;%APM_HOME%\jmxbridge\lib\gmbal-api-only.jar;%APM_HOME%\jmxbridge\lib\ha-api.jar;%APM_HOME%\jmxbridge\lib\javax.activation-api.jar;%APM_HOME%\jmxbridge\lib\axb-core.jar;%APM_HOME%\jmxbridge\lib\jaxb-impl.jar;%APM_HOME%\jmxbridge\lib\jaxws-rt.jar;%APM_HOME%\jmxbridge\lib\management-api.jar;%APM_HOME%\jmxbridge\lib\mimepull.jar;%APM_HOME%\jmxbridge\lib\policy.jar;%APM_HOME%\jmxbridge\lib\saaj-impl.jar;%APM_HOME%\jmxbridge\lib\stax-ex.jar;%APM_HOME%\jmxbridge\lib\streambuffer.jar;%APM_HOME%\jmxbridge\lib\javax.annotation-api.jar;%APM_HOME%\jmxbridge\lib\WildFly-8.2\jboss-cli-client.jar


;Jboss 7.4
param02 = %APM_HOME%\jmxbridge\SolarWinds.JMX.Bridge.jar;%APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1.jar;%APM_HOME%\jmxbridge\lib\log4j-api-2.17.1.jar;%APM_HOME%\jmxbridge\lib\log4j-core-2.17.1.jar;%APM_HOME%\jmxbridge\lib\wlclient.jar;%APM_HOME%\jmxbridge\lib\wljmxclient.jar;%APM_HOME%\jmxbridge\lib\javax.xml.soap-api.jar;%APM_HOME%\jmxbridge\lib\jaxb-api.jar;%APM_HOME%\jmxbridge\lib\jaxws-api.jar;%APM_HOME%\jmxbridge\lib\jsr181-api.jar.jar;%APM_HOME%\jmxbridge\lib\gmbal-api-only.jar;%APM_HOME%\jmxbridge\lib\ha-api.jar;%APM_HOME%\jmxbridge\lib\javax.activation-api.jar;%APM_HOME%\jmxbridge\lib\axb-core.jar;%APM_HOME%\jmxbridge\lib\jaxb-impl.jar;%APM_HOME%\jmxbridge\lib\jaxws-rt.jar;%APM_HOME%\jmxbridge\lib\management-api.jar;%APM_HOME%\jmxbridge\lib\mimepull.jar;%APM_HOME%\jmxbridge\lib\policy.jar;%APM_HOME%\jmxbridge\lib\saaj-impl.jar;%APM_HOME%\jmxbridge\lib\stax-ex.jar;%APM_HOME%\jmxbridge\lib\streambuffer.jar;%APM_HOME%\jmxbridge\lib\javax.annotation-api.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.4\jboss-cli-client.jar


;Jboss 7.1.1
;param02 = %APM_HOME%\jmxbridge\SolarWinds.JMX.Bridge.jar;%APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1.jar;%APM_HOME%\jmxbridge\lib\wlclient.jar;%APM_HOME%\jmxbridge\lib\wljmxclient.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\remoting-jmx-1.0.2.Final.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\jboss-remoting-3.2.3.GA.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\jboss-logging-3.1.0.GA.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\xnio-api-3.0.3.GA.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\xnio-nio-3.0.3.GA.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\jboss-sasl-1.0.0.Final.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\jboss-marshalling-1.3.11.GA.jar;%APM_HOME%\jmxbridge\lib\JBoss-7.1\jboss-marshalling-river-1.3.11.GA.jar

Use the Orion Service Manager or Windows Services Manager to start the SolarWinds JMX Bridge Service.

If you encounter issues when restarting the SolarWinds JMX Bridge Service, please try the following steps:

Review jsl64.ini to confirm that Log4j references, such as %APM_HOME%\jmxbridge\lib\log4j-1.2-api-2.17.1, reflect the correct version of Log4j.
Check if the JMX Bridge service is enabled in the Windows Service Manager.
Close the Orion Service Manager and then launch it again.

Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment. You elect to use third-party content at your own risk, and you will be solely responsible for the incorporation of the same if any.