NetPath graph issue: All timeout nodes between probe and endpoint

10/9/2018 4:42 PM

10/6/2022 1:11 PM

The NetPath graph seems to be missing many nodes.

NetPath only captures timeout nodes between the NetPath probe and the endpoint.

Network Performance Monitor

Possible Firewall issue

Step 1: Check if the next node to the NetPath probe computer is a firewall

Is it a firewall?

• Yes: Go to Step 2.
• No: Go to Step 3.

Step 2: Check if the firewall is creating logs for NetPath probing packets

1. Find the endpoint IP address and TCP port from the service that has the issue.
2. Find the IP address of the NetPath agent that probes the service.
3. Locate the firewall that the NetPath probing traffic traverses. The firewall may or may not display in the NetPath graph.
4. Make sure the log is enabled for Drop rules, including Default and Implicit Drop rules.
5. Based on the firewall manual:
   1. For the outbound direction from agent to endpoint, find the firewall drop logs for TCP packets from the IP address of the NetPath agent and any source port, to the endpoint IP address and port of the service with the issue.
   2. For the inbound direction from endpoint to agent, find the firewall drop logs for ICMP Type 11 packets from any IP address to the IP address of the NetPath agent.

Check firewall logs

Open Check Point SmartView Tracker > All Records > Add filters.

For outbound NetPath probing packets, add the filter for:

• Service: TCP service specified in NetPath Services
• Source: NetPath probe
• Destination: Endpoint

For inbound NetPath probing packets, add the filter for:

• ICMP Type 11
• Destination: NetPath probe

Checkpoint

Palo Alto

SonicWall


Cisco ASA

The log option must be enabled for rules that can allow or deny NetPath probing traffic.

Did you find the logs?

• Yes: This is a potential firewall issue. Locate the drop rule from the log, and consult with a firewall engineer to fix it.
• No: Go to Step 3.

Step 3: Check if the IP ID masking rule is enabled for a Check Point firewall

1. Open the Check Point SmartDashboard.
2. Navigate to IPS > Protections.
3. Search for IP ID Masking.
4. Check if the rule is Active.

Is the IP ID masking rule enabled?

• Yes: Add the IP address of the NetPath probe to the exception: Edit IP ID Masking > Network Exceptions > New > Add NetPath Probe to Source Network Condition.
• No: Go to Step 4.
• I don't have a Check Point firewall: Go to Step 4.

Step 4: Check if the NetPath probe is running on AWS

Is the NetPath probe running on Amazon Web Services (AWS), Google Compute Engine (GCE), or other Linux-based virtualization?

• Yes: Go to Step 5.
• No: Go to Step 6.

Step 5: Check the security policy

Does the security allow ICMP packets?

• Yes: Follow this workaround for each AWS probe:
  1. Open C:\ProgramData\Solarwinds\Orion\NetPath\NetPathAgent.cfg with a text editor as an administrator.
  2. Save a copy of original NetPathAgent.cfg file as a backup.
  3. Stop the SolarWinds Job Engine 2 service.
  4. Change the following settings in the NetPathAgent.cfg file:

     "EnableAdvancedBpf": false,
     "TracerouteTimeout": 500,

  5. Save the file.
  6. Start the SolarWinds Job Engine 2 service.
• No: See the Cloud Environment section of NetPath requirements .

Step 6: Capture NetPath probing packets on the NetPath probe computer

For NPM 12.0

1. Download and install Wireshark. (© 2020 <Wireshark>, available at <https://www.wireshark.org/>, obtained on <October 9, 2018>)
2. Find the IP address port from the service that has issue.
3. Find the IP address of the NetPath probe that probes the service.
4. Select the interface for your NetPath outgoing traffic.
5. Apply the icmp Capture Filter.

   

6. Apply the following display filter:

   icmp.type == 11 and ip.dst == endpoint_ip

   Replace endpoint_ip with your IP address.

   

For NPM 12.0.1 and later

1. Open the NetPath Service page with ?debug appended to the end of the URL.
2. Edit the path you want to troubleshoot.
3. Select Enable logging, and click Save.
4. Wait for two probing intervals for the selected path.
5. Check the pcap file in the polling engine (not the Agent computer) where the selected path is collected from.

   

Were the packets captured?

• Yes: Follow this workaround for each AWS probe:
  1. Open C:\ProgramData\Solarwinds\Orion\NetPath\NetPathAgent.cfg with a text editor as an administrator.
  2. Save a copy of original NetPathAgent.cfg file as a backup.
  3. Stop the SolarWinds Job Engine 2 service.
  4. Change the following settings in the NetPathAgent.cfg file:

     "EnableAdvancedBpf": false,
     "TracerouteTimeout": 500,

  5. Save the file.
  6. Start the SolarWinds Job Engine 2 service.
• No: Submit a ticket to technical support.

 

Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment.  You elect to use third party content at your own risk, and you will be solely responsible for the incorporation of the same, if any.