How to troubleshoot SolarWinds NetPath Probing

1. Overview
2. Environment
3. What is probing?
4. Log file locations
5. Enable Debug level in the Log Adjuster
6. Capture packets using Wireshark
   1. Set a filter in Wireshark
   2. Export filtered packets
7. Probing configuration
8. Common troubleshooting scenarios
   1. Capture packets
   2. Create paths
      1. How to find the probing type
      2. What to check in the log
   3. Generate performance issues
   4. Show edge packet loss
9. Exceptions to ignore
10. Troubleshooting steps
    1. The page is blank
    2. "No data found" message
    3. The graph shows only two nodes connected by a dotted line
    4. The browser cannot get data from the NetPath server
    5. The packet loss at the endpoint is too high
    6. Graph completion ratio is low
    7. Same IP in multiple hops in debug mode
    8. No contact information for a node
    9. The latency changes a lot
    10. TTL mismatch
11. FAQs

What is probing?

Probing is the foundation of the Netpath. It provides the data for building path graph, generating performance issues, and calculating status. If there is a problem with probing, users will not see complete data in the graph. The probing occurs for each probing interval.

Log file locations

The probing-related log is at C:\ProgramData\Solarwinds\Logs\NetPath. The log file contains probing details for all monitored services. The files under C:\ProgramData\Solarwinds\Logs\NetPath\Results contain summarized information of probing.

Enable Debug level in the Log Adjuster 

When troubleshooting a probing issue, enable Debug level for NetPath probing in the log adjuster to log more details.

Capture packets using Wireshark 

Sometimes, we need packet capture to investigate the customer issue. To reduce the size of the capture file or avoid disclosing customer-sensitive information, we can set the filter to export only related packets.

Set a filter in Wireshark 

Export filtered packets 

Probing configuration 

In most cases, the default configuration for probing is sufficient. In some special cases, the configuration can be changed to work around some customer issues. The configuration file is: C:\ProgramData\Solarwinds\Orion\NetPath\NetPathAgent.cfg

Please don't change configurations not explained here.

All of the settings are global and apply to all services on the same machine.

• NumberOfRoundsPerProbeStandard

  In standard-based probing, the number of paths we get with the same source port to consolidate them into a single consistent path. Default is 3.

  If you see the message "failed to consolidate" in the NetPath probing log very often, you can decrease this value to 2 to see if you get graphs more often at route inspector.

  Changing this to 1 can result in a graph with extra links between a node in the path to the service endpoint. 

• NumberOfRoundsPerProbeConnection

  In connection-based probing, the number of paths we get with the same source port to consolidate them into a single consistent path. Default is 1.

  Increase this value if the graph has extra links between a node in the path to the service endpoint, and the probing type is connection-based.

• EnableConnectionBasedProbing

  When it is false, always use standard-based probing in a probing interval. Otherwise, use connection-based probing first unless prohibited by other conditions. Details of probing types are in the Create Paths section below.

• TracerouteTimeout

  The time to wait for the response after sending a probing packet with a particular TTL. The default is 1000 ms. Increasing this value will slow down probing. Decreasing it could cause a TTL mismatch (the hop becomes timed out, but its response comes back later, which is matched to the wrong probing packet with different TTL).

***MAKE SURE TO RESTART SERVICES AFTER CHANGING ANYTHING ABOVE***

Common troubleshooting scenarios 

NetPath can have issues in the following areas. This article focuses only on the probing related areas (in blue below).

• Capture packets
• Create paths
• Generate performance issues
• Build graph based on multiple paths from probing (not covered here)
• Calculate status and derive issues (not covered here)
• Save graph into database (not covered here)
• Client retrieves graph from database (not covered here)
• Client side aggregation for nodes within the same Autonomous system (not covered here)

Capture packets

After the probe is installed, it's possible that the probe cannot receive any response packets, and therefore cannot generate the path graph. 

The symptom is that there is no graph at the route inspector. There are several possible reasons:

1. A DLL is missing.
   • Go to Control Panel > Uninstall or change a program, and check the version of "Microsoft Visual C++ 2013 Redistributable (x86)". Verify that version 12.0.30501 is there. NetPath uses this version.
   • Check the NetPath debug level log at C:\ProgramData\Solarwinds\Logs\NetPath. See if no response packets were received (no messages with "Created hop: ").
2. There is an exception at opening the WinPcap device. The work-around is to restart the machine.

   

3. The NetPath log looks like it is working, but there is no graph in route inspector. It is possible the license has expired. In collector log at C:\ProgramData\Solarwinds\Collector\Logs\Collector.Service.log, look for the following message:

   [STP DatabaseProcessing Thread #1] DEBUG
SolarWinds.Licensing.Framework.Diagnostic.Logger - Evaluation expired
SolarWinds.Licensing.Framework.EvaluationExpiredException: Evaluation license has expired.
at SolarWinds.Licensing.Framework.LicenseManager.GetEvalLicense(ILicense[] licenses)
at SolarWinds.Licensing.Framework.LicenseManager.GetLicensesInternal(String productName, ProductVersionData productVersion, Int32 licenseVersion, DateTime productRtmDate, Boolean interactive, Boolean reload)

Create paths 

The path discovery uses two approaches: connection-based and the standard-based.

In connection-based probing, we probe paths within a TCP connection. We always have paths to present to the user.

Some services might not allow a TCP connection to last without sending data. If this happens, we will switch to standard-based probing. This decision of using standard-based probing is cached with a timeout. So in the next round of probing, standard-based probing is used without wasting time on connection-based probing. When the cache times out, connection-based probing is used first again.

In standard-based probing, if we find a consistent path, we present the path to the user. Otherwise, the result is discarded and probing continues. If too many results are discarded, there may be no data in a probing interval.

Connection based probing is more firewall-friendly. It is the preferred probing type. 

How to find the probing type 

1. Enter ?debug after the URL to open the debugger.
2. Click the Traces tab.
3. Click Load Traces to display a list of traces.
4. Click Show Hops to display more information about a trace.
5. The Mode shows whether the trace was obtained using connection-based probing or standard-based probing

What to check in the log 

Look for error and warning messages in NetPath probing log.

Notes:

• Paths After Retest Timeouts does not indicate a problem. It is to log the generated path for a service.

• The following error indicates that we find different IP in the same TTL when finding path, and will discard this probing result due to inconsistent result. If it does not happen often, it is of no concern. If it happens very often, that could be the cause of no graph in a probing interval.

  [16] WARN p;SolarWinds.NetPath.Agent.Probe.Traceroute.Util.RouteComparator - Found multiple IPs X.X.X.1,X.X.X.2 at TTL 13 ...failed to consolidate hops.

Where there is no graph for a probing interval, use the service host name or IP to search the log at debug level, and determine if there is a response related to this service in the log (search for "Original hops for" and "RouteComparator - ConsolidatedPath") to see if the hops has been collected for that service and if we can find consistent path.

A common cause is failed to consolidate the paths. If you find such message in NetPath debug log, a workaround is to reduce the number of rounds per test. For example, set NumberOfRoundsPerProbeStandard = 2 or even 1.

Generate performance issues 

When there is a performance problem, NetPath measures the packet loss and latency and shows the status of each node and link along the path. The status helps users isolate where the problem is. One example is below.

If there are multiple places that show problems, check for several consecutive probing intervals to see where the problem happens most often. And always focus on the first node or edge that shows the issue. Resolve that one, then work on the next one.

Show edge packet loss 

You can show edge packet loss in debug mode, so you can quickly find what edge is having the most serious issue. Look at several intervals to confirm.

Exceptions to ignore 

1. Connection based traceroute failed. It is common that an endpoint sends FIN or RESET to terminate a connection during our probing. It should not cause any issues with getting data.

   WARN SolarWinds.NetPath.Agent.Probe.Traceroute.ConnectionBasedRunner - NetPath_10000_W_15DF9681 Connection based traceroute failed with Endpoint: 208.109.4.218 on Port 80. Captured Reset, Acknowledgment packet. Something is not right. Will fall back to standard probing

   SolarWinds.NetPath.Agent.Probe.Traceroute.Sender.TracerouteException: Captured Reset, Acknowledgment packet. Something is not right. Will fall back to standard probing

   at SolarWinds.NetPath.Agent.Probe.Traceroute.Sender.TcpAckSender.FoundReplyPacket(Packet packet, IPacketCommunicatorWrapper communicator, Int64 eslapsedTime)

2. Cannot resolve IP address to DNS name.

   The message "DEBUG SolarWinds.NetPath.Agent.Probe.Traceroute.Util.DnsResolver - Encounter errors when resolving IP 54.239.108.118. System.Net.Sockets.SocketException: No such host is known" does not cause a  "no data found" issue. You can ignore it.

3. Cannot load all types in SolarWinds.Orion.Common.SerializationHelper. You can ignore this exception, which does not cause issues.

   "[STP SmartThreadPool Thread #2] DEBUG SolarWinds.Orion.Common.SerializationHelper - Unable to load all types from 'SolarWinds.NetPath.Utility, Version=1.0.0.2182, Culture=neutral, PublicKeyToken=null'. System.IO.FileNotFoundException: Could not load file or assembly 'System.Web.Http, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. File name: 'System.Web.Http, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'"

Troubleshooting steps 

Notes:

• If you enable debug mode in log files, they change very quickly with many services. If you can disable most services and focus on one service, reading the log will be much easier. If you cannot, make a copy of the log files so you can correlate the history of a service in multiple log files.
• The configuration change mentioned below is a global setting, and applies to all services. 

Depending on what the route inspector page displays, following the appropriate troubleshooting steps in the following sections:

• The page is blank
• "No data found" message
• The graph shows only two nodes connected by a dotted line
• The browser cannot get data from the NetPath server
• The packet loss at the endpoint is too high
• Graph completion ratio is low
• Same IP in multiple hops in debug mode
• No contact information for a node
• The endpoint is much closer to the probe due to a proxy
• The latency changes a lot
• TTL mismatch

The page is blank 

If the route inspector page shows nothing, click the  button to center the graph.

"No data found" message 

The message "No Data Found" means there is no data collected and saved in database.

If this message is displayed, follow these guidelines to troubleshoot the problem:

1. If it is the first interval, it could be that probing is not done yet. So wait for the next interval.
2. If there is still "No Data Found", it could be that probing has an issue, and never generates results to save into the database, or that there is a problem with the connection to the database.
   1. Check the NetPath log file at C:\ProgramData\Solarwinds\Logs\NetPath to see if there are exceptions.
      • If you don't see "Created hop:", then packet capture probably has an issue. See Capture packets for details.

      • Look for the message "TracerouteJob completed".

        • If the details look like the message below, it indicates that the probing finished successfully:

          [STP SmartThreadPool Thread #10] DEBUG SolarWinds.NetPath.Pollers.Traceroute.TraceroutePoller - TracerouteJob completed. TestResult: Endpoint:www.amazon.com, EndpointPort:80 AgentId: a103fcac-2943-47ab-aba2-5e7056b50a8c, PublicIp:255.255.255.255, UtcStartTime:4/20/2016 2:12:41 AM,
CompletionRatio=15.63238
UniqueEdgeCount=146
UniquePathCount=7
SentPackets=267
LostPackets=0
ConnectionBased=False

        • If no details are shown, it indicates that no result is found:

          [STP SmartThreadPool Thread #10] DEBUG SolarWinds.NetPath.Pollers.Traceroute.TraceroutePoller - TracerouteJob completed. TestResult: None

      • The warning message "No Path results for Endpoint" also indicates a probing issue. Check the message before this to get the specific reason why probing failed.

   2. If you are sure the probing finishes successfully, check the collector log at C:\ProgramData\Solarwinds\Collector\Logs\Collector.Service.log. 

      • Determine if the license is valid.

        • Valid licenses show a message like

          [STP DatabaseProcessing Thread #33] DEBUG SolarWinds.Orion.Core.Common.ProductLicenseWrapper - Valid licenses for NPM 1.0.0.2182 

        • Expired licenses show a message like:

          DEBUG SolarWinds.Licensing.Framework.Diagnostic.Logger - Evaluation expired
SolarWinds.Licensing.Framework.EvaluationExpiredException: Evaluation license has expired.
at SolarWinds.Licensing.Framework.LicenseManager.GetEvalLicense(ILicense[] licenses)
at SolarWinds.Licensing.Framework.LicenseManager.GetLicensesInternal(String productName, ProductVersionData productVersion, Int32 licenseVersion, DateTime productRtmDate, Boolean interactive, Boolean reload)

      • Determine if the collector gets the probing result from polling, and if the result is stored in the database.

        [STP DatabaseProcessing Thread #33] DEBUG SolarWinds.Collector.DBProcessing.DBProcessingWorkerWithAccess - Start commit 33 - NetPath.Test.Generic

        [STP DatabaseProcessing Thread #33] DEBUG SolarWinds.NetPath.Collector.DAL.TracerouteResultDAL - Firing BGP requests

        [23] DEBUG SolarWinds.NetPath.Collector.DAL.Helper.TracerouteGraphBuilder - 46 Networks out of 46 requests were found in local BGP resolver and added to graph

        [26] DEBUG SolarWinds.Orion.Common.DbHelper - SQL:
INSERT INTO [dbo].[NetPath_Tests] ([ExecutedAt],[ServiceMonitorId],[ProbeId],[ProbeStatus],[CompletionRatio],[UniqueEdgeCount],
[UniquePathCount],[TotalTraceCount],[TotalEdgeCount],[PortClosedTraceCount],[DestUnreachedTraceCount],
[DnsFailureTraceCount],[RouteChangeCount],[EdgeChangeCount],[Rtt],[RttMin],[PacketLoss],[StatusValue],
[Graph],[SentPackets],[LostPackets],[ConnectionBased])
VALUES (@ExecutedAt,@ServiceMonitorId,@ProbeId,@ProbeStatus,@CompletionRatio,@UniqueEdgeCount,
@UniquePathCount,@TotalTraceCount,@TotalEdgeCount,@PortClosedTraceCount,@DestUnreachedTraceCount,
@DnsFailureTraceCount,@RouteChangeCount,@EdgeChangeCount,@Rtt,@RttMin,@PacketLoss,@StatusValue,
@Graph,@SentPackets,@LostPackets,@ConnectionBased)
(ExecutedAt=4/20/2016 2:32:44 AM, ServiceMonitorId=33, ProbeId=1, ProbeStatus=Up, CompletionRatio=100, UniqueEdgeCount=30, UniquePathCount=6, TotalTraceCount=22, TotalEdgeCount=144, PortClosedTraceCount=0, DestUnreachedTraceCount=0, DnsFailureTraceCount=0, RouteChangeCount=0, EdgeChangeCount=0, Rtt=117.136363636364, RttMin=97, PacketLoss=12.5, StatusValue=Warning, Graph=System.Byte[], SentPackets=490, LostPackets=40, ConnectionBased=False)

        [60] DEBUG SolarWinds.NetPath.Collector.DAL.TracerouteResultDAL - Traceresult was stored.
[STP DatabaseProcessing Thread #33] DEBUG SolarWinds.Collector.DBProcessing.DBProcessingWorkerWithAccess - Commit of 33 - NetPath.Test.Generic .... 1 items in 00:00:00.8280112

3. If there are multiple intervals that have no data, the collector service may have a problem.

   1. Check the event viewer system log. It shows error messages during that interval.

      The description for Event ID 7031 from source Service Control Manager cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

      If the event originated on another computer, the display information had to be saved with the event.

      The following information was included with the event:

      SolarWinds Collector Service
1
60000
1
Restart the service

      The handle is invalid

      

   2. Check the collector log at C:\ProgramData\Solarwinds\Collector\Logs. If the area that corresponds to the interval with missing data contains the following exception, it is caused by wrong input in centralized settings for the NetPath.Collector group.

      ERROR SolarWinds.Collector.BusinessLayer.EntityWatcher - PollingPlan process failed NetPath.Test.Generic
System.TypeInitializationException: The type initializer for 'SolarWinds.NetPath.Collector.CollectorSettings' threw an exception. ---> System.FormatException: Input string was not in a correct format.
at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider)
at SolarWinds.Settings.TypeHelper.Convert(Object value, Type targetType)
at SolarWinds.Settings.SettingsBase.Apply(Dictionary`2 settings)
at SolarWinds.Settings.SettingsBase..ctor()
at SolarWinds.NetPath.Collector.CollectorSettings..cctor()
--- End of inner exception stack trace ---
at SolarWinds.NetPath.Collector.ResultsRecorder.RecorderConfig.GetConfig()
at SolarWinds.NetPath.Collector.DAL.TestDAL.PollingDescriptions(IPollingEngine engine)
at SolarWinds.Collector.BusinessLayer.EntityWatcher.ProcessPollingPlan(PollingPlanPlugin plugin, IPollingEngine engine, EngineChangeKeeper keeper)
at SolarWinds.Collector.BusinessLayer.EntityWatcher.ProcessEngine(IPollingEngine engine, Func`2 matchEntityPrefix)

   3. Correct the input in the centralized setting page at https://<OrionServer>/Orion/admin/advancedconfiguration/Global.aspx. Then the collector service can start, and the data shows up again.

The graph shows only two nodes connected by a dotted line 

There is a graph, but it shows only two nodes (the probe and the endpoint) connected by a dotted line.

1. It could be a DNS issue that the configured host name cannot be resolved to a valid IPv4 address. We don't support IPV6 for now.

2. It could be that the interface to send out probing packets cannot be found.

   [STP SmartThreadPool Thread #0] ERROR SolarWinds.NetPath.Agent.Probe.Traceroute.Util.LocalIpMacResolver - Found no matching interface.
[STP SmartThreadPool Thread #0] ERROR SolarWinds.NetPath.Agent.Probe.Traceroute.Jobs.TcpTracerouteJobFactory - Failed to find local ip and mac address. Not able to send out probe packets.
[STP SmartThreadPool Thread #0] ERROR SolarWinds.NetPath.Pollers.Traceroute.TraceroutePoller - Failed to create traceroute job (AggregateE)
System.NotSupportedException: Failed to find local ip and mac address. Not able to send out probe packets.
at SolarWinds.NetPath.Agent.Probe.Traceroute.Jobs.TcpTracerouteJobFactory.<CreateSenderSettings>d__9.MoveNext()

3. It could because probing cannot find consistent path between the probe and the endpoint.
   1. Check the NetPath log at C:\ProgramData\Solarwinds\Logs\NetPath, and see if "Failed to Consolidate" is logged many times.
   2. If so, edit C:\ProgramData\Solarwinds\Orion\NetPath\NetPathAgent.cfg, and then restart services:
      • Change the "NumberOfRoundsPerProbeStandard" value from 3 to 1
      • Make sure "NumberOfRoundsPerProbeConnection" is 1

4. It could because the probe cannot receive packets.

   Check the NetPath log to search for the message "Original hops for". If the message is similar to the example below (every hop is a timeout), it means the probe cannot capture packets. See Capture Packets for details.

   [STP SmartThreadPool Thread #14] DEBUG SolarWinds.NetPath.Agent.Probe.Traceroute.StandardRunner - Original hops for 54.239.26.128:
[192.168.250.1 (SP:1, PL:0, TTL:0, Status:StartingPoint, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:1, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:2, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:3, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:4, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:5, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:6, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:7, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:8, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:9, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:10, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:11, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:12, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:13, Status:Timeout, DR:False, EP:False) ]->[null (SP:1, PL:100, TTL:14, Status:Timeout, DR:False, EP:False) ]->[

The browser cannot get data from the NetPath server 

• Web Server Error - This indicates that the client was unable to access the web service or the web service generated an error. You can use a tool like Fiddler to see what the web server returned - if anything. If the response takes longer than 15 seconds to return, then the issue is likely on the server - possibly the database.
• Unknown - This occurs when the client was able to contact the web service but the web service returned null instead of a valid graph. Investigate the cause on the web server and database.

The packet loss at the endpoint is too high 

The packet loss may be caused by our probing pattern: sending multiple probing packets in the same TCP session to endpoint in short time. So the server sometimes responds, and sometimes does not. Change the NumberOfRoundsPerProbeStand to 1 in C:\ProgramData\Solarwinds\Orion\NetPath\NetPathAgent.cfg, and see if the endpoint packet loss is zero.

Graph completion ratio is low 

In debug mode, you see that the test completion ratio is low. It means there are many paths not discovered yet.

1. It is possible that the probing interval is too small to discover all paths in one probing interval.

   Increase the probing interval or change the "NumberOfRoundsPerProbeStandard" value from 3 to 1 .

2. It is possible the probing has an error and cannot use the full probing interval. Check the NetPath log for clues.

Same IP in multiple hops in debug mode 

If you see the same IP address in multiple hops in debug mode it may result in much simpler topology than the real topology.

We have encountered several cases like this. From packet capture, the router in the middle of path sends back ICMP for multiple TTL in connection based probing. It is likely that the user configured that router to handle TTL expires attack (more information is at TTL Expiry Attack Identification and Mitigation). As the result, we cannot discover many nodes along the path.

Set EnableConnectionBasedProbing=false and restart the services. See if the path to the endpoint shows more hops with IP addresses. 

No contact information for a node 

When you click on a node, there is no phone number or email for any of the nodes.

1. First see if TCP port 43 is opened at the firewall. NetPath sends WhoIs query to the remote server through this port.

2. Determine if the BGP service is working:

   Check C:\ProgramData\Solarwinds\Logs\Orion\Bgp.BusinessLayer.log to see if the service is up. Enable debug to see if it sends out requests and get responses.

3. Determine if the BGP client sends out BGP requests:

   In C:\ProgramData\Solarwinds\Collector\Logs\Collector.Service.log, look for "Firing BGP requests".

The latency changes a lot 

The service is mapped to many different IP addresses. And the latency to one endpoint IP is different from the latency to another endpoint IP. Click to the endpoints with the different latencies to see if their IP addresses are the same.

Create a service using one of the service's IP address, and the latency is consistent. This means the changing latency is not an issue. It is caused by the fact that a single endpoint service is mapped to multiple IP addresses.

TTL mismatch 

It is possible that there is an extra edge that connects a node in the middle to the endpoint directly, and the customer confirms that there is no such direct connection.

This could happen because the response from endpoint is mismatched with the probing packet at the wrong TTL. The graph may look similar to the graph in the issue "The endpoint is much closer to the probe due to a proxy". However, the cause is very different.

In the proxy situation, a proxy caches traffic and responds on behalf of the endpoint. It could be a WAN Optimization appliance. It only happens in connection-based probing.

In the TTL mismatch case, the real endpoint responds (not another node responds on behalf of the endpoint), but when NetPath matches the response to the probing request, it cannot find enough information and matches it to the wrong request. Then the extra edge shows in the graph.

• Packet capture is required to verify whether the edge is caused by a proxy or a TTL mismatch. 
  • In a proxy situation, the responses from the endpoint IP address have different TTL values for short trace (total number of hops in a trace is small) versus a long trace.
  • In a TTL mismatch situation, the responses from the endpoint IP address have the same TTL value for a short trace (total number of hops in a trace is small) versus a long trace.
• There could be several ways to avoid this situation.
  • If the extra edge (usually, the that trace contains this extra edge has many fewer hops than the trace that contains the correct path) is discovered in standard probing, change "SendResetPerRoundStandard" from false to true, and see if it removes the extra edge.
  • If the extra edge is discovered in connection-based probing and the latency on the extra edge to the endpoint is pretty high (more than 200 ms), increase "TracerouteTimeout" to 2000. This may fix the issue.

FAQs 

We want to test the path multiple times to make sure the path discovered is consistent. If we change it to be 1 by default, it is possible there are extra edges between nodes occasionally. 

1. Why not set the default "NumberOfRoundsPerProbeStandard" value to 1 instead of 3?

   We want to test the path multiple times to make sure the path discovered is consistent. If we change it to be 1 by default, it is possible there are extra edges between nodes occasionally.

2. Why not set EnableConnectionBasedProbing = false by default?

   Connection-based probing is more firewall friendly. It is less likely to trigger a security policy and cause probing packets to be dropped. The packet loss estimation tends to be more accurate.