Diagnosing Chart and Graph Data Gaps
Orion data gaps can be caused by a number of different reasons:
-
SNMP requests refused or timing out on the remote device
-
ICMP (PING) timeouts or refusals on the remote device
-
SQL server resource over stressed
-
Polling intervals not set at the time span requested by the chart or graph
If you find that both SNMP and ICMP charts show the same gaps, continue through the diagnostic exercise. If they do not show the same data gaps, ensure you have enabled SNMP and ICMP on the remote device and that your SNMP credentials are correct.
The following procedure helps you determine if the SQL Server hardware is capable of collecting and inserting the data as quickly as you are asking the data to return. If you find that the disk queue length is greater than twice the number of physical drives within the SQL Server, you can take advantage of the Polls Per Second Tuning utility, or you can use a different database server with less I/O latency and faster drives.
To ensure your SQL Server resources can keep up with polled data insertions:
-
Log on to the database server with an administrator account.
-
Click Start > Run.
-
Type
perfmon in the Run dialog, and then click OK.
-
Ensure System Monitor is selected in the left Console Root tree.
-
Click Avg Disk Queue Length in the table below the graph.
-
Ensure the Average value is less than twice the number of physical hard drives in your SQL Server.
Note: If the value exceeds twice the number of hard drives in the database server, then the server is caching read/write requests until it has sufficient resources to complete the request.
If your average disk queue length is within an acceptable range, continue to the "Reviewing Polling Completion Percentage" section.
If the database server average disk queue length is acceptable, use the Orion Database Manager to review the percentage of received replies versus SNMP requests (polling completion percentage).
To view the polling completion percentage:
-
Log on to the Orion server.
-
Click Start > All Programs > SolarWinds Orion > Database Utilities > Database Manager.
-
Expand your Orion database in the left navigation tree, and then click Engines.
-
Click Table > Query Table in the menu bar.
-
Type
Select PollingCompletion from Engines in the Query window, and then click Refresh.
-
Review the PollingCompletion value.
Note: The PollingCompletion value should be greater than 99.00000, though it will never reach 100%. Orion is sending more requests to your devices than they can respond to if this value is less than 99%. The polling interval on one or more devices should be adjusted.
If you need to adjust your polling interval, continue to the "Adjusting Database Summarization and Polling Intervals" section.
To improve performance, consider adjusting your database summarization and polling intervals. In the following sections, you can review the default settings and decide when to consider changing them.
To review your settings:
-
Log on to the Orion server.
-
Click Start > All Programs > SolarWinds Orion > System Manager.
-
Click Settings in the menu bar.
You can use the Database tab to tell Orion how frequently to combine granular statistics into more general statistics; this is called summarizing data. Orion summarizes statistics to keep the database at a manageable size. Frequent polling, a large number of elements (nodes, interfaces and volumes), or Orion modules data can cause rapid growth.
For individuals with limited exposure to databases, this tool is one way to manage database size. Once data is summarized, it can not be unsummarized. You may lose a negligible degree of granularity, which can be seen as gaps in historical data if you report or attempt to chart at a shorter interval than you have granular data. For example, if you attempt to chart data over the last month at 5 minute intervals and you have left the summarization settings at their defaults, you will see hourly statistics. The following are the default and recommended values:
-
Detailed statistics are summarized into hourly statistics after 7 days
-
Hourly statistics are summarized into daily statistics after 30 days
-
Daily statistics are deleted from the database after 365 days
-
Network Events are deleted from the database after 30 days
You can use the settings on the Polling tab to dictate how Orion checks the up-down status of elements. Orion checks up-down status every 120 seconds. For nodes, this information is collected by ICMP ping. For interfaces and volumes, this information is collected through an SNMP request. The Rediscover Resources interval governs how long Orion waits before looking for an element that has not respond to the initial poll.
When adjusting these settings to improve performance, consider clicking Apply these settings to All to reset all element values to the adjusted values. You can readjust individual element polling times on a case-by-case basis by right-clicking the element in System Manager, and then clicking Node Details, Interface Details, or Volume Details as applicable. For more information about launching System Manager, see the "Adjusting Database Summarization and Polling Intervals" section.
After adjusting these values, consider balancing status requests using the Polls Per Second Tuning utility to ensure you send the correct number of requests. If you send too few requests, you can create gaps in your data. For more information, see the "Polls Per Second Utility" section.
The following values provide the default values for the Polling tab:
-
Default Poll Interval for Nodes: every 120 seconds
-
Default Poll Interval for Interfaces: every 120 seconds
-
Default Poll Interval for Volumes: every 120 seconds
-
Rediscover Resources: every 30 minutes
You can use the Statistics tab to govern how frequently Orion requests data from different elements using SNMP. SNMP data includes memory availability, CPU usage, thermal levels among other detailed device data.
SNMP data requires more bandwidth than simple up-down status, therefore statistics are collected every ten minutes by default. Though MIBs can report a subset of this data at 2 to 5 second intervals, most of the data does not need to be updated at less than 10 minute intervals. If you have volumes that you know are failing, collecting SNMP statistics at shorter intervals may be justified. Orion can report SNMP data every minute. The default value for each element type varies and collecting data more frequently than the recommended intervals requires hardware that exceeds the minimum specifications.
Orion can only collect SNMP data from a node or volume that sends SNMP data using the same SNMP version for which Orion listens. SNMP version can be changed on the Node Details dialog. Interfaces do not send SNMP responses, Orion parses interface data from the data received from nodes.
To modify individual node or volume SNMP data versions:
-
Log on to the Orion server.
-
Click Start > All Programs > SolarWinds Orion > System Manager.
-
Select the node that you want to modify or the node that contains the volume on which you want to report in the left hand navigation pane, and then click Nodes > Node Details in the menu bar.
-
Provide the appropriate SNMP version in the Version field of the SNMP grouping, and then provide the appropriate credentials in the Community String and Read-Write Community String fields.
When changing settings on the Statistics tab, consider clicking Apply these settings to All to apply your settings to all elements in Orion. You can modify these settings individually, as stated above.
The following values reflect the default values for this tab:
-
Collect node statistics every 10 minutes
-
Collect interface statistics every 9 minutes
-
Collect volume statistics every 15 minutes
The Polls Per Second Tuning utility offers you the ability to balance status and statistics polling intervals. Status polls provide the up-down status of an element. Statistics polls return detailed information, for example, memory usage, CPU load, and core temperature. For more information about the different types of polling, see "Exploring Polling Interval Settings" and "Exploring Statistics Collection Settings" in the "Adjusting Database Summarization and Polling Intervals" section.
Consider the following scenario:
You have 200 nodes you want to monitor at 10 minute intervals.
-
Orion could send 200 requests in the first second of the first minute, and then remain idle for 9 minutes.
-
Or, Orion can send 20 requests every minute.
By choosing the second option, Orion polls all the nodes but with less overall network load. Consider the network load savings when the number of nodes, interfaces and volumes increases by an ordinal, as well as when statistics are requested in addition to the status of each element.
The Polls Per Second Tuning utility calculates the total number of Status polls requested, and then divides that number by the time period found in "Exploring Polling Interval Settings" in the "Adjusting Database Summarization and Polling Intervals" section. The utility displays the current number of polls per second and recommends an ideal value for each setting. If the current number of polls is higher than the recommended number, then Orion is sending out excess requests and creating unnecessary network traffic. If the current number of polls is lower than the recommended number, then Orion can not gather all of the requested data and the polling completion percentage will be below 99%. As noted in the "Reviewing Polling Completion Percentage" section, a percentage below 99 can create report and graph data gaps. A similar calculation is performed for Statistics.
Anytime the status or statistics polling interval changes, Orion needs to recalculate the number of polls per second using this tool. If you are taking advantage of more than one polling engine on a network, run the utility on each polling engine whenever polling intervals change.
Warning: When you click OK to apply changes, the Orion service restarts, temporarily disabling monitoring and causing alerts to be triggered.
To launch the Polls Per Second Tuning utility, click Start > All Programs > SolarWinds Orion > Advanced Features > Polls Per Second Tuning.
Due to the variable nature of each network, the default values may not be ideal. The following list provides the default values:
-
Maximum Node and Interface Status Polls: 30 Per Second
-
Maximum Statics Collections: 30 Per Second
Copyright� 1995-2008 SolarWinds.net, Inc., all rights reserved worldwide.
Copyright� 1995-2008 SolarWinds.net, Inc., all rights reserved worldwide. No part of this document may be reproduced by any means nor modified, decompiled, disassembled, published or distributed, in whole or in part, or translated to any electronic medium or other means without the written consent of SolarWinds. All right, title and interest in and to the software and documentation are and shall remain the exclusive property of SolarWinds and its licensors. SolarWinds Orion�, SolarWinds Cirrus�, and SolarWinds Toolset� are trademarks of SolarWinds and SolarWinds.net� and the SolarWinds logo are registered trademarks of SolarWinds. All other trademarks contained in this document and in the Software are the property of their respective owners.
SOLARWINDS DISCLAIMS ALL WARRANTIES, CONDITIONS OR OTHER TERMS, EXPRESS OR IMPLIED, STATUTORY OR OTHERWISE, ON SOFTWARE AND DOCUMENTATION FURNISHED HEREUNDER INCLUDING WITHOUT LIMITATION THE WARRANTIES OF DESIGN, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL SOLARWINDS, ITS SUPPLIERS OR ITS LICENSORS BE LIABLE FOR ANY DAMAGES, WHETHER ARISING IN TORT, CONTRACT OR ANY OTHER LEGAL THEORY EVEN IF SOLARWINDS HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Microsoft�, Windows 2000, Windows 2003, Windows XP, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.