Basics of Network Monitoring

1. Overview:

Networks have evolved from being a flat network where there were only a handful of elements. Everything was connected—to a more complex design where there are a lot more technologies, such as cloud, wireless, remote users, VPN, IoT, mobile devices, and so on.

In spite of all the evolution that has occurred, one factor that has been constant is the need for network monitoring. Monitoring allows network admins to know what is going on in their network, be it with their WAN, LAN, VoIP, MPLS, and other connections or the state of various network elements or nodes such as the access, distribution and core switches, routers, firewalls, servers, client systems and so on.

Before you begin with network monitoring, it is necessary to understand networking in general, as well as essentials about Windows® systems which is the major OS used in enterprises worldwide. Knowledge about the essentials of networking and the elements that make up a computer network helps with better network management and monitoring.

2. General Networking:

A network is a collection of devices that are connected and can communicate with one another over a common transport or communication protocol. Here communication can refer to the transfer of data among users or instructions between nodes in the network, such as computers, mobile devices, output devices, management elements, servers, routing and switching devices, etc.

Networks can be categorized based on the geo area they span as LAN, WAN, or Internet. Further, the design or topology of a network too can differ based on user and organizational requirements, such as star, ring, bus, mesh, etc.

Whatever be the design or the topology, every network follows a reference design as described in the OSI model for data transmission and communication. Open System Interconnection (OSI) is a reference model for a network and describes how information from an application installed on a device or system moves through various nodes in the network to another device within the same network or to an external network. There are many components that make a network and enables communication between various nodes, such as network addresses, data transport & communication protocols, and methods used for transfer of packets between nodes within the same network or different networks. Below are some of the basic components that are part of every computer network and these also are the vectors that form the essentials of network monitoring.

  1. IP address and subnetting
    An IP address is the reference label assigned to each node in a network and is used by other nodes for location and communication. Further, IP addresses are binary numbers, but are stored in human readable format, either as an IPv4 address or IPv6 address. The elements with an IP address that make up a network can be divided into different sub networks based on the device type, location, access, etc. The devices in the same subnet all have a common network prefix defined in its IP address.
  2. Switching and Routing
    Switching refers to the process in which data is divided into smaller packets before they are sent and transported over the network. Routing is the act of finding a path for the packets that form data to traverse from a source node in one network to a destination node in a different network.
  3. Domain Name System (DNS)
    Each element in a network, in addition to an IP address, can also have a reference name. This allows a user to communicate with a resource using an easy to remember alphabetical name rather than a difficult to remember IP address. DNS maps the name of a resource to its physical IP address or translates a physical IP address to a name.
  4. Dynamic Host Configuration Protocol (DHCP)
    DHCP is a network protocol that allows a management server (DHCP server) to dynamically assign an IP address to the resources in its network. Without DHCP, network admins would have to assign IP addresses for each host in their network manually, making management of IP addresses difficult.

3. General Windows Monitoring elements:

Enterprises use various business applications that are installed on servers within the enterprise network or datacenter to provide services to hosts within the organization. There are also additional network and user management such as the DNS, Active Directory, DHCP, etc., that are provided from servers. Additionally, users or clients in an organization too require an Operating System. Among the multiple choices available for an Operating System, Windows based OS’s are the most widely used, both for server as well as for client host requirements in an enterprise.

The presence of business applications on servers necessitates their constant monitoring for visibility about resource usages, such as memory, disk space, cache, CPU, and more. Monitoring also helps identify possible issues that are affecting server performance. In addition to servers, client devices too require monitoring to provide a trouble-free experience to the end-user.

Windows based systems can provide data to monitoring systems which then processes and uses the data to report on the performance and health of the servers, and host machines. The data that is used for monitoring can be collected from a Windows machine using any of the available methods discussed below.

  1. Performance counters
    Microsoft Windows Server includes OS performance counters that are enabled by default. These performance counters provide data about system performance, such as data on cache, memory, disk, processor, etc., which can be used by server performance monitoring applications.
  2. Windows Management Instrumentation (WMI)
    WMI is a Microsoft feature that allows for accessing management information about the status of computer systems. In addition, WMI supports actions, such as configuration & changing system properties, permissions, scheduling processes, etc. WMI can be used to manage servers as well as client machines locally and remotely.
  3. Eventlog
    Microsoft Windows OS generates a variety of event logs that contain information about events that occur on a system, such as application events (eg. data loss, or any significant problem with application performance), security events (failed logons, attempts to access secure files, security log tampering etc.), or system events. These logs can be monitored with a monitoring system to identify possible issues with server and client systems.

4. General Monitoring techniques and protocols:

Now that you know what makes up a network and the components available for Windows monitoring, let us look at general monitoring techniques used by network and systems admins.

In order to successfully monitor your network or even server and systems, the availability of the below options are necessary:

  • Data or information from various elements in the network. Data includes information about the working, current status & performance, and health of the element being monitored.
  • An application or monitoring software must be able to collect, process, and present data in a user friendly format. Software should even alert users about impending problems based on thresholds.
  • A protocol or method for transmitting information between the monitored element and the monitoring software.

Information collected from the network helps with better management and control over the network, identification of possible network issues before they cause downtime, and quick resolution of issues when something goes wrong. In short, constant monitoring will help create a high performing network.

Below are some of the general techniques available for monitoring. These techniques are used for collection of monitoring data from the network.

  1. Ping
    This is a network admin tool that is used to test the reachability and availability of a host in an IP network. The data from ping results can determine whether a host in the network is active or not. Furthermore, it can measure the transmission time and packet loss when communicating with a host.
  2. Simple Network Management Protocol (SNMP)
    SNMP is a network management protocol that is used for exchanging information between hosts in a network that includes network monitoring software. This is the most widely used protocol for management and monitoring of the network and includes the below components: Managed device: The node in the network that supports SNMP and access to specific information.
    • Managed device: The node in the network that supports SNMP and access to specific information.
    • Agent: A software that is part of the monitored device. An agent has access to the MIB (management information database) of the device and allows NMS systems to read and write to the MIB.
    • Network Management System (NMS): An application on a system that monitors and controls the managed devices through the agent using SNMP commands.

    SNMP data is collected or sent to a managed device, either by polling or using traps. Traps allow an agent to send information to an NMS about events on the device.

    The MIB holds information about the structure of the data on a device for management. The MIBs contain OID (object identifiers) which is the actual identifier for the variable to be read from the device or set on the device.

  3. Syslog
    Syslog (not to be confused with Windows Eventlog), is a message logging system that allows for a device to send event notifications in IP networks. The information from these messages can be used for system management, as well as security auditing. Syslogs are supported on a variety of devices ranging from printers to routers, and firewalls.
  4. Leveraging the power of scripts
    In networks where an NMS is not available for monitoring, or the existing NMS does not support specific functions or even extend the functionality of the existing NMS tool, network admins can make use of scripts. Scripts use common commands, such as ping, netstat, lynx, snmpwalk, etc., that are supported by most network elements to perform an action, such as collecting information from elements, making changes to device configurations, or perform a scheduled task. Bash scripts, Perl, etc. are common scripting tools used by network admins.

5. Database Basics:

Database is a collection of data or information that is structured. Every database involves a DBMS (Database Management System) which is a software application that performs actions, such as data creation, updates, retrieval or deletion based on user, or other application input. In addition to the data management functions, DBMS provides for data security, helps with data backup & recovery, and maintains data integrity. The actual data and the DBMS, because of their close relation, are sometimes referred together as database. Some of the popular DBMS in the market today are MySQL, Microsoft SQL, PostgeSQL, Oracle, DB2, SAP ASE and others.

Database and their related DBMS are usually run on dedicated servers, referred to as database servers. These servers may leverage RAID technology available on storage arrays for redundancy and performance.

  1. Relational database design
    While there have been multiple database models, the most popular ones in the market have all used the relation database model (RDBMS). An RDBMS allows users to create and maintain all data in objects called tables. Each table is a collection of related data entry and consists of rows and columns. The table structure of RDBMS allows for viewing the same database in multiple ways.
  2. SQL queries
    Structured Query Language (SQL) is a standard language for accessing information or data from databases. SQL queries can be used to perform actions, such as create, delete, update, and other manipulations to data stored in a database.

6. Tips / Resources

  1. Monitoring 101
  2. Networking Tutorials for Beginners
  3. Network Monitoring Tool