Monitor syslog messages with Kiwi Syslog Server NG

A syslog message is a message in standardized format using System Logging Protocol (syslog) that network devices use to communicate. Network devices—such as routers, switches, firewalls, and servers—use syslog messages to send information about their status or important events, so they’re extremely important for network troubleshooting.

The key for taking advantage of syslog messages for network monitoring and troubleshooting is to have a good syslog server. A syslog server can centralize syslog messages from your syslog-capable devices and allow you to access, search, or filter the messages (and usually a lot more). For this, the syslog-capable devices need to be configured to send the syslog messages to a syslog server.

Syslog messages are used mainly by network devices with Linux and Unix operating systems. By default, syslog messages are sent via UDP (User Datagram Protocol), which is a connectionless protocol, so there’s no guarantee the message arrived successfully. However, some devices can also use a connection-oriented protocol—TCP (Transmission Control Protocol)—which helps ensure the message delivery.

What are syslog messages used for?

Syslog messages are typically used by network and system administrators for early detection and troubleshooting of a possible issue for a network device. Syslog messages provide essential information about network device status and important events capable of having a negative impact on the standard operation of a network. Together with SNMP traps, syslog messages are a basic means of communication for network devices, such as routers, switches, firewalls, and servers. In a typical network, thousands of syslog messages and SNMP traps are generated every minute, which makes their usability for network monitoring without a centralized solution impossible. Both types of messages can be collected by a syslog server, which acts as a central place for all the logs network devices generate. A syslog server offers an easy way to access, search, and filter logs, and it’s a crucial part of log management.

Syslog messages have three main parts:

HEADER (identification information) SD (structured data) MSG (the actual message)

Header: The header of a syslog message includes identification information such as version, time stamp, hostname, IP address of the device, process ID, and message priority (PRI). Syslog message priority is a calculated value that helps classify syslog messages, determine the overall importance of the message, and assign an appropriate reaction, if needed.

Structed data: This part of a syslog message is designed to provide a well-defined and easily parseable data format. Since the message itself is in a free-text format, it can be challenging to extract relevant information from it. Structured data offers a way to provide additional valuable information about a syslog message (such as traffic counters or IP addresses) in a more friendly format for further data processing.

Message: This part of a syslog message includes the actual message in a free-text format and provides information about the event. Usually, a UNICODE character set encoded with UTF-8 is used in syslog messages.

PRI: The priority of a syslog message is calculated as a combination of two variables: facility and severity.

The facility code specifies the type of system that generated the message. It can have a numerical value between 0 and 23 based on 15 predefined values and eight values that can be defined locally:

Number - Facility Description

0 - Kernel Messages

1 - User-Level Messages

2 - Mail System

3 - System Daemons

4 - Security/Authorization Messages

5 - Messages Generated by syslogd

6 - Line Printer Subsystem

7 - Network News Subsystem

8 - UUCP Subsystem

9 - Clock Daemon

10 - Security/Authorization Messages

11 - FTP Daemon

12 - NTP Subsystem

13 - Log Audit

14 - Log Alert

15 - Clock Daemon

16 - 23 - Locally Used

Severity: This variable specifies the importance of the message itself and can have a numerical value between zero and seven (from emergency to debug-level messages).

The priority of a syslog message is calculated as follows:

Priority = Facility * 8 + Severity

For example, an emergency kernel message would have a priority value of 0. The lower the priority value, the higher the importance of the message.

A good syslog server allows you to identify messages with high priority and adequately react to the situation, whether it means sending an email notification to a network administrator or running an external script.

There are eight severity levels used for categorizing syslog messages. The description of each severity level according to The Syslog Protocol RFC 5424 is as follows:

Numerical Code - Severity
0 - Emergency: system is unusable
1 - Alert: action must be taken immediately
2 - Critical: critical conditions
3 - Error: error conditions
4 - Warning: warning conditions
5 - Notice: normal but significant condition
6 - Informational: informational messages
7 - Debug: debug-level messages

It’s unlikely you’ll receive emergency messages, as these usually mean the system is down and it can’t send any messages. On the other side, debug messages are usually used during development and don’t typically impact your network operations, so you might want to get notified about these.

Like the priority level, a good syslog server should allow you to set up rules to react to syslog messages according to their severity levels.

You can monitor syslog messages more effectively in the Kiwi Syslog Server NG by defining rules and using filters. This robust syslog monitor offers the ability to define unlimited number of rules (consisting of unlimited number of filters and actions) so you can process and respond to syslog messages according to your criteria and needs. Kiwi Syslog Server NG offers keyboard shortcuts to simplify deleting, inserting, copying, pasting, moving, renaming, and auto-naming your rules, filters, actions, and schedules.

Rules tell Kiwi Syslog Server NG how to process incoming syslog messages, including which messages trigger which actions. If a rule applies to a log message, Kiwi Syslog Server NG will compare the message to each filter in the rule, starting with at the top. If any filter condition is false, Kiwi Syslog Server NG will stop processing the rule and apply the next rule to the message. However, if every condition in a filter is true, Kiwi Syslog Server NG will repeat the process with the following filter. If a message passes every filter in a rule, Kiwi Syslog Server NG will begin performing all your actions in order. Once it’s finished with all the filters and actions in your first rule, Kiwi Syslog Server NG will move on to the following rule, so applying rules in order is essential.

Adding rules to determine which actions occur after a message is received is easy. To start, select Setup from the main menu.. Then click Rules and Add Rule in the Kiwi Syslog Server NG dialogue box to add a new rule to the tree. Finally, name the rule, add rule filters and rule actions, and save your changes by clicking OK. After creating a rule, you can easily export it to share with another Kiwi Syslog Server NG.

You can set filters to control whether a message triggers a rule’s actions. SolarWinds Kiwi Syslog Server NG enables you to filter messages based on IP address, priority, time of day, hostname, input source, regular expressions, and message text. Once you’ve created your filters, Kiwi Syslog Server NG will automatically apply them in the order they’re listed, but if you forgo filters, every message will trigger an action.

You can configure Kiwi Syslog Server NG to perform a specific action when a message passes through all of a rule’s filters.

Common actions include:

• Running a script or external program
• Sending an email
• Logging incoming messages to a file, Papertrail™, or Loggly^®
• Sending a syslog message or an SNMP trap
• Resting counters and flags
• Displaying a message

Configuring your syslog-capable devices to start sending messages to Kiwi Syslog Server NG for syslog monitoring is easy. To start, ensure your device has its message logging capabilities enabled. Fortunately, most devices capable of generating syslog messages automatically enable logging, but it’s still a good idea to double-check. Then, set up your device to send syslog messages to a port (usually port 514) on the computer with Kiwi Syslog Server NG.

The RFC standard 5426 named port 514 as the default port for syslog messages. Kiwi Syslog Server NG will listen for User Datagram Protocol (UDP) messages on port 514 by default. However, if this doesn’t suit your needs, you can easily configure your Kiwi Syslog Server’s settings to listen for Transmission Control Protocol (TCP) messages, secure TCP messages, and Simple Network Management Protocol (SNMP) traps instead of UDP messages. You can configure Kiwi Syslog Server NG to listen for UDP, TCP, secure TCP, or SNMP messages on a different port.

To configure UDP input options, open the Kiwi Syslog Server NG Settings section  under Setup in the main menu. Here, under the Inputs menu item,click UDP, and specify the port where you’d like to listen for UDP messages. Any port value between 1 and 65535 will work if the device transmitting the syslog message supports the new port number. It’s best for most people to leave the Bind to address field blank and allow your UDP socket to listen for messages on all interfaces. However, specifying the IP address in the Bind to address field will allow you to limit binding to a specific interface. You can establish which decoding method will be applied to any incoming data by selecting an encoding format from the drop-down menu or entering the code’s page number under the Data encoding section. After configuring your settings, save changes by clicking Apply.

Configuring TCP, secure TCP, and SNMP trap input options is just as simple. Instead of clicking UDP under the Inputs, select TCP or SNMP. You can then configure your settings. For example, the default port for TCP syslog messages is 1468, but you can choose a different port number. As with UDP messages, you can alter the Bind to address field and data encoding format. Then, specify your message delimiters, also known as separators, which signify which character or sequence of characters split a TCP stream into separate syslog messages.

As a network or system engineer, you’ll want to use a syslog management tool to collect and monitor syslog messages from your network’s devices. Kiwi Syslog Server NG can collect syslog data from an unlimited number of devices, so you can easily monitor all your switches, firewalls, and routers.

In addition to monitoring syslog messages, Kiwi Syslog Server NG can collect Simple Network Management Protocol (SNMP) traps from Unix, Linux, and Windows systems, enabling you to view essential information across your IT infrastructure in a centralized location.

You can view your data in real-time with the user-friendly syslog web-based console from anywhere in the world with web access. Kiwi Syslog Server NG has 21 customizable views and syslog statistics graphs, so you can quickly understand and troubleshoot network or device performance issues. You can filter syslog messages by host IP address, priority, hostname, or time of day to locate crucial messages.

Beyond simply collecting and monitoring syslog messages, SNMP traps, and Windows event logs, Kiwi Syslog Server NG can respond to syslog messages thanks to its built-in actions.

Other Kiwi Syslog Server NG advantages include the ability to:

• Archive syslog messages on disks, files, or ODBC-compliant databases
• Forward messages to other SolarWinds IT management tools like Loggly, Papertrail, Security Event Manager (SEM), and Network Performance Monitor (NPM)
• Keep your inbox clear thanks to Kiwi Syslog Server’s advanced message buffering capabilities
• Store, archive, and cleanup logs to help demonstrate compliance with SOX, PCI-DSS, and HIPAA