I‘ve already walked through the basics of building alert logic. Now let’s look at a couple of additional options to build more advanced alerts.
If you look to the bottom of the screen, you’ll see a check box for adding a timer to the alert logic.
I can click the checkbox and then enter whatever interval I want. This is useful for filtering out anomalies and temporary spikes from more serious alert events.
For instance, rather than notifying me immediately when a node goes down, by adding a timer I’m saying not only does the node have to go down, but it has to stay down for 10 minutes before I want to see an alert. On secondary devices that aren’t mission critical, that might make sense to help control the amount of alerts you have to deal with.
Anytime I’m alerting on things like CPU, memory, or interface utilization that tend to be very dynamic with lots of spikes and drops, I’ll probably want to include a timer as well. I don’t want to be alerted every time my utilization spikes if it immediately goes back down. But if the utilizations stays high, then I want to know about it.
Below the condition timer, there is an advanced options section.
I’m going to click the plus to expand advanced options, then click the check box to enable complex conditions. Now two things have just happened. First, I have a new checkbox under my condition timer. This new checkbox allows me to look for multiple alert events before triggering an alert.
Now instead of alerting if a single node goes down, I won’t be alerted unless at least five nodes have gone down. Below the checkboxes, there is a button to add a new section.
By adding another section to my alert, I can add a completely new set of alert logic conditions.
Notice this new section is identical to my first logic section,
including at the top of the section the dropdown box to specify the object type I want to reference with my new logic. So in the top logic section I can be looking at nodes, and in the bottom section I can be looking at something completely different, like applications or interfaces.
Now there are two ways that you’ll use the additional logic sections. The first is to simply expand on your alert logic. So in this case maybe I want to see how network traffic to my server is affecting my server’s performance.
So in my first section, I’m look for high memory utilization on my server.
Then in the second section, I’m going to choose to alert on the Interface object type rather than a node, and I’ll limit the scope of this section to the network interface on my server.
Now for my trigger condition I’m going to dig a little deeper into the database. I’m going to click on Browse all Fields,
and under Receive Utilization threshold,
I’m going to choose critical value reached.
Before I click select though, notice above that the critical value is set to 90, which means it’s looking for my receive utilization to hit 90%. At the bottom of the section, I again have a condition timer, so I could set a separate timer for my interface utilization or just leave it unchecked.
So this alert says if my memory utilization is high, check to see if high network traffic may be the cause. I wouldn’t be able to build this alert in a single logic section.This is an example of expanding the alert logic using complex conditions. The second way that you will use complex conditions is to suppress alerts.
Let me show you an example. I want to alert on a critical interface on one of my switches. And if the interface status is anything other than UP, I want to know about it.
However, if the entire switch goes down, then alerting on the interface might not make as much sense because in that case I have other things to worry about than just the interface.
So in my secondary section, I’m going to choose the switch the interface is on, and I’ll say the switch has to be UP. Now when the alert runs, it will check the status of my critical interface first, but then it will check the status of the switch. If the switch is down, then this secondary logic section will effectively suppress the alert.
You can add as many additional logic sections as you need and can create very complex alerts. But do note that each logic section is joined by an AND statement.
You can add a timer to the and statement by changing from AND to And then after in the dropdown, but it is still an AND statement. So the trigger conditions in each of the logic sections must be met in order to trigger an alert.
Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment. You elect to use third-party content at your own risk, and you will be solely responsible for the incorporation of the same if any.