OTADD

When otadd -c 10.1.1.1 (for example) is run from \OverTime\bin\ it queries the given IP address via SNMP for information about the device. Otadd firstly resolves the SNMP SysName. If no further parameters are given then the database for the devices is created to monitor the traffic loads through its interfaces (these are know as I/O collections). I/O collections involve ifInOctets, ifOutOctets, ifInErrors, ifOutErrors, CiscoAvgBusy5 and sysuptime are in the I/O class. If the device happens to be special cased (like the sensor array) then entries specific to that device are created. All other collections are considered as special user defined collections. For I/O collections on device at IP address 10.1.1.1 with SNMP read community string of "public" the following commands will achieve this:

Otadd -c public 10.1.1.1

OR

Otadd -c public 10.1.1.1 "" ""

To monitor just instance 3 and 4

Otadd -c public 10.1.1.1 "" "3,4"

To monitor all instances of a specific OID

Otadd -c public 10.1.1.1 .1.3.6.1.4.1.x.y ""

To add just instances 4 and 5 from this OID

Otadd -c public 10.1.1.1 .1.3.6.1.4.1.x.y "4,5"

To monitor I/O instances 3 and 4 and OID .1.3.6.1.4.1.x.y instances 6 and 7

Otadd -c public 10.1.1.1 "" "3,4" .1.3.6.1.4.1.x.y "6,7"

To monitor I/O instances 3 and 4 and OID .1.3.6.1.4.1.x.y instances 6 and 7 as well as OID .1.3.6.1.4.1.w.z all instances

Otadd -c public 10.1.1.1 "" "3,4" .1.3.6.1.4.1.x.y "6,7" .1.3.6.1.4.1.w.z ""

For I/O collections, the device is queried for all available interfaces that meet certain criteria. I.E. the interface must be administratively up, not be any of: a ds1, a loopback, a Null, a Dialer, a CPU and have a reasonable interface speed >0 and <400Mb/sec. The acceptable interfaces are then queried for their speed, their names setup in the device, their maximum bandwidth and any associated IP address all of which then automatically appear in the OverTime master configuration file for this device and will ultimately appear on the web page and its graphs.

If the master configuration file for the device already existed and the legends, y axis label or graph title were changed for user-defined collections, then these changes are preserved by otadd.

Otadd writes an devname.col file that is then used by devgetupd to perform the collections.

Otadd also writes the master configuration file in OverTime/cfgs. This file’s name is the device’s SNMP sysName (or its IP address if no name is available either of which will be referred to as sysName within this document) with an extension of “.mst”. (See above for the definition of this file).

Otadd then updates overtime.csv (see later) and the master HTML file, index.html.

Otadd creates the directories for the RRD databases to be stored in OverTime/data/ under the devices sysName directory.

Otadd creates the directories for the graphics to be held in the web server’s area OverTime/ under the devices sysName directory.

Otadd sets up the RRDs based on what it finds and other user setting within overtime.cfg.

User-defined collections cause otadd to create RRDs based on the OID. These databases are also configured based on the collection frequency setup when the RRD is initially created. IT IS NOT CHANGED while the RRD database exists. The RRDs are given names that commence with sysName and then followed by the SNMP interface number for I/O collections. For user-defined collections, the RRD is called sysName and then the OID followed by the instance discovered by “snmpwalking” the device for this OID.

External/Non-SNMP Collections

OverTime can be used to record and display collections gathered by your own application or script and not just by OverTime's snmpCollect service/daemon. These types of collections are termed "external or non-SNMP collections" in OverTime. You use non-SNMP collections to monitor from your server any kind of statistical value. Maybe you want to monitor your web server’s response time or the number of cars going past your window. If your server can count them, then OverTime can help you record them. This is what a graph of web server response times looks like:

Setting Up a new TCP port Collection.

An example of this type of collection will help to make the process clear. Let's assume that you would like to monitor the response times for loading the main web page from two web sites (www.myco.com and www.myotherco.com) and you want to see the results on your OverTime page under the label ‘Web_Responses”.

You can use this check_port_pl.txt or this time2open.pl script as a starting point.

1) Copy the check_port.pl script and give it a name like http_time.pl on your OverTime server.

2) Edit the new script and change the settings at the top. The settings that you need to modify are the TCP port number, the string to send when the port opens and the string that you expect to be returned if the test is successful. Leaving the ‘testing’ value set to 1 will help you test the process. For our example, we need to make these changes:
$port=80; # 80 for http - User chooses what they want!
$send = "GET index.html\n"; # What do you want sent
$expect="\<\/html\>"; # What you expect to be returned if it worked.
$testing=1; # 1 if we're testing, 0 if we're really collecting
$threshold = 0.5 # A threshold of 0.5 seconds to respond
$runcmd = "echo Threshold breached."; # Command to run when threshold is breached
$script_name = "http_time"; # The name of this script

3) Test your new script with a command like ‘http_time.pl Web-Responses www.myco.com www.myotherco.com”

4) When you have this working correctly, remember to edit the script and set $testing=0

5) To add your new collection to OverTime you run the otadd command:
“otadd --external G /var/opt/OverTime/cfgs/http_time.pl Web_Responses www.myco.com www.myotherco.com”.
This instructs OverTime to create the appropriate configuration files and also update the PingTime-xxx command file.
The “--external” tells otadd to create an external/non-SNMP collection configuration.
The “G” indicates if the numbers collected are a gauge. Alternatively, it could have been a "C" for counter. Counters are for values that continuously increase like the count of packets through an interface. Gauge is for values that are absolute like the current temperature or the current response time.
The "/var/opt/OverTime/cfgs/http_time.pl" (or on Windows "c:\OverTime\bin\http_time.pl") is the command, script or executable to run, including its full path.
“Web_Responses” is the name that will be used on the main OverTime HTML page and must also be a valid directory name. It may be a new name or the name of an existing OverTime device.
“www.myco.com www.myotherco.com” is the list of targets to be tested.

6) Once this has completed, you will be able to browse to results via the OverTime index.html page. (Remember, you won't see any results until the first collection has completed)

If "sysName" does not already appear in the index.html page for OverTime, then it will be added by otadd.

RRD datafiles will are created and the configuration of the regularly run (via 'at' or 'cron') command "PingTime-300" (see PingTime) is modified.

When PingTime-300 is next run, it then calls this command:

“/var/opt/OverTime/cfgs/http_time.pl Web_Responses www.myco.com www.myotherco.com”.

You can download the sample Perl script, check_port.pl and it looks like this:

__________________________________________________________________________________________

#!/usr/local/bin/perl

# Web server checking looks like this:

$port=80; # 80 for http - User chooses what they want!

$send = "GET /index.html\n"; # What do you want sent

$expect="\<\/html\>"; # What you expect to be returned if it worked.

$testing=1; # 1 if we are testing , 0 if we're really collecting

$threshold = 0.5; # A threshold of 0.5 seconds to respond

$runcmd = "echo Threshold breached."; # Command to run when threshold is breached

$script_name = "http_time"; # The name of this script

######################## Probably won't need to change anything below here ##################

use IO::Socket;

use Time::HiRes qw(gettimeofday);

for($ind=1; $ind< @ARGV; ++$ind){

$resptime = "U";

# Start the timer here!

# Open the socket

$start = now();

$soket = IO::Socket::INET->new(Proto => "tcp", PeerAddr => $ARGV[$ind] ,PeerPort => $port, Timeout => 2, );

$resptime="U"; # Failed by default

if ( $! eq '' ) { # Everything OK so far

$soket->autoflush(1);

# Send some protocol dependant data to the server

print $soket "$send"; # Send something to the port for it to respond

# Read back the results

while(<$soket>){

if($testing > 0){ print; }#Print all the responses if testing is set greater than 1

# Check for something in the result?

if(/$expect/) {

$resptime = now() - $start;

if ($testing) { print "Expected response of '$expect' was seen in $resptime seconds for $ARGV[$ind]\n"; }

} # Success

}

close $soket;

} else {

# Tell someone we've had a problem

print ("Port $port to $ARGV[$ind] did not open\n");

}

# End the timer here

# if $resptime=="U" then the time is not valid so don't check the thresholds.

# If the threshold has been breached, run a command!

if($resptime ne "U") {

if($resptime > $threshold) {`$runcmd`; }

} elsif ($testing) { print "Expected response of '$expect' was NOT seen for $ARGV[$ind]\n"; }

if ($testing > 0) {

`"/OverTime/bin/devupd"`; # Just run an update with no arguments - should see the usage

} else {

`"/OverTime/bin/devupd" $ARGV[0] $script_name $ARGV[$ind] Now $resptime`;

}

}

sub now {

my(@tmp) = gettimeofday();

return $tmp[0] + ($tmp[1] / 1000000.0);

}

__________________________________________________________________________________________

You can download the sample Perl script, #!/usr/bin/perl

use strict;

use warnings;

use Getopt::Std;

use IO::Socket::INET;

use Time::HiRes qw(gettimeofday);

my ($start,$resptime,$port,$host,$expect,$testing,$threshold,$runcmd);

my($portn,$script_name,$devupdcmd,$ind);

$| = 1;

$port=53; # 80 for http, 25 for SMTp, 53 for DNS - User chooses what they want!

$testing=0; # 1 if we are testing , 0 if we're really collecting

$threshold = 0.5; # A threshold of 0.5 seconds to respond

$runcmd = "cmd /c echo Threshold breached."; # Command to run when threshold is breached

$script_name = "time2open"; # The name of this script

$devupdcmd = "\"\\OverTime\\bin\\devupd\""; # The update command

######################## Probably won't need to change anything below here ##################

for($ind=1; $ind< @ARGV; ++$ind){

($host,$portn)=split /\./,$ARGV[$ind];

$resptime="U"; # Failed by default

$port = ($portn eq "dns" ) ? 53 : $portn; # The name 'dns' does not resolve

# Start the timer here!

# Open the socket

$start = now();

my $soket = new IO::Socket::INET->new( PeerAddr=> $host, PeerPort=> $port, Proto=> 'tcp', Timeout=> 30, );

if ( $soket ) { # Everything OK so far

$soket->autoflush(1);

$resptime = now() - $start; # End the timer here

## while(<$soket>){ print; last;}

shutdown($soket,2);

close $soket;

}

# if $resptime=="U" then the time is not valid so don't check the thresholds.

# If the threshold has been breached, run a command!

if($resptime ne "U") {

if($resptime > $threshold) {print `$runcmd`; }

}

if ($testing > 0) {

if($resptime ne "U") {

print "Response time of $resptime observered for $host on $portn\n";

### print`$devupdcmd`; # Just run an update with no arguments - should see the usage

} else {

# Tell someone we've had a problem

print ("Port $portn to $host did not open\n");

}

} else {

print `$devupdcmd $ARGV[0] $script_name $host.$portn Now $resptime`;

}

}

sub now {

my(@tmp) = gettimeofday();

return $tmp[0] + ($tmp[1] / 1000000.0);

}

Need to install Hi-Res on Windows?

Download nmake15 (http://download.microsoft.com/download/vc15/Patch/1.52/W95/EN-US/Nmake15.exe) and

HiRes can be downloaded from http://ppm.activestate.com/PPMPackages/zips/6xx-builds-only/Time-HiRes.zip

Make sure your Perl\bin is in your PATH. If not, set it up in your environment.

unzip Time-HiRes.zip into a temporary directory and then cd to that directory.

Finally, run “ppm install Time-HiRes.ppd” and your Hi-Res will be installed.