DevOps Sandbox: September 2015

I needed to install SolarWinds Log and Event Manager in hundreds of Linux servers for my client.

But there were some obstacles to over come as it was not a normal software installation.

I needed to adept my puppet manifest to declare all my specific requirements and of course make sure to run installation unattended.

Requirements

So...the sweet requirements:

1. if the old LEM is installed, uninstall it

2. before uninstall the old version, stop the service

3. if directory couldn't delete, force delete it

4. download the installer file from the designed server and path

5. since it's unattended installation, download a defined response file as well

6. since the setup file is not able to take the response file with absolute path, execute the installer in a directory where response file has been downloaded

7. for some unknown reasons, installer was starting to execute before response file get downloaded, so I needed to make sure response file is a requirement for installer

8. and finally, do all of these steps for all the servers except one!

Manifest

Based on all the 8 points above, I came up with the following puppet manifest:

node default {
if $hostname != 'yumrepo01.example.com' {

exec { "stop_old_lem":
command => "/etc/init.d/swlem-agent stop",
}

file { "/usr/local/contego/ContegoSPOP":
ensure => "absent",
force => "true",
require => Exec['stop_old_lem'],
}

file {'/tmp/installer.properties':
ensure => present,
mode => '0644',
owner => 'root',
group => 'root',
source => 'puppet:///modules/download/installer.properties',
}

file {'/tmp/setup_soalrwinds_lem_64bit.bin':
ensure => present,
mode => '0755',
owner => 'root',
group => 'root',
source => 'puppet:///modules/download/setup_soalrwinds_lem_64bit.bin',
}

exec { "install_lem":
cwd => '/tmp',
command => "/tmp/setup_soalrwinds_lem_64bit.bin -i silent",
require => File['/tmp/installer.properties'],
}

}

}

When you have monitoring system in place for the Linux environment, if you want to make a full use of it, you need to consider using SNMP protocol.

Net-SNMP needs to be installed in your Linux boxes and be configured properly.

Installation:

SNMP can be installed via yum command on Redhat/CentOS or "zypper" command in Suse servers:

# yum install net-snmp.x86_64

SNMP MIBs

With standard SNMP configuration, you don't get much details monitoring information you would like to have. So SNMP needs to be configured properly to produce all sort of SNMP MIBS.

A standard Linux SNMP config produce following limited MIBs only if we snmpwalk on it:

# snmpwalk -mALL -v1 -cpublic 10.8.230.42

.1.3.6.1.2.1.1.1.0 = STRING: "Linux hostname 2.6.18-371.el5 #1 SMP Thu Sep 5 21:21:44 EDT 2013 x86_64"
.1.3.6.1.2.1.1.2.0 = OID: 1.3.6.1.4.1.8072.3.2.10
.1.3.6.1.2.1.1.3.0 = TIMETICKS: 59756051
.1.3.6.1.2.1.1.4.0 = STRING: "Root <root@localhost> (configure /etc/snmp/snmp.local.conf)"
.1.3.6.1.2.1.1.5.0 = STRING: "hostname"
.1.3.6.1.2.1.1.6.0 = STRING: "Unknown (edit /etc/snmp/snmpd.conf)"
.1.3.6.1.2.1.1.8.0 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.2.1 = OID: 1.3.6.1.6.3.1
.1.3.6.1.2.1.1.9.1.2.2 = OID: 1.3.6.1.2.1.49
.1.3.6.1.2.1.1.9.1.2.3 = OID: 1.3.6.1.2.1.4
.1.3.6.1.2.1.1.9.1.2.4 = OID: 1.3.6.1.2.1.50
.1.3.6.1.2.1.1.9.1.2.5 = OID: 1.3.6.1.6.3.16.2.2.1
.1.3.6.1.2.1.1.9.1.2.6 = OID: 1.3.6.1.6.3.10.3.1.1
.1.3.6.1.2.1.1.9.1.2.7 = OID: 1.3.6.1.6.3.11.3.1.1
.1.3.6.1.2.1.1.9.1.2.8 = OID: 1.3.6.1.6.3.15.2.1.1
.1.3.6.1.2.1.1.9.1.3.1 = STRING: "The MIB module for SNMPv2 entities"
.1.3.6.1.2.1.1.9.1.3.2 = STRING: "The MIB module for managing TCP implementations"
.1.3.6.1.2.1.1.9.1.3.3 = STRING: "The MIB module for managing IP and ICMP implementations"
.1.3.6.1.2.1.1.9.1.3.4 = STRING: "The MIB module for managing UDP implementations"
.1.3.6.1.2.1.1.9.1.3.5 = STRING: "View-based Access Control Model for SNMP."
.1.3.6.1.2.1.1.9.1.3.6 = STRING: "The SNMP Management Architecture MIB."
.1.3.6.1.2.1.1.9.1.3.7 = STRING: "The MIB for Message Processing and Dispatching."
.1.3.6.1.2.1.1.9.1.3.8 = STRING: "The management information definitions for the SNMP User-based Security Model."
.1.3.6.1.2.1.1.9.1.4.1 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.2 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.3 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.4 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.5 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.6 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.7 = TIMETICKS: 0
.1.3.6.1.2.1.1.9.1.4.8 = TIMETICKS: 0
.1.3.6.1.2.1.25.1.1.0 = TIMETICKS: 502897869
.1.3.6.1.2.1.25.1.1.0 = No more variables left in this MIB View (It is past the end of the MIB tree)

While for making full use of SNMP power, the config should lead to produce following loads of MIBs:(around 130 pages! showing only first 2 pages below)

# snmpwalk -mALL -v1 -cpublic 10.8.230.42

.1.3.6.1.2.1.1.1.0 = STRING: "Linux hostname 2.6.32-431.el6.x86_64 #1 SMP Sun Nov 10 22:19:54 EST 2013 x86_64"

.1.3.6.1.2.1.1.2.0 = OID: 1.3.6.1.4.1.8072.3.2.10

.1.3.6.1.2.1.1.3.0 = TIMETICKS: 60013154

.1.3.6.1.2.1.1.4.0 = STRING: "Root <root@localhost> (configure /etc/snmp/snmp.local.conf)"

.1.3.6.1.2.1.1.5.0 = STRING: "hostname"

.1.3.6.1.2.1.1.6.0 = STRING: "Unknown (edit /etc/snmp/snmpd.conf)"

.1.3.6.1.2.1.1.8.0 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.2.1 = OID: 1.3.6.1.6.3.11.2.3.1.1

.1.3.6.1.2.1.1.9.1.2.2 = OID: 1.3.6.1.6.3.15.2.1.1

.1.3.6.1.2.1.1.9.1.2.3 = OID: 1.3.6.1.6.3.10.3.1.1

.1.3.6.1.2.1.1.9.1.2.4 = OID: 1.3.6.1.6.3.1

.1.3.6.1.2.1.1.9.1.2.5 = OID: 1.3.6.1.2.1.49

.1.3.6.1.2.1.1.9.1.2.6 = OID: 1.3.6.1.2.1.4

.1.3.6.1.2.1.1.9.1.2.7 = OID: 1.3.6.1.2.1.50

.1.3.6.1.2.1.1.9.1.2.8 = OID: 1.3.6.1.6.3.16.2.2.1

.1.3.6.1.2.1.1.9.1.3.1 = STRING: "The MIB for Message Processing and Dispatching."

.1.3.6.1.2.1.1.9.1.3.2 = STRING: "The MIB for Message Processing and Dispatching."

.1.3.6.1.2.1.1.9.1.3.3 = STRING: "The SNMP Management Architecture MIB."

.1.3.6.1.2.1.1.9.1.3.4 = STRING: "The MIB module for SNMPv2 entities"

.1.3.6.1.2.1.1.9.1.3.5 = STRING: "The MIB module for managing TCP implementations"

.1.3.6.1.2.1.1.9.1.3.6 = STRING: "The MIB module for managing IP and ICMP implementations"

.1.3.6.1.2.1.1.9.1.3.7 = STRING: "The MIB module for managing UDP implementations"

.1.3.6.1.2.1.1.9.1.3.8 = STRING: "View-based Access Control Model for SNMP."

.1.3.6.1.2.1.1.9.1.4.1 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.2 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.3 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.4 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.5 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.6 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.7 = TIMETICKS: 1

.1.3.6.1.2.1.1.9.1.4.8 = TIMETICKS: 1

.1.3.6.1.2.1.2.1.0 = INTEGER: 2

.1.3.6.1.2.1.2.2.1.1.1 = INTEGER: 1

.1.3.6.1.2.1.2.2.1.1.2 = INTEGER: 2

.1.3.6.1.2.1.2.2.1.2.1 = STRING: "lo"

.1.3.6.1.2.1.2.2.1.2.2 = STRING: "eth0"

.1.3.6.1.2.1.2.2.1.3.1 = INTEGER: 24

.1.3.6.1.2.1.2.2.1.3.2 = INTEGER: 6

.1.3.6.1.2.1.2.2.1.4.1 = INTEGER: 16436

.1.3.6.1.2.1.2.2.1.4.2 = INTEGER: 1500

.1.3.6.1.2.1.2.2.1.5.1 = GAUGE32: 10000000

.1.3.6.1.2.1.2.2.1.5.2 = GAUGE32: 4294967295

.1.3.6.1.2.1.2.2.1.6.1 = STRING: ""

.1.3.6.1.2.1.2.2.1.6.2 = HEX-STRING: 005056B73BF7

.1.3.6.1.2.1.2.2.1.7.1 = INTEGER: 1

.1.3.6.1.2.1.2.2.1.7.2 = INTEGER: 1

.1.3.6.1.2.1.2.2.1.8.1 = INTEGER: 1

.1.3.6.1.2.1.2.2.1.8.2 = INTEGER: 1

.1.3.6.1.2.1.2.2.1.9.1 = TIMETICKS: 0

.1.3.6.1.2.1.2.2.1.9.2 = TIMETICKS: 0

.1.3.6.1.2.1.2.2.1.10.1 = COUNTER32: 18452015

.1.3.6.1.2.1.2.2.1.10.2 = COUNTER32: 3287903299

.1.3.6.1.2.1.2.2.1.11.1 = COUNTER32: 59529

.1.3.6.1.2.1.2.2.1.11.2 = COUNTER32: 23225997

.1.3.6.1.2.1.2.2.1.12.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.12.2 = COUNTER32: 1

.1.3.6.1.2.1.2.2.1.13.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.13.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.14.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.14.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.15.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.15.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.16.1 = COUNTER32: 18452015

.1.3.6.1.2.1.2.2.1.16.2 = COUNTER32: 1686413129

.1.3.6.1.2.1.2.2.1.17.1 = COUNTER32: 59529

.1.3.6.1.2.1.2.2.1.17.2 = COUNTER32: 19108930

.1.3.6.1.2.1.2.2.1.18.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.18.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.19.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.19.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.20.1 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.20.2 = COUNTER32: 0

.1.3.6.1.2.1.2.2.1.21.1 = GAUGE32: 0

.1.3.6.1.2.1.2.2.1.21.2 = GAUGE32: 0

.1.3.6.1.2.1.2.2.1.22.1 = OID: 0.0

.1.3.6.1.2.1.2.2.1.22.2 = OID: 0.0

.1.3.6.1.2.1.3.1.1.1.2.1.10.8.227.53 = INTEGER: 2

.1.3.6.1.2.1.3.1.1.1.2.1.10.8.227.62 = INTEGER: 2

.1.3.6.1.2.1.3.1.1.2.2.1.10.8.227.53 = HEX-STRING: 005056B73237

.1.3.6.1.2.1.3.1.1.2.2.1.10.8.227.62 = HEX-STRING: B0FAEB566CC1

.1.3.6.1.2.1.3.1.1.3.2.1.10.8.227.53 = IPADDRESS: 10.8.227.53

.1.3.6.1.2.1.3.1.1.3.2.1.10.8.227.62 = IPADDRESS: 10.8.227.62

.1.3.6.1.2.1.4.1.0 = INTEGER: 2

.1.3.6.1.2.1.4.2.0 = INTEGER: 64

.1.3.6.1.2.1.4.3.0 = COUNTER32: 18615701

.1.3.6.1.2.1.4.4.0 = COUNTER32: 0

.1.3.6.1.2.1.4.5.0 = COUNTER32: 88698

.1.3.6.1.2.1.4.6.0 = COUNTER32: 0

.1.3.6.1.2.1.4.7.0 = COUNTER32: 0

.1.3.6.1.2.1.4.8.0 = COUNTER32: 0

.1.3.6.1.2.1.4.9.0 = COUNTER32: 18526999

.1.3.6.1.2.1.4.10.0 = COUNTER32: 18853234

.1.3.6.1.2.1.4.11.0 = COUNTER32: 6

.1.3.6.1.2.1.4.12.0 = COUNTER32: 0

.1.3.6.1.2.1.4.13.0 = INTEGER: 30

.1.3.6.1.2.1.4.14.0 = COUNTER32: 6

.1.3.6.1.2.1.4.15.0 = COUNTER32: 3

.1.3.6.1.2.1.4.16.0 = COUNTER32: 0

.1.3.6.1.2.1.4.17.0 = COUNTER32: 1184

.1.3.6.1.2.1.4.18.0 = COUNTER32: 0

.1.3.6.1.2.1.4.19.0 = COUNTER32: 2368

.1.3.6.1.2.1.4.20.1.1.10.8.227.50 = IPADDRESS: 10.8.227.50

.1.3.6.1.2.1.4.20.1.1.127.0.0.1 = IPADDRESS: 127.0.0.1

.1.3.6.1.2.1.4.20.1.2.10.8.227.50 = INTEGER: 2

.1.3.6.1.2.1.4.20.1.2.127.0.0.1 = INTEGER: 1

.1.3.6.1.2.1.4.20.1.3.10.8.227.50 = IPADDRESS: 255.255.255.192

.1.3.6.1.2.1.4.20.1.3.127.0.0.1 = IPADDRESS: 255.0.0.0

.1.3.6.1.2.1.4.20.1.4.10.8.227.50 = INTEGER: 1

.1.3.6.1.2.1.4.20.1.4.127.0.0.1 = INTEGER: 0

.1.3.6.1.2.1.4.21.1.1.0.0.0.0 = IPADDRESS: 0.0.0.0

.1.3.6.1.2.1.4.21.1.1.10.8.227.0 = IPADDRESS: 10.8.227.0

.1.3.6.1.2.1.4.21.1.1.169.254.0.0 = IPADDRESS: 169.254.0.0

.1.3.6.1.2.1.4.21.1.2.0.0.0.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.2.10.8.227.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.2.169.254.0.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.3.0.0.0.0 = INTEGER: 1

.1.3.6.1.2.1.4.21.1.3.10.8.227.0 = INTEGER: 0

.1.3.6.1.2.1.4.21.1.3.169.254.0.0 = INTEGER: 0

.1.3.6.1.2.1.4.21.1.7.0.0.0.0 = IPADDRESS: 10.8.227.62

.1.3.6.1.2.1.4.21.1.7.10.8.227.0 = IPADDRESS: 0.0.0.0

.1.3.6.1.2.1.4.21.1.7.169.254.0.0 = IPADDRESS: 0.0.0.0

.1.3.6.1.2.1.4.21.1.8.0.0.0.0 = INTEGER: 4

.1.3.6.1.2.1.4.21.1.8.10.8.227.0 = INTEGER: 3

.1.3.6.1.2.1.4.21.1.8.169.254.0.0 = INTEGER: 3

.1.3.6.1.2.1.4.21.1.9.0.0.0.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.9.10.8.227.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.9.169.254.0.0 = INTEGER: 2

.1.3.6.1.2.1.4.21.1.11.0.0.0.0 = IPADDRESS: 0.0.0.0

.1.3.6.1.2.1.4.21.1.11.10.8.227.0 = IPADDRESS: 255.255.255.192

.1.3.6.1.2.1.4.21.1.11.169.254.0.0 = IPADDRESS: 255.255.0.0

.1.3.6.1.2.1.4.21.1.13.0.0.0.0 = OID: 0.0

.1.3.6.1.2.1.4.21.1.13.10.8.227.0 = OID: 0.0

.1.3.6.1.2.1.4.21.1.13.169.254.0.0 = OID: 0.0

.1.3.6.1.2.1.4.22.1.1.2.10.8.227.53 = INTEGER: 2

Below is example of some MIBs related to CPU, Memory, Disks, list of running processes and list of installed packages details:

CPU MIBs

.1.3.6.1.2.1.25.3.2.1.3.768 = STRING: "GenuineIntel: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"

.1.3.6.1.2.1.25.3.2.1.3.769 = STRING: "GenuineIntel: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"

.1.3.6.1.2.1.25.3.2.1.3.770 = STRING: "GenuineIntel: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"

.1.3.6.1.2.1.25.3.2.1.3.771 = STRING: "GenuineIntel: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz"

.1.3.6.1.4.1.2021.10.1.2.1 = STRING: "Load-1"

.1.3.6.1.4.1.2021.10.1.2.2 = STRING: "Load-5"

.1.3.6.1.4.1.2021.10.1.2.3 = STRING: "Load-15"

Memory MIBs

.1.3.6.1.2.1.25.2.3.1.3.1 = STRING: "Physical memory".1.3.6.1.2.1.25.2.3.1.3.3 = STRING: "Virtual memory".1.3.6.1.2.1.25.2.3.1.3.6 = STRING: "Memory buffers".1.3.6.1.2.1.25.2.3.1.3.7 = STRING: "Cached memory".1.3.6.1.2.1.25.2.3.1.3.10 = STRING: "Swap space"

Disk MIBs

.1.3.6.1.2.1.25.3.2.1.3.1552 = STRING: "SCSI disk (/dev/sda)"

.1.3.6.1.2.1.25.3.2.1.3.1553 = STRING: "SCSI disk (/dev/sdb)"

.1.3.6.1.2.1.25.3.2.1.3.1554 = STRING: "SCSI disk (/dev/sdc)"

.1.3.6.1.2.1.25.3.2.1.3.1600 = STRING: "LVM volume (vg00-lvvar)"

.1.3.6.1.2.1.25.3.2.1.3.1616 = STRING: "LVM volume (vg00-lvusr)"

.1.3.6.1.2.1.25.3.2.1.3.1632 = STRING: "LVM volume (vg00-lvhome)"

.1.3.6.1.2.1.25.3.2.1.3.1648 = STRING: "LVM volume (vg00-lvopt)"

.1.3.6.1.2.1.25.3.2.1.3.1664 = STRING: "LVM volume (vg00-lvtmp)"

.1.3.6.1.2.1.25.3.2.1.3.1680 = STRING: "LVM volume (vgerp-lvtrans)"

.1.3.6.1.2.1.25.3.2.1.3.1696 = STRING: "LVM volume (vgerp-interfacelv)"

.1.3.6.1.2.1.25.3.2.1.3.1712 = STRING: "LVM volume (vgdata-lvsaplog)"

.1.3.6.1.2.1.25.3.2.1.3.1728 = STRING: "LVM volume (vgdata-lvsapvol)"

.1.3.6.1.2.1.25.3.2.1.3.1744 = STRING: "LVM volume (vgdata-lvsapdata)"

.1.3.6.1.2.1.25.3.2.1.3.1760 = STRING: "LVM volume (vg00-lvswap)"

.1.3.6.1.2.1.25.3.2.1.3.1776 = STRING: "LVM volume (vg00-lvroot)"

.1.3.6.1.2.1.25.2.3.1.3.1 = STRING: "Physical memory"

.1.3.6.1.2.1.25.2.3.1.3.3 = STRING: "Virtual memory"

.1.3.6.1.2.1.25.2.3.1.3.6 = STRING: "Memory buffers"

.1.3.6.1.2.1.25.2.3.1.3.7 = STRING: "Cached memory"

.1.3.6.1.2.1.25.2.3.1.3.10 = STRING: "Swap space"

.1.3.6.1.2.1.25.2.3.1.3.31 = STRING: "/"

.1.3.6.1.2.1.25.2.3.1.3.35 = STRING: "/dev/shm"

.1.3.6.1.2.1.25.2.3.1.3.36 = STRING: "/boot"

.1.3.6.1.2.1.25.2.3.1.3.37 = STRING: "/home"

.1.3.6.1.2.1.25.2.3.1.3.38 = STRING: "/opt"

.1.3.6.1.2.1.25.2.3.1.3.39 = STRING: "/usr"

.1.3.6.1.2.1.25.2.3.1.3.40 = STRING: "/var"

.1.3.6.1.2.1.25.2.3.1.3.41 = STRING: "/sap"

.1.3.6.1.2.1.25.2.3.1.3.42 = STRING: "/sapdb"

.1.3.6.1.2.1.25.2.3.1.3.43 = STRING: "/sapmnt"

.1.3.6.1.2.1.25.2.3.1.3.44 = STRING: "/usr/sap"

.1.3.6.1.2.1.25.2.3.1.3.45 = STRING: "/sapdb/ZQ1/saplog"

.1.3.6.1.2.1.25.2.3.1.3.46 = STRING: "/sapdb/ZQ1/sapdata"

.1.3.6.1.2.1.25.2.3.1.3.47 = STRING: "/usr/sap/trans"

.1.3.6.1.2.1.25.2.3.1.3.48 = STRING: "/usr/sap/interfaces"

.1.3.6.1.2.1.25.2.3.1.3.51 = STRING: "/mnt/infra_Exchange"

.1.3.6.1.2.1.25.2.3.1.3.52 = STRING: "/home/ConcurPGP"

.1.3.6.1.2.1.25.3.8.1.3.21 = STRING: "//sp01/SharedDrives/Infra_Exchange"

.1.3.6.1.2.1.25.3.8.1.3.22 = STRING: "//sp01/shareddrives/concur_pgp"

Running Processes MIBs

We are able to monitor running process via SNMP:

.1.3.6.1.2.1.25.4.2.1.4.1 = STRING: "/sbin/init"

.1.3.6.1.2.1.25.4.2.1.4.2 = STRING: "kthreadd"

.1.3.6.1.2.1.25.4.2.1.4.3 = STRING: "migration/0"

.1.3.6.1.2.1.25.4.2.1.4.4 = STRING: "ksoftirqd/0"

.1.3.6.1.2.1.25.4.2.1.4.5 = STRING: "migration/0"

.1.3.6.1.2.1.25.4.2.1.4.6 = STRING: "watchdog/0"

.1.3.6.1.2.1.25.4.2.1.4.7 = STRING: "migration/1"

.1.3.6.1.2.1.25.4.2.1.4.8 = STRING: "migration/1"

.1.3.6.1.2.1.25.4.2.1.4.9 = STRING: "ksoftirqd/1"

.1.3.6.1.2.1.25.4.2.1.4.10 = STRING: "watchdog/1"

.1.3.6.1.2.1.25.4.2.1.4.11 = STRING: "migration/2"

.1.3.6.1.2.1.25.4.2.1.4.12 = STRING: "migration/2"

.1.3.6.1.2.1.25.4.2.1.4.13 = STRING: "ksoftirqd/2"

.1.3.6.1.2.1.25.4.2.1.4.14 = STRING: "watchdog/2"

.1.3.6.1.2.1.25.4.2.1.4.15 = STRING: "migration/3"

.1.3.6.1.2.1.25.4.2.1.4.16 = STRING: "migration/3"

.1.3.6.1.2.1.25.4.2.1.4.17 = STRING: "ksoftirqd/3"

.1.3.6.1.2.1.25.4.2.1.4.18 = STRING: "watchdog/3"

.1.3.6.1.2.1.25.4.2.1.4.112 = STRING: "kstriped"

.1.3.6.1.2.1.25.4.2.1.4.208 = STRING: "scsi_eh_0"

.1.3.6.1.2.1.25.4.2.1.4.22229 = STRING: "/usr/sap/hostctrl/exe/saphostexec"

.1.3.6.1.2.1.25.4.2.1.4.22247 = STRING: "/usr/sap/hostctrl/exe/sapstartsrv"

.1.3.6.1.2.1.25.4.2.1.4.22328 = STRING: "dw.sap_DVEBMGS00"

.1.3.6.1.2.1.25.4.2.1.4.22467 = STRING: "/usr/sap/ZQ1/ASCS01/exe/sapstartsrv"

.1.3.6.1.2.1.25.4.2.1.4.22680 = STRING: "/usr/sap/ZQ1/DVEBMGS00/exe/sapstartsrv"

.1.3.6.1.2.1.25.4.2.1.4.22776 = STRING: "flush-253:9"

.1.3.6.1.2.1.25.4.2.1.4.23546 = STRING: "flush-253:3"

.1.3.6.1.2.1.25.4.2.1.4.24299 = STRING: "/usr/sap/hostctrl/exe/saposcol"

"/sapdb/programs/pgm/niserver"

Installed packages MIBS

SNMP gives us the list of installed packages in the servers:

.1.3.6.1.2.1.25.6.3.1.2.1 = STRING: "lapack-3.2.1-4.el6"

.1.3.6.1.2.1.25.6.3.1.2.2 = STRING: "PackageKit-gstreamer-plugin-0.5.8-21.el6"

.1.3.6.1.2.1.25.6.3.1.2.3 = STRING: "pnm2ppa-1.04-19.el6"

.1.3.6.1.2.1.25.6.3.1.2.4 = STRING: "system-config-firewall-1.2.27-5.el6"

.1.3.6.1.2.1.25.6.3.1.2.5 = STRING: "xorg-x11-drv-openchrome-0.3.0-3.20120806git.el6"

.1.3.6.1.2.1.25.6.3.1.2.6 = STRING: "cdrdao-1.2.3-4.el6"

.1.3.6.1.2.1.25.6.3.1.2.7 = STRING: "libcurl-7.19.7-37.el6_4"

.1.3.6.1.2.1.25.6.3.1.2.8 = STRING: "basesystem-10.0-4.el6"

.1.3.6.1.2.1.25.6.3.1.2.9 = STRING: "krb5-pkinit-openssl-1.10.3-10.el6_4.6"

.1.3.6.1.2.1.25.6.3.1.2.10 = STRING: "mingetty-1.08-5.el6"

.1.3.6.1.2.1.25.6.3.1.2.11 = STRING: "xorg-x11-drv-fpit-1.4.0-5.el6"

.1.3.6.1.2.1.25.6.3.1.2.12 = STRING: "python-rtslib-2.1.fb21-3.el6_4"

.1.3.6.1.2.1.25.6.3.1.2.13 = STRING: "gstreamer-plugins-bad-free-0.10.19-2.el6"

.1.3.6.1.2.1.25.6.3.1.2.14 = STRING: "gnome-power-manager-2.28.3-7.el6_4"

.1.3.6.1.2.1.25.6.3.1.2.15 = STRING: "libwmf-lite-0.2.8.4-22.el6"

.1.3.6.1.2.1.25.6.3.1.2.16 = STRING: "ntpdate-4.2.6p5-1.el6"

.1.3.6.1.2.1.25.6.3.1.2.17 = STRING: "pth-2.0.7-9.3.el6"

.1.3.6.1.2.1.25.6.3.1.2.18 = STRING: "PackageKit-yum-plugin-0.5.8-21.el6"

.1.3.6.1.2.1.25.6.3.1.2.19 = STRING: "libusb-0.1.12-23.el6"

.1.3.6.1.2.1.25.6.3.1.2.20 = STRING: "gnome-utils-libs-2.28.1-10.el6"

.1.3.6.1.2.1.25.6.3.1.2.21 = STRING: "kernel-firmware-2.6.32-431.el6"

.1.3.6.1.2.1.25.6.3.1.2.22 = STRING: "jasper-1.900.1-15.el6_1.1"

.1.3.6.1.2.1.25.6.3.1.2.23 = STRING: "ustr-1.0.4-9.1.el6"

.1.3.6.1.2.1.25.6.3.1.2.24 = STRING: "amanda-client-2.6.1p2-8.el6"

.1.3.6.1.2.1.25.6.3.1.2.25 = STRING: "abrt-libs-2.0.8-21.el6"

.1.3.6.1.2.1.25.6.3.1.2.26 = STRING: "nss-softokn-freebl-3.14.3-9.el6"

.1.3.6.1.2.1.25.6.3.1.2.27 = STRING: "libatasmart-0.17-4.el6_2"

Configuration

Following changes need to be done in a standard snmpd.conf file to make it produce all monitoring details.

# vim /etc/snmp/snmpd.conf

In the following section, comment out first 2 lines started with "view"and add the third line to your config file:

# Make at least snmpwalk -v 1 localhost -c public system fast again.

# name incl/excl subtree mask(optional)

--> comment out #view systemview included .1.3.6.1.2.1.1

--> comment out #view systemview included .1.3.6.1.2.1.25.1.1

--> Add view all included .1 80

Before:

view systemview included .1.3.6.1.2.1.1

view systemview included .1.3.6.1.2.1.25.1.1

After:

#view systemview included .1.3.6.1.2.1.1

#view systemview included .1.3.6.1.2.1.25.1.1

view all included .1 80

And in the following section, change the "systemview" to "all" in the line starts with "access":

####

# Finally, grant the group read-only access to the systemview view.

# group context sec.model sec.level prefix read write notif

--> access notConfigGroup "" any noauth exact all none none

Before:

access notConfigGroup "" any noauth exact systemview none none

After:

access notConfigGroup "" any noauth exact all none none

and finally restart the snmpd daemon for new changes to take effect:

# /etc/init.d/snmpd restart

Stopping snmpd: [ OK ]

Starting snmpd: [ OK ]

DevOps Sandbox

Pages

21 September 2015

A puppet manifest with some challenging requirements to do the proper modification in hundreds of servers in one shot!

Requirements

Manifest

7 September 2015

SNMP Configuration Full Monitoring