Free performance monitoring and capacity planning for IBM Power™ platform

Installation

Prerequisites

  • LPAR2RRD can be hosted on any Unix or even Windows server, so far it has been tested on AIX 6.1/5.3/5.2, SuSE Linux, RedHad and Ubuntu on Intel
  • it is not necessary to have a dedicated box for it It is a good idea to use some management server like NIM server
  • disk space requirements : each monitored LPAR needs roughly 2MB of the disk space, if you use dual HMC setup then 4MB (since 3.10 with new retention 8MB)
  • Perl : arbitrary version comming with the OS (tested with 5.8.0, 5.8.2), it just requires TimeDate module (installation procedure is below)
  • Apache or any other Web server
  • SSH : OpenSSH or any other commercial SSH (Tested with OpenSSH & F-Secure SSH)
  • RRDTool, tested with 1.0.33, 1.0.42 and 1.2.15 , generally should work with all versions (on Linux use 1.2.27 at least)
  • Power5 servers need to be at least at SF240_201 firmware level, since utilization data collection is supported since that version. (all POWER6/7 are ok)
Download page contains links to required packages.

Installation procedure

  • install RRDTool, here are download links even with its prerequisites
  • create preferably a dedicated user (lpar2rrd) on a server which will own the tool and will have access to HMC/IVM servers via ssh-keys
  • add a user on Unix under root account 
    • # useradd -m lpar2rrd
  • increase memory limits for user lpar2rrd (memory to 512MB, stack to 256MB) and web user, all under root 
    • # chuser  data=1048576 lpar2rrd
# chuser  stack=524288 lpar2rrd
# chuser  data=524288  nobody 	# assure it is really nobody user: 
# chuser  stack=262144 nobody   # ps -ef|egrep -i "httpd|apache"
  • allow utilization data collection (either via HMC GUI or globally via following cmd as hscroot) 
    • # ssh hmc1 -l hscroot 
hscroot@hmc1:~> chlparutil -r config -s 60
hscroot@hmc1:~> lslparutil -r config  -F name,sample_rate
PWR6A-9117-MMA-SN103A4B0,60
PWR6B-9117-MMA-SN103B5C0,60
  • Use 3600 for HMC older than 7.3.3
    use padmin account for IVM, or sysadmin for SDMC
  • create a HMC account with role hmcviewer on the HMC server and allow ssh-keys access from a LPAR2RRD hosted server login to HMC as hscroot
    • # ssh -l hscroot hmc1
$ mkhmcusr -u lpar2rrd -a hmcviewer --passwd abc1234
  • IVM: under padmin account with role "view":
    • mkuser -attr pgrp=view lpar2rrd
  • SDMC: under sysadmin account with role "SMMonitor":
    • smcli mkuser -u lpar2rrd -g smmon -p abc1234
smcli chuser -u lpar2rrd -e expire 9999 # do not let it expire
  • create the user ssh-key on a LPAR2RRD hosted server for lpar2rrd user: 
    • $ su - lpar2rrd
$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/lpar2rrd/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/lpar2rrd/.ssh/id_dsa.
Your public key has been saved in /home/lpar2rrd/.ssh/id_dsa.pub.
The key fingerprint is:
c0:05:85:ff:9c:90:39:da:d7:48:33:35:4f:cf:1a:21 lpar2rrd@p550-2  
$ cat /home/lpar2rrd/.ssh/id_dsa.pub 
'ssh-dss AAAAB3NzaC1kc3MAAACBALvbupPLnqy6UfJjgMG5SRnnFDfD013OtBxFv8P7qoHfGKCG0Vu2
IhNCYttpYMmsMR+BWADo4c9oT7r92raLfVDjNW8uO5C5fatK305+sNqazbT91HDDNDKQnbpoKUqybVRC2
BQbPR8ESh+ws3uHdxXSQKwOSyrHO6Nwkkx8/h4TAAAAFQDZgT9MxrEUQg4uOwxhciwwdoOO8QAAAIAH37
x06Ia4FiCdlk3U9vuOI0QEvCKMXL+ZPFNoRcgiqiCnR2WeiaG5qM+odYWk/F/owV0Y/DmkmrrjPrgDADj
S1uKoSy+NMg803+4cS1B06EMtia+RmMbWkr+kQZ90WvB8C5cxofzcllnQjztQxVrZnZPmMI73/SQdS7QB
U9a9WwAAAIA3VAdMvP+ZO3zs57FaznMySoszxgNWxuINlcP61TQuyuCp5Bq3mFXwxFwJWPpqpbW5yHj9N
1+3pscSsFxEQb9YVvb1oL9c5QdPDnmXZK8BHHmNZMn+ftuzJKHgR2lynaxQ4hxTnz+xT0ywGwokeuuCnr
4G3Bftc70Yn9vCNjuzGQ== lpar2rrd@p550-2'
  • paste the pub key just created to mkauthkeys command on HMC (as hscroot, all in one line, without carriage returns):
    • hscroot@hmc1:~> mkauthkeys -u lpar2rrd -a 'ssh-dss AAAAB3NzaC1kc3MAAACBALvbupPL
nqy6UfJjgMG5SRnnFDfD013OtBxFv8P7qoHfGKCG0Vu2IhNCYttpYMmsMR+BWADo4c9oT7r92raLfVDjNW
8uO5C5fatK305+sNqazbT91HDDNDKQnbpoKUqybVRC2BQbPR8ESh+ws3uHdxXSQKwOSyrHO6Nwkkx8/h4T
AAAAFQDZgT9MxrEUQg4uOwxhciwwdoOO8QAAAIAH37x06Ia4FiCdlk3U9vuOI0QEvCKMXL+ZPFNoRcgiqi
CnR2WeiaG5qM+odYWk/F/owV0Y/DmkmrrjPrgDADjS1uKoSy+NMg803+4cS1B06EMtia+RmMbWkr+kQZ90
WvB8C5cxofzcllnQjztQxVrZnZPmMI73/SQdS7QBU9a9WwAAAIA3VAdMvP+ZO3zs57FaznMySoszxgNWxu
INlcP61TQuyuCp5Bq3mFXwxFwJWPpqpbW5yHj9N1+3pscSsFxEQb9YVvb1oL9c5QdPDnmXZK8BHHmNZ{{M
n+ftuzJKHgR2lynaxQ4hxTnz+xT0ywGwokeuuCnr4G3Bftc70Yn9vCNjuzGQ== lpar2rrd@p550-2'
  • on SDMC same as above only under sysadmin account
  • On IVM systems you need at first log in as lpar2rrd user an then:
    • hscroot@hmc1:~> mkauthkeys -a 'ssh-dss AAAAB3NzaC1kc3MAAACBALvbupPLnqy6UfJjgMG5SRnn
FDfD013OtBxFv8P7qoHfGKCG0Vu2IhNCYttpYMmsMR+BWADo4c9oT7r92raLfVDjNW8uO5C5fatK305+sNq
azbT91HDDNDKQnbpoKUqybVRC2BQbPR8ESh+ws3uHdxXSQKwOSyrHO6Nwkkx8/h4TAAAAFQDZgT9MxrEUQg
4uOwxhciwwdoOO8QAAAIAH37x06Ia4FiCdlk3U9vuOI0QEvCKMXL+ZPFNoRcgiqiCnR2WeiaG5qM+odYWk/
F/owV0Y/DmkmrrjPrgDADjS1uKoSy+NMg803+4cS1B06EMtia+RmMbWkr+kQZ90WvB8C5cxofzcllnQjztQ
xVrZnZPmMI73/SQdS7QBU9a9WwAAAIA3VAdMvP+ZO3zs57FaznMySoszxgNWxuINlcP61TQuyuCp5Bq3mFX
wxFwJWPpqpbW5yHj9N1+3pscSsFxEQb9YVvb1oL9c5QdPDnmXZK8BHHmNZ{{Mn+ftuzJKHgR2lynaxQ4hxT
nz+xT0ywGwokeuuCnr4G3Bftc70Yn9vCNjuzGQ== lpar2rrd@p550-2'
  • test the connection between the LPAR2RRD hosted server and HMC (should not request password any more) as lpar2rrd user:
    • $ ssh -l lpar2rrd hmc1
The authenticity of host 'hmc1 (9.138.236.97)' can′t be established.
RSA key fingerprint is 97:e1:a5:c2:58:3c:c6:0c:d1:6f:4d:6e:62:9c:68:4b.
Are you sure you want to continue connecting (yes/no)? yes
hmc1:~>
  • when you use a commercial SSH distribution (like F-Secure ssh) which supports only SSH2 format (above example was for OpenSSH format), then you need to convert public keys onto OpenSSH format before uploading the keys into HMC. Convert SSH2 to OpenSSH format and after that use it in mkauthkeys: 
    • $ ssh-keygen -i -f your-ssh2.pub > your-openssh.pub
  • install RRDTool prerequisites : freetype2, libart_lgpl, libpng, zlib as root 
    • # rpm -ivh freetype2-2.1.7-5.aix5.1.ppc.rpm libart_lgpl-2.3.17-4.aix5.1.ppc.rpm
         libpng-1.2.8-6.aix5.1.ppc.rpm zlib-1.2.3-4.aix5.1.ppc.rpm
  • install RRDTool as root
    • # rpm -ivh rrdtool-1.2.13-1.perl58.aix5.2.ppc.rpm
  • make sure RRDTool is compiled with Perl support and the version of Perl is the right one if not and LPAR2RRD complains that it cannot find RRDp.pm then adjust PERL5LIB parameter in lpar2rrd.cfg after installation (ignore it now, just do it in case of an error)
    • # ls -l /opt/freeware/lib/perl/5.8.0
total 16
-r--r--r--   1 root     system         5048 May 04 2006  RRDp.pm
drwxr-xr-x   3 root     system          256 Jan 23 2007  aix-thread-multi
  • when RRD.pm is not there then you will need to change default PERL5LIB setting in lpar2rrd.cfg afterwards to point to the right directory
  • install Perl as root 
    • # rpm -ivh perl-5.8.2-1.aix5.1.ppc.rpm
  • install TimeDate Perl module as root
    test whether is already installed or not:
    • # perl -MDate::Parse -le 'print (str2time("12/24/2006"))'
  1166914800
  • means it is installed, if there is no output or an error then install it as per below
    • # umask 022 	# just to prevent permission problems ...
# gzip -d TimeDate-1.16.tar.gz; tar xvf  TimeDate-1.16.tar; cd  TimeDate-1.16
# perl ./Makefile.PL
# make test
# make install
  • It sometimes happens that the module is installed with read/write rights only for root, so no one else can use it (it is related to umask setting).
    • # ls -ld /usr/opt/perl5/lib/site_perl/5.8.2/Date
drwxr-----  3 root  system  256 Nov 24 2006  /usr/opt/perl5/lib/site_perl/5.8.2/Date
  • fix:
    • # chmod 755 /usr/opt/perl5/lib/site_perl/5.8.8/Date/Language
# chmod 755 /usr/opt/perl5/lib/site_perl/5.8.8/Date/
# find /usr/opt/perl5/lib/site_perl/5.8.8/Date -exec chmod o+r {} \;
  • Assure you have Perl module XML::Simple in place
    AIX distributions contain it, Linux usualy not
    • # perl -MXML::Simple -le 'XMLin(" ")'
  File does not exist:  at -e line 1
    This output is ok, the module is in place.
    If you get "Can't locate XML/Simple.pm in @INC" then yo have 2 options:
    • You do not need CSV output from historical reports, then set EXPORT_TO_CSV=0 in etc/lpar2rrd.cfg later on
    • You need CSV output. Then install XML::Simple and its dependeces from CPAN (note there could be quite a lot of deps)
  • install and configure Apache or any other web server as root 
    • # rpm -ivh expat-1.95.7-4.aix5.1.ppc.rpm apache-1.3.31-1.aix5.1.ppc.rpm
# vi /etc/opt/freeware/apache/httpd.conf
    • it might be in a different location like /opt/freeware/etc/httpd/httpd.conf, /etc/httpd/httpd.conf, /etc/httpd/conf/httpd.conf ...)
    • append at the end following: 
      • # NB: The "Alias" line below must NOT be used if you have
#     the LPAR2RRD webfiles as the root URL. In that case,
#     you should instead set this:
#
#          DocumentRoot /home/lpar2rrd/lpar2rrd/www/
Alias /lpar2rrd/  "/home/lpar2rrd/lpar2rrd/www/"
<Directory "/home/lpar2rrd/lpar2rrd/www/">
	Options Indexes FollowSymLinks Includes MultiViews
    Order allow,deny
    Allow from all
</Directory>

# CGI-BIN
ScriptAlias /lpar2rrd-cgi/ "/home/lpar2rrd/lpar2rrd/lpar2rrd-cgi/"
<Directory "/home/lpar2rrd/lpar2rrd/lpar2rrd-cgi">
    AllowOverride None
    Options ExecCGI Includes FollowSymLinks
    Order allow,deny
    Allow from all
</Directory>

# /opt/freeware/apache/sbin/apachectl start
  • If you append that exactly, then you will find LPAR2RRD here : http://your_web_server/lpar2rrd/
    Do not forget to ensure that Apache starts after the OS reboot (there must be a starting script in /etc/rc.... )
  • install LPAR2RRD
    • # su - lpar2rrd
$ tar xvf lpar2rrd-3.XX.tar
$ cd lpar2rrd-3.XX
$ ./install.sh
$ cd /home/lpar2rrd/lpar2rrd
  • configure LPAR2RRD parameters in lpar2rrd.cfg, install process should do most of that automaticaly
    Here is the list of parameters which need to be reviewed: 
    • $ vi /home/lpar2rrd/lpar2rrd/etc/lpar2rrd.cfg  
WEBDIR=/home/lpar2rrd/lpar2rrd/www  
HMC_USER=lpar2rrd  
HMC_LIST="hmc1 sdmc1 ivm1"  
PERL=/usr/bin/perl  
RRDTOOL=/opt/freeware/bin/rrdtool
  • Now you might check sample rates of utilization data collection on servers attached to configured HMCs (must not be 0!) 
    • $ cd ./bin
$ ./sample_rate.sh  
Going to check HMC as user lpar2rrd
hmc1:PWR6A-9117-MMA-SN103A4B0 sample rate : 60, OK  
hmc1:PWR6B-9117-MMA-SN103B5C0 sample rate : 60, OK
  • To allow "real-time" refresh on demand, the WEB user (apache, nobody, ..) needs to have access to LPAR2RRD ssh keys
    • find out WEB user
    • $ ps -ef|egrep "apache|httpd"|grep -v grep|awk '{print $1}'|grep -v "root"|head -1 
nobody
    • under root user copy ssh keys and change ownership to the WEB user
    • # cp /home/lpar2rrd/.ssh/id_dsa /home/lpar2rrd/.ssh/realt_dsa  
# chown nobody /home/lpar2rrd/.ssh/realt_dsa  
# chmod 600 /home/lpar2rrd/.ssh/realt_dsa
  • Assure you have enough of disk space on the fs where is LPAR2RRD installed
    Roughly you might count 5MB/lpar (10MB/lpar in dual HMC setup) == 1GB/100lpars 
    df -g /home
  • now, you can start the tool itself
    Note that the first run might take a while especially if there is already allowed utilization data collection for a long time
    • $ ./load.sh
  • check graphs via web
    point the web browser on the page which was chosen as the target one http://your_web_server/lpar2rrd/
  • schedule to run LPAR2RRD from lpar2rrd crontab
    once an hour seems to be reasonable time, do not use less as LPAR2RD might run for 1/2 an hour if you have a big environment 
    • $ crontab -e  
0 * * * * /home/lpar2rrd/lpar2rrd/load.sh > /home/lpar2rrd/lpar2rrd/load.out 2>&1 
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/lpar2rrd/lpar2rrd/load_hea.sh >   	
		/home/lpar2rrd/lpar2rrd/load_hea.out 2>&1
  • You might need to add lpar2rrd user into /var/adm/cron/cron.allow if crontab command fails as root user.
    • # echo "lpar2rrd" >> /var/adm/cron/cron.allow
  • If you have any problems with Historical reports, Refresh or CPU Workload Estimator then check:
    • /var/opt/freeware/apache/logs/error_log # Apache error log
    • /var/opt/freeware/apache/logs/access_log # Apache access log
    • /home/lpar2rrd/lpar2rrd/logs/error-cgi.log # LPAR2RRD CGI-BIN log

follow link for upgrade steps »