X11 forwarding over SSH on firewalled CentOS host

I had a few issues with X11 forwarding over SSH on one of my CentOS hosts. After a bit of fiddling, I discovered that there were a couple of things I hadn’t taken into account.

I’d set my putty session up to allow X11 fowarding, and set the X display location to “localhost”. On the server, I installed xclock and its dependencies for testing, and set the following in /etc/ssh/sshd_config:

X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost yes

I restarted sshd, however this still wasn’t working.

In short, I was missing two things:

1) xauth wasn’t installed. This is required!
2) I wasn’t allowing connections to localhost in my iptables config. This was fixed in my ruleset with:

iptables -A INPUT -i lo -j ACCEPT

sshd was restarted after installing xauth and adding the firewall rule and it now works a treat!

Building Smokeping on CentOS and running without root permissions.

I needed smokeping to run some tests, but every guide online alluded to using root for the application user which is really not ideal. I’m still not happy about setuid for fping, but as long as you provide at least basic authentication on the web front end, it shouldn’t be too much of a problem when it comes to audit.

The sections mention swapping between smokeping and root users. This will typically involve a lot of ctrl + D and sudo su – commands. Use common sense in the procedure below for this and use the “id” command to check who you are effectively logged in as. I know you can chown -R but I chose to use the actual user and then lock it down once finished to avoid confusing permission oversights. :p

I have run through this procedure myself and it works on a CentOS minimal install. There was a frustrating issue I had with getting echoping built with ssl support. Running configure and running into library errors usually points to development versions not being installed, NOT the normal, user libraries. In this case, openssl-devel and popt-devel. Thanks to my good friend Cody for pointing this out before I went crazy! :D

smokeping install – CentOS 6
With credit to WeDebugYou.com for the base on which I built this procedure.


# 1) Create smokeping non-priveleged user

sudo su -
useradd -m -s /bin/bash smokeping

# 2) Install prerequisite packages

yum install wget
cd /tmp
wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
rpm -Uvh rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm

# Required
yum install mod_fcgid httpd httpd-devel rrdtool fping wget curl bind-utils gcc make  
yum install perl perl-Net-Telnet perl-Net-DNS perl-LDAP perl-libwww-perl perl-RadiusPerl 
yum install perl-IO-Socket-SSL perl-Socket6 perl-CGI-SpeedyCGI perl-FCGI perl-RRD-Simple
yum install perl-CGI-SpeedyCGI perl-ExtUtils-MakeMaker


# 3) Install smokeping

cd /home/smokeping
mkdir smokeping
chown smokeping:apache smokeping	# Allow apache to read the dir later
chmod 750 smokeping			# Allow apache to read the dir later
su - smokeping
wget http://oss.oetiker.ch/smokeping/pub/smokeping-2.6.9.tar.gz

tar -zxvf smokeping-2.6.9.tar.gz -C /home/smokeping
cd smokeping-2.6.9/setup
./build-perl-modules.sh

cp -r ../thirdparty /home/smokeping/smokeping
cd ..
./configure --prefix=/home/smokeping/smokeping
make install


Create missing folders
cd /home/smokeping/smokeping
mkdir data var cache


# 4) Add startup script

[go back to root with ctrl + d ]

# Copy and paste from the line below into /etc/init.d/smokeping

#!/bin/sh
#
# smokeping    This starts and stops the smokeping daemon
# chkconfig: 345 98 11
# description: Start/Stop the smokeping daemon
# processname: smokeping
# Source function library.
. /etc/rc.d/init.d/functions

SMOKEPING=/home/smokeping/smokeping/bin/smokeping
SMOKEPINGUSER=smokeping
LOCKF=/var/lock/subsys/smokeping
CONFIG=/home/smokeping/smokeping/etc/config

[ -f $SMOKEPING ] || exit 0
[ -f $CONFIG ] || exit 0

RETVAL=0

case "$1" in
  start)
        echo -n $"Starting SMOKEPING: "
        daemon --user $SMOKEPINGUSER $SMOKEPING
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch $LOCKF
        ;;
  stop)
        echo -n $"Stopping SMOKEPING: "
        killproc $SMOKEPING
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && rm -f $LOCKF
        ;;
  status)
        status smokeping
        RETVAL=$?
        ;;
  reload)
        echo -n $"Reloading SMOKEPING: "
        killproc $SMOKEPING -HUP
        RETVAL=$?
        echo
        ;;
  restart)
        $0 stop
        sleep 3
        $0 start
        RETVAL=$?
        ;;
  condrestart)
        if [ -f $LOCKF ]; then
                $0 stop
                sleep 3
                $0 start
                RETVAL=$?
        fi
        ;;
  *)
        echo $"Usage: $0 {start|stop|status|restart|reload|condrestart}"
        exit 1
esac

# Stop copying!

chmod 755 /etc/init.d/smokeping


# 5) Back to smokeping user to set up config (probably ctrl+D rather than sudo su command)

sudo su - smokeping
cd /home/smokeping/smokeping/etc
for foo in *.dist; do cp $foo `basename $foo .dist`; done
vi config

change the following:

owner    = Jean Debogue
contact  = noc@jeandebogue.com
cgiurl   = http://graph.mydomain.com/smokeping/smokeping.cgi

imgcache = /home/smokeping/smokeping/cache
datadir  = /home/smokeping/smokeping/data
piddir  = /home/smokeping/smokeping/var
smokemail = /home/smokeping/smokeping/etc/smokemail.dist
tmail = /home/smokeping/smokeping/etc/tmail.dist

*** Presentation ***

template = /home/smokeping/smokeping/etc/basepage.html.dist

:wq!


# 6) Configure smokeping for apache

sudo su -

cd /home/smokeping/smokeping
ln -s /home/smokeping/smokeping/cache /home/smokeping/smokeping/htdocs/cache
chown -R apache cache
chown -R apache data

Now edit apache config…

#Add these lines into the file /etc/httpd/conf.d/smokeping.conf (ignore the
#Auth stuff and htpasswd if you don't want to password protect the dir)
#I do this out of paranoia.


ScriptAlias /smokeping/smokeping.cgi /home/smokeping/smokeping/htdocs/smokeping.fcgi.dist
Alias /smokeping /home/smokeping/smokeping/htdocs

< Directory "/home/smokeping/smokeping/htdocs">
        Options FollowSymLinks
        AuthType Basic
        AuthName "Smokeping"
        AuthBasicProvider file
        AuthUserFile /etc/httpd/passwd/passwords
        Require valid-user
< /Directory>


#Create Auth password if you need it...
mkdir /etc/httpd/passwd
htpasswd -c /etc/httpd/passwd/passwords webuser

[specify password, eg: smokeping]

chmod 600 /etc/httpd/passwd/passwords


#################### IF YOU HAVEN'T ALREADY SET APACHE UP PROPERLY DO IT NOW #####################
vi /etc/httpd/conf/httpd.conf

# IP below is an example!

Listen 192.168.1.25:80
DocumentRoot "/var/www/html"

And the rest…


# 7) setuid for fping
chown root:root /usr/sbin/fping
chmod u+s /usr/sbin/fping

edit /home/smokeping/smokeping/etc/config

remove the slaves section as we're not using that.

Replace the following from probes section

============================================================

*** Probes ***

+ FPing
binary = /usr/sbin/fping
packetsize = 750
step = 60

+ DNS
 binary = /usr/bin/dig
 server = 8.8.8.8
 pings = 3
 forks = 5

# Use these after you've compiled echoping
#+ EchoPingHttp
# binary = /usr/bin/echoping
# pings = 5
# forks = 5
# offset = 50%
# ipversion = 4
# url = /
#
#+ EchoPingHttps
# binary = /usr/local/bin/echoping
# pings = 5
# forks = 5
# offset = 50%
# ipversion = 4

+ Curl
 # probe-specific variables
 binary = /usr/bin/curl
 pings = 5

# a default for this target-specific variable
urlformat = http://%host%/


*** Targets ***

probe = FPing

menu = Top
title = Network Latency Grapher
remark = Welcome to the SmokePing website of SubnetZero.org!

 + network
 menu = Net latency
 title = Network latency (ICMP pings)

 ++ www1
 host = www.google.com

 + services
 menu = Service latency
 title = Service latency (DNS, HTTP)

 ++ DNS
 probe = DNS
 menu = DNS latency
 title = Service latency (DNS)

 +++ www1
 host = www.google.com

 + HTTP
 probe = Curl
 menu = http full page
 title = HTTP latency

 ++ www1
 host = www.google.com




==============================================

Double check permissions are ok:

ls -l /home/smokeping
drwxr-x---. 11 smokeping apache      4096 Jul  3 22:52 smokeping

cd /home/smokeping/smokeping
ls -l
drwxrwxr-x. 2 smokeping smokeping 4096 Jul  3 22:41 bin
drwxrwxr-x. 6 apache    smokeping 4096 Jul  3 23:55 cache
drwxrwxr-x. 6 apache    smokeping 4096 Jul  3 23:14 data
drwxrwxr-x. 3 smokeping smokeping 4096 Jul  3 23:13 etc
drwxrwxr-x. 3 smokeping smokeping 4096 Jul  3 23:28 htdocs
drwxrwxr-x. 3 smokeping smokeping 4096 Jul  3 22:42 lib
drwxrwxr-x. 3 smokeping smokeping 4096 Jul  3 22:42 share
drwxrwxr-x. 6 smokeping smokeping 4096 Jul  3 22:40 thirdparty
drwxrwxr-x. 2 smokeping smokeping 4096 Jul  3 23:13 var


If necessary..
chown -R apache cache
chown -R apache data

#8) Start smokeping and apache
/etc/init.d/smokeping start
/etc/init.d/httpd start


#### TEST AND IF ALL IS OK.... #######

sudo su -

chkconfig --add smokeping
chkconfig httpd on
chkconfig smokeping on

#lock down smokeping user

usermod -s /bin/false smokeping

======================================================

#9) Optional... build echoping with ssl

# If you locked down smokeping user already then do
usermod -s /bin/bash smokeping

sudo su - 
yum install openssl-devel popt-devel


sudo su - smokeping  
cd /tmp
wget http://downloads.sourceforge.net/project/echoping/echoping/6.0.2/echoping-6.0.2.tar.gz
tar zxvf echoping-6.0.2.tar.gz
cd echoping-6.0.2
./configure --prefix /usr/local/ --enable-icp --with-ssl --without-libidn


sudo su - 
cd /tmp/echoping-6.0.2
make test 
make install

#test echoping

/usr/local/bin/echoping -C -h /dana-na/auth/url_default/welcome.cgi rcseu.rabobank.com


#lock down smokeping user
usermod -s /bin/false smokeping

Note that if you want to use SSL (EchoPingHttps) probes, you MUST refer to the /usr/local/bin/echoping binary now instead
of the one that may already be installed.

This is a fantastic tool but it does have some peculiarities. I don’t agree with the default graph scaling, which you’ll soon see clips out the “smoke” peaks. Changing unison_tolerance for targets can work around this to an extent but you have to remember that graphs scale to the median. You will also probably want to review the RRDTool database aggregation if you want high resolution polling.

Here are my changes in the config file for 1 minute resolution… If I’ve done anything wrong, I’ll be happy to correct this post.

# consfn mrhb steps total

AVERAGE  0.5   1  10080 # 7 days of 1 min = 86400 seconds in a day. So 86400 * 7 / (step value 60) = 10080
AVERAGE  0.5   5  8064  # 4 weeks of 5 min =  2016*5 min aggregates in a week = 8064
    MIN  0.5   5  8064
    MAX  0.5   5  8064
AVERAGE  0.5  60  2016 # 3 months of 1 hour = 24*60min aggregates in a day * 7days * 12 weeks = 2016
    MAX  0.5 144  2016
    MIN  0.5 144  2016

CentOS 6.3 NNMi Installation for the Lazy

Can’t be bothered to RTFM? Here is a quickstart guide to installing NNMi9 on CentOS 6.3 (it’s essentially the same procedure as on RHEL). This is handy for getting a test instance up and running quickly. Obviously, for a full-blown production environment, you should always read the deployment guide and release notes.

Install Base CentOS build with /var/opt/OV and /opt/OV filesystems and hardware sized to the requirements of your environment (see below).

Nodes   /opt/OV   /var/opt/OV  CPUs  RAM  -Xmx_value
250     3GB       10GB         2     4GB  -Xmx2g
250-3K  3GB       30GB         10    8GB  -Xmx4g
3K-8K   3GB       40GB         25    16GB -Xmx8g
8K-18K  3GB       60GB         40    24GB -Xmx12g
18K-25K 3GB       80GB         40    32GB -Xmx16g

Download the HP Public Keys and import them.

rpm --import hpPublicKey.pub
rpm --import hpPublicKey2048.pub

Install required packages and disable iptables for the most pain-free installation (you’re installing this in a trusted environment, right?)

yum install compat-libstdc++-33.i686
yum install compat-libstdc++-33.x86_64
yum install glibc.x86_64
yum install libXi.i686
yum install compat-libstdc++-33.x86_64
yum install xorg-x11-apps-7.4-10.el6.x86_64
yum install xorg-x11-server-Xorg.x86_64
yum install xorg-x11-server-Xorg.x86_64
yum install libX11.i686
yum install libX11.x86_64
yum groupinstall "X Window System"

chkconfig iptables off

Edit /etc/sysctl.conf and adjust/add parameters according to the HP Deployment Documentation.

vi /etc/sysctl.conf

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296

# NNMi settings for UDP receive and send buffer sizes
net.core.rmem_max = 8388608
net.core.wmem_max = 8388608

:wq!

/sbin/sysctl -p

Update /etc/security/limits.conf as follows:

 vi /etc/security/limits.conf

*       soft    nofile  4096
*       hard    nofile  4096

Install NNMI from an XClient. This is highly annoying but easy enough – you should be able to tunnel X11 over SSH in PuTTY’s properties. You will first need to allow X11 forwarding (X11Forwarding yes) in /etc/ssh/sshd_config and restart sshd.

If you only have sudo access to root (common in enterprise environments), then you will find that this is a show-stopper. You need to do the following (don’t blindly copy and paste, you need to copy your own xauth information):

[user@nnmiserver ~] xauth list
nnmi.localdomain.com/unix:10  MIT-MAGIC-COOKIE-1  58682a5bb5a4f731ae15c186ff3d68f8
[user@nnmiserver ~] sudo su -
[root@nnmiserver ~] xauth add nnmi.localdomain.com/unix:10  MIT-MAGIC-COOKIE-1  58682a5bb5a4f731ae15c186ff3d68f8

Avoid using ReflectionX on Windows 7, as this has a habit of crashing/hanging during installation; this caused massive headaches and took a while to figure out! Xming/Exceed are fine.

Assuming you’ve downloaded the iso, mount it as follows:

mkdir /nnminstall
mount -o loop -t iso9660 Software,_NNM_i_Linux_9.20_Eng_TB768-15004.iso /nnmistall
cd /nnminstall
./setup.bin

Once installed and running, tune the ovjboss Xmx value as per the table at the top of this article.

ovstop -c ovjboss
vi /var/opt/OV/shared/nnm/conf/props/ovjboss.jvmargs

# –Xms: Initial Java Heap Size
# –Xmx: Maximum Java Heap Size
# –Xss: Java stack size (default to OS-supplied value)
#
–Xms128m
–Xmx2048m
#–Xss128m

:wq!

ovstart -c ovjboss

From here, download and install the latest patch RPMs to bring it up to the current version. :)