Saving/Mailing Large “show tech” Output

Assuming you’re using a unix/linux management server, this is a quick and easy way to get a large “show tech” output to a file and mail it off elsewhere, assuming . If it’s leaving your organisation then you should either use 7zip or winzip or similar on your windows workstation to encrypt as the standard unix zip utility does not provide strong encryption.

me@mgmt-server# script lonsw02-tech.txt
Script started, file is lonsw02-tech.txt
me@mgmt-server# ssh lonsw02
password:
LONSW02> en
password:
LONSW02# term len 0
LONSW02# show tech-support
< output omitted >
LONSW02# exit
me@mgmt-server# ^D  ( CTRL + D )
Script done, file is lonsw02-tech.txt
me@mgmt-server# unix2dos lonsw02-tech.txt
unix2dos: converting file test.txt to DOS format ...
me@mgmt-server# uuencode lonsw02-tech.txt lonsw02-tech.txt | mailx -s "Show tech" me@mycompany.com

If your management server isn’t configured as a mail client, that can be fixed on RHEL with:

# vi /etc/sysconfig/sendmail

DAEMON=no

# vi /etc/mail/submit.cf

D{MTAHost}[smtprelay.mycompany.com]

Reload Sendmail

# service sendmail reload

Syslog-ng and log rotation

I decided that I would change the way that system logs are archived via a custom (ie: non logrotate) script, however, I managed to forget that using unix “mv” means that the inode of the new filename is the same as the old filemane. rm the new moved temporary file and you’re in trouble.

When rotating logs without logrotate it’s probably best to use a command that will read until end of file properly (such as gzip) then truncate (eg cp /dev/null targetfile) the original. This way, loss is minimal and inode doesn’t change. I thought I’d be smart, move the log and then gzip it, hoping that syslog-ng would be smart enough to create a new file and hence new inode. Unfortuanately, this meant that syslog-ng kept logging to the moved file, then threw a fit and stopped logging altogether once the mv’d temporary file was gzipped off elsewhere and then deleted.

Summary:

– Use an archiving solution that properly reads through to EOF before you truncate the original file.
– Use a postrotate action with logrotate to send a HUP signal to the process (eg: syslog-ng) after archiving/moving each file. copytruncate may be a useful option with logrotate here too.

I’ve seen a few posts on forums that blame syslog-ng, but the fact is that this is the way that UNIX behaves.

Using syslog-ng to split logs by facility and/or hostname

An example (not exhaustive) syslog-ng file to filter out logs on local6 facility to specific files using the $HOST macro. If you’d configured all your firewalls to log at local6 (facility 22) then this might be of use. Also takes in other remote syslogs from our network devices and puts them into a single file (and excludes local6 as we already have those elsewhere).

BEWARE of some vendors using local facilities for their syslogs. Juniper and F5 tend to do this a lot. You may want to override syslog facilities on the devices themselves.

Permissions should be changed to your preference.

The line we’re interested in is:

destination dst_fwlog { file("/netlogs/messages.firewall.$HOST" perm (0644)); };

Example config below:

options { long_hostnames(off);
        sync(0);
        owner(root);
        group(root);
        use_dns(persist_only);
        dns_cache_hosts(/etc/hosts);
};

source src_local {
        internal();
        unix-stream("/dev/log");
        file("/proc/kmsg");
};

source src_remote{
        tcp(max-connections(50));
        udp();
};

destination dst_messages { file("/var/log/messages" perm(0644)); };
destination dst_authpriv { file("/var/log/secure" perm(0600)); };
destination dst_maillog { file("/var/log/maillog" perm(0600)); };
destination dst_cron { file("/var/log/cron" perm(0600)); };
destination dst_alltty { usertty("*"); };

destination dst_netlog { file("/netlogs/messages" perm(0644)); };
destination dst_fwlog { file("/netlogs/messages.firewall.$HOST" perm (0644)); };

filter f_syslog { not facility(authpriv, mail, cron, local6); };
filter f_authpriv { facility(auth, authpriv); };
filter f_mail { facility(mail); };
filter f_cron { facility(cron); };
filter f_local5 { facility(local5); };
filter f_local6 { facility(local6); };
filter f_info_to_notice { level(info .. notice); };
filter f_emerg { level(emerg); };

# Local Syslog
log { source(src_local); filter(f_syslog); destination(dst_messages); };

# Local Authpriv
log { source(src_local); filter(f_authpriv); destination(dst_authpriv); };

# Local Mail
log { source(src_local); filter(f_mail); destination(dst_maillog); };

# Local Cron
log { source(src_local); filter(f_cron); destination(dst_cron); };

# Local Emerg
log { source(src_local); filter(f_emerg); destination(dst_alltty); };

# Remote Syslog Network Devices
log { source(src_remote); filter(f_syslog); destination(dst_netlog); };

# Remote syslog Firewalls (local6 override on all devices)
log { source(src_remote); filter(f_local6); destination(dst_fwlog); };

Port redirection in iptables

Running applications as a non-privileged user typically means that you can’t listen on port 1023 or below. You can work around this with iptables, however, by forwarding the privileged port to another higher port. This works well for http.

If using an iptables firewall rules text file, add the following section (the config below redirects TCP port 80 to 8000):

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]

#Redirect 80 to 8000
-A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8000

COMMIT

I keep my rules in a separate file (old habit) so do iptables-restore < /etc/iptables-firewall-rules (s) then do service iptables save From the command line, you'd do:

iptables -t nat -A PREROUTING -p tcp –dport 80 -j REDIRECT –to-port 8000

Obviously you need a rule to allow connectivity to the redirected port.