This news item struck me today: Bar-peeing is a new fenomenon in Holland (in Dutch)
Quote:
“In a busy, dirty pub with beer all over, I don’t mind. What you pee is mostly beer anyway”, says an anonymous Rotterdammer.
Must be Heineken, methinks…
Bar-peeing in Holland
June 3rd, 2009Puppet and Nagios
May 11th, 2009Although not very well documented, the integration between Puppet and Nagios can be very useful, especially in a highly modularized environment with components shared between multiple instances, such as a webfarm.
I intend on writing a little tutorial page on this site with my recent insights on Puppet and Nagios integration, but one thing I won’t keep unmentioned is how to deal with lots and lots of Puppet defined servicechecks.
By default, collected nagios_servicecheck objects are all stored in one config file. As soon as you reach several hundred defined servicechecks, however, the puppetclient on your nagios server will tend to be slower and slower. A puppetrun that takes up as much as an hour is no exception anymore. This is perfectly explainable. Puppet has to verify whether or not a servicecheck definition has changed since the previous puppetrun. It does so by scanning the nagios_servicechecks.cfg file for each and every defined service.
The best way to overcome this issue is to use a config directory instead of a config file. To do this, specify a “target” argument to the nagios_servicecheck:
@@nagios_service {"PING-$fqdn":
ensure => present,
host_name => $fqdn,
...
target => "/etc/nagios/nagios_services.d/ping-$fqdn.cfg"
}
This way, for each and every defined servicecheck, puppet no longer has to scan a textfile containing thousands of lines, but only verify the existence of a configfile and make sure the content (just a couple of lines) is still up to date.
This will give you a speedup in the 50-fold range.
Of course, you can do the same thing with nagios_host, nagios_command, etc.. objects, but the largest gain, you’ll find will be with the servicecheck optimisation.
The FOSDEM report
February 26th, 2009It’s about time I penned about FOSDEM ‘09. It’s been some years since I last attended FOSDEM, but I’m glad we made it, even though we could only go for one day.
Bdale Garbee’s keynote was entertaining, but I actually wanted to see the Augeas talk as I want to start using Augeas in my Puppet setup. Raphaël Pinson did a good job in his introductory talk.
I also wanted to attend Remko and stpeter’s talk on XMPP as well as Dave Cridland’s XMPP talk.
Remko in full action
All in all, FOSDEM hasn’t changed much: it’s an interesting conference, but most of all it’s a way to meet up again with the people you don’t always get to see in real life anymore…
Shopping horror
February 23rd, 2009Shopping on the Dell online shop is absolute horror. Tons of choices without any explanation, like this one:
How am I supposed to know what this is?
What about this one:
Most stupid option, when selecting a LED backlit display, I get this error:
That’s right! I should have chosen the “No camera with microphone for LED backlit display”. Reminds me of the joke about a man that asks for a ham and cheese sandwich without butter to with the clerk responds they only have ham and cheese sandwiches without margarine…
Big news!
January 19th, 2009I’m finally going to FOSDEM again. A couple of reasons:
- The programme looks really interesting, especially the RoR track, MySQL track and XMPP track.
- It’ll be a nice opportunity to meet up with some geeky friends.
- Some Puppet users want to meet up.
- An is getting more and more interested in IT, so this is a very good way to lure her in all this!
So I hope to see you all at FOSDEM!
RIP Rick Wright
September 16th, 2008I just wanted to express my sadness with the passing away of Richard Wright and gratitude for all the great music he’s left us with.
Though I really rediscovered Pink Floyd only recently, I have grown to like their music to the point of (unintendedly) annoying my colleagues and friends with their music.
For those that have no idea what I’m talking about: shame on you, go look where the Scissor Sisters, Beasty Boys, Placebo and lots of others got their inspiration.
See you at the great gig in the sky, Mr. Wright!
I don’t want SPAM!
May 27th, 2008Today, I took the drastic step of dropping all mails marked as spam. I finally valued the time I loose when sifting through my spam folder looking for false positives before emptying the folder over the risk of false positives.
From now on, people should learn to write emails that don’t look like spam!
Aaaahhhh….
Identifying external FC disks
May 14th, 2008So, you got a server linked to a SAN box. You define some LUNs on that SAN box. You link the LUNs to the server and you want to start using the LUNs on the server.
Let’s see:
xen02:/dev# ls -l sd*
brw-rw---- 1 root floppy 8, 0 2008-04-15 16:51 sda
brw-rw---- 1 root floppy 8, 1 2008-04-15 16:51 sda1
brw-rw---- 1 root disk 8, 16 2008-04-15 16:51 sdb
brw-rw---- 1 root disk 8, 32 2008-04-15 16:51 sdc
brw-rw---- 1 root disk 8, 48 2008-04-15 16:51 sdd
brw-rw---- 1 root disk 8, 64 2008-04-15 16:51 sde
brw-rw---- 1 root disk 8, 80 2008-04-15 16:51 sdf
brw-rw---- 1 root disk 8, 96 2008-05-09 13:35 sdg
brw-rw---- 1 root disk 8, 112 2008-05-09 13:35 sdh
brw-rw---- 1 root disk 8, 128 2008-05-09 13:35 sdi
brw-rw---- 1 root disk 8, 144 2008-05-09 13:35 sdj
brw-rw---- 1 root disk 8, 160 2008-05-09 13:35 sdk
brw-rw---- 1 root disk 8, 176 2008-05-09 13:35 sdl
brw-rw---- 1 root disk 8, 192 2008-05-09 13:35 sdm
brw-rw---- 1 root disk 8, 208 2008-05-09 13:35 sdn
brw-rw---- 1 root disk 8, 224 2008-05-09 13:52 sdo
brw-rw---- 1 root disk 8, 240 2008-05-09 13:52 sdp
brw-rw---- 1 root disk 65, 0 2008-05-09 13:54 sdq
Phew, how to identify which drive corresponds to which LUN, especially if you have some equal-sized LUNs.
Fortunately, we have this:
xen02:/dev/disk/by-id# ls -l
total 0
lrwxrwxrwx 1 root root 9 2008-05-14 16:11 scsi-1 > ../../sdl
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 scsi-3600a0b8000320d200000038e47e1430a > ../../sdb
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 scsi-3600a0b8000320d200000039047e3f21f > ../../sdd
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 scsi-3600a0b8000320d200000039447f4f916 > ../../sdf
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000320d200000041f48073120 > ../../sdh
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000320d200000042348076e8c > ../../sdj
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000320d200000043748206a2e > ../../sdn
lrwxrwxrwx 1 root root 9 2008-05-09 13:52 scsi-3600a0b8000320d200000043a48244540 > ../../sdp
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 scsi-3600a0b8000322cba0000067c47e3f289 > ../../sdc
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 scsi-3600a0b8000322cba0000067e47e3f2cc > ../../sde
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000322cba000007ca48076f1b > ../../sdi
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000322cba000007da480dc9cd > ../../sdk
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 scsi-3600a0b8000322cba000007de480f5683 > ../../sdm
lrwxrwxrwx 1 root root 9 2008-05-09 13:54 scsi-3600a0b8000322cba000007e0482445e4 > ../../sdq
lrwxrwxrwx 1 root root 9 2008-05-09 13:52 scsi-3600a0b8000322cba000007e248244632 > ../../sdo
lrwxrwxrwx 1 root root 9 2008-05-14 16:11 scsi-3600a0b8000322cba00000890482afe68 > ../../sdg <---
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 usb-M-Sys_uDiskOnChip_0F801A713040492E > ../../sda
lrwxrwxrwx 1 root root 10 2008-04-15 16:51 usb-M-Sys_uDiskOnChip_0F801A713040492E-part1 > ../../sda1
Exactly, by using the /dev/disk/by-id virtual directory, you can see which disk id corresponds to which LUN, as you can see in the next (partial) screenshot:
That’s nice! Now we know that our LUN called dpmgmt-root corresponds to 60:0a:0b:80:00:32:2c:ba:00:00:08:90:48:2a:fe:68 which, according to our second listing, corresponds to /dev/sdg.
Using such long device paths isn’t really convenient though, so let’s take this a little further.
Format the device using your preferred filesystem and label your filesystem:
xen02:/dev/disk/by-id# mkfs.ext3 -L dpmgt-root scsi-3600a0b8000322cba00000890482afe68
mke2fs 1.40-WIP (14-Nov-2006)
scsi-3600a0b8000322cba00000890482afe68 is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=dpmgt-root
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
...
Reboot your system and check this out:
xen02:/dev/disk/by-label# ls -l
total 0
lrwxrwxrwx 1 root root 9 2008-05-14 17:28 dpmgmt-root > ../../sdg
lrwxrwxrwx 1 root root 10 2008-04-15 16:51 root > ../../sda1
lrwxrwxrwx 1 root root 9 2008-05-09 13:35 semail-root > ../../sdn
lrwxrwxrwx 1 root root 9 2008-04-15 16:51 tempvm-root > ../../sdf
So now, we can address our device as /dev/disk/by-label/dpmgmt-root on every machine that has access to that LUN, now matter what the actual real device path is on that machine.
What’s even better, is that you even don’t even have to reboot to update /dev/disk/by-label/
Just trigger udev to reload the system information on /dev/sdg:
xen06:/dev/disk/by-label# echo add > /sys/block/sdg/uevent
xen06:/dev/disk/by-label# ls
dpmgmt-root root semail-root tempvm-root
Et voilà!
SSH storm – updated
May 13th, 2008The last couple of days, it seems there’s some kind of ssh botnet trying to spread out. Since I installed DenyHosts some weeks ago, I usually got 5-10 notifications of blocked IP addresses. Last weekend however, I got more than 200 notifications.
Although I feel rather safe having installed DenyHosts (which I urge you to install on every SSH accessible server), as a lot of hosts out there aren’t protected, I fear a new botnet is in the making.
Clearly, someone thinks I need to get more junk in my mailbox. Let me tell you though, 1400 spam mails a day is enough already…
That’s why I wrote a very rudimentary script that hooks into DenyHosts and queries a whois server for an abuse email and sends it an email when found.
Update:
I didn’t notice the difference between the PLUGIN_DENY setting and the mail behaviour of DenyHosts.
By default, DenyHost will send you an email everytime it adds a new host to /etc/hosts.deny, whereas the PLUGIN_DENY script will be invoked every time it adds or readds a host to /etc/hosts.deny. That’s why I now first grep a file with hosts whose hostmaster I already notified of the abuse
The script is ridiculously easy:
#!/bin/bash
# Get parameter
IP=$1
# Check whether we've already seen this host.
if `grep $IP /var/lib/denyhosts/notified_abuse > /dev/null` ;
then
echo host already seen
exit
else
echo new host, added to logfile
echo "`date` $IP" >> /var/lib/denyhosts/notified_abuse
fi
# Try to lookup the abuse mailbox
abuse=`whois $IP|grep ^abuse-mailbox:| tail -n 1 |sed -e "s/abuse-mailbox: //"`
# if found an abuse mailbox, send a mail.
if [ "x$abuse" != "x" ];
then
cat << EOF | mail -a "From: Pieter Barrezeele <xxxxxx@xxxxxxxxxx.be>" -s "SSH brute force attack from $IP" $abuse
Dear Sir/Madam,
Today we experienced an SSH brute force attack originating
from $IP, a host under your responsibility. This probably means
the host in question is compromised.
Please take action to stop this host from attacking us again.
Thanks in advance,
Pieter Barrezeele
PS: this is an automated mail, any errors in this mail are caused by parsing errors.
EOF
fi
I’d advise you to send the mails to yourself for a few days until you see only new hosts are added. Alternatively, you could copy the contents of /etc/hosts.deny into /var/lib/denyhosts/notified_abuse as well.