Code is Love

Hacking CakePHP’s request_handler.php to accept Compressed XML

by on Apr.03, 2009, under Uncategorized

Disclaimer: There are undoubtedly cleaner ways of accomplishing this task, which is why I didn’t consider submitting this code back to CakePHP. Feel free to take my problem and come up with your own better solution — this worked for me, and certainly there are others who could benefit from it.

I was having a rough time trying to figure out how I was going to get CakePHP to handle compressed input. In my case, it was from the OCS Inventory Agent, which outputs zlib compressed XML server inventory reports to the host of your choosing. I was essentially doing some REST work between the OCS client and CakePHP — and CakePHP was handing off the request to request_handler properly thanks to the very simple suggested Routes configuration. The problem was that request handler didn’t know what to do with the mime-type “application/x-compress” (i.e. zlib compression), never mind detect the xml inside it and then magically do what it does with plan text XML.

Patch is here. request_handler looks for the “application/x-compress” Content-Type, decompresses the input (POST), then uses simplexml_load_string to determine if the contents are XML since the other check is simply checking for an xml Content-Type. Then the content is turned into an instance of CakePHP’s Xml object, which is then assigned to the $data property of the controller automagically.

Thanks to this post for assistance with handling CRC’s since not all zlib compressed POSTs are created equally. Also thanks to the people who wrote wireshark, which saved me as I debugged HTTP headers in this endeavor.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , more...

PCMCIA card won’t work – ‘cs: warning: no high memory space available!’

by on Jun.27, 2008, under Uncategorized

I recently found my 3G card, a Sierra Wireless Aircard 860 I use to connect to my cell carrier, not working after well over a year of flawless execution. Poking through syslog I found:

‘cs: warning: no high memory space available!’

and indeed, the card was not recognized when you put it in, which was new and unusual. After some trial and error and a lot of googling, this solved my problem:

Add this to /etc/pcmcia/config.opts

include memory 0x40000000-0x40000fff

Boom – now it works. Slide the card into the slot and instantly the TTy is assigned and I’m good to go.

My Hardware:
Dell Latitude D620
Sierra Wireless Aircard 860

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , more...

What domains are awaiting mail from my server?

by on Jun.10, 2008, under Uncategorized

Here’s a less than elegant solution to get a summary of the number of emails per domain that have mail sitting in your postfix queue.

#!/bin/sh
postqueue -p|egrep '^([[:space:]])+([[:alnum:]]|.|-|_)+@([[:alnum:]]|-)+.([[:alnum:]]+|.|-)*([[:alpha:]]){2,4}$'|awk '{ print $1 }'
|awk -F@ '{ print $2 }'|sort|uniq -i -c
| awk 'BEGIN {sum=0} {sum += $1} {print $0} END {printf("---------------------------n "sum" Total E-mails that are properly formattedn")}'

Thank you, awk

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Comments Off :, , , , more...

Flickr-ify your Gnome Screen Saver

by on Oct.28, 2007, under Uncategorized

I’ve been reading about the estarling, a digital photo frame that will sit on a wireless network and grab photos from an RSS feed, on say, Flickr, or Picasaweb. What a fantastic idea this is — and really it’s the product I’ve been waiting to come to market for a long time if its features work as promised. Imagine: stick it in the home of your grandparents, parents, etc. – and _you_ can send the photos to their frame – keeping them up to date with all the latest photos from the lives of their family and friends.

To go along with this, I wanted a simple screen saver for my new install of Ubuntu Feisty-Fawn that did the same thing — grabbed latest images from an RSS feed on the web and then serve them when the screen was locked, or the system idle for more than a few minutes. It seemed like every Mac in my office has been taunting me with a feature like this, but maybe all those photo slideshows I’ve been seeing are from local images.

So, this is nothing short of a hack, but if you want this functionality on your desktop, maybe you’ll think it’s worth it like I did. The end result is Gnome Pictures Screen Saver with a modified version of bashpodder that grabs the latest photos from any feed(s) I specify to become the content for my screensaver.

The script takes a config file which consists of the URL to the feed you want to subscribe to and the xsl file which pulls out the URLs to the photos themselves from the feed. Currently RSS feeds/xsl transformations from Gallery, Picasaweb, and Flickr have been tested. When it runs, it checks the feed against files previously downloaded and only downloads the new stuff. It could be run from cron or perhaps when you log in. I suggest running it by hand the first time to make sure it is doing what you expect.

Package available in this thread:

Prerequisite pkg: libwww-perl (I used GET instead of wget – long story)

To install:

$tar -xzvf rss_photos.tar.gz

Edit rss_photos.sh and change the values of CONF_FILE and LOG_FILE to a permanent location. Create the CONF_FILE you just specified.

# Lines in CONF_FILE should be of the pipe-delimited format:
# |
# e.g.
# http://api.flickr.com/services/feeds/photos_public .gne?id=YOURIDHERE&lang=en-us&format=atom|parse_flickr_enclosure.xsl

Obviously, you will need to enable the “Pictures Folder” screen saver in Gnome for this to work – the script simply grabs photos for the screen saver.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Comments Off :, , , , , , , , more...

Gnome Terminal Profiles for Connecting to Different Hosts

by on Jan.19, 2007, under Uncategorized

I was really surprised that I couldn’t find a how-to to configure Gnome’s Terminal program to have various profiles to connect to different hosts. For those familiar, I was looking for something PuTTY like, where when I launched the program, I would have a list of hosts to connect to, rather than just opening a local shell and running the same ssh commands to different hosts each time I need to login to a remote system. Yes, I have heard that PuTTY runs on linux, but I really wanted to use Gnome Terminal since I use it every day and want to use more standard tools in my daily Linux life.

What I did find, however, was the Gnome Terminal Launcher Applet. This is an applet that runs in your Gnome tray/panel with a drop down menu for each of your terminal profiles. Sweet! I ran into a problem though: if I just put “ssh ” in the profile, ssh wouldn’t use my keys already in memory thanks to keychain and the ssh agent, of course. So, here’s where my hack comes into play — and might save you the same minor hassle.

Create a shell script in your home directory called terminal-helper with the following contents:

#!/bin/sh
. .bash_profile
ssh $1

so I get the benefit of the keychain lines pre-existing in my .bash_profile when it is sourced, and it just connects to whatever host I pass to the script (thanks, Diego, for removing some obvious clunkiness from the original implementation). In my Gnome Terminal profiles, on the “Title and Command” tab, I specify “Run command as a login shell”, “Run a custom command instead of my shell”, and then specify the command:
/home/myuser/terminal-helper <yourusernamehere>@<yourhostnamehere>

Now I can connect to whatever host I have created a profile for just by clicking on a drop down menu. 10-15 keystrokes per host saved! Heh. Enjoy.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , more...

OpenOffice, Avery 8250 Labels, and the Mail Merge

by on Dec.11, 2006, under Uncategorized

So this weekend it was time to get our Holiday Cards out and rather than hand-write 100+ addresses, it seemed to make a lot of sense to mail merge our addresses from a database onto store-bought labels to the tune of 30 per sheet. While in the end this may have been faster to hand-write, if you’re here now, it won’t for you, because as usual, I’m hoping my agony saves you some time.

The labels we happen to have in the office were Avery 8250. If you are using OpenOffice and the 8250s, you probably already know that OpenOffice doesn’t have the 8250 in it’s drop-down in the label-making dialog. Yeah. So after trying to get downloaded template files to do what I wanted, eventually just started tweaking the dimensions of the labels in the OOffice label dialog and got satisfaction this way. But let me rewind…

We happen to have our family address book in a MySQL DB already — but OOffice will accept a number of sources for your data: Evolution, LDAP, Thunderbird, etc.. First I had to figure out how to get the data source into Open Office. While I can’t remember all the gory details, the gist of it is this: installed unixODBC, the odbc stuff for MySQL, edited two files in /etc (see link below), and then was able to connect with username and password from OpenOffice to add the data source. I also had to install openoffice.org-base (OO’s DB package). Then it was time to create some labels.

With OO open, choose File->New->Labels. Here you can choose the data source, format the layout of the fields, etc.. This is where I discovered that not only were the Avery labels I had not in the drop-down, but that Avery pretty much stinks for having no support for OO that I could find on their pretty lousy website. Cheers to the people at Worldlabel.com who had a fair number of OO templates on their site, even though they didn’t have the ones I needed. What I ended up doing was opening the format tab and adjusting the values to what I believed them to be – hint: use a ruler to get started plus the dimensions listed on the packaging. Print on plain paper and hold your output up to a sheet of the real labels in front of a light to test alignment until you get it right.

The Measurements I ended up using (almost perfect – need minor adjustment, but “worked”):
Horizontal Pitch: 2.83″
Vertical Pitch: 1.00″
Width: 2.63″
Height: 0.90″
Left Margin: 0.25″
Top Margin: 0.55″
Columns: 3
Rows: 10

Don’t forget to save these settings — and on the “Options” tab, Make sure “Synchronize Contents” is checked before clicking “New Document”. If in the resulting document you select all (ctrl-a for the keyboard addicts out there), you can then change the font and font size in the first label and then click “syncronize labels” in the dialog window to make the rest follow this format. When you are ready, choosing print should populate the values into the labels from your db — and I to not waste paper or labels, do 1 sheet at a time until you have the alignment right, and then you can print them all at once. Hope this saves someone some time – it sure took a lot of mine to figure out.

References:
Directions I loosely followed to setup ODBC for OOffice/MySQL
Use of MySQL in OOffice 1.0 (PDF)

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment : more...

Fedora Core 6: Compiz, Beryl, Nvidia & Broadcom, Oh Boy

by on Dec.02, 2006, under Uncategorized

So, I finally decided it was time to upgrade to a modern Linux distro — and I was pushed by the fact that some experimentation with dmraid from way back had left my system in a sorry state. A coworker was already running Fedora Core 6 on his laptop and while I don’t always love Fedora for it’s lack of longevity per release, I am a bit addicted to the bleeding edge.

In the last week I have installed FC6 Twice. Once on a homebrew desktop:

Asus K8N-E
AMD Athlon 64 3400
1 GB RAM
2 200 GB Maxtor SATA 133 Drives in dmraid mirror config through motherboard based controller
OEM Graphics card w/ nVidia Corporation NV34 [GeForce FX 5200]
Onboard Gb NIC
etc.
etc.

and a laptop:
Dell Lattitude d620
Intel Core 2 Duo T7200
nVidia Corporation Quadro NVS 110M / GeForce Go 7300
Broadcom Corporation Dell Wireless 1390 WLAN Mini-PCI Card
1 GB RAM
80 GB 5400 RPM SATA drive

The first install had a number of small hiccups — some related to hardware installed that was, well, kinda broken, and some minor anaconda explosions. None of these were that big a deal, but one lesson learned would be don’t configure an IP on your system if you are them going to add extra repositories to install from (this part has another network config screen, and it clearly didn’t like that I had already chosen to give myself a fixed IP). Run with dhcp until after the install, then set your static IP.

The second install went great — until I ran into a problem luckily already witnessed in my office — a stupid Linksys device that couldn’t handle the TCP window scaling properly. Thank you: “echo 0 > /proc/sys/net/ipv4/tcp_window_scaling”. The symptom was HORRENDOUSLY slow network performance before making this tweak (think downloading a few rpm updates all day).

I knew that eye-candy in gnome was finally working because I had stumbled across this news a while back, so I had to try it. Fedora comes pre-installed with the compiz package which, so long as your graphics card supports it, does some nice GUI effects, like rubbery windows, each desktop on it’s own side of a rotating cube, etc.. Really cool stuff, and to my amazement it mostly worked right out of the box. Linux is finally catching up in the UI world to the likes of Apple, so for the people who don’t generally live at the command line, this is a big deal. Nice work, everybody who had a hand in design/coding/testing/documenting/etc./etc.. This rocks. Amazing stuff.

A couple bumps in the compiz road that were/are being faced: the closed-source NVidia driver is necessary for those of us with nVidia chipsets as far as I can tell. None of this really works on the Xen kernel, currently. Also, running said nvidia driver causes IRQ conflicts with the Broadcom/Dell WiFi, and the wi-fi is currently unusable with either the ndiswrapper or the reverse engineered bcm43xx driver. Note: ndiswrapper works for some indeterminate amount of time before either dropping the wireless or freezing the whole system requiring hard reboot.

Another thing I discovered in all of this is Beryl. Whoa Cool. It basically takes the GUI effects in your desktop to the next level.

So, if you haven’t seen what desktop Linux has been up to in a while, go out of your way to see all the new stuff in Compiz and/or Beryl on a system near you – you won’t be disappointed.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , more...

Linux: How many processors do I have?

by on Sep.13, 2006, under Uncategorized

On Linux, you can get a ton of information about the physical hardware you are running right from the command line. Simply cat’ing /proc/cpuinfo will give you lots of gory details about the processors you have installed. Unfortunately, it gets a little confusing thanks to hyperthreading, dual cores, etc.. So here’s a simple command to find out how many physical processors you have installed.

$ dmesg | grep "Physical Processor" | awk '{ print $5 }' |sort -nu|wc -l

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , more...

Variable Variables: Double Interpolation in Bash

by on Sep.12, 2006, under Uncategorized

Ok, so most of you are wondering why I’d be even bothering to do double interpolation in Bash rather than a real language. Well, I already had some scripts in progress and I didn’t want to start from scratch. What is double interpolation and why would I bother?

In my case, I knew that I was going to have a dynamic list (an array), and that for every array member, there would be a number of other things I would need to know about it (yes, like a multi-dimensional array).

Perhaps we want to do some simple monitoring of an unknown amount of servers in each data center and for each server we need to know how to reach it — the IP address, and the way we want to reach it to test (i.e.http/ping/etc.). So ideally, we need to be able to keep a separate config file which just holds X number of shell variables with server configurations in it — but that the contents of that config file might be wildly different (number of servers, server specs, etc.) between environments and keep the script simple.

So, in one environment we want to monitor the servers, moe, larry, and curly (woop, woop, woop) :

We write a config file that has one array for the server names, then for each of those we define variables to hold the IP and check-type.

The config file, servers.cfg, looks like:

SERVERS="moe larry curly"

moe_IP=10.10.10.10
moe_CHECKTYPE=http

larry_IP=10.10.10.11
larry_CHECKTYPE=ping

curly_IP=10.10.10.12
curly_CHECKTYPE=ping

Then we write a script which just loops through the given servers and does the check based on whatever check-type and IP we’ve configured.

#!/bin/bash

# Source our config file
. servers.cfg

# Loop through the servers and test them!
for server in $SERVERS;
do
IP=$(eval echo $`echo ${server}`_IP);
CHECKTYPE=$(eval echo $`echo ${server}`_CHECKTYPE);

if [ "$CHECKTYPE" = "http" ];
then
# Do dome kind of http check – this is oversimplified for this example
wget $IP
elseif [ "$CHECKTYPE" = "http" ]
then
# Ping it! Normally we would test the success/failure of the ping here
ping $IP
fi

done

The beauty of what we have here is that we don’t need to reconfigure the script to add or remove servers, just maintain a simple config file.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , , more...

PHP’s apache_note: Oft-overlooked gem

by on May.09, 2006, under Uncategorized

Problem: You need to log cookies and you want to be able to tie your users to the sites they came from.

Solution: PHP’s apache_note and a little apache config tweak.

It goes something like this. Cookies are generally served when a user first visits a website. They can be used for all kinds of things, yes, even some of them evil. Typically, though, they allow a webmaster/sysadmin/etc. to do better user tracking than IP tracking alone. IP tracking is riddled with inherent problems – like proxies, IP Changes between sessions, etc.. Cookies have their own set of challenges, but modern browsers support them because they are an important web technology – especially when it comes to personalization. Back to the problem.

When a browser holds a cookie for a domain, it sends the value of that cookie with a web request. When the browser makes its first ever request to a new website, it has not yet been cookied. So, let’s say that you are the webmaster, and you wanted to see which parts of your site are being trafficked by people from a certain site that links to you. Well, even though you are uniquely cookieing users, it isn’t until the second request that you will get the cookie back, so even if you are logging the cookies that are coming in with requests — you can’t pull the lines in your logfile by cookie to see where users came from — because when the browser made the first request with all that valuable referrer information in it, it didn’t have a cookie!! Well hold on – here comes the magic (ok no magic, just someone’s good old fashioned open source kick-ass code to the rescue, Rasmus Lerdorf’s, I think). Enter PHP’s apache_note function.

apache_note() is an Apache-specific function which gets and sets values in a request’s notes table.” What does this mean to you? You can write things to the apache logs on a per-request basis — like the cookie value you are setting. The code might look like this:


<?php
// tag every new visitor with a unique ID
$domain = ".mydomain.com";

if (isset($_COOKIE['ID'])) {
// add error handling here for evil cookie values - don't ever trust user input!
apache_note("CID", $_COOKIE['ID']);
} else {
$randval = rand();
$id = md5($randval . time() . $_SERVER['REMOTE_ADDR'] . $_SERVER['HTTP_USER_AGENT']);
setcookie("ID", $id, time()+60*60*24*365*20, "/", $domain, 0);
apache_note("SENT_ID", $id);
}
?>

And the matching apache config lines might look like:
LogFormat “%h %l %u %t “%r” %>s %b “%{Referer}i”
“%{User-Agent}i” %{CID}n %{SENT_ID}n” combined_with_cookies
CustomLog logs/www.mydomain.com-access_log
combined_with_cookies

Restart apache and now you’ll see you are capturing outgoing cookie id’s and the incoming id’s. No more referrers lost in the shuffle – you can confidently tell where users from site X are going on your site.

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
Leave a Comment :, , , more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...