Author: Bobby Rockers

The great growling engine of change

To implement SSL through a Perl Catalyst application it is necessary to use an SSL proxy to relay the HTTP requests through HTTPS.  This setup also means we can use fastcgi for lightweight web calls instead of a full http server with all the overhead that requires.  That said, it has not been entirely straightforward setting-up the proxy.  Therefor, I have started some documentation on getting my current setup running.

This tutorial was what I used to get my companies Catalyst/CouchDB application running on a non-local environment because the official Catalyst tutorial was somewhat… lacking.

To get started we configure fastcgi.  Catalyst kindly provides a fastcgi handler as part of our build screens during project creation. To be able to use the fastcgi handler with a http proxie we need to setup Catalyst to use an internal ip port (much like using sockets or an internal bus) and then we configure our http proxy to listen to forward requests on that internal port.  A quick example test looks something like this:

script/myapp_fastcgi.pl -l 127.0.0.1:8100 -n 5

For long term use you will want to setup the system to run it as a service and start that service during boot.  Here is an example that works for sysinit on Fedora:

#! /bin/sh
### BEGIN INIT INFO
# Provides: catalyst-projectname
# Required-Start: $local_fs $network
# Required-Stop: $local_fs $network
# Default-Start: 3 5
# Default-Stop: 0 1 2 4 6
# Short-Description: Starts the FastCGI app server for the “projectname” catalyst site
# Description: The FastCGI application server for the “projectname” catalyst site
### END INIT INFO

# Source function library.
. /etc/rc.d/init.d/functions

# Source networking configuration.
. /etc/sysconfig/network

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
SITE_HOME=/usr/share/nginx/html
DAEMON=$SITE_HOME/script/projectname_fastcgi.pl
OPTS=”-l 127.0.0.1:8100 -n 5″
NAME=projectname
DESC=”projectname Application Server”
USER=apache

test -f $DAEMON || exit 0
# Check that networking is up.
[ “$NETWORKING” = “no” ] && exit 0

# set -e
lockfile=”/var/lock/subsys/projectname”
pidfile=”/var/run/${NAME}.pid”

start_daemon()
{
echo -n “Starting $DESC: ”
# echo “$NAME.”
daemon $DAEMON $OPTS
retval=$?
echo
[ $retval -eq 0 ] && touch $lockfile
return $retval
}

stop_daemon()
{
echo -n “Stopping $DESC: ”
# echo “$NAME.”
killproc -p $pidfile $NAME
retval=$?
echo
[ $retval -eq 0 ] && rm -f $lockfile
return $retval
}

reload_daemon()
{
# echo “$NAME.”
stop_daemon
start_daemon
}

case “$1” in
start)
start_daemon
;;
stop)
stop_daemon
;;
reload)
reload_daemon
;;
restart|force-reload)
stop_daemon
sleep 5
start_daemon
;;
*)
N=/etc/init.d/$NAME
echo “Usage: $N {start|stop|reload|restart|force-reload}” >&2
exit 1
;;
esac

exit 0

I will post more soon, but for now this was what I needed to get things started.

the whole world is here for me to see

A self inflicted hard reboot caused a bad block write on my Fedora 16 laptop the other day.  Usually this isn’t much of a problem (I have been using Linux everyday for about 12 years now) but I discovered something; something I didn’t know until AFTER the reboot.

Fedora 16 has changed to a using GPT disk labels instead of the old standard DOS partition labels.  While this is a HUGE improvement over a system that has been in place over 30 years now; it does lend to some problems when debugging issues because I have not used this format before.

So over the last couple weeks I have lived with some bad blocks and simply exited out of the rescue boot to complete the boot process (as I tried, without any luck, to fix my problem the old way.)  Well, the solution finally presented itself.  Not only will the updated Linux fsck command fix the problem now, but this solution will fix most system hard drive sector issues; and it is easy.

From the rescue command prompt type:

blkid

To identify the block partitions that are present on the system.  Read your crash error message and identify the bad partition by name; and then locate the partition name in the results of blkid.  Finally run:

fsck -y  /dev/mapper/root.hd-1

Replacing /dev/mapper/root.hd-1 with the full correct path of the drive partition name provided by blkid.  Then finish your boot and go back to the rest of your wonderful Linux experience.

The Hand That Feeds

I have been doing a fair amount of programming lately and undoubtedly this leads to counterproductive lifestyle behaviors while I am deep “in the code.”  Obvious examples include eating pizza three or more times a day, drinking a couple gallons of coffee (always black) at a sitting, failure to exercise (or even leave my chair for that matter), and listening to music at volumes that are generally reserved for the decks of aircraft carriers.  In most cases the music is some mix of metal, techno, or industrial (or in this case all three.)  Because of recent soundtracks for “The Girl with the Dragon Tattoo” and “The Social Network”, I have degenerated to listening to everything ever produced by Trent Reznor.  A habit, I am curtain, will continue until I burn it out of my system and move on to something moderately less abusive… like maybe The Prodigy??!?

So as not to be alone in my depravity, here is the current Top 11 list of favorite Nine Inch Nails/Trent Reznor songs in order of preference:

  1. The Hand That Feeds, With Teeth
  2. Dead Souls, The Crow Soundtrack
  3. Immigrant Song, The Girl With The Dragon Tattoo Soundtrack
  4. Head Like A Hole, Pretty Hate Machine
  5. In Motion, The Social Network Soundtrack
  6. Just Like You Imagined, The Fragile
  7. We’re In This Together, The Fragile
  8. Terrible Lie, Pretty Hate Machine
  9. Hurt, The Downward Spiral
  10. Closer, The Downward Spiral
  11. Only, With Teeth

And while it probably deserves its own list here is an excellent sampling of some of Trent’s slower tempo ballads:

  1. Hurt, The Downward Spiral
  2. Something I Can Never Have, Natural Born Killers Soundtrack
  3. Leaving Hope, And All That Could Have Been

Fear Lying Upon a Pallet

Almost all of of my recent work has been using NoSQL solutions, my favorite of which is Couchdb.  Easily the best feature of Couch is the RESTful JSON API that it uses to provide data.  Because you get your DB queries back to you directly as JavaScript objects, you don’t have to worry about application servers or middle tier systems for N-Tier development.  This is HUGE and make the whole web development (and given that most mobile applications are actually web apps) must cleaner, faster, and more functional for the end user.

Couch does have a couple weaknesses.  The one that has been giving me the most headaches is the lack of documentation for their parameters that the server can handle as part of the JSON View (map/reduce) phase. So here are a number that I have found useful over the last few months.  I will update this list as I find more.

  • key=abc The most commonly passed option to a given couchdb view.  This provides a way to select a single unique (well, I guess probably unique) key for a given view.  That said, view keys DON’T HAVE TO BE UNIQUE in couchdb.  Meaning, that if more than one result returns with abc this will also return those multiple results.
  • keys=[abc,123,ACC] A JSON encoded list of keys to use in a given map/reduce function.  Basically the same as above but without the need to call multiple network queries.
  • startkey=abc Used with endkey=abC to provide reange selection for a given view.  startkey will accept (as valid input) anything that would be valid in a standard couchdb view key, even JSON objects.  So think startkey=[a,]&endkey=[a,{}] to get a range of all keys=[a,somethingElse].
  • endkey=abC Counterpart of startkey, see the above reference.  One thing to note, it is generally better to specify another object and the end of a range if you want to inclusively select a range.  So {} is a better end range value than ZZZZZZZZ is.
  • limit=100 Select on the first N number of results.  This parameter is particularly useful for paginated return results (like “showing 1-100 of 399.)  Reduces network bandwidth for a given request.  Because map/reduce functions are cached upon request, the response time for the server isn’t any faster, but there is less data to download.
  • skip=100 Work with the above parameter limit to return a group result set.  For example you can limit the return result to 100 documents starting from 101 going through 200 (think email counts in gmail) with the ?limit=100&skip=100.
  • descending=true Reverses the return result order.  Will also work with limit, skip, startkey, etc…
  • group=true The default result for a given map/reduce function (which has been re-reduced) is a total, i.e. a single number.  In my case this is seldom the result I am actually looking for so this command provides the bridge between the full re-reduce and what I is most commonly sought, the groups result.  Your final results when this option have been passed it to return the reduced functions grouped by the map keys.  Instead of a single row with {key:null, value:9999} you will get multiple rows with the key being the name of the map key i.e [{key:”bob”,value:”444″},{key:”tom”,value:555}].  If you created design documents and view them inside of Futon, group=true is the default.  Which can be a little confusing when you actually try and make a JSON request and find you get a different result.
  • group_level=2 An alternative to the above parameter is the group_level option which will actual group the resulting reduce by the number of levels specified IF you key is an array matching at least that many arguments.  While the example above is for two levels the number can be as many array places as your key has.  This become particularly helpful when working with years and dates.  For a detailed example checkout this post.  That said, group=true is the functional equivalent of group_level=exact.
  • reduce=false Turn OFF the reduce function on a given map/reduce query.  This is the default if not reduce is defined but you can override it on views that DO have a reduce function if you only want the results of the map.
  • include_docs=true For map queries (that do not have a corresponding view) this option will include the original document in the final result set.  This means the structure of your JSON rows object will be {_id, key, value, doc} instead of the normal {_id, key, value}.  This will save you an additional JSON request if you are using the map query as a lookup for a particular database query.

Landed on Us

The new graphical boot splash for Linux is a program call Plymouth.  It provides feature like kernel mode graphics, flicker free boot messages, full boot logging, and… animation. The install is pretty simply, as the root user do the following:

yum -y install plymouth-theme-*
plymouth-set-default-theme –list (to see a list of all installed plymouth themes)
plymouth-set-default-theme nameOfMyTheme -R

Of particular note is that the -R is different from earlier installs of plymouth that required you run the command plymouth-rebuild-initrd.  Most tutorials online list the old way of rebuilding plymouth and following them will leave you with an unchanged system.

One of the nice features of plymouth is that the boot splash is loaded before the actual boot process when the initial RAM disk image is being loaded.  This means you get the pretty boot image while you are doing things like entering your hard drive decryption pass phrase.

that the ripest might fall

Git is simply amazing when it comes to branching and merging.  I probably have half a dozen branches, each with a unique feature I am working on.  Git makes combining and working with branches so easy that it simply seems natural to store test-functionality on different branches, across multiple repositories, and between different developers.

…and HOLY CRAP is it fast!

That said I hit a problem today that I had to hunt down the answer to, so I am posting it here for easy reference in the future.  The basic issue is that whenever you create a new branch that is then pushed to someone else as a remote branch, git automatically (as would be logical) associates those branches together because they share the same name.  This makes future pushes easy and allows other users to continually get updates when you have commits you want to make available.

The problem occurs when you try to PULL from that new remote branch (because the users or repository has made some of their own changes.)  Git does NOT automatically associate pulls with the remote branch of the same name, even though this associates pushes.  So how do you fix this?  The error message says to modify your config file and look at the git-pull man page, but that could quickly cause insanity based solely on the extent of the complexity of the command set for git.  I probably spent an hour looking through documentation.

The answer was, like you didn’t know where this was going, Google.  Ultimately you can the problem with a fairly simple command that WILL AUTOMATICALLY update your config.

git branch --set-upstream yourBranchName origin/yourBranchName

And that is it! Hopefully that saves someone the time that I lost.

Another fix I ran across comes from an initial pull from a remote repository that is empty. Because the original repository (empty) shares no files in common with my local repository (that I have added files to.) Git gets upset. The error generated looks something like this:

No refs in common and none specified; doing nothing.
Perhaps you should specify a branch such as ‘master’.
fatal: The remote end hung up unexpectedly

It can be fixed with the following command:

git push origin master

Then all remaining pushes should be fine as now you have a shared reference to use for future pushes.

Geniuses remove it

I am starting to believe that in the Poettering household, simplicity was considered a cancer that must be tortured and destroyed with extreme vigor.  Systemd is quickly becoming thoroughly ubiquitous in Linux systems everywhere.  While Systemd tries to do everything for everybody (it is supposed to eventually replace sysvinit, chkconfig, automount, logging, cron, and a whole host of other things) ultimately the primary intent of Systemd is to speed up the boot process.  It does this job exceedingly well.  This concern about boot time is a direct response to the speed of which other Unix based OSes boot and reference material even explicitly points to Apple as a reference.

That said, sysvinit did have one thing going for it… IT WAS SIMPLE.  Heck, just getting a list of available services is a pain in the ass now and generally requires looking up documentation just to remember how to do it.  Simple actions in systemd are annoyingly complex with a cheat sheet that looks like it was written by a Perl regular expression programmer on acid.  I will be the first to admit that systemd-analyze plot is pretty awesome and, considering that systemd was designed by the same guy who created PulseAudio, we should probably be thankful that it isn’t even MORE complex.  But still, something just seems wrong about using an all-for-everything program on an OS that was designed to be simple and efficient.