Category: Technology

to create a space for them

OK, first I have a new favorite quote:

Concentrated power is not rendered harmless by the good intentions of those who create it.
–Milton Friedman

And second… Well, this was a topic I had not expected to be posting about again but the last couple weeks I have found myself spending more and more time building RPM packages for Fedora.  Thankfully the development stack (and documentation) for Fedora is noticeably better than it was for Redhat 9.  So, in my usual fashon, I am listing some of the more useful information I have RECENTLY come across for building RPM packages on Fedora 16.

  • Recommended Method for adding Users & Groups — A Fedora wiki page that discusses the best way to add new users to a system during the rpm install process.  There is no recommendation for REMOVING users during uninstall.  Additionally, rpmlint will scream about un-registered users if you don’t provide reference users for rpmlint.  This bugzilla report discusses how to best alleviate that problem.
  • Packaging Tricks — A stupidly useful Fedora wiki article discussing common issues/fixes for doing package builds.  Some of them are simply look-up problems (like knowing group package groups are available.)   Some of the information is much more advanced package configuration tips (like converting badly encoded files to UTF-8.)  All are really helpful.
  • Frequently Made Mistakes — In the same vein as the Packaging Tricks but specifically focused on problematic RPM methodology.  One correction on this page.  The correct location for checking SPEC files from other Fedora packages is not correctly listed (Fedora doesn’t use CVS anymore.)  The correct location is in their git repository.
  • Creating Sub-packages — Is a very early stage draft document on the Fedora Documentation website that discusses how to best create multiple sub-packages from a given SPEC file.  I had been needing good documentation on this process and this seems to be the start of it.
  • RPM Groups — Raw list of valid RPM package groups.
  • How to Make RPM Packages — Exactly what the name implies.  Probably the best starting point for Fedora Linux software packagers.
  • rpm –showrc — This command will list all the current Macros defined for the rpm build environment.  It even includes your custom local setup.  It is a great place to grep for path information and to verify directory locations for installation.  It has probably been around forever but I honestly didn’t know about it until a couple days ago.
  • rpmdev-setuptree is one of several tools available in the rpmdevtools package (yum install rpmdevtools.)   Running this command will setup a local build directory in THAT USERS home directory (as you should NEVER build packages as root using the system wide build directory.) Additionally it will create a stock .rpmmacros config file.  You will still want to define your own %packager and %vendor macros.
  • Package Guidelines – The definitive guide from Redhat on creating Fedora/Redhat rpm files for distribution.
  • RPM Dev Tools – Web listing of some of the new automated packing tools for RPM based distributions.  Things like creating your default build environment and spec file format checker.
  • CPAN2RPM – A tool for building rpm files from the Comprehensive Perl Archiving Network.  While tools like cpanplus work well for package installation, I prefer the flexibility and consistency of rpm packages and this is a nice way to be able to use rpm files for CPAN modules.
  • cpanspec – Another tool for building spec files (and therefore rpm packages) from cpan repository information.  Generally I use cpan2rpm to create a basic package and then modify the spec file to work anyway, so this might be a better option.

Proud of the ones I’ve got

Got a few really handy debug/development features inside of Catalyst that I have been using lately.

  1. RenderView‘s dump_info parameter.  Basically when using the default view close handler in Catalyst (i.e. Catalyst::Action::RenderView) you can add ?dump_info=1 to the end of any URL in your application server to get the exception dump.  While this will need to be turned off in any live environment, it is crazy helpful during development or when hunting down a bug in your code.
  2. StackTrace  is Catalyst plug-in (i.e. Catalyst::Plugin::StackTrace) that gives better detailed information to the server during the request process.  You add it to the primary project module library under lib/MyProjectName.pm INSIDE the use Catalyst parameter.
  3. If you are using Template Toolkit for your Catalyst template layouts you can include the following hook as well.  [% USE Dumper(Indent=1) -%]  will allow you to Data::Dumper variables to the browser window for debugging purposes.  After specifying the use statement you can simply [% Dumper.dump(variablename) %] out anything you want to see.  Again, bad for production, but really really good for development.
  4. Finally, one of my favorites; if you run DBIC_TRACE=1 script/myapp_server.pl -r for your development server (instead of the standard script/myapp_server.pl) you get automatic refreshes AND full DBIC traces of your error messages.  Think detailed messages with full function states when your server crashes.  The best part is that you simply don’t use the DBIC_TRAC option on production to remove the functionality when you are live.

And if it’s a bad script…

JavaScript has a couple features that I wasn’t aware of until I saw a recent video on Node.js.

  • — splice – A function to pull out a single element from an array.  So [1,2,3,4,5].splice(1,1); will return 2.  Additionally the array will afterwords look like [1,3,4,5].
  • — indexOf – Need to find out what a specific index number for a given value is (basically a reverse index look-up) something like [8,7,6,5,4].indexOf(7); will return a 1.  I actually thought indexOf was a function of the HTML DOM for some reason.
  • — delete – Built in function to delete a specific index of an array.  Something like a=[5,6,7,8]; delete a[0] will remove the individual entry for a[0].  Now depending on environment delete will shorten the array for that element, some environments will simply turn that array element to null effectively blanking out that one entry.

All I must do is find it

The essence of success is that it is never necessary to think of a new idea oneself. It is far better to wait until somebody else does it, and then to copy him in every detail, except his mistakes.
–Aubrey Menen

Two quick things things I wanted to post.  I have used rsync for along time for file backups but generally speaking rysnc is a spectacular tool for efficiently coping files.  Not only does it only transfer the changes between a two files but, it also has a bunch of other useful features.  One I need just recently, a copy command with a progress bar.  So check this out:

rsync -v –progress

Yup, that is it. Simple, clean, efficient, and usable from any scripting interface.

The second thing I wanted to link was an article posted on the Linux Blog related to removing new lines characters from text files.  The post mentions how to due this using tools like tr, awk, perl, sed, and C.  Again, very handy for scripting on Unix systems.

Too remote nor too familiar

Discovered a freakishly use Linux application today that I am already installing on some of my virtual machines.  It is called scponly and functionally what it does is give a user account scp/sftp access to a folder location, under a given user credintials, without actual shell access.  Additionally you can manage the log-in account the same way you do every other account (ala adduser, passwd, chmod, etc.)

Here is a specific example.  Pretend you want to setup sftp access to your apache server (because we all know ftp is terribly insecure.)  Assuming your web root is located at /var/www (example for fedora) you would do the following:

  • Install scponly (yum install scponly)
  • Copy the shell into your allow system shells (which scponly >> /etc/shells)
  • Create a new user (useradd -s /usr/bin/scponly -d /var/www -M sftpUserLogInName)
  • Set user password for scp/sftp sessions (passwd sftpUserLogInName)

Now you can log in remotely using standard ssh to make sure you file transfer is secure.  Directory listing even work under sftp.  Most “quality” ftp clients support sftp (check out fireftp if you use Firefox) so client support is easily available regardless of platform.  The best part, when you want to change permissions or modify the account password you do it with the standard Linux user manager suite, just like every other user on your system.

Lux et Veritas

From the earliest days of software technology, the term hacker has been used to describe someone who was particularly proficient with technology.  While the term “hacker” has come to mean something malicious to those with only a cursory understanding of geek culture, it is still held as a term of high regard among those who know otherwise.  Lately I have gotten the question from students in my IT/IS classes concerning how best to become a “hacker”.  While the question they are asking is not precisely the information they want to know, I have decided to answer the question correctly.

While this may seem like a silly exercise, it actually has a long and storied tradition inside of hacker communities.  There are a number of well known “guides” that have circulated the halls of old Usenet and dusty Jargon files from many years back with contributions from such luminaries as Bruce Parens, Eric S. Raymond, and Larry Wall.  Basically I am going to offer a stupidly basic outline of what I would like to have learned, in the order I would like to have learned it in, to have developed my skill-set (or lack thereof) in the quickest fashion possible.

*One caveat is that I have changed this list 8 times in the last 3 weeks (which is why it has taken so long to post) and added another step just minutes before posting again.  My point is that I will probably continue making updates for a while and I would encourage anyone comments or suggestions on how it can be improved.

  • Step 1: Start by visiting/read these websites, netcasts, mailing lists, IRC forums, & newsgroups.. everyday!  ArsTechnica,  Phrack, SecList, Schneier on Security, OWASP, ITSecurity, cDc, Toms Hardware, HackerNew, and the LiquidMatrix Blog.  Listen to every episode of Security Now!  For extra credit read everything you can handle from the SANS Reading Room.  If you don’t understand what you are reading, Google it!
  • Step 2: The best resources you can possibly have as an IT Professional are OTHER IT Professionals.  This means you need to be part of your local community by networking with other professionals.  Nationally there is ACM, AITP, and IEEE. MORE importantly are your local groups & chapters.  In the Oklahoma City area we have OKC JavaScript Developers Group, OKC LugNuts, Okcruby, DC405, Oklahoma Game Developers, OK InfraGuard, ISSAOKC, and OKCJug, just to name a few.  Join a couple groups and start attending monthly meetings.
  • Step 3: Build your own computer.  Check out the ArsTechnica System Guide to make sure you have compatible hardware that will actually be cost effective and then purchase the parts at PriceWatch (be sure to check the vendor ratings before you buy.)  Yes, you could actually buy a system “kit” from someplace like NewEgg but the value of this exercise is in learning the hows and whys of each component.
  • Step 4: Install a Unix style OS on your newly built system… and/or every other system you can get your hands on.  I recommend either Fedora or Ubuntu to start with as they are the easiest to begin using but in reality any *nix style will work (i.e. FreeBSD, Linux mint, OpenSuse, OpenSolaris, etc.)  No hacker I know lives in Windows (well… unless they are gaming.)  Linux OSes provide access to a system on a level you simply will NEVER get on Windows.  This is partly because of the power of the Unix system model and partly because hackers develop for hackers on Linux/Unix.  After you install your new OS, use it, everyday, as your primary OS.
  • Step 5: Make your own website… from scratch.  Go get a $6.00/month HostMonster account, decide on a domain name, learn FTP, setup an email account, and start making a website!  It is a cheep way to get some direct experience doing many of the most common web task.  Finally, got get a copy of the Lama, Pythonista, or Rails Tutorial and go through the ENTIRE tutorial thus creating your first dynamic website content.
  • Step 6: Continue building your library.  Nobody can know everything there is to know about technology.  Having quick access to information that you have (at the very least) read before is a HUGE help.  Additionally, going through the programming and hacking literature will give you the best hands-on experience necessary to take the next steps.  You should own/have read as many of these as possible:
  • Step 7: Pick a transformative technology and learn how to use it. “Give me six hours to chop down a tree and I will spend the first four sharpening the axe.” — Abraham Lincoln.   In many ways this is the basic activity that all IS/IT employees do on a daily basis.  Technology solutions have a tendency to be very darwinian.  Therefore, you should start by picking tools that have stood the test of time.  Very few technology resources have lasted longer than 4 or 5 years.  Tools like
  • Step 8: Find an Open Source project that interests you, pick a problem, and fix it.  Meaningful contributions to Open Source projects now rank as more valuable than experience with a fortune 500 company and by volunteering yourself to something used by hundreds or thousands of people, you increase your credibility as a contributor to the community.  Often contributors get their names listed in the software itself (imagine being able to show a prospective employer you name in the Firefox contributors tab.)  It doesn’t have to be software development either, many projects need volunteers to answer forum questions, update documentation, translate it to other languages, or simply test and report bugs.
  • Step 9: …and then teach.  No amount of experience will develop your skills, help you network, or let you enjoy technology more than passing on the things you have learned.  Write articles or blog posts, offer to teach a computer class at your local library, or do mentoring to grade school kids.  Sometimes this step can even turn itself into a decent part-time job.
  • Step 10: Download the Linux From Scratch book and build your very own custom operating system from source code.  You will learn more about how operating systems work than 99.9% of IT professionals do.  Doing your own OS build will change the way you see systems and how to protect them.  The results of a LFS build is not the system itself (you will probably go back to using Fedora shortly after finishing) but that you will NEVER go back to be just a technology user…

A Part of the Maine

Even if I knew that tomorrow the world would go to pieces, I would still plant my apple tree.
-Martin Luther

While checking one of my WordPress sites I noticed an update for one of my favorite plug-ins, The Events Calendar.  Everything was broken after the update so I when to the support site to get things running.  Long story short, I got the site working again based on the support recommendations they had but regardless, some functionality was still missing.  The reason for this was related to a huge shift in the underlying focus of the WordPress post design.

the issue you state about other plugins integrating has more to do with WordPress being in a period of flux between having everything be either posts or in its own table and authors fully adopting custom post types. The core WordPress team is placing a heavy emphasis on CPTs and most major plugin authors are moving over… As more and more plugins make the move, the integration you want to see will return in a much more powerful and controlled manner.
–Shane Pearlman

The Events Calendar has converted over to a new design method called Custom Post Types (CPT) whereby individual modules can define their own post “types” instead of adding functionality to default post type already available in WordPress.

This drew my attention because the new version of The Event Calendar provides a “Professional” version, with more functionality, that can be purchased. I haven’t actually seen much software in the WordPress universe that followed this model (and I don’t believe The Event Calendar did until this new update occurred.)

My concerns were confirmed with some web searching:

Custom post types aren’t really meant for that use [¦] Custom post types are great for things that are more or less catalogued: products (in an e-commerce site), listings for a real estate site, etc. For regular content creation as described [by Chris], you can already do [that] by using custom taxonomies and/or stylesheets to make post templates.

Some part of the WordPress team has been pushing these these CPT’s and it looks like they have been doing it primarily to capitalize on the success of a Free Software program. I suspect that most developers who are interested in focusing on this kind of feature set are probably not Free Software developers but are, instead, quasi-open source developers running Macs who would be making iPhone apps if they knew something more useful than PHP.

This kind of monetizing has become massively popular with the success of Apple’s App Store and Google’s Market. A significant number of developers who have built very popular software stacks on top of Open Source applications are looking for ways to turn that work into cash flow, and I don’t blame them. That said…

…I really don’t like something about it. Maybe it is because I left a “free” blog application because it stopped being free (I learned a valuable lesson between Free Software and free software and I still have a bad taste in my mouth about it.) Maybe it is because I have actually contributed work to a number of WordPress plug-ins and would NOT have done so if I had known my efforts were going to help someone else make money. Maybe it is because the new version of The Events Calendar actually broke a lot of functionality in the name of changing their platform model to a for-profit design and now I have to use an unsupported version of the software until I find another or I write one myself.

Whatever the reason, Open Source software is loosing something of itself if this is actually the intent of their focus, and we are all the less because of it.

Linux is Magic

Any sufficiently advanced technology is indistinguishable from magic.
–Arthur C. Clarke

It has been entirely too long since I last ranted about how truly amazing Linux is. I have three different problems in the last 24 hours that all resolved themselves via a fairly simple Linux hack. There is some of links that were useful for resolving my problems:

  • Convert MS/Word to PDF OpenOffice/LibreOffice macro for automated doc to pdf conversion. Use a simply bash script to use call the macro without starting a GUI instance of oowriter.
  • wvWare & antiword — More examples of MS Word command line processing.
  • testdisk — Boot-able Linux CD Rom for fixing broken partition tables, corrupted MBRs, and recovering lost files.  I used to have to do this by hand with fdisk but testdisk makes it MUCH MUCH easier!
  • Repair Broken Grub Multi-boot — Stupidly simply tutorial for using Yast to repair a broken Grub install on OpenSuse.  Includes information on fixing the Windows boot options for Grub as well.
  • zipsplit — Got a zip file too large to upload/transfer/email?  Use zipsplit to split the zip file into multiple files base on size.  Careful, the size is specified in bytes so a 300mb files would be split like zipsplit -n 300000000 myfile.zip.  BTW it is significantly FASTER than re-zipping a file or even unzipping a file.

If you can prove you don’t need it

For years, I have watched the number of technology companies that operate without debt.  The trend has always been popular among IT/IS companies because of the fundamental instability of intellectual property over hard assets.  The logic is hard to argue with.  If everything you “own” of value only has value as a direct cause of its perceived importance, then a shift of public perception doesn’t just hurt your brand, but fundamentally devalues your property.

Think of it this way; if tomorrow everyone stopped trusting Google for their search results (say, you know, someone found out their code sent all our personal information to the Chinese) then overnight they could loose 95% of their US market share.  How much is Google’s code-base worth at that point?  Currently Google is trading at 183 billion so a 95% loss in usage would probably translate to a market value somewhere south of 3-5 billion.

Physical assets don’t behave the same way.  1,000,000 lbs of steal doesn’t just loose 98% of its value overnight.  Even in heavily over-inflated markets things like… I don’t know… homes, don’t loose 98% of their value.  People may be upset that their 350,000 home is now worth 260,000 but just image if one NIGHT your $350,000 home was worth $7,000.  THAT is the danger for companies whose primary assets are intellectual property.

I will give you another concrete example.  Once upon a time there was a company who made A LOT of money in the energy trading business.  Basically the company had sold off almost ALL its physical assets because they made so much money acting as a broker for energy trading.  Think of them as the stock market (or eBay) for energy.  The only problem was that their principal value lay in the fact that people trusted them, trusted their market, trusted their systems, and trusted their software.  Then one day  it was demonstrated that this company lied, cheated, and stole in almost every way you could imagine.  Enron’s stock dropped from $90 to just under $1 in a matter of weeks.  Basically, Enron’s major asset was trust, which it lost, and the company disintegrated overnight.

So how does a company protect itself from such quick devaluation?  The same way you and I protect ourselves from economic turbulence; a big savings account and as little debt as possible.   Microsoft, for example, is famous for “saving” close to a billion dollars a month… yes, a MONTH!  At the same time, Microsoft doesn’t borrow money.  I have been told, by people I put NO trust in to know this information, that they don’t even lease the copiers.  Competitors who want to beat Microsoft can certainly do so, but it will not be an easy fight.  That kind of financial position means that competitors must beat them dollar for dollar, customer for customer, year in and year out… for YEARS!

So who else do you know that doesn’t use debt?  Here are are couple names both in IT and outside of it.  Accenture, Activision Blizzard, Apple, Bed Bath & Beyond, Broadcom , Citrix Systems, eBay, Gap, Google, Infosys Technologies, Juniper Networks, Marvel Technology Group, Qualcomm, Research In Motion, Stryker, Texas Instruments, and Yahoo.  Want to see something more amazing?  Check out those companies 1, 3, and 5 year average returns compared to the market average!

I think it was Warren Buffett who said, “Leverage [i.e. debt] is a funny thing, people who don’t understand it shouldn’t use it; and those who do, don’t.”

13 Deadly Sins

The “Deadly Sins” from P. J. Brown’sWriting Interactive Compilers and Interpreters Wiley 1979.

–to code before you think.
–to assume the user has all the knowledge the compiler writer has.
–to not write proper documentation.
–to ignore language standards.
–to treat error diagnosis as an afterthought.
–to equate the unlikely with the impossible.
–to make the encoding of the compiler dependent on its data formats.
–to use numbers for objects that are not numbers.
–to pretend you are catering to everyone at the same time.
–to have no strategy for processing break-ins.
–(A break-in is when you interrupt an interactive compiler, and then possibly continue it later. This is meaningful in an environment in which the compiler is run dynamically, such as many LISP and some BASIC environments. It is not meaningful for typical uses of C/C++ (although there was at least one interactive C environment according to Chris Lattner).)
–to rate the beauty of mathematics above the usability of your compiler.
–to let any error go undetected.
–to leave users to find the errors in your compiler.

This entry comes to us from the GCC Wiki.