blog: Easy_install Isn't

I got a shiny new Linux desktop at work yesterday (yay!), and went about the wonderful adventure of setting up my development environment. Emacs - check. Firefox, Firebug - check. Pylons - here we go.

Here's my first issue: setuptools makes verbose, mixed-case, ugly directories. I don't like it making everything ugly. In fact, it makes it harder to develop with multiple versions of libraries than just using good old symlinks.

OK, so I'll confine it to it's own purgatory inside of ~/.ez/ where it can make UglyDirectoryNames-1.0.3c2b1-DEV/ to its heart's content.

Except... easy_install won't install to a directory that's not on my sys.path.

Excuse me? Let me deal with the sys.path, on my own time, and just do as you're told.

So far, my "easy" installation has consisted of:

• apt-get python-setuptools - actually, this is the only easy step.

• Making a ~/.ez/ directory for it to live in

• Making a .pydistutils.cfg with a prefix=~/.ez in the [install] section. (Which, by the way, is wicked hard to find in Google.)

• Finally, easy_install Pylons. easy_install complains about my PYTHONPATH. I tell it to shut up, yell at my workstation a bit, and try again. No luck.

• Set up my $PYTHONPATH in my ~/.bashrc to include ~/.ez/lib/python2.5/gratuitously/deep/directory/structure/site-packages/

• Start installing Pylons. Headers are missing: apt-get install build-essential and python-dev. Okay, not setuptools' fault.

• Get all the way to WebHelpers, which requires setuptools 0.6c7, not the 0.6c6 that apt-get installed.

• apt-get remove setuptools...

• wget http://peak.telecommunity.com/dist/ez_setup.py

• python ez_setup.py

• alias obscenely_hard_install=easy_install

• obscenely_hard_install Pylons

Now, parts of this is my own fault for not RTFHugeHTMLPage-ing. Yeah, I know there's some Unix or Python convention that easy_install is following in making a 'prefix' directory mean 'the first tiny bit of the prefix, not the bulk of the path that comes after the prefix'. So, can you make a --real-prefix setting? Oh, right, I can instead use --install-dir and --script-dir, both of which are not given nearly enough press by Google.

But can I nuke my easy_install installation to redo it with --install-dir and --script-dir without going through the same amount of pain? Forgive me if I'm not feeling enthusiastic about trying.

I'm just going to be disgruntled and write about it online instead. But, I'm also going to show how I prefer to do things.

A Better Way

Now, this only works if you have real symlinks. So, this is oriented towards, from general to specific, Linux, Debian and Ubuntu, but anything at least marginally POSIXy should work. OSX, for example, should be just fine.

Setting Up Your .bashrc

Inside of your .bashrc, you want to have:

export PYTHONPATH=$PYTHONPATH:~/.pylibs

I like to have the directory hidden. Out of sight, out of mind; but still quick to get to when debugging.

Get Your Libraries

I also have a ~/libs or ~/.libs directory to actually put my copies of the upstream code. I like to leave it versioned so I can 'svn up' or 'hg pull' (etc etc) to get the newest revision. Within it, I'll create a directory for each module, and then pull down each version that I need - usually, trunk, plus at least one stable tag. So, now my directory structure is looking like:

~/.pylibs/
~/libs/
    sqla/
        trunk/
        0.4.0/
    django/
        trunk/
        queryset-refactor/
    ...

Symlink your Favorite Version

Like so:

ln -s ~/libs/sqla/trunk/lib/sqlalchemy ~/.pylibs/
ln -s ~/libs/django/queryset-refactor/django ~/.pylibs/

Switching Versions

You have two ways of switching versions: either environment-wide, or in your code. Changing the version in the environment works something like this:

rm ~/.pylibs/django
ln -s ~/libs/django/trunk/django ~/.pylibs/

In your code, do this instead:

import sys
sys.path.insert(0, "/home/adam/libs/django/trunk")
import django

This should all be familiar to you if you use Python. So I guess I'm just saying... use what's familiar.

But - but - but...

But It's Not Cross-Platform!

True. There is probably a cross-platform solution that is much better, but I don't know it. I'd have written about that, but I can hardly write about things that I don't know exist.

But You Just Made This Up!

True.

But Setuptools is a Standard!

I just prefer Python's built-in packaging. Which is also a standard, sort of, just with shorter sys.paths and prettier directory names.

But this doesn't handle compiling C extensions!

This is actually a problem. I do run setup.py with those, but they're relatively few and far between. This is for your bread-and-butter pure-Python library.