pip - python package management

If you’ve ever developed a Python application before, you may be familiar with the litany of packages you need to get everything running. You might start off your web app with Django then add in Jinja2 to get some template speedups. From there, you might want some additional speed so you install packages like Cython, MarksupSafe, and simplejson for their C-extensions. After that, you realize you might want to dabble around with PyMongo as well. You go back to hacking for a while until you start thinking to yourself... how am I ever going to manage all these 3rd party packages? Luckily for us, things aren’t as dire as they seem. Python package management has gotten a lot better over the last few years thanks in part to PyPi, Python’s central package repository and a tool called “pip”

http://www.pip-installer.org/en/latest/index.html

In essence, pip is a python package installer

Things pip can do

• Manage python packages much like apt-get manages system packages
  
• Automatically build C-extensions as required
  
• Automatically upgrade/downgrade packages based on your specs

Here’s a quick cheatsheet to the commands we most commonly use

Enumerating installed packages

$ pip freeze

Uninstalling packages

$ sudo pip uninstall Django

Installation - By name and version

$ sudo pip install Django==1.2.5 pycrypto==2.0.1 simplejson==2.1.5

Installation - From a requirements file

A requirements file is a text file that literally looks like the following. This makes it trivial to version control what python packages/versions we have running in our production systems.

Django==1.2.5

pycrypto==2.0.1

simplejson==2.1.5

$ sudo pip install -r my-requirements.txt

We found "pip" to be VERY fast at enumerating already installed packages. To see this for yourself, re-run the above command! We currently have about 35 packages in production and "pip" can enumerate all of these in about half a second.

Installation - Using a private PyPi server

All commands thus far have managed to magically discover these packages/versions versions and install them on our behalf. Behind the scenes, "pip" installs from PyPi, a public HTTP server that has meta-information regarding all these packages. You can think of PyPi as the closest thing Python has to a central package repository. Most of the time, we can rely on PyPi being up. To isolate yourself from the occasional PyPi hiccup though, you can setup your own private PyPi server. To install packages using your own private server, run

$ sudo pip install [-r my-requirements.txt] [package==version] --index-url http://your-pypi-server:8001/simple

Setting up your own private PyPi server has some benefits, you can

• Isolate yourself from PyPi going down
  
• Download all packages over a local network connection (Speedy!)
  
• Manage custom-modified packages and have them install as part of the standard requirements process - for example, if you needed to hack Tornado

Setting up a private PyPi server is a bit beyond the scope of this article but here’s an article that definitely helped us on our way

http://brandonkonkle.com/blog/2010/mar/25/creating-personal-pypi-chishop/

Thanks to “pip” we’ve managed to tame our package installation process. Hopefully, after reading this article you will have too :)