Tuesday, August 16, 2011

pip - python package management

If you’ve ever developed a Python application before, you may be familiar with the litany of packages you need to get everything running. You might start off your web app with Django then add in Jinja2 to get some template speedups. From there, you might want some additional speed so you install packages like Cython, MarksupSafe, and simplejson for their C-extensions. After that, you realize you might want to dabble around with PyMongo as well. You go back to hacking for a while until you start thinking to yourself... how am I ever going to manage all these 3rd party packages? Luckily for us, things aren’t as dire as they seem. Python package management has gotten a lot better over the last few years thanks in part to PyPi, Python’s central package repository and a tool called “pip”


In essence, pip is a python package installer

Things pip can do
  • Manage python packages much like apt-get manages system packages
  • Automatically build C-extensions as required
  • Automatically upgrade/downgrade packages based on your specs

Here’s a quick cheatsheet to the commands we most commonly use

Enumerating installed packages

$ pip freeze

Uninstalling packages

$ sudo pip uninstall Django

Installation - By name and version

$ sudo pip install Django==1.2.5 pycrypto==2.0.1 simplejson==2.1.5

Installation - From a requirements file

A requirements file is a text file that literally looks like the following. This makes it trivial to version control what python packages/versions we have running in our production systems.
Django==1.2.5
pycrypto==2.0.1
simplejson==2.1.5
$ sudo pip install -r my-requirements.txt

We found "pip" to be VERY fast at enumerating already installed packages. To see this for yourself, re-run the above command! We currently have about 35 packages in production and
"pip" can enumerate all of these in about half a second.

Installation - Using a private PyPi server

All commands thus far have managed to magically discover these packages/versions versions and install them on our behalf. Behind the scenes, "pip" installs from PyPi, a public HTTP server that has meta-information regarding all these packages. You can think of PyPi as the closest thing Python has to a central package repository. Most of the time, we can rely on PyPi being up. To isolate yourself from the occasional PyPi hiccup though, you can setup your own private PyPi server. To install packages using your own private server, run

$ sudo pip install [-r my-requirements.txt] [package==version] --index-url http://your-pypi-server:8001/simple

Setting up your own private PyPi server has some benefits, you can
  • Isolate yourself from PyPi going down
  • Download all packages over a local network connection (Speedy!)
  • Manage custom-modified packages and have them install as part of the standard requirements process - for example, if you needed to hack Tornado
Setting up a private PyPi server is a bit beyond the scope of this article but here’s an article that definitely helped us on our way



Thanks to “pip” we’ve managed to tame our package installation process. Hopefully, after reading this article you will have too :)

63 comments:

  1. You should use virtualenv with pip and make your life even better.

    ReplyDelete
  2. One thing to be aware of is that while it is convenient to build packages with C extensions on your production machines, it is better to establish your packaging system in a way that building is different from deployment. A lot of people never really hit that pain point, but if you even think you might run into a situation where you could be releasing some C code, it is to your advantage to see how that effects deployment. Similarly, if you use non-python applications, creating a system for deployment and dependency management that isn't entirely Python focused can help you there.

    Fortunately, pip is a good step towards this as it does like to use simple tarballs that are extracted over a filesystem, much like debs or rpms.

    ReplyDelete
  3. Nice post. Thank you.

    ReplyDelete
  4. I would love to keep track of your posts, it is really a useful source of information, wish you success. It's great, wish you success in the next blog, this is a post that we all should read at least once.
    360 jogo jogar, jogos friv gratis, cá koi mini, games io name

    ReplyDelete
  5. ويتم التخلص من العقارب عن طريق رشها مع جميع المبيدات الحشرية المخصصة لها ، والتي تعمل على التخلص من جميع العقارب السامة.
    افضل شركة مكافحة حشرات
    شركة مكافحه النمل الابيض بالدوادمي
    شركة مكافحه حشرات بالدلم
    شركة مكافحه النمل الابيض ببيشة

    ReplyDelete
  6. I really love the quality writing as offered on this post.

    www.caramembuatwebsiteku.com

    ReplyDelete
  7. Thanks for sharing your inspiring blog the information was very useful to my project ….


    norton.com/setup | norton.com/setup | norton.com/setup | mcafee.com/activate



    ReplyDelete
  8. What a superb post! I have no words to describe this post because everthing is clear with your wonderfull words. I really feel out of world reading your post, it is full of fresh and usefull. I really appreciate, keep the work continue.

    norton.com/setup ! norton.com/setup

    ReplyDelete
  9. Greetings, I’m John. I’m a writer living in London, UK. I am a fan of technology, writing. You can visit my store.
    Mcafee.com/activate
    office.com/setup
    Mcafee.com/activate

    ReplyDelete
  10. trendmicro.com/activation - Follow steps to download, install, and activate Trend Micro product. Type the product key by clickingtrendmicro.com/activation.
    Avast Activate- Get free or paid Avast antivirus. Download, install, and activate it from avast.com/activate

    ReplyDelete
  11. Greetings, I’m John. I’m a writer living in London, UK. I am a fan of technology, writing. You can visit my store.
    Mcafee.com/activate
    Mcafee.com/activate
    Mcafee.com/activate
    Mcafee.com/activate

    ReplyDelete

  12. You need to activate your norton products?. you have to visit norton official website for download norton setup then log in your norton account with email and password. follow few steps to activate your norton setup for any help visit website.

    norton.com/setup

    ReplyDelete

  13. Dragon NaturallySpeaking remains the benchmark for Windows-based speech-to-text technology.
    dragon naturally speaking | dragon naturallyspeaking

    ReplyDelete
  14. McAfee service offers trusted safety to protect data and devices. So, you can shop, surf & keep all your devices safe online with the convenience of a single subscription.
    www mcafee activate | mcafee login my account |
    www mcafee com login
    mcafee livesafe login | my mcafee account |
    mcafee activation code |
    mcafee.com my account

    ReplyDelete
  15. McAfee Antivirus protect The basic antivirus program will do all the essential security measure www.mcafee.com/activate always protect your devices McAfee Antivirus applications which are easily available in the market and also on mcafee.com/login .McAfee antivirus offers protection to your important documents and secures your folder and files on these devices. www.mcafee.com login the most well-known antivirus for security and privacy purpose. If you wish to get McAfee antivirus on the operational system, then follow detail by detail guide by and download McAfee mcafee.com my account.
    mcafee.com my account antivirus offers protection to your important documents and secures your folder and files on these devices. It's the most well-known antivirus for security and privacy purpose mcafee activate product key

    ReplyDelete
  16. We are Offering Escort in kolkata. we've high-class, most beautiful and well
    experienced females who make your gay and night.
    if you are intrested than you can contact: Kolkata Escorts

    ReplyDelete
  17. The Escort in Hyderabad can spruce up entire in an Indian outfit. For example,

    envision our fit bodied young ladies spruced up in an Indian low abdomen sari.

    Goodness, man! it previously removed your faculties. That half naked midsection

    can give you the appeal you require to do some activity in the bed.
    if you are intrested than you can contact: Escort service in Hyderabad

    ReplyDelete
  18. Microsoft Office Suite of products developed by Microsoft that includes Microsoft Word, Excel, Access, PowerPoint, and Outlook. Get some easy steps for easy downloading, installing, activating, and re-installing the Microsoft Office suite by visiting the following links.
    Office.com/setup

    ReplyDelete
  19. eset internet security can be installed quickly on each device if the user follows the correct procedure. You can follow the instructions below to install the setup on your devices such as Windows PC, Mac, or mobile device. Best IT security solutions for your home and business devices. Try ESET antivirus and internet security solutions for Windows, Android, Mac or Linux OS.

    ReplyDelete

  20. Enter Norton Setup Product Key and present Norton antivirus on your PC. click Norton.com/setup Enter a New Product Key.Norton.com/setup is the great antivirus software tool that protect your system.Download Norton setup to install Norton antivirus on your computer. Activate your Norton security key for Norton setup.
    Norton.com/setup

    ReplyDelete
  21. We will tell you how you can install McAfee.com/activate in your computer. We will help your Macafi product for installation.

    ReplyDelete
  22. Epson Printer assistance available 24*7. If you are facing any issues in printer you will get expert assistance through Epson Printer helpline number for fixing all kind of printer related issues within seconds.

    Epson printer support

    ReplyDelete
  23. Mcafee.Com/Activate – Get started with Mcafee setup with easy steps to download and install on Windows, Mac, iOS, and Android. For activation visit
    mcafee.com/activate

    ReplyDelete
  24. Visit McAfee Activate official site www.mcafee.com/activatein any program.

    Enter your 25 digits McAfee Activation Product Key.

    Sign in to your McAfee account with accreditations.

    Snap Submit and Choose "Nation and language".

    Once Logged in, Download your Setup in download organizer.

    Run application to introduce, You're good to go to go!
    for more information can visit:
    mcafee.com/activate

    ReplyDelete

  25. If you suddenly face Mcafee antivirus installation error then you can instantly contact this support service team members of Mcafee and find a solution of complex issues related to Mcafee antivirus.
    mcafee.com/activate

    ReplyDelete
  26. Visit McAfee official website www.mcafee.com/activate. Enter your 25 digits Mcafee activation Product Key Enter your Email and password Click Submit and Log in to your account. Once Logged in, Download your Setup in download folder. Run application to install, you’re all set to go!
    mcafee.com/activate

    ReplyDelete
  27. Thank you so much for this wonderful Post and all the best for your future. I hope to see more post from you. I am satisfied with the arrangement of your post. You are really a talented person I have ever seen.

    mcafee.com/activate

    ReplyDelete
  28. Protects users personal and confidential information safe in devices. The internet accessing devices need security which trusted mcafee antivirus gives you. With protecting antivirus gives your machine protection from malware, viruses and other harmful infected viruses. Visit
    mcafee.com/activate

    ReplyDelete
  29. Thanks for taking time to share with us such a great article. I found such a significant number of fascinating stuff with regards to your blog particularly its discussion. keep doing awesome.
    McAfee.com/Activate | McAfee.com/Activate

    ReplyDelete
  30. Really great article, Glad to read the article. It is very informative for us. Thanks for posting.
    SEO company in Delhi | SEO services in Delhi | SEO company in Gurgaon

    ReplyDelete
  31. Visit mcafee.com/activate to download the McAfee setup on your device. Install it and activate the subscription on www.mcafee.com/activate.

    ReplyDelete
  32. Download the MS Office setup on office.com/setup and easily activate your Office subscription by redeeming the activation key on www.office.com/setup.

    ReplyDelete
  33. norton.com/setup-Get the Norton setup by downloading, installing on the Windows and Mac. Activate the Norton by entering the 25-digit alphanumeric product key at www.norton.com/setup.

    ReplyDelete
  34. webroot.com/safe   is known worldwide for the services that it provides to solve the queries of its customers.  Ask Me Rating  is a critical step ensuring we always have the most recent and accurate details on our web site.

    ReplyDelete
  35. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man,Keep it up.
    Webroot.com/safe | central.bitdefender.com | webroot.com/safe

    ReplyDelete
  36. mcafee.com/activate | www.mcafee.com/activate | mcafee activate |
    McAfee.com/activate - McAfee is among the most utilized and most established antivirus programming accessible available, and naturally so. The antivirus is fit for fighting off current infections and predominant malware to give all-adjust security on various gadgets.

    ReplyDelete
  37. mcafee.com/activate | mcafee activate | mcafee login |
    McAfee backing you up. It offers an assortment of security items like firewalls, hostile to spyware, and unsafe site cautioning. In the event that you need to take a few to get back some composure of every one of these capacities, at that point you need to explore to www.mcafee.com/activate. You can download, introduce, and activate an able McAfee programming for your framework on the website.

    ReplyDelete
  38. Norton antivirus guarantees excellent virus protection without damaging the device performance while going through the scanning process. Norton antivirus protects the system from infected sites, online browsing, viruses, Trojans, and other harmful software. It also blocks the harmful websites, which can inject code to damage all the data and files on the system. Firstly, purchase the Norton
    Norton.com/setup | Norton setup
    http://antivirus-norton.com/
    norton.com/setup
    norton setup
    norton download
    norton account

    ReplyDelete
  39. McAfee.com/Activate – Nowadays, there are loads of individuals confronting the issue of destructive infections or malware on their PCs. So as to annihilate such an issue, clients need to download and introduce an antivirus to secure the framework. McAfee is a standout amongst other antivirus programming accessible in the market that can assist the clients with getting free of the critical step. mcafee.com/activate | mcafee activate

    ReplyDelete
  40. Dear admin, first of all, thank you so much for sharing such useful information with us, I really like your writing style, you are doing such amazing work and your information is very useful for me, I have also something here to share. norton.com/setup
    www.norton.com/setup, Norton product key

    ReplyDelete
  41. visit norton official website for download norton setup then log in your norton

    account with email and password. follow few steps to activate your norton setup

    for any help visit website.

    norton.com/setup

    ReplyDelete
  42. choose mcafee antivirus for your system

    McAfee is exceptional among other security gadgets for any of your devices. This isn't just an ordinary instrument that clears little contaminations and reports bugs.

    In any case, this is one whole full security course of action for your Device. Macintosh or Windows just as it works with various tablets and even mobile phones also.

    With security being the central point of convergence of the McAfee security systems, it in like manner saves you from potential threats and alerts you, To guarantee

    your device with mcafee.
    visit:
    mcafee.com/activate

    ReplyDelete