Thursday, September 15, 2011

A Hodgepodge of Python

Here at Adku, we do most of our development in Python. Python is concise, expressive, and well suited to rapid prototyping and readable code. I must admit though, I was initially skeptical of the choice to use Python. In my mind, dynamic typing = bugs, interpreted language = slow. Ahhhhh! But slowly my misgivings have disappeared, and I’ve come to really appreciate the Python Language for the wide coverage of its standard library, and the elegant design of many of the idioms.
For the rest of this post, I’d like to go over three of my favorite python features.
The timeit module
My initial worries about python’s performance versus traditional compiled languages have turned out to be irrelevant*. But in the few times when I’ve had some cause for worry, I found python’s timeit module to be a great help. Unlike other profiling modules, timeit can be run strictly from the command line. So we can quickly answer questions such as the following.
how much is xrange was faster than range?
:~/ $ python -m timeit 'for i in xrange(1000): continue'
100000 loops, best of 3: 19.1 usec per loop
:~/ $ python -m timeit 'for i in range(1000): continue'
10000 loops, best of 3: 24.6 usec per loop
What is the fastest way to reverse a list?
:~/ $ python -m timeit 'l=range(1000)' 'l[::-1]'
100000 loops, best of 3: 12.3 usec per loop
:~/ $ python -m timeit 'l=range(1000)' 'l.reverse()'
100000 loops, best of 3: 8.85 usec per loop
And as you’ll notice from the second example, even simple multiline programs can be timed; just separate lines with quotes.
* a poor algorithm seems to be more often responsible.
** Matt just pointed out to me that more detailed profiling can be obtained with a cProfile (two)liner
python -m cProfile (your script path) > timing.txt
cat timing.txt | sort -snk 4 | tail -n 50

You can read about zip here: The way I like to think about it is: if nested for loops are like walking through a matrix element by element, zip allows you to just “zip” down the diagonal.
Generally, zip is used to combine columns or rows of data. But there are two uses cases which I think are really cool.
Matrix transpose:
Suppose we are given a matrix expressed as a nested list:
a = [[...],...[...]]
Then we can find the transpose of this matrix by writing
Grouping a list:
Suppose we have a list
a = [x1, x2, x3, …, xn]
Then we can group it into [(x1,x2), (x3,x4), … (xn-1,xn)] by writing
Replace 2 with any integer k, and you can make k groups.
The main thing to understand in both of these examples is the * operator. In python, * has the usual meaning of multiplication when it’s a binary operator, but as a unary operator it unzip a list and pass it as arguments to a function. Aside from that though, both of these examples just require some staring and thinking to understand. But it’s worth it! I promise
With collections.defaultdict, you can set a default value to the dictionaries you create. No more
a = {}
if not ‘foo’ in a:
a[‘foo’] = 0
a[‘foo’] += 1
Just do
a = collections.defaultdict(int)
a[‘foo’] += 1
If you want, you can also do nested dictionaries. For example,
a = collections.defaultdict(lambda: collections.defaultdict(int))
will initialized a “matrix” of all zeroes.
That’s it for now. I would love to hear about more python features in the comments.


  1. a = {}
    a['foo'] = a.get('foo', 0) + 1

    1. The effectiveness of IEEE Project Domains depends very much on the situation in which they are applied. In order to further improve IEEE Final Year Project Domains practices we need to explicitly describe and utilise our knowledge about software domains of software engineering Final Year Project Domains for CSE technologies. This paper suggests a modelling formalism for supporting systematic reuse of software engineering technologies during planning of software projects and improvement programmes in Final Year Project Centers in Chennai.

      Software management seeks for decision support to identify technologies like JavaScript that meet best the goals and characteristics of a software project or improvement programme. JavaScript Training in Chennai Accessible experiences and repositories that effectively guide that technology selection are still lacking.

      Aim of technology domain analysis is to describe the class of context situations (e.g., kinds of JavaScript software projects) in which a software engineering technology JavaScript Training in Chennai can be applied successfully

  2. Very cool stuff. Post more.... :)

  3. @Matt: defaultdict is faster in many real use situations (after you factor out the cost to import a module).

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(1000): d.get(i, 0)'
    1000 loops, best of 3: 887 usec per loop

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(1000): d[i]'
    1000 loops, best of 3: 1.74 msec per loop

    From that alone you'd be reasonable to prefer your dict.get method. But consider the more common use case (IMO) in which you hit the dictionary at the same key many times.

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(100): [d[i] for j in xrange(100)]'
    100 loops, best of 3: 4.37 msec per loop

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(100): [d.get(i, 0) for j in xrange(100)]'
    100 loops, best of 3: 9.69 msec per loop

    defaultdict(int) is twice as fast ;)

  4. The article was great, I accidentally saw it and I was really attracted to it, hoping it would be well known. This post is very useful, I admire the author, hope you will succeed in the next post, I will continue to follow your post.
    Games io 2019, Jogos para crianças 2019, Jogos online 360, cá koi mini

  5. توفر الشركة عددًا كبيرًا من الخدمات المميزة ، بما في ذلك الخبرة الكبيرة لجميع العمال الذين يعملون في الشركة طوال الوقت ، وتعمل على توفير وتوفير عدد كبير من المبيدات الحشرية الفعالة والمميتة لجميع الحشرات. .

    شركة مكافحة حشرات بالرس
    شركة رش مبيدات بالرس
    ارخص شركة مكافحة حشرات

  6. Super site! I am Loving it!! Will return yet again, I'm taking your sustenance, what's more, Thanks. Will surely wrote an article for this on my site

  7. بموقع مؤسسة الحرمــين فخدماتنا ليس لها بديل واسعارنا ليس لها مثيل ،ولدينا فريق عمل يتصل مع العملاء على جسور الثقه

    والصدق والامانه فى العمل ، وهدفنا هو ارضاؤك وراحتك ، لا تقلق ونحن معك

    شركه عزل فوم بالجبيل
    لا تجهد نفسك ونحن تحت امرك ورهن اشارتك .
    أبرز خدمات مؤسسة الحرمــين للمقاولات العامة بالدمام والرياض

    شركه عزل فوم بالدمام

    شركه كشف تسربات المياه بالاحساء

  8. المساعدة هنا وكل ما يلزم للحصول على مناشف جميلة ونظيفة هو معرفة الأسباب التي تسبب المشاكل وتعلم كيفية غسل المناشف ومناشف الحمام بشكل صحيح.
    شركة تنظيف
    شركة تنسيق حدائق بالرياض
    شركة تنظيف منازل بالخرج
    شركة تنظيف فلل بالرياض

  9. شركة لحام خزانات المياه بالرياض
    افضل شركة متخصصة في لحام خزانات المياه بالرياض نقوم بعمل لحام لجميع الكسور والشروخ المتواجدة بجدران الخزانات والاجناب للحفاظ علي المياه التي تهدر كل يوم شركة لحام خزانات المياه بالرياض لديها الخبرة الكبيرة التي تجعلها أفضل شركة متخصصة في لحام خزانات المياه الفايبر والخزانات البلاستيك بالرياض
    عند ظهور مشاكل بالخزانات الفايبر يجب الاتصال سريعا علي شركة العمران للحام خزانات المياه بالرياض لحل تلك المشكلة سريعا للحفاظ علي المكان المكسور وعدم اتساعة مرة أخري شركة صيانة ولحام خزانات المياه بالرياض هي شركة لديها جميع انواع المواد التي يمكن ان تقوم بأصلاح الخزانات الفايبر والخزانات البلاستيك بكل سهولة ويسر
    لحام خزانات المياه بالرياض تتوفر لدينا افضل انواع اللحامات التي تعيش طولا بدون مشاكل مع تعرضها للشمض وتعرضها للامطار شركة لحام خزانات المياه بالرياض لديها دائما الجديد
    لحام خزانات المياه
    اصلاح خزانات المياه
    شركة اصلاح الخزانات بالرياض
    شركة لحام الخزانات بالرياض
    لحام الخزانات بالرياض
    تصليح الخزانات بالرياض

  10. If you want to watch Oscars 2020 online Free. Right place for guide how to Academy Awards 2020 live free. Oscars red carpet Live Online
    Oscars 2020 Live
    How does Ruby feel about Oscar as an Oscar?pic. 6 replies 52 retweets 584 likes. Reply. 6. Retweet. 52. Retweeted
    Oscars 2020 Live Stream
    The Oscars 2020 Short Film (Live Action) nominations have been announced! Five films are now competing to take
    Oscars 2020 Live Stream Free

  11. خدمات فني ستلايت بجميع محافظات وضواحيها محترف فني ستلايت اكثر من 15 عام خبرة في مجال الستلايت, من تركيب ستلايت وبرمجة رسيفر والصيانة وكل الخدمات التي تطلبونها نقوم بتوفيرها لكم فني ستلايت على مدار 24 ساعة بجميع مناطق وعلي مدار أيام الأسبوع 24 ساعه, نصل لبابكم بأسرع وقت ونقدم أفضل الخدمات بدقة وخبرة ومهارة عالية مع الكفالة, وبزمن بسيط.