Thursday, September 15, 2011

A Hodgepodge of Python


Here at Adku, we do most of our development in Python. Python is concise, expressive, and well suited to rapid prototyping and readable code. I must admit though, I was initially skeptical of the choice to use Python. In my mind, dynamic typing = bugs, interpreted language = slow. Ahhhhh! But slowly my misgivings have disappeared, and I’ve come to really appreciate the Python Language for the wide coverage of its standard library, and the elegant design of many of the idioms.
For the rest of this post, I’d like to go over three of my favorite python features.
The timeit module
My initial worries about python’s performance versus traditional compiled languages have turned out to be irrelevant*. But in the few times when I’ve had some cause for worry, I found python’s timeit module to be a great help. Unlike other profiling modules, timeit can be run strictly from the command line. So we can quickly answer questions such as the following.
how much is xrange was faster than range?
:~/ $ python -m timeit 'for i in xrange(1000): continue'
100000 loops, best of 3: 19.1 usec per loop
:~/ $ python -m timeit 'for i in range(1000): continue'
10000 loops, best of 3: 24.6 usec per loop
What is the fastest way to reverse a list?
:~/ $ python -m timeit 'l=range(1000)' 'l[::-1]'
100000 loops, best of 3: 12.3 usec per loop
:~/ $ python -m timeit 'l=range(1000)' 'l.reverse()'
100000 loops, best of 3: 8.85 usec per loop
And as you’ll notice from the second example, even simple multiline programs can be timed; just separate lines with quotes.
* a poor algorithm seems to be more often responsible.
** Matt just pointed out to me that more detailed profiling can be obtained with a cProfile (two)liner
python -m cProfile (your script path) > timing.txt
cat timing.txt | sort -snk 4 | tail -n 50



zip
You can read about zip here: http://docs.python.org/library/functions.html#zip. The way I like to think about it is: if nested for loops are like walking through a matrix element by element, zip allows you to just “zip” down the diagonal.
Generally, zip is used to combine columns or rows of data. But there are two uses cases which I think are really cool.
Matrix transpose:
Suppose we are given a matrix expressed as a nested list:
a = [[...],...[...]]
Then we can find the transpose of this matrix by writing
zip(*a)
Grouping a list:
Suppose we have a list
a = [x1, x2, x3, …, xn]
Then we can group it into [(x1,x2), (x3,x4), … (xn-1,xn)] by writing
zip(*[iter(a)]*2)
Replace 2 with any integer k, and you can make k groups.
The main thing to understand in both of these examples is the * operator. In python, * has the usual meaning of multiplication when it’s a binary operator, but as a unary operator it unzip a list and pass it as arguments to a function. Aside from that though, both of these examples just require some staring and thinking to understand. But it’s worth it! I promise
collections.defaultdict
With collections.defaultdict, you can set a default value to the dictionaries you create. No more
a = {}
if not ‘foo’ in a:
a[‘foo’] = 0
else:
a[‘foo’] += 1
Just do
a = collections.defaultdict(int)
a[‘foo’] += 1
...
If you want, you can also do nested dictionaries. For example,
a = collections.defaultdict(lambda: collections.defaultdict(int))
will initialized a “matrix” of all zeroes.
That’s it for now. I would love to hear about more python features in the comments.

66 comments:

  1. a = {}
    a['foo'] = a.get('foo', 0) + 1

    ReplyDelete
  2. Very cool stuff. Post more.... :)
    Thanks

    ReplyDelete
  3. @Matt: defaultdict is faster in many real use situations (after you factor out the cost to import a module).

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(1000): d.get(i, 0)'
    1000 loops, best of 3: 887 usec per loop

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(1000): d[i]'
    1000 loops, best of 3: 1.74 msec per loop

    From that alone you'd be reasonable to prefer your dict.get method. But consider the more common use case (IMO) in which you hit the dictionary at the same key many times.

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(100): [d[i] for j in xrange(100)]'
    100 loops, best of 3: 4.37 msec per loop

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(100): [d.get(i, 0) for j in xrange(100)]'
    100 loops, best of 3: 9.69 msec per loop

    defaultdict(int) is twice as fast ;)

    ReplyDelete
  4. توفر الشركة عددًا كبيرًا من الخدمات المميزة ، بما في ذلك الخبرة الكبيرة لجميع العمال الذين يعملون في الشركة طوال الوقت ، وتعمل على توفير وتوفير عدد كبير من المبيدات الحشرية الفعالة والمميتة لجميع الحشرات. .

    شركة مكافحة حشرات بالرس
    شركة رش مبيدات بالرس
    ارخص شركة مكافحة حشرات

    ReplyDelete
  5. Super site! I am Loving it!! Will return yet again, I'm taking your sustenance, what's more, Thanks. Will surely wrote an article for this on my site

    http://www.caramembuatwebsiteku.com/kenapa-membeli-domain-lama/

    ReplyDelete
  6. المساعدة هنا وكل ما يلزم للحصول على مناشف جميلة ونظيفة هو معرفة الأسباب التي تسبب المشاكل وتعلم كيفية غسل المناشف ومناشف الحمام بشكل صحيح.
    شركة تنظيف
    شركة تنسيق حدائق بالرياض
    شركة تنظيف منازل بالخرج
    شركة تنظيف فلل بالرياض

    ReplyDelete
  7. خدمات فني ستلايت بجميع محافظات وضواحيها محترف فني ستلايت اكثر من 15 عام خبرة في مجال الستلايت, من تركيب ستلايت وبرمجة رسيفر والصيانة وكل الخدمات التي تطلبونها نقوم بتوفيرها لكم فني ستلايت على مدار 24 ساعة بجميع مناطق وعلي مدار أيام الأسبوع 24 ساعه, نصل لبابكم بأسرع وقت ونقدم أفضل الخدمات بدقة وخبرة ومهارة عالية مع الكفالة, وبزمن بسيط.

    ReplyDelete
  8. Thanks for sharing it is important for me. I also searched for that from here. Visit our site AVG contacteren

    ReplyDelete
  9. hi, your post is very helpful for me. Finally, I found exactly what i want. If need information regarding printers then you can visit our site Xerox Printer ondersteuning for help.

    ReplyDelete
  10. hi, Your post is very helpful for me, If you want to know more about antivirus then you can visit our site Canon Printer contacteren for help.

    ReplyDelete
  11. hi, Your post is very helpful for me,finally i got exactly what I want. If you want to know more about antivirus then you can visit our site Bitdefender belgie help.


    Commenting as lilyloo180@gmail.com
    Comment as:

    ReplyDelete
  12. Top quality blog with excellent writing skills and information was very valuable thank you.
    Data Science Course in Hyderabad 360DigiTMG

    ReplyDelete
  13. Writing in style and getting good compliments on the article is hard enough, to be honest, but you did it so calmly and with such a great feeling and got the job done. This item is owned with style and I give it a nice compliment. Better!
    Cyber Security Training in Bangalore

    ReplyDelete
  14. Python programming supports numerous styles such as functional programming, imperative and object-oriented styles. Here are the top five reasons why a computer programmer must learn the Python language: data science course in india

    ReplyDelete
  15. We are really grateful for your blog post. You will find a lot of approaches after visiting your post. I was exactly searching for. Thanks for such post and please keep it up. Great work AOL Desktop Gold Update Error

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. This is a very nice one and gives in-depth information. I am really happy with the quality and presentation of the article. I’d really like to appreciate the efforts you get with writing this post. Thanks for sharing.
    Python Training in Delhi

    ReplyDelete
  20. Priya Chaudhary Label is online designer clothing store to buy Kurti/Kurta, tops, shirts, dresses, pants, palazzos, dupattas, pajama, kaftans & mask for Women .
    Designer Kurtis
    Best online designer clothing stores
    kurti for women
    shirts for women
    Dupatta for women
    Kurta sets for women
    palazzo for women
    dresses for women
    pants for women

    ReplyDelete
  21. Have you been looking for the Zelle customer service support, than you have arrived at the right destination. At this destination you will find the customer service help from zelle, and if you found some issues in using zelle, then you can resolve them by just a click on the given link.

    ReplyDelete
  22. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.
    servicenow training in hyderabad

    ReplyDelete
  23. Very good message. I stumbled across your blog and wanted to say that I really enjoyed reading your articles. Anyway, I will subscribe to your feed and hope you post again soon.ai training in jaipur

    ReplyDelete
  24. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing
    data analytics courses in hyderabad

    ReplyDelete
  25. Two full thumbs up for this magneficent article of yours. I've really enjoyed reading this article today and I think this might be one of the best article that I've read yet. Please, keep this work going on in the same quality. ethical hacking institute in delhi

    ReplyDelete



  26. Such a very useful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. data science institute in nagpur


    ReplyDelete
  27. This blog was really great, never seen a great blog like this before. i think i'm gonna share this to my friends..
    ethical hacking training in hyderabad

    ReplyDelete
  28. Wonderful blog post. This is absolute magic from you! I have never seen a more wonderful post than this one. You've really made my day today with this. I hope you keep this up! data science course in delhi with placement

    ReplyDelete
  29. Great tips and very easy to understand. This will definitely be very useful for me when I get a chance to start my blog. cyber security course in delhi

    ReplyDelete
  30. I recently found a lot of useful information on your website, especially this blog page. Among the lots of comments on your articles. Thanks for sharing.
    best data science training in hyderabad

    ReplyDelete
  31. The information you have posted is very useful. The sites you have referred was good. Thanks for sharing.
    servicenow training in hyderabad

    ReplyDelete
  32. Writing with style and getting good compliments on the article is quite hard, to be honest.But you've done it so calmly and with so cool feeling and you've nailed the job. This article is possessed with style and I am giving good compliment. Best!
    best aws training in hyderabad

    ReplyDelete
  33. Your work is very good and I appreciate you and hopping for some more informative posts cloud computing training in noida

    ReplyDelete
  34. Thank You for providing us with such an insightful information through this blog.
    servicenow training in hyderabad

    ReplyDelete
  35. Thank you for sharing this wonderful blog, I read that Post and got it fine and informative. Please share more like that...
    Ethical Hacking Institute in Bangalore

    ReplyDelete
  36. I wanted to thank you for this great read!! I definitely enjoying every little bit of it waiting for next one.
    Data Analytics Course in Chennai

    ReplyDelete
  37. Hey dude, what kind of blog theme are you using? I want it to use on my blog sbobet login too , Psilocybe Cubensis. I think this is among the most significant information for me. And i am glad reading your article. But want to remark on few general things, The web site style is perfect, the articles is really excellent Good job, cheers!

    ReplyDelete
  38. Really impressed! Information shared was very helpful Your website is very valuable. Thanks for sharing..
    Business Analytics Course in Bangalore

    ReplyDelete
  39. Digital Marketing Course can be a vital tool to cultivate your career and business. In this course work, you will discover what is online marketing, why it is significant, how all businesses are turning from traditional marketing to this online branding.

    For more details, visit our website -Best & Affordable Digital Marketing Course in Chennai

    ReplyDelete
  40. Really an awesome blog and informative content. Keep posting more blogs with us. Thank you.
    Data Science Training in Hyderabad

    ReplyDelete
  41. Gangaur Realtech is a professionally managed organisation specializing in real estate services where integrated services are provided by professionals to its clients seeking increased value by owning, occupying or investing in real estate. data science course in kanpur

    ReplyDelete
  42. Thank you so much for sharing this information with everyone. You truly understand what you're talking about! Bookmarked. You can also get help from my site =). It is possible to have an agreement to change the hyperlink between us! smm panel

    ReplyDelete
  43. You can tell when your projects make a difference from the crowd. There's something unique about the projects. It appears to me that they are all amazing! the smm panel

    ReplyDelete
  44. This is an excellent article. Thanks for sharing this information. I will be visiting your blog regularly for the latest articles. I will be visiting your blog regularly to see some of the latest posts.

    Data Science Training in Ernakulam

    ReplyDelete
  45. I got to this amazing site not long ago. I actually captured with the piece of resources you have got here. Thanks for making such wonderful blog page.
    Business Analytics Course in Indore

    ReplyDelete
  46. 360DigiTMG, the top-rated organisation among the most prestigious industries around the world, is an educational destination for those looking to pursue their dreams around the globe. The company is changing careers of many people through constant improvement, 360DigiTMG provides an outstanding learning experience and distinguishes itself from the pack. 360DigiTMG is a prominent global presence by offering world-class training. Its main office is in India and subsidiaries across Malaysia, USA, East Asia, Australia, Uk, Netherlands, and the Middle East.

    ReplyDelete
  47. Really impressed! Everything is very open and very clear clarification of issues. It contains truly facts. Your website is very valuable. Thanks for sharing. data analytics course in mysore

    ReplyDelete
  48. It is the superset of data mining in which data is collected. It is then cleansed with the help of statistical algorithms to transform it into a model that can efficiently represent data.

    ReplyDelete
  49. https://www.blogger.com/comment.g?blogID=25324432&postID=7917940124762182894&page=3&token=1654059366629

    ReplyDelete
  50. It would help if you thought that the data scientists are the highest-paid employees in a company.
    data science course in kochi

    ReplyDelete
  51. Thanks for sharing the information. Such a very useful article. Very interesting to read this article. I would like to thank you for the efforts you had made for writing this awesome article.
    separation agreement virginia
    new jersey divorce laws property division
    new jersey divorce litigation

    ReplyDelete
  52. AparentLink, the associate advertising an affiliate community web site complicated software than making a normal website for business intelligence.

    ReplyDelete
  53. I do not feel comfortable providing rewritten content promoting affiliate marketing or other online money-making schemes. However, I'm happy to have a constructive discussion on more positive topics that bring value to society.


    Bigg Boss Malayalam

    ReplyDelete