Thursday, September 15, 2011

A Hodgepodge of Python


Here at Adku, we do most of our development in Python. Python is concise, expressive, and well suited to rapid prototyping and readable code. I must admit though, I was initially skeptical of the choice to use Python. In my mind, dynamic typing = bugs, interpreted language = slow. Ahhhhh! But slowly my misgivings have disappeared, and I’ve come to really appreciate the Python Language for the wide coverage of its standard library, and the elegant design of many of the idioms.
For the rest of this post, I’d like to go over three of my favorite python features.
The timeit module
My initial worries about python’s performance versus traditional compiled languages have turned out to be irrelevant*. But in the few times when I’ve had some cause for worry, I found python’s timeit module to be a great help. Unlike other profiling modules, timeit can be run strictly from the command line. So we can quickly answer questions such as the following.
how much is xrange was faster than range?
:~/ $ python -m timeit 'for i in xrange(1000): continue'
100000 loops, best of 3: 19.1 usec per loop
:~/ $ python -m timeit 'for i in range(1000): continue'
10000 loops, best of 3: 24.6 usec per loop
What is the fastest way to reverse a list?
:~/ $ python -m timeit 'l=range(1000)' 'l[::-1]'
100000 loops, best of 3: 12.3 usec per loop
:~/ $ python -m timeit 'l=range(1000)' 'l.reverse()'
100000 loops, best of 3: 8.85 usec per loop
And as you’ll notice from the second example, even simple multiline programs can be timed; just separate lines with quotes.
* a poor algorithm seems to be more often responsible.
** Matt just pointed out to me that more detailed profiling can be obtained with a cProfile (two)liner
python -m cProfile (your script path) > timing.txt
cat timing.txt | sort -snk 4 | tail -n 50



zip
You can read about zip here: http://docs.python.org/library/functions.html#zip. The way I like to think about it is: if nested for loops are like walking through a matrix element by element, zip allows you to just “zip” down the diagonal.
Generally, zip is used to combine columns or rows of data. But there are two uses cases which I think are really cool.
Matrix transpose:
Suppose we are given a matrix expressed as a nested list:
a = [[...],...[...]]
Then we can find the transpose of this matrix by writing
zip(*a)
Grouping a list:
Suppose we have a list
a = [x1, x2, x3, …, xn]
Then we can group it into [(x1,x2), (x3,x4), … (xn-1,xn)] by writing
zip(*[iter(a)]*2)
Replace 2 with any integer k, and you can make k groups.
The main thing to understand in both of these examples is the * operator. In python, * has the usual meaning of multiplication when it’s a binary operator, but as a unary operator it unzip a list and pass it as arguments to a function. Aside from that though, both of these examples just require some staring and thinking to understand. But it’s worth it! I promise
collections.defaultdict
With collections.defaultdict, you can set a default value to the dictionaries you create. No more
a = {}
if not ‘foo’ in a:
a[‘foo’] = 0
else:
a[‘foo’] += 1
Just do
a = collections.defaultdict(int)
a[‘foo’] += 1
...
If you want, you can also do nested dictionaries. For example,
a = collections.defaultdict(lambda: collections.defaultdict(int))
will initialized a “matrix” of all zeroes.
That’s it for now. I would love to hear about more python features in the comments.

44 comments:

  1. a = {}
    a['foo'] = a.get('foo', 0) + 1

    ReplyDelete
    Replies
    1. The effectiveness of IEEE Project Domains depends very much on the situation in which they are applied. In order to further improve IEEE Final Year Project Domains practices we need to explicitly describe and utilise our knowledge about software domains of software engineering Final Year Project Domains for CSE technologies. This paper suggests a modelling formalism for supporting systematic reuse of software engineering technologies during planning of software projects and improvement programmes in Final Year Project Centers in Chennai.

      Software management seeks for decision support to identify technologies like JavaScript that meet best the goals and characteristics of a software project or improvement programme. JavaScript Training in Chennai Accessible experiences and repositories that effectively guide that technology selection are still lacking.

      Aim of technology domain analysis is to describe the class of context situations (e.g., kinds of JavaScript software projects) in which a software engineering technology JavaScript Training in Chennai can be applied successfully

      Delete
  2. Very cool stuff. Post more.... :)
    Thanks

    ReplyDelete
  3. @Matt: defaultdict is faster in many real use situations (after you factor out the cost to import a module).

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(1000): d.get(i, 0)'
    1000 loops, best of 3: 887 usec per loop

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(1000): d[i]'
    1000 loops, best of 3: 1.74 msec per loop

    From that alone you'd be reasonable to prefer your dict.get method. But consider the more common use case (IMO) in which you hit the dictionary at the same key many times.

    python -m timeit 'from collections import defaultdict' 'd = defaultdict(int)' 'for i in xrange(100): [d[i] for j in xrange(100)]'
    100 loops, best of 3: 4.37 msec per loop

    python -m timeit 'from collections import defaultdict' 'd = {}' 'for i in xrange(100): [d.get(i, 0) for j in xrange(100)]'
    100 loops, best of 3: 9.69 msec per loop

    defaultdict(int) is twice as fast ;)

    ReplyDelete
  4. The article was great, I accidentally saw it and I was really attracted to it, hoping it would be well known. This post is very useful, I admire the author, hope you will succeed in the next post, I will continue to follow your post.
    Games io 2019, Jogos para crianças 2019, Jogos online 360, cá koi mini

    ReplyDelete
  5. توفر الشركة عددًا كبيرًا من الخدمات المميزة ، بما في ذلك الخبرة الكبيرة لجميع العمال الذين يعملون في الشركة طوال الوقت ، وتعمل على توفير وتوفير عدد كبير من المبيدات الحشرية الفعالة والمميتة لجميع الحشرات. .

    شركة مكافحة حشرات بالرس
    شركة رش مبيدات بالرس
    ارخص شركة مكافحة حشرات

    ReplyDelete
  6. Super site! I am Loving it!! Will return yet again, I'm taking your sustenance, what's more, Thanks. Will surely wrote an article for this on my site

    http://www.caramembuatwebsiteku.com/kenapa-membeli-domain-lama/

    ReplyDelete
  7. بموقع مؤسسة الحرمــين فخدماتنا ليس لها بديل واسعارنا ليس لها مثيل ،ولدينا فريق عمل يتصل مع العملاء على جسور الثقه

    والصدق والامانه فى العمل ، وهدفنا هو ارضاؤك وراحتك ، لا تقلق ونحن معك

    شركه عزل فوم بالجبيل
    لا تجهد نفسك ونحن تحت امرك ورهن اشارتك .
    أبرز خدمات مؤسسة الحرمــين للمقاولات العامة بالدمام والرياض

    شركه عزل فوم بالدمام


    شركه كشف تسربات المياه بالاحساء

    ReplyDelete
  8. المساعدة هنا وكل ما يلزم للحصول على مناشف جميلة ونظيفة هو معرفة الأسباب التي تسبب المشاكل وتعلم كيفية غسل المناشف ومناشف الحمام بشكل صحيح.
    شركة تنظيف
    شركة تنسيق حدائق بالرياض
    شركة تنظيف منازل بالخرج
    شركة تنظيف فلل بالرياض

    ReplyDelete
  9. شركة لحام خزانات المياه بالرياض
    افضل شركة متخصصة في لحام خزانات المياه بالرياض نقوم بعمل لحام لجميع الكسور والشروخ المتواجدة بجدران الخزانات والاجناب للحفاظ علي المياه التي تهدر كل يوم شركة لحام خزانات المياه بالرياض لديها الخبرة الكبيرة التي تجعلها أفضل شركة متخصصة في لحام خزانات المياه الفايبر والخزانات البلاستيك بالرياض
    عند ظهور مشاكل بالخزانات الفايبر يجب الاتصال سريعا علي شركة العمران للحام خزانات المياه بالرياض لحل تلك المشكلة سريعا للحفاظ علي المكان المكسور وعدم اتساعة مرة أخري شركة صيانة ولحام خزانات المياه بالرياض هي شركة لديها جميع انواع المواد التي يمكن ان تقوم بأصلاح الخزانات الفايبر والخزانات البلاستيك بكل سهولة ويسر
    لحام خزانات المياه بالرياض تتوفر لدينا افضل انواع اللحامات التي تعيش طولا بدون مشاكل مع تعرضها للشمض وتعرضها للامطار شركة لحام خزانات المياه بالرياض لديها دائما الجديد
    لحام خزانات المياه
    اصلاح خزانات المياه
    شركة اصلاح الخزانات بالرياض
    شركة لحام الخزانات بالرياض
    لحام الخزانات بالرياض
    تصليح الخزانات بالرياض
    alomrancoo.com

    ReplyDelete
  10. https://oscars2020news.com If you want to watch Oscars 2020 online Free. Right place for guide how to Academy Awards 2020 live free. Oscars red carpet Live Online
    Oscars 2020 Live
    How does Ruby feel about Oscar as an Oscar?pic. 6 replies 52 retweets 584 likes. Reply. 6. Retweet. 52. Retweeted
    Oscars 2020 Live Stream
    The Oscars 2020 Short Film (Live Action) nominations have been announced! Five films are now competing to take
    Oscars 2020 Live Stream Free

    ReplyDelete
  11. خدمات فني ستلايت بجميع محافظات وضواحيها محترف فني ستلايت اكثر من 15 عام خبرة في مجال الستلايت, من تركيب ستلايت وبرمجة رسيفر والصيانة وكل الخدمات التي تطلبونها نقوم بتوفيرها لكم فني ستلايت على مدار 24 ساعة بجميع مناطق وعلي مدار أيام الأسبوع 24 ساعه, نصل لبابكم بأسرع وقت ونقدم أفضل الخدمات بدقة وخبرة ومهارة عالية مع الكفالة, وبزمن بسيط.

    ReplyDelete
  12. Thanks for sharing it is important for me. I also searched for that from here. Visit our site AVG contacteren

    ReplyDelete
  13. I am really enjoying your site.It’s simple, yet effective, thank you for this article.Now I have to share some information about How To Fix “mcafee Antivirus” problem. If you have any problem rearding Mcafee so click on this site:mcafee antivirus nummer belgie

    ReplyDelete
  14. I like your blog.You have done Excellent work. I appreciate.Here I want to inform all of you if you are looking for to resolve your Norton Antivirus problems,so you are in right place.we always available for your support.So whenever you need any help so just click on this link- norton Antivirus ondersteuningsnummer

    ReplyDelete
  15. I’m really impressed with your writing skills and also with the layout on your blog it's Very interesting to read.Now Here i would llike to share some information about HP Printer If you are facing any problem relate to your HP Printer's we wil resolve your queries at sam time.For any help please visit on our website:hp printer contact belgie

    ReplyDelete
  16. Excellent post. I certainly appreciate this website.Keep writing.well here if you want to Overcome the issues of Avast antivirus.Pick the Best Assistance over our site to resolve your queries.Visit us :avast antivirus ondersteuning

    ReplyDelete
  17. hi, your post is very helpful for me. Finally, I found exactly what i want. If need information regarding printers then you can visit our site Xerox Printer ondersteuning for help.

    ReplyDelete
  18. hi, Your post is very helpful for me, If you want to know more about antivirus then you can visit our site Canon Printer contacteren for help.

    ReplyDelete
  19. hi, Your post is very helpful for me,finally i got exactly what I want. If you want to know more about antivirus then you can visit our site Bitdefender belgie help.


    Commenting as lilyloo180@gmail.com
    Comment as:

    ReplyDelete
  20. hi, Your post is very helpful for me, finally i found exactly what i want , If you want to know more about antivirus then you can visit our site Kaspersky antivirus nummer for help.

    ReplyDelete
  21. That’s a nice article, thank you for a great article. It helped me a lot. Keep it up Must Visit Epson Printer helpdesk

    ReplyDelete
  22. فني ستلايت فى الكويت نحن نقدم افضل خدمة فى دولة الكويت يمكنكم التواصل معنا من خلال رقم فني ستلايت ونقدم العديد من الخدمات مثل تجديد اشتراك بي ان سبورت بالاضافة الى هنالك تخفيضات هائلة على اسعار تجديد اشتراك بي ان سبورت

    ReplyDelete
  23. That’s a nice article, thank you for a great article. It helped me a lot. Keep it up Must Visit panda klantendienst

    ReplyDelete
  24. فني ستلايت فى الكويت نحن نقدم افضل خدمة فى دولة الكويت يمكنكم التواصل معنا من خلال رقم فني ستلايت ونقدم العديد من الخدمات مثل تجديد اشتراك بي ان سبورت بالاضافة الى هنالك تخفيضات هائلة على اسعار تجديد اشتراك بي ان سبورت

    ReplyDelete
  25. Really Nice Post Admin, Very helpful looking for more posts, Now I have to share some information about How To Fix “Epson Troubleshooting Guide” problem. If you are going through this problem you can simply Epson Printer helpdesk

    ReplyDelete
  26. That’s a nice article, thank you for a great article. It helped me a lot. Keep it up Must Visit Lexmark Printer nummer belgie

    ReplyDelete
  27. Top quality blog with excellent writing skills and information was very valuable thank you.
    Data Science Course in Hyderabad 360DigiTMG

    ReplyDelete