Comprehensions: scary, or amazing?
Judging by this post, I think you’ll find my beliefs fall into the second camp. At first though, they might seem a bit confusing to someone who has never seen them before.
Once you fully understand what’s happening though, the power you can wield is fantastic.
This post assumes some basic knowledge of Python, and programming in general, such as loops, basic data structures (maps/dictionaries, lists/arrays) and conditional branching (if/else/or/not/and, etc…)
Let’s get started!
So first off, what’s a comprehension? It’s simply a way to express a statement. It finds its roots in mathematics, where comprehensions are an everyday thing, and it has been extended as a general construct for many programming languages. In short, it’s another way to express something. Comprehensions always involve some looping construct and can optionally include potential predicates, e.g. branching (flow control).
One thing to note, is that list comprehensions will ultimately result in the same type of results as a traditional for loop, but a comprehension can change the way you think about the same problem. This is where they shine, and because of their terseness, they provide very efficient programming.
Let’s setup a quick print function for testing output.
def print_all(thing):
print '-------------------'
print thing
Now, it’s important to note that comprehensions operate per data structure type. That means we can have list comprehensions or dictionary comprehensions, or even set comprehensions!. I’ll ditch set comprehensions because at face value, they look very similar to lists, except their values must be unique (in set theory notation, this is standard).
List Comprehensions
# We could define an array manually,
# using what might be called a "list literal"
nums = [1, 2, 3, 4, 5]
# or we could do it the the lazy (better) way with comprehensions:
nums = [num for num in xrange(10)]
print_all(nums)
# >>> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# How about odd only, times itself?
nums_odd = [num * num for num in xrange(10) if num % 2 != 0]
print_all(nums_odd)
# >>> [1, 9, 25, 49, 81]
some_evens = [x for x in range(100) if x % 2 == 0 and x > 10]
print_all(some_evens)
# >>> [12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 72, 74, 76, 78, 80, 82, 84, 86, 88, 90, 92, 94, 96, 98]
# Or two separate lists, with a randomized choice combined into a new word!
nouns = ['cat', 'dog', 'human', 'robot', 'dinosaur']
verbs = ['jumping', 'sitting', 'attacking', 'barking', 'purring', 'thinking']
import random
combos = ['And here we have: {} {}!'.format(
random.choice(verbs),
random.choice(nouns)) for verb in verbs]
print_all(combos)
# >>> ['And here we have: attacking robot!',
# 'And here we have: jumping robot!',
# 'And here we have: thinking dinosaur!',
# 'And here we have: attacking robot!',
# 'And here we have: purring cat!',
# 'And here we have: jumping cat!']
# Or use and combine other functions, like pythons
# hex() and unichr() to combine into a tuple of encodings:
nums_encoded = [(unichr(num), hex(num)) for num in nums]
print_all(nums_encoded)
# >>> [(u'\x00', '0x0'), (u'\x01', '0x1'), (u'\x02', '0x2'), (u'\x03', '0x3'), (u'\x04', '0x4'), (u'\x05', '0x5'), (u'\x06', '0x6'), (u'\x07', '0x7'), (u'\x08', '0x8'), (u'\t', '0x9')]
# How about filtering based on some custom filter?
# First we need our own filter
def filter_no_vowels(word):
"""In order to filter something,
the predicate must either return true or false,
so we can have any filter function imaginable,
as long as its test returns a Boolean"""
import re
has_vowels = re.search(r'a|e|a|i|o', word)
if has_vowels:
return False
else:
return True
# Kind of relevant
words = ['cts', 'cats', 'potato', 'spaceship', 'rckt', 'ntro', 'ntrstng']
no_vowels = filter(filter_no_vowels, words)
print_all(no_vowels)
# >>> ['cts', 'rckt', 'ntrstng']
Dictionary Comprehensions
Okay! We got through that okay. Now that we’ve experimented with lists, let’s try another important structure: dictionaries. I’ll assume you know these, but just in case, a dictionary has a key and value that can be unique – whereas a list must be number based for it’s keys. There are other details beyond that, but we’ll forget about them for now.
# Let's get started with a dictionary.
animals_one = {
'Cats': True,
'Dogs': True,
'Birds': 'maybe',
'Raptors': 'hell yes',
'Iguanas': False
}
# Note: here we'll use the **enumerate** function, which will allow
# us to access the keys and values for a dictionary,
# while also using a generator, which is very performant.
# Here we'll grab only the key
new_val = [index for index, key in enumerate(animals_one)]
print_all(new_val)
# >>> [0, 1, 2, 3, 4]
# And here we'll grab both,
# but we'll store each item in an array of tuples.
new_val_both = [(index, key) for index, key in enumerate(animals_one)]
print_all(new_val_both)
# >>> [(0, 'Cats'), (1, 'Raptors'), (2, 'Dogs'), (3, 'Iguanas'), (4, 'Birds')]
# Let's do that again, but only for items that are true
new_val_true = [(index, key) for index, key in enumerate(animals_one) if key is True]
print_all(new_val_true)
# >>> []
# Let's try **modifying** data as we reassign it
animals = {
'Cats': 100,
'Dogs': 10,
'Birds': 45,
'Raptors': 2,
'Iguanas': 1
}
# We'll grab the real value, by accessing the
# key and index (aka subscription) then multiply it
new_val_nums = [(index, animals[key] * animals[key]) for index, key in enumerate(animals)]
print_all(new_val_nums)
# >>> [(0, 10000), (1, 4), (2, 100), (3, 1), (4, 2025)]
# Or how about the first letter of each type plus the amount divided by 2?
# Remember, no dividing by zero!
new_val_fst_div = [(key[0], animals[key] / 2) for index, key in enumerate(animals) if animals[key] != 0]
print_all(new_val_fst_div)
# >>> [('C', 50), ('R', 1), ('D', 5), ('I', 0), ('B', 22)]
# We can keep piling on operations, though it's
# important to strike a balance between brevity and clarity.
new_val_ops = [(animals[key] * animals[key] / 3) for index, key in enumerate(animals) if key != 0]
print_all(new_val_ops)
# >>> [3333, 1, 33, 0, 675]
# We can even combine two lists, making nested comprehensions:
pet_descs = {
'Cats': 'Soft, sleek creatures that generally only eat and sleep.',
'Dogs': 'Cute, lovable, companions',
'Birds': 'Annoying but interesting ex-dinosaurs',
'Raptors': 'Just plain awesome. Actual dinosaurs.',
'Iguanas': 'Ugly, scaly, smelly, but interesting. They often wear sunglasses.'
}
# And call a function on the result if we like.
new_combined = zip(
(pet_descs[key] for index, key in enumerate(pet_descs)),
(animals[key] for index, key in enumerate(animals)))
print_all(new_combined)
# >>> [
# ('Soft, sleek creatures that generally only eat and sleep.', 100),
# ('Just plain awesome. Actual dinosaurs.', 2),
# ('Cute, lovable, companions', 10),
# ('Ugly, scaly, smelly, but interesting. They often wear sunglasses.', 1),
# ('Annoying but interesting ex-dinosaurs', 45)]
# Or even concatenate a string to the first value:
# And call a function on the result if we like.
new_combined_s = zip(
('%s: %s ' % (key, pet_descs[key]) for index, key in enumerate(pet_descs)),
('There are %s of \'em' % animals[key] for index, key in enumerate(animals)))
print_all(new_combined_s)
# >>> [
# ('Cats: Soft, sleek creatures that generally only eat and sleep. ', "There are 100 of 'em"),
# ('Raptors: Just plain awesome. Actual dinosaurs. ', "There are 2 of 'em"),
# ('Dogs: Cute, lovable, companions ', "There are 10 of 'em"),
# ('Iguanas: Ugly, scaly, smelly, but interesting. They often wear sunglasses. ', "There are 1 of 'em"),
# ('Birds: Annoying but interesting ex-dinosaurs ', "There are 45 of 'em")]
Set comprehensions, tuples and beyond…
We’ve explored major uses, and even combined tuples into our comprehensions, but you can see that a lot can be possible. Try experimenting with more!