Python Generator Expressions

Intermediate

A generator expression is a compact expression that creates a generator iterator. It uses comprehension-like syntax, but parentheses make it a generator expression instead of a list, set, or dictionary comprehension.

The basic form

The general form is (expression for item in iterable). The expression is evaluated as iteration advances, so the generator expression does not build all of its results up front.

numbers = range(1, 4)
squares = (number * number for number in numbers)

for square in squares:
    print(square)

Generator expression vs. list comprehension

A list comprehension uses square brackets and creates a list immediately. A generator expression uses parentheses and creates a generator iterator. Choose the form that matches whether you need a materialized list or a value-producing iterator.

numbers = range(1, 4)

squares_list = [number * number for number in numbers]
squares_generator = (number * number for number in numbers)

print(squares_list)
print(list(squares_generator))

Generator expressions are iterators

A generator expression produces a generator iterator. You can advance it with next() or consume it with a for loop. Like other iterators, it becomes exhausted after its values have been consumed.

numbers = range(3)
squares = (number * number for number in numbers)

print(next(squares))
print(next(squares))
print(next(squares))

Using an expression with a function

When a function call has one generator-expression argument, Python allows the parentheses belonging to the generator expression to be omitted. The resulting call still receives a generator iterator.

numbers = range(1, 5)
total = sum(number * number for number in numbers)

print(total)

Choose the right form

Use a generator expression when you want values to be produced during iteration and do not need the complete collection at once. Use a list comprehension when you specifically need a list. A generator expression is not simply a shorter spelling for a list comprehension; it produces a different kind of object with different evaluation and storage behavior.

Generator expressions and generator functions

Both generator expressions and generator functions produce generator iterators. Generator expressions are convenient for a compact single expression. Use a generator function when the production logic needs statements, branching, multiple yield points, or other control flow.

def squares_up_to(limit):
    for number in range(1, limit + 1):
        yield number * number

expression = (number * number for number in range(1, 4))
function = squares_up_to(3)

print(list(expression))
print(list(function))