Learning

Python Generators

Learn how generator functions and generator expressions create iterators, how yield suspends and resumes execution, and when generators are useful.

Levels: Intermediate ~25 min read

Intermediate

A generator is a convenient way to create an iterator. Instead of writing __iter__() and __next__() yourself, you write a function that uses yield. Python pauses that function at each yield and resumes it when the generator is asked for another value.

A generator function uses yield

A function containing a yield expression is a generator function. Calling it does not run the function body to completion. It returns a generator iterator that can produce values when iteration begins.

python

def count_up_to(limit):
    number = 1
    while number <= limit:
        yield number
        number += 1

for number in count_up_to(3):
    print(number)

Calling the generator function creates an iterator; the for loop asks it for values one at a time.

yield pauses and resumes execution

When a generator reaches yield, it produces that value and suspends its execution state. Local variables keep their values. The next request resumes execution after the yield.

python

def steps():
    print("start")
    yield 1
    print("middle")
    yield 2
    print("end")

generator = steps()
print(next(generator))
print(next(generator))
try:
    next(generator)
except StopIteration:
    print("finished")

The print statements show that the generator resumes exactly where its previous yield left execution.

Generators are iterators

A generator object implements the iterator protocol. You can use it with for, pass it to next(), and let normal iteration stop when the generator finishes.

python

generator = (number * 2 for number in range(3))

print(next(generator))
print(next(generator))
print(list(generator))

A generator expression also produces a generator iterator. Once values have been consumed, later iteration continues from the current position.

return ends a generator

A return statement ends a generator. When the generator finishes, iteration raises StopIteration. A return value is available as the value carried by that StopIteration, although ordinary for loops do not expose it.

python

def numbers():
    yield 1
    yield 2
    return "done"

generator = numbers()
print(next(generator))
print(next(generator))
try:
    next(generator)
except StopIteration as error:
    print(error.value)

The generator returns a final value when it finishes; normal for-loop iteration simply stops.

Generator expressions

A generator expression has comprehension-like syntax but produces a generator iterator instead of constructing the complete result immediately. Parentheses are used around the expression.

python

squares = (number * number for number in range(5))

print(next(squares))
print(next(squares))
print(list(squares))

The generator expression produces values as they are requested rather than creating a list of all values.

Choose generators for incremental production

Generators are useful when values can be produced incrementally. They can make iteration code shorter and avoid storing all produced values at once. They do not automatically make every computation faster, and a list is still appropriate when you need a reusable collection of all results.

python

def even_numbers(limit):
    for number in range(limit):
        if number % 2 == 0:
            yield number

for number in even_numbers(8):
    print(number)

This generator produces each matching value only when iteration requests it.

Generators and iterator mechanics

You do not normally implement __iter__() and __next__() for a generator. Python creates the generator iterator machinery for a generator function. This is the main reason generators are a compact way to build iterators.

Methods such as send(), throw(), and close() are part of the generator iterator API, but they are beyond the first generator lesson. They become useful when controlling suspended generator execution explicitly.

Was this page helpful?

Yes No

Mark this learning resource complete to track your learning progress.