Python Iterators

Intermediate

An iterator is an object that produces values one at a time. Python uses iterators throughout the language, including when a for loop processes an iterable.

Iterable and iterator are different

An iterable is an object you can pass to iter or use in a for loop. Calling iter() on an iterable returns an iterator. The iterator keeps track of where it is in the sequence of values.

numbers = [10, 20, 30]
iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

An iterable can usually produce a fresh iterator when you call iter() again. An iterator, by contrast, keeps its current position and is consumed as you advance it.

Use next to get the next value

The built-in next() asks an iterator for its next value. When there are no more values, the iterator raises StopIteration. A for loop handles that stopping condition for you.

iterator = iter(["red", "green"])

print(next(iterator))
print(next(iterator))

try:
    next(iterator)
except StopIteration:
    print("No more values")

Stop a callable iterator with a sentinel

The built-in iter(callable, sentinel) form is different from iter(iterable). Python calls the callable with no arguments each time the iterator needs a value. The returned value is yielded unless it compares equal to the sentinel; that matching value stops iteration and is not yielded.

For a text stream, readline() returns an empty string at end-of-file. This in-memory example uses StringIO so it can be run without reading or changing a real file.

from io import StringIO

stream = StringIO("Ada\nGrace\nLinus\n")

for line in iter(stream.readline, ""):
    print(line.strip())

A blank line is not the same as end-of-file: readline() returns '\n' for a blank line, but '' when no more data is available. Choose a sentinel that represents the actual stopping condition for your callable.

Let for use an iterator

The for statement obtains an iterator from its iterable and repeatedly requests values until iteration finishes. You normally do not need to call next() yourself when a for loop expresses the task clearly.

numbers = [1, 2, 3]

for number in numbers:
    print(number)

Understand the iterator protocol

An iterator provides __iter__() and __next__() methods. __iter__() returns the iterator itself, while __next__() returns the next value or raises StopIteration when iteration is finished.

class CountDown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self

    def __next__(self):
        if self.current == 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value

for number in CountDown(3):
    print(number)

This protocol is why many different Python objects can work with the same for syntax. The loop does not need to know how the iterator stores its state.

An iterator is usually consumed

Once an iterator reaches the end, calling next() again keeps raising StopIteration. If you need to iterate over the original collection again, create a new iterator from the iterable.

numbers = [1, 2, 3]
iterator = iter(numbers)

print(list(iterator))
print(list(iterator))

Iterables do not have to be sequences

Iteration is broader than indexing. A for loop can consume sets, dictionaries, strings, files, and many other iterable objects even when they do not behave like a sequence with integer indexes.

word = "foxipa"

for character in word:
    print(character)

Choose the abstraction that fits

Use an iterable when you want to represent values that can be traversed. Use an iterator when the current traversal position needs to be maintained and values should be produced one at a time. For everyday loops, prefer the clear for form unless you specifically need manual control with next().