Textbook of Python

A generator is a special kind of function that produces a sequence of values lazily, one at a time, using the yield keyword instead of return. When a generator function is called, it does not execute its body immediately; instead, it returns a generator object that implements the iterator protocol (__iter__ and __next__). Each call to next() on the generator resumes execution from where it last yielded, runs until the next yield, and suspends again.

Generators are Python's primary tool for lazy evaluation. They are ideal for processing large datasets, infinite sequences, or pipelines where building the entire collection in memory would be wasteful or impossible. A generator that reads a 10 GB file line by line uses memory proportional to one line, not the entire file.

Generator expressions ((x**2 for x in range(10))) provide a compact syntax analogous to list comprehensions but producing values lazily. The yield from statement (Python 3.3+) delegates to a sub-generator, enabling clean composition of generators. Generators are closely related to coroutines — yield can both produce and receive values — though modern Python async programming uses async def and await rather than generator-based coroutines.