Textbook of Python

A generator expression is a compact syntax for creating a generator object. It looks exactly like a list comprehension but uses parentheses instead of square brackets: (x**2 for x in range(1000000)) creates a generator that yields squared values lazily, one at a time, without building a million-element list in memory.

Generator expressions are ideal when you need to iterate over a transformed sequence exactly once and do not need random access or the ability to re-iterate. They are commonly passed directly to functions that consume iterables: sum(x**2 for x in data), max(len(line) for line in file), ','.join(str(x) for x in items). When a generator expression is the sole argument to a function, the outer parentheses can be omitted: sum(x for x in data).

Under the hood, a generator expression creates a generator function and immediately calls it. It shares the same suspension and resumption mechanics as generator functions defined with yield. The memory advantage is significant for large datasets: a list comprehension over a million items allocates a million-element list; a generator expression allocates only the generator state (a few hundred bytes). Use list comprehensions when you need the entire list; use generator expressions when you only need to iterate once.