Textbook of Python

A process is an independent instance of a running program with its own memory space, file descriptors, and Python interpreter. Unlike threads, processes do not share memory by default, so they are not affected by the GIL — each process has its own GIL. This makes processes the standard approach for CPU-bound parallelism in Python.

The multiprocessing module provides an API deliberately similar to threading: you create Process objects with target functions, start them, and join them. It also provides inter-process communication primitives — Queue, Pipe, Value, Array, and Manager — for sharing data between processes. Data passed between processes must be picklable (serialisable with the pickle module).

The main trade-off is overhead: creating a process is more expensive than creating a thread (each process needs its own Python interpreter and memory space), and inter-process communication is slower than shared memory. For most use cases, concurrent.futures.ProcessPoolExecutor provides a simpler interface. The multiprocessing module also supports Pool objects that manage a pool of worker processes and provide map() and apply_async() for distributing work.