Python › Comprehensions & Generators

How would you profile whether a generator expression is actually providing a measurable memory or performance benefit over a list comprehension in a specific use case?

Advanced

Answer

Use the 'tracemalloc' module (or sys.getsizeof() for a rough comparison) to measure actual memory usage of a materialized list versus a generator object, and 'timeit' to compare execution time for the specific consuming operation — the benefit is usually most measurable for LARGE datasets combined with EARLY TERMINATION or streaming consumption, not for small collections consumed in full anyway.
import sys

list_version = [n * n for n in range(1_000_000)]
gen_version = (n * n for n in range(1_000_000))

print(sys.getsizeof(list_version))  # several MB
print(sys.getsizeof(gen_version))   # ~200 bytes, regardless of the range size
Real-world example Deciding whether converting a hot-path list comprehension into a generator expression is actually worth the (minor) readability trade-off for a given dataset size.

Common follow-ups: At roughly what collection size does the memory difference between a list and a generator become practically significant?