15 questions found
How would you profile memory allocations specifically (not just CPU time) to find which objects are consuming the most memory in a running process?
Advanced
Combine tracemalloc's snapshot statistics grouped by 'traceback' (not just 'lineno') to see the FULL call stack responsible for each allocation, or use a dedicated tool like objgraph to visualize reference chains keeping specific objects alive — CPU profilers like cProfile don't measure memory at all, so a dedicated memory-focused tool is required.
import tracemalloc
tracemalloc.start(25) # capture up to 25 frames of traceback per allocation
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics("traceback")
for stat in top_stats[:3]:
print(stat)
for line in stat.traceback.format():
print(line)
Real-world example
Diagnosing exactly which code path is responsible for the largest chunk of memory usage in a long-running data processing service.
Common follow-ups: How does objgraph's show_backrefs() help find WHY a specific object is still being kept alive by the garbage collector?
Memory Management & Garbage Collection
How do you use conditional breakpoints in pdb to pause execution only when a specific condition is met, avoiding manually stepping through many unrelated iterations?
Advanced
Use `pdb.set_trace()` guarded by an 'if' condition directly in your code (the simplest approach), OR set a conditional breakpoint via the 'break' command with a trailing condition when running the debugger interactively (e.g., `break 42, x > 100`), pausing only when that specific condition becomes true.
def process_items(items):
for i, item in enumerate(items):
if item.value > 1000: # only pause for the specific problematic case
breakpoint()
process(item)
# Or interactively in pdb: break myfile.py:42, item.value > 1000
Real-world example
Debugging a bug that only manifests for a specific, hard-to-reach value deep inside a large loop, without manually stepping through every prior iteration.
Common follow-ups: How do you set a conditional breakpoint directly from the interactive pdb prompt using the 'break' command's condition syntax?
Iterators & the Iterator Protocol
How would you use line_profiler (a third-party tool) to get LINE-BY-LINE timing information within a specific function, more granular than cProfile's function-level data?
Advanced
Decorate the target function with @profile (line_profiler's marker decorator) and run the script with `kernprof -l -v script.py` — this produces a report showing the exact TIME SPENT on EACH INDIVIDUAL LINE within that function, pinpointing precisely which line (not just which function) is the actual bottleneck.
@profile # requires running via kernprof, not a normal Python run
def process_data(data):
cleaned = [x.strip() for x in data] # line-level timing shown for THIS line
filtered = [x for x in cleaned if x] # and separately for THIS line
return sorted(filtered)
# Run with: kernprof -l -v script.py
Real-world example
Pinpointing the EXACT line inside a slow function responsible for most of its execution time, beyond what function-level profiling shows.
Common follow-ups: Why can't you simply 'import' the @profile decorator normally, and what does kernprof do to make it available?
Debugging & Profiling
How would you use py-spy (a sampling profiler) to profile a running Python process in PRODUCTION without restarting it or adding any code changes?
Advanced
py-spy attaches to an ALREADY-RUNNING Python process from OUTSIDE it (using OS-level process introspection, no code changes or restarts needed) and samples the call stack periodically, producing either a live top-style view (`py-spy top --pid <pid>`) or a flame graph (`py-spy record`) — ideal for diagnosing performance issues in a live production service.
# Attach to a running process by PID, no code modification needed
py-spy top --pid 12345
# Record a flame graph over 30 seconds
py-spy record -o profile.svg --pid 12345 --duration 30
Real-world example
Diagnosing why a production web server is suddenly consuming excessive CPU, without restarting the service or deploying profiling code.
Common follow-ups: Why is a SAMPLING profiler like py-spy generally considered safer for production use than a deterministic profiler like cProfile?
Debugging & Profiling
How do you use Python's built-in 'warnings' module to surface deprecated or risky code paths during debugging?
Intermediate
warnings.warn(message, category) emits a runtime warning without stopping execution, and running Python with -W error (or filterwarnings('error')) turns warnings into exceptions during testing, helping catch deprecated API usage before it becomes a hard failure in a future version.
import warnings
def old_function():
warnings.warn("old_function is deprecated, use new_function instead", DeprecationWarning)
return new_function()
# During testing: python -W error::DeprecationWarning script.py -- turns the warning into an exception
Real-world example
Flagging a deprecated internal API during a migration so callers get an early, non-fatal signal to update before it's removed.
Common follow-ups: How do you configure a custom warnings filter to show a warning only once, instead of every time it's triggered?
Exception Handling