15 questions found
How would you implement a concurrency-limited task queue in Node.js to process many async tasks without overwhelming a downstream resource?
Advanced
Running Promise.all() over thousands of items sends every request essentially simultaneously, which can overwhelm a downstream database or API and exhaust connections -- a concurrency-limited queue instead processes only N tasks at a time, starting a new one immediately as each one completes, keeping throughput high while capping the maximum simultaneous load; libraries like p-limit or a hand-rolled worker-pool pattern both implement this.
async function processWithConcurrencyLimit(items, limit, fn) {
const results = [];
const executing = new Set();
for (const item of items) {
const p = fn(item).then(r => { executing.delete(p); return r; });
executing.add(p);
results.push(p);
if (executing.size >= limit) await Promise.race(executing);
}
return Promise.all(results);
}
Real-world example
A data-migration script updating 100,000 records via an external API uses a concurrency-limited queue capped at 10 simultaneous requests, avoiding the rate-limit errors and connection exhaustion that occurred when an earlier version of the script fired all 100,000 requests via a single Promise.all().
Common follow-ups: How do you choose an appropriate concurrency limit for a given downstream dependency?;How does this pattern compare to using a proper job queue backed by Redis or RabbitMQ for the same problem at larger scale?
Performance Optimization & Profiling;Background Jobs & Queues
What is the difference between micro tasks and macro tasks in Node.js's event loop, and how does this affect Promise callback ordering?
Intermediate
Microtasks (Promise .then()/.catch()/.finally() callbacks, and process.nextTick() callbacks which have their own even-higher-priority queue) are processed completely -- draining the entire microtask queue -- after each single macrotask, before the event loop moves on to the next phase. Macrotasks include timers (setTimeout/setInterval), I/O callbacks, and setImmediate() callbacks, each processed one at a time per event loop phase. This is why a resolved promise's .then() callback always runs before a setTimeout(fn, 0) callback, even though both appear to be scheduled for 'immediately'.
console.log('1');
setTimeout(() => console.log('2 (macrotask)'), 0);
Promise.resolve().then(() => console.log('3 (microtask)'));
console.log('4');
// Output order: 1, 4, 3, 2 -- microtasks always drain before the next macrotask
Real-world example
A developer debugging unexpected ordering in a logging system realizes that a Promise-based logger's output was appearing before a setTimeout-scheduled cleanup task's output specifically because of this microtask/macrotask distinction, not because of any bug in either piece of code.
Common follow-ups: Where does process.nextTick() fit relative to regular Promise microtasks in terms of priority?;Can an application accidentally starve macrotasks by continuously scheduling new microtasks?
Event Loop & Non-blocking IO;Advanced Node.js
How do you convert a Node.js callback-based function into a Promise-based one manually, without using util.promisify()?
Beginner
You wrap the callback-based call inside 'new Promise((resolve, reject) => {...})', calling resolve() with the successful result inside the callback, or reject() with the error if the callback's first (error-first) argument is truthy -- this is exactly the pattern util.promisify() automates, but writing it manually is useful when the function doesn't follow the standard error-first convention or needs custom handling.
function readFilePromise(path) {
return new Promise((resolve, reject) => {
fs.readFile(path, 'utf-8', (err, data) => {
if (err) reject(err);
else resolve(data);
});
});
}
const content = await readFilePromise('config.json');
Real-world example
A team integrating with a legacy internal SDK that uses a non-standard callback signature (success, data, error instead of error-first) manually wraps each needed method in a Promise constructor, since util.promisify() can't correctly handle that non-standard callback order automatically.
Common follow-ups: What happens if both resolve() and reject() are accidentally called inside the same Promise executor?;How would you add a timeout to a manually wrapped Promise so it doesn't hang forever if the callback is never invoked?
Async Patterns;Core Node.js Modules
What is 'fire-and-forget' async code, and why is it generally discouraged in production Node.js applications?
Advanced
Fire-and-forget means calling an async function without awaiting or attaching a .catch() to its returned promise -- the calling code moves on immediately without knowing whether the operation succeeded, failed, or is still in progress. This is discouraged because any rejection becomes an unhandled promise rejection (potentially crashing the process on modern Node.js), and because the calling code has no way to guarantee the operation actually completed before, say, the process exits or a response is sent.
// Fire-and-forget: risky, errors are silently unhandled
function handleRequest(req, res) {
logAnalyticsEvent(req); // not awaited -- if this rejects, it's an unhandled rejection
res.json({ status: 'ok' });
}
// Safer: explicitly handle the case where you intentionally don't want to block
function handleRequest(req, res) {
logAnalyticsEvent(req).catch(err => logger.error('Analytics logging failed', err));
res.json({ status: 'ok' });
}
Real-world example
A team notices intermittent process crashes traced to a fire-and-forget analytics-logging call that occasionally rejected due to a network blip; adding an explicit .catch() to swallow and log that specific, genuinely non-critical failure resolves the crashes without needing to await the call and slow down the response.
Common follow-ups: In what situation is fire-and-forget actually the correct, deliberate choice rather than a bug?;How does this interact with Node's default unhandled-rejection termination behavior discussed earlier?
Error Handling;Advanced Node.js
What is the purpose of the 'node:timers/promises' module, and how does it simplify working with delays in async code?
Beginner
node:timers/promises provides promise-returning versions of setTimeout, setImmediate, and setInterval, letting them be used directly with await instead of needing to be manually wrapped in a Promise constructor -- this removes a very common piece of boilerplate that used to appear at the top of countless Node.js files.
const { setTimeout: sleep } = require('node:timers/promises');
async function retryLater() {
console.log('Waiting 2 seconds...');
await sleep(2000);
console.log('Done waiting.');
}
Real-world example
A rate-limited API client uses timers/promises' setTimeout directly with await to pause between retries, replacing a small hand-written 'function delay(ms) { return new Promise(resolve => setTimeout(resolve, ms)); }' helper that used to be copy-pasted into nearly every project.
Common follow-ups: How does timers/promises' setInterval as an async iterator differ from the traditional callback-based setInterval?;Can these promise-based timers be cancelled, and if so how?
Async Patterns;Core Node.js Modules