File System & File Processing
15 questions found
What is the difference between fs.readFile() and fs.createReadStream() for reading a file in Node.js?
Beginner
fs.readFile() reads an entire file into memory at once before invoking its callback with the complete contents -- simple, but problematic for large files since the whole file must fit in memory simultaneously. fs.createReadStream() reads the file incrementally in configurable chunks, emitting 'data' events as each chunk becomes available, using a bounded, predictable amount of memory regardless of the file's total size, making it the appropriate choice for processing large files.
// Loads the entire file into memory
fs.readFile('huge-file.csv', 'utf-8', (err, data) => { /* entire file in `data` */ });
// Processes incrementally, bounded memory usage
fs.createReadStream('huge-file.csv').on('data', (chunk) => { /* process one chunk at a time */ });
Real-world example
A log-analysis tool processing multi-gigabyte log files switches from fs.readFile() (which was crashing with an out-of-memory error) to fs.createReadStream(), processing the file line by line with bounded memory regardless of the file's total size.
Common follow-ups: At what file size does the tradeoff meaningfully shift from readFile() being fine to streaming being necessary?;How would you process a stream line-by-line rather than in raw chunks?
Streams & Buffers;Performance Optimization & Profiling
How would you recursively read all files within a directory tree in Node.js?
Intermediate
fs.readdir() with the 'recursive' option (Node 20+) returns every file and subdirectory path within a directory tree in one call, or for finer control (like filtering or processing files as they're discovered rather than all at once), you can manually walk the tree, calling fs.readdir() on each directory and recursing into any subdirectories found.
const fs = require('node:fs/promises');
const files = await fs.readdir('./project', { recursive: true, withFileTypes: true });
const jsFiles = files.filter(f => f.isFile() && f.name.endsWith('.js'));
Real-world example
A code-linting tool recursively scans an entire project directory for all .js files using fs.readdir with the recursive option, avoiding the need to manually implement tree-walking logic that used to require a third-party package like glob.
Common follow-ups: How does the recursive option's behavior differ across Node.js versions where it wasn't yet available?;What's the performance consideration of reading an entire deep directory tree into memory at once versus processing it incrementally?
CLI Tools & Scripting with Node.js;Path & OS Modules
How would you implement a file-watching feature in Node.js using fs.watch(), and what are its known cross-platform limitations?
Advanced
fs.watch() monitors a file or directory for changes, emitting events when something is modified, renamed, or removed -- but its behavior is notoriously inconsistent across operating systems (event types and reliability differ between Linux, macOS, and Windows, and it can occasionally miss rapid successive changes or fire duplicate events), which is why many production tools use a more robust third-party library like chokidar that normalizes these platform differences.
const fs = require('node:fs');
fs.watch('./config', (eventType, filename) => {
console.log(`${eventType} detected on ${filename}`);
reloadConfig();
});
// More robust cross-platform alternative
const chokidar = require('chokidar');
chokidar.watch('./config').on('change', reloadConfig);
Real-world example
A development server's hot-reload feature initially used fs.watch() directly, but switched to chokidar after developers on different operating systems reported inconsistent reload behavior, since chokidar normalizes the platform-specific quirks fs.watch() exposes directly.
Common follow-ups: What specific inconsistencies does fs.watch() have between Linux's inotify and macOS's FSEvents backends?;How does chokidar's polling fallback mode trade reliability for higher resource usage?
CLI Tools & Scripting with Node.js;Core Node.js Modules
How would you safely check if a file exists in Node.js before attempting to read it, avoiding a race condition?
Intermediate
Rather than checking existence with fs.access() or fs.exists() and then separately attempting to read the file (a pattern vulnerable to a race condition -- a TOCTOU, time-of-check to time-of-use bug -- where the file could be deleted between the check and the actual read), the recommended approach is to simply attempt the read operation directly and handle the specific 'ENOENT' (file not found) error if it occurs, since the operation and the error handling happen atomically together.
// Race-condition-prone: file could vanish between the check and the read
if (fs.existsSync(path)) { const data = fs.readFileSync(path); }
// Correct: attempt directly, handle the specific error
try {
const data = await fs.promises.readFile(path, 'utf-8');
} catch (err) {
if (err.code === 'ENOENT') { /* handle missing file specifically */ }
else throw err;
}
Real-world example
A configuration-loading function that previously checked fs.existsSync() before reading occasionally still threw an unexpected error in a containerized environment where a mounted config volume briefly became unavailable between the check and the read; refactoring to attempt the read directly and catch ENOENT eliminated the race condition entirely.
Common follow-ups: Why is this TOCTOU pattern considered a genuine bug class rather than just a theoretical edge case?;What other Node.js filesystem operations have similar race-condition risks worth being aware of?
Error Handling;Advanced Node.js
How would you implement atomic file writes in Node.js to avoid leaving a file in a corrupted, half-written state if the process crashes mid-write?
Advanced
A direct write to a file's final path risks leaving it partially written (corrupted) if the process crashes or is killed mid-operation -- the standard atomic-write pattern instead writes the complete new content to a temporary file first, then uses fs.rename() (which is an atomic operation at the filesystem level on most systems) to move the temp file into the final destination path, ensuring readers only ever see either the complete old version or the complete new version, never a partial write.
const fs = require('node:fs/promises');
async function atomicWriteFile(finalPath, content) {
const tempPath = `${finalPath}.tmp-${process.pid}`;
await fs.writeFile(tempPath, content);
await fs.rename(tempPath, finalPath); // atomic on most filesystems
}
Real-world example
A configuration-persistence service switches from writing directly to config.json to writing to a temp file and renaming it into place, eliminating a rare but real bug where a process crash mid-write had occasionally left the production config file truncated and unparseable.
Common follow-ups: Why specifically is fs.rename() atomic while a direct fs.writeFile() to the final path isn't?;Does this atomic-rename guarantee hold across all filesystems and operating systems, or are there exceptions?
Error Handling;Advanced Node.js
What is the difference between fs.appendFile() and opening a file with the 'a' flag and writing to it manually?
Intermediate
fs.appendFile() is a convenience method that internally opens the file in append mode, writes the data, and closes the file, all in one call -- functionally equivalent to manually opening a file descriptor with fs.open(path, 'a'), writing with fs.write(), and closing with fs.close(), but appendFile() is simpler for one-off appends, while manually managing the file descriptor is preferable when writing many times to the same file, avoiding the overhead of repeatedly opening and closing it.
// Simple one-off append
await fs.promises.appendFile('log.txt', 'New log entry\n');
// More efficient for many repeated writes to the same file
const fd = await fs.promises.open('log.txt', 'a');
for (const entry of manyLogEntries) { await fd.appendFile(entry + '\n'); }
await fd.close();
Real-world example
A logging utility writing thousands of log entries per minute keeps a single open file descriptor via fs.open() rather than calling fs.appendFile() (which reopens and recloses the file every single time) for each entry, significantly reducing file-system overhead under high log volume.
Common follow-ups: What's the actual performance difference between repeatedly using appendFile() versus keeping a file descriptor open, under realistic load?;What happens if two separate processes both try to append to the same file concurrently?
Performance Optimization & Profiling;Logging & Monitoring
How would you process a very large CSV file in Node.js without loading the entire file into memory?
Advanced
Combining a readable file stream with a line-by-line or CSV-specific parsing library (like csv-parse in streaming mode) lets you process each row as it's read, rather than waiting for and holding the entire file in memory -- this is essential for files too large to fit comfortably in memory, and lets processing begin immediately rather than waiting for the entire file to load first.
const fs = require('node:fs');
const { parse } = require('csv-parse');
fs.createReadStream('huge-data.csv')
.pipe(parse({ columns: true }))
.on('data', (row) => { processRow(row); })
.on('end', () => console.log('Finished processing'));
Real-world example
A data-import tool processes a multi-gigabyte CSV file of customer records by streaming it through csv-parse and inserting each row into the database as it's parsed, rather than attempting to load and parse the entire file into an array first, which had previously caused out-of-memory crashes.
Common follow-ups: How would you add backpressure-aware batching, inserting rows in batches of 1000 rather than one at a time, to this streaming pipeline?;What happens if a single malformed row in the middle of a huge file causes a parsing error?
Streams & Buffers;Databases & ORMs (MongoDB/Mongoose
SQL/Sequelize)
What is the difference between fs.stat() and fs.lstat(), and when does the distinction matter?
Intermediate
fs.stat() returns file information, automatically following symbolic links to report on the actual target file they point to. fs.lstat() returns information about the symbolic link itself (its size, type) rather than the file it points to -- the distinction matters specifically when working with symlinks and you need to know whether a given path is itself a symlink, rather than transparently resolving through it to whatever it points to.
const fs = require('node:fs/promises');
const stats = await fs.lstat('shortcut.txt');
if (stats.isSymbolicLink()) {
const target = await fs.readlink('shortcut.txt');
console.log('This is a symlink pointing to:', target);
}
Real-world example
A file-organization tool that needs to detect and specifically handle symbolic links (rather than silently following them, which could create infinite loops with circular symlinks) uses fs.lstat() to identify symlinks before deciding how to process them.
Common follow-ups: What specific problem, like an infinite loop, could occur if a file-tree-walking tool doesn't account for symbolic links at all?;What information does fs.Stats provide beyond just file size and type, like timestamps?
CLI Tools & Scripting with Node.js;Advanced Node.js
How would you implement file locking in Node.js to prevent two processes from writing to the same file simultaneously?
Advanced
Node.js doesn't provide built-in cross-process file locking directly, so implementations typically use a third-party library (like proper-lockfile) that creates a separate lock file (or uses OS-level advisory locking where available) alongside the target file -- a process attempting to write first acquires the lock (waiting or failing if another process already holds it), performs its write, and then releases the lock, preventing concurrent, conflicting writes from corrupting the target file.
const lockfile = require('proper-lockfile');
const release = await lockfile.lock('shared-data.json');
try {
const data = JSON.parse(await fs.promises.readFile('shared-data.json', 'utf-8'));
data.count += 1;
await fs.promises.writeFile('shared-data.json', JSON.stringify(data));
} finally {
await release();
}
Real-world example
Two separate cron jobs that both occasionally needed to update the same shared JSON state file were causing intermittent data corruption from concurrent writes, until the team added proper-lockfile-based locking to ensure only one process could write to the file at a time.
Common follow-ups: What happens if a process crashes while still holding a lock -- how do lock libraries typically handle stale lock recovery?;Would using a proper database instead of a shared file sidestep this problem entirely, and when is a file-based approach still preferable?
Error Handling;Databases & ORMs (MongoDB/Mongoose
SQL/Sequelize)
How would you copy a large file efficiently in Node.js, and what's the difference between fs.copyFile() and manually piping streams?
Intermediate
fs.copyFile() (or fs.cp() for directories) performs an efficient, often OS-optimized file copy in a single call, suitable for most use cases. Manually piping a read stream to a write stream (fs.createReadStream(src).pipe(fs.createWriteStream(dest))) offers more control -- useful if you need to transform the data during the copy (like compressing it), but generally has more overhead than the OS-level optimized copy that fs.copyFile() can leverage for a plain, unmodified copy.
// Simple, efficient copy
await fs.promises.copyFile('source.txt', 'destination.txt');
// Manual piping, useful when transforming data during the copy
fs.createReadStream('source.txt').pipe(zlib.createGzip()).pipe(fs.createWriteStream('destination.txt.gz'));
Real-world example
A backup utility uses fs.copyFile() for straightforward file duplication (letting the OS handle the copy as efficiently as possible), but switches to manual stream piping specifically when it needs to simultaneously compress files during the backup process.
Common follow-ups: Does fs.copyFile() actually use OS-level fast-copy mechanisms like copy_file_range on Linux, and how does that affect performance for very large files?;What happens if the destination file already exists when using fs.copyFile()?
Streams & Buffers;Performance Optimization & Profiling