File Uploads & Media Processing

15 questions found

What is multipart/form-data, and why does a Node.js server need special middleware (like multer) to handle file uploads?

Beginner
multipart/form-data is the HTTP content type browsers use to submit forms containing file uploads, encoding each form field (including file contents) as a separate 'part' within the request body, separated by a boundary string -- Express's built-in body-parsing middleware (express.json/urlencoded) doesn't understand this format, so a dedicated middleware like multer is needed to parse the multipart body, extract uploaded files (saving them to disk or memory), and populate req.file/req.files with the parsed results.
const multer = require('multer');
const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('avatar'), (req, res) => {
  console.log(req.file); // { filename, path, size, mimetype, ... }
  res.json({ success: true });
});
Real-world example A profile-picture upload endpoint uses multer configured to save uploaded files to a temporary directory, then processes the resulting req.file to move it to permanent storage after validating its size and type.

Common follow-ups: Why can't express.json() parse a multipart/form-data request the way it parses a JSON body?;What's the difference between multer's disk storage and memory storage engines?

HTTP & Web Servers;Express & Middleware

How would you validate and restrict uploaded file types and sizes using multer to prevent malicious or oversized uploads?

Intermediate
multer accepts a 'fileFilter' function to reject files based on their reported MIME type or extension before they're even fully processed, and a 'limits' option to cap the maximum file size, preventing a malicious or careless client from uploading an unexpectedly huge file or a disallowed file type (like an executable disguised with an image extension).
const upload = multer({
  limits: { fileSize: 5 * 1024 * 1024 }, // 5MB max
  fileFilter: (req, file, cb) => {
    const allowedTypes = ['image/jpeg', 'image/png'];
    if (!allowedTypes.includes(file.mimetype)) return cb(new Error('Invalid file type'));
    cb(null, true);
  },
});
Real-world example An image-upload API rejects any file over 5MB or that isn't a JPEG or PNG at the multer middleware level, preventing wasted bandwidth and storage from oversized or inappropriate file types before the request even reaches the actual route handler logic.

Common follow-ups: Why is relying solely on the client-reported MIME type insufficient for genuinely verifying a file's actual content type?;How would you additionally verify a file's real type by inspecting its actual binary content (magic bytes) rather than trusting the reported MIME type?

Security;Express & Middleware

How would you process and resize an uploaded image in Node.js using the sharp library, and why is sharp generally preferred over pure-JavaScript alternatives?

Advanced
sharp is a high-performance image-processing library built on the native libvips library, dramatically faster and more memory-efficient than pure-JavaScript image manipulation for common operations like resizing, format conversion, and compression -- since libvips processes images in a streaming fashion internally, sharp can resize very large images without loading the entire uncompressed bitmap into memory the way some pure-JS libraries need to.
const sharp = require('sharp');

await sharp(uploadedFilePath)
  .resize(800, 600, { fit: 'inside' })
  .jpeg({ quality: 80 })
  .toFile('resized-output.jpg');
Real-world example An image-hosting service generates thumbnail, medium, and large versions of every uploaded photo using sharp, processing even very high-resolution uploaded images quickly and with modest memory usage thanks to libvips's efficient internal streaming architecture.

Common follow-ups: How does sharp's performance and memory usage compare to a pure-JavaScript library like Jimp for the same resizing operation?;What image formats and operations does sharp support beyond basic resizing, like format conversion or watermarking?

Streams & Buffers;Advanced Node.js

How would you implement direct-to-S3 file uploads from a client, avoiding routing large file uploads through your Node.js server entirely?

Advanced
Rather than having the client upload a file to your Node.js server, which then re-uploads it to S3 (doubling bandwidth usage and tying up server resources for the duration of potentially large uploads), you generate a pre-signed S3 URL server-side (a temporary, cryptographically-signed URL granting time-limited permission to upload directly to a specific S3 location) and return it to the client, which then uploads the file directly to S3 using that URL, with your server never touching the actual file bytes at all.
const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');
const { getSignedUrl } = require('@aws-sdk/s3-request-presigner');

app.post('/upload-url', async (req, res) => {
  const command = new PutObjectCommand({ Bucket: 'my-bucket', Key: `uploads/${crypto.randomUUID()}` });
  const url = await getSignedUrl(s3Client, command, { expiresIn: 300 });
  res.json({ uploadUrl: url });
});
Real-world example A video-sharing platform generates a pre-signed S3 URL for each upload request, letting users upload multi-gigabyte video files directly from their browser to S3, completely bypassing the Node.js server for the actual (potentially very slow) file transfer.

Common follow-ups: How does this pre-signed URL approach limit what the client is allowed to do (like restricting the file size or content type) despite the server never seeing the actual upload?;What's the tradeoff of this approach in terms of being able to validate or process the file before it's stored?

Cloud & DevOps;Security

How would you stream a large file upload directly to processing (like virus scanning or transcoding) without first saving the entire file to disk?

Intermediate
Rather than using multer's disk storage (which writes the complete file to disk first) you can use multer's memory storage for smaller files, or for genuinely large files, use a library that exposes the upload as a readable stream directly (like busboy, which multer itself is built on), piping that stream directly into a processing pipeline (a virus scanner, a transcoder, cloud storage) as data arrives, without ever needing the complete file to exist on local disk at any point.
const busboy = require('busboy');

app.post('/upload', (req, res) => {
  const bb = busboy({ headers: req.headers });
  bb.on('file', (name, fileStream) => {
    fileStream.pipe(s3UploadStream); // streams directly through to S3, no local disk write
  });
  req.pipe(bb);
});
Real-world example A media-processing service streams uploaded video files directly into an ffmpeg transcoding pipeline as they're received, never writing the potentially multi-gigabyte original file to local disk at all, reducing both disk I/O and processing latency.

Common follow-ups: What's the tradeoff of this streaming approach if a processing step later in the pipeline needs to seek backward within the file, which streams don't naturally support?;How would you handle validating a file's type before processing begins, given you only have the first chunk of a stream to inspect initially?

Streams & Buffers;Cloud & DevOps

What is the difference between multer's diskStorage and memoryStorage engines?

Beginner
diskStorage writes uploaded files directly to a specified directory on disk as they're received, returning a file path in req.file, suitable for larger files where holding the entire content in memory would be wasteful. memoryStorage keeps the uploaded file entirely in a Buffer in memory (available as req.file.buffer), convenient for smaller files that will be immediately processed or forwarded elsewhere (like directly to cloud storage) without ever needing to touch the local disk.
// Disk storage: writes to disk, file path available
const upload = multer({ storage: multer.diskStorage({ destination: 'uploads/' }) });

// Memory storage: buffer available directly, no disk write
const upload = multer({ storage: multer.memoryStorage() });
app.post('/upload', upload.single('file'), (req, res) => {
  s3.putObject({ Body: req.file.buffer, Key: req.file.originalname });
});
Real-world example An avatar-upload endpoint uses memoryStorage since uploaded images are small and immediately forwarded to S3 without ever needing a local disk copy, while a large-video-upload endpoint uses diskStorage to avoid holding potentially gigabyte-sized files entirely in memory.

Common follow-ups: What's the memory risk of using memoryStorage for an endpoint that might receive very large files?;How would you clean up temporary files left on disk by diskStorage after they've been processed?

Streams & Buffers;Cloud & DevOps

How would you generate a video thumbnail in Node.js using ffmpeg via a library like fluent-ffmpeg?

Advanced
fluent-ffmpeg provides a Node.js-friendly wrapper around the ffmpeg command-line tool (which must be installed separately on the system or bundled via a package like ffmpeg-static), letting you programmatically extract a frame at a specific timestamp from a video file and save it as an image, commonly used to automatically generate a representative thumbnail for uploaded video content.
const ffmpeg = require('fluent-ffmpeg');

ffmpeg('input-video.mp4')
  .screenshots({
    timestamps: ['00:00:05'],
    filename: 'thumbnail.png',
    folder: './thumbnails',
  })
  .on('end', () => console.log('Thumbnail generated'));
Real-world example A video-sharing platform automatically generates a thumbnail from the 5-second mark of every uploaded video using fluent-ffmpeg, giving users a visual preview in their video library without requiring them to manually select or upload a separate thumbnail image.

Common follow-ups: Since ffmpeg processing is CPU-intensive, should this work run on the main Node.js process or be offloaded elsewhere?;What's the licensing consideration of bundling ffmpeg with a commercial application?

Background Jobs & Queues;Clustering & Worker Threads

Why should CPU-intensive media processing like image resizing or video transcoding typically not run directly inside an Express request handler?

Intermediate
Media processing operations like resizing large images or transcoding video are genuinely CPU-intensive, and running them synchronously (or even via a Promise awaited directly) inside a request handler ties up that handler's processing time and, more importantly, can block the event loop if the underlying library isn't itself offloading work to a native thread pool -- the recommended pattern is to accept the upload quickly, enqueue the actual processing as a background job, and respond to the client immediately with a 202 Accepted and a way to check on the job's progress.
app.post('/upload', upload.single('video'), async (req, res) => {
  const jobId = await videoProcessingQueue.add('transcode', { filePath: req.file.path });
  res.status(202).json({ jobId, status: 'processing' });
});
Real-world example A video-upload endpoint immediately enqueues a transcoding job and responds with a 202 status and job ID, letting users check back later for the processed result, rather than making the HTTP request hang for however long a large video's transcoding actually takes.

Common follow-ups: How would sharp's or ffmpeg's own internal use of native threads change this calculus somewhat?;How does the client find out when the background processing job has completed?

Background Jobs & Queues;Event Loop & Non-blocking IO

How would you validate an uploaded image's actual dimensions and reject ones that don't meet minimum or maximum size requirements?

Intermediate
After the file is uploaded (but before accepting it as valid), you can use an image-processing library like sharp to read the image's actual metadata (width and height) and compare it against your application's requirements, rejecting and cleaning up the uploaded file if it doesn't meet the criteria, rather than relying solely on file-size limits which don't reflect actual image dimensions.
const sharp = require('sharp');

const metadata = await sharp(req.file.path).metadata();
if (metadata.width < 200 || metadata.height < 200) {
  await fs.promises.unlink(req.file.path); // clean up the rejected upload
  return res.status(400).json({ error: 'Image must be at least 200x200 pixels' });
}
Real-world example A profile-picture upload feature rejects images smaller than 200x200 pixels after checking the actual decoded image metadata with sharp, cleaning up the temporarily-saved file immediately rather than leaving rejected uploads accumulating on disk.

Common follow-ups: Why is checking actual image dimensions via a library more reliable than trying to infer size from file size alone?;What's an appropriate cleanup strategy for uploaded files that fail validation, to avoid accumulating orphaned files over time?

Error Handling;File System (fs) Module

What is the security risk of trusting a client-supplied filename for an uploaded file, and how do you mitigate it?

Advanced
A client-supplied original filename could contain path traversal sequences (like '../../etc/passwd') or other malicious characters that, if used directly to construct a file path on the server, could let an attacker write files outside the intended upload directory -- the mitigation is to never use the client-supplied filename directly for the actual storage path, instead generating a new, safe, randomly-generated filename (like a UUID) server-side, optionally preserving only the validated file extension.
const crypto = require('node:crypto');
const path = require('node:path');

const safeExtension = path.extname(req.file.originalname).toLowerCase();
const allowedExtensions = ['.jpg', '.png', '.gif'];
if (!allowedExtensions.includes(safeExtension)) throw new Error('Invalid file extension');

const safeFilename = `${crypto.randomUUID()}${safeExtension}`; // never uses the raw client filename
Real-world example A security review flags that an upload endpoint was using the client-provided filename directly to construct the storage path, a path-traversal vulnerability; the fix generates a random UUID-based filename server-side, only carrying over a validated, allow-listed file extension from the original name.

Common follow-ups: What specific path-traversal payload would exploit an endpoint that uses the raw client filename directly?;Why is validating the extension against an allow-list safer than checking it against a deny-list of dangerous extensions?

Security;File System (fs) Module

Showing 1–10 of 15