Uploads and Downloads
Stream files in chunks, cap the size as you read, set the content type, and never trust the client's filename.
shutil.copyfileobj(fsrc, fdst, length=0) # or resp.iter_content(chunk_size=8192)Files are the place where web code runs out of memory, fills disks, and gets a path traversal. All three come from the same habit: treating a file as a value you can hold.
Stream, do not load
Copy in chunks. shutil.copyfileobj(fsrc, fdst, length=0) is the loop already written for you — read a block, write a block — and a plain src.read() with no size is the whole file in RAM. On the client side, requests needs stream=True and then resp.iter_content(chunk_size=8192); httpx uses with httpx.stream("GET", url) as r: and r.iter_bytes(). Ten concurrent requests for a 500MB file is 5GB of memory if you buffer, and a steady trickle if you stream.
Limit the size yourself
Count the bytes you have read and abort at your own ceiling, answering 413 (HTTPStatus.CONTENT_TOO_LARGE, renamed from REQUEST_ENTITY_TOO_LARGE in Python 3.13). Content-Length is a claim, and a chunked upload does not have one at all. Flask's MAX_CONTENT_LENGTH raises RequestEntityTooLarge for you.
The filename is hostile input
A client can send ../../home/you/.bashrc, a Windows path, a name of 4,000 characters, or one that differs from an existing file only by Unicode normalisation. Never join it to a directory. Generate your own stored name and keep what they sent as metadata you display but never resolve. Werkzeug's secure_filename turns that example into home_username_.bashrc if you need the original-ish name.
For the response, mimetypes.guess_file_type(path, *, strict=True) (added in 3.13) gives you a Content-Type, and Content-Disposition: attachment; filename="..." asks the browser to save rather than render.
Example
When to use it
- Letting users attach a PDF without the request filling your container's memory.
- Pulling a multi-gigabyte export from a vendor on a small instance.
- Rejecting an oversized upload before it fills the disk.
- Fixing an endpoint where a crafted filename could overwrite a file outside the upload directory.
More examples
Downloading without holding the file
The pattern for a job that pulls files from anywhere. The .part rename is the part people leave out and then debug for an afternoon: without it, a crash mid-download leaves a truncated file with the final name.
import httpx
import shutil
from pathlib import Path
def download(url: str, dest: Path, max_bytes: int = 200 * 1024 * 1024) -> int:
total = 0
# Write to a temporary name and rename at the end, so a failed download
# never leaves a half file where something else will pick it up.
tmp = dest.with_suffix(dest.suffix + ".part")
with httpx.stream("GET", url, timeout=30.0, follow_redirects=True) as r:
r.raise_for_status()
with tmp.open("wb") as f:
for chunk in r.iter_bytes(): # default chunk size, already sensible
total += len(chunk)
if total > max_bytes:
raise ValueError("response exceeded the size limit")
f.write(chunk)
tmp.replace(dest) # atomic on the same filesystem
return total
# requests: with requests.get(url, stream=True, timeout=30) as r:
# shutil.copyfileobj(r.raw, f) # note: r.raw skips decompression
# ... prefer r.iter_content(chunk_size=8192), which decodes properly.Accepting an upload, with both limits
A complete upload handler. Two things to notice: the stored name is generated so no filename can ever influence a path, and the byte counter runs during the stream, because a limit checked after the write has already filled the disk.
import uuid
from pathlib import Path
from fastapi import FastAPI, File, HTTPException, UploadFile
app = FastAPI()
UPLOADS = Path("/var/app/uploads")
ALLOWED = {"image/png": ".png", "image/jpeg": ".jpg", "application/pdf": ".pdf"}
MAX_BYTES = 10 * 1024 * 1024
@app.post("/attachments", status_code=201)
async def upload(file: UploadFile = File(...)): # needs python-multipart
# Trust the declared type only to reject early; it is client-supplied too.
if file.content_type not in ALLOWED:
raise HTTPException(status_code=415, detail="unsupported media type")
# Your own name, your own extension. file.filename is never joined to a path.
stored = UPLOADS / f"{uuid.uuid4().hex}{ALLOWED[file.content_type]}"
total = 0
with stored.open("wb") as out:
while chunk := await file.read(1024 * 1024):
total += len(chunk)
if total > MAX_BYTES:
stored.unlink(missing_ok=True) # do not keep the partial file
raise HTTPException(status_code=413, detail="file too large")
out.write(chunk)
return {"id": stored.stem, "bytes": total, "original_name": file.filename}Serving a file back with the right headers
Reach for it when you are returning a file from your own storage. application/octet-stream as the fallback is deliberate: an unknown type served as text/html is a stored cross-site-scripting bug, and Content-Disposition tells the browser to save rather than execute.
import mimetypes
from pathlib import Path
def download_headers(path: Path, download_name: str) -> dict:
# guess_file_type was added in 3.13; before that, guess_type(str(path)).
# Passing a path to guess_type is soft deprecated as of 3.13.
content_type, encoding = mimetypes.guess_file_type(path)
safe_name = download_name.replace('"', "").replace("\n", "").replace("\r", "")
headers = {
"Content-Type": content_type or "application/octet-stream",
"Content-Length": str(path.stat().st_size),
# Strip quotes and newlines from the name before quoting it: an
# unescaped one lets a client inject extra headers here.
"Content-Disposition": f'attachment; filename="{safe_name}"',
}
if encoding:
headers["Content-Encoding"] = encoding
return headers
print(download_headers(Path("report.pdf"), "Q1 report.pdf"))
# Frameworks wrap this: flask.send_from_directory(dir, name, as_attachment=True),
# fastapi.responses.FileResponse(path, filename=name).
Discussion