Sitemap & robots
Generate sitemap.xml and robots.txt with special files that export a config or data.
Syntax
app/sitemap.ts and app/robots.tsNext.js can generate sitemap.xml and robots.txt from code. Add app/sitemap.ts that default-exports an array of URLs, and app/robots.ts that returns crawler rules. Next serves them at the standard paths automatically.
Example
// app/sitemap.ts
import type { MetadataRoute } from 'next';
export default function sitemap(): MetadataRoute.Sitemap {
return [
{ url: 'https://example.com', lastModified: new Date() },
{ url: 'https://example.com/blog', lastModified: new Date() },
];
}
// app/robots.ts
import type { MetadataRoute } from 'next';
export function robots(): MetadataRoute.Robots {
return {
rules: { userAgent: '*', allow: '/' },
sitemap: 'https://example.com/sitemap.xml',
};
}When to use it
- A content site generates a sitemap.xml automatically from all published post slugs so search engines discover every URL.
- A SaaS app blocks all /api/* routes from crawlers via a programmatic robots.ts file.
- An e-commerce site generates a large sitemap split by product category using Next.js sitemap generation functions.
More examples
Static sitemap.ts
Exporting from app/sitemap.ts generates /sitemap.xml at build time with correct content-type headers.
// app/sitemap.ts
import type { MetadataRoute } from 'next';
export default function sitemap(): MetadataRoute.Sitemap {
return [
{ url: 'https://acme.com/', lastModified: new Date() },
{ url: 'https://acme.com/about', lastModified: new Date() },
{ url: 'https://acme.com/pricing', lastModified: new Date() },
];
}Dynamic sitemap from database
Async sitemap functions can query the database and return all published URLs for search engine indexing.
// app/sitemap.ts
import type { MetadataRoute } from 'next';
import { getPosts } from '@/lib/posts';
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
const posts = await getPosts();
return posts.map(p => ({
url: `https://acme.com/blog/${p.slug}`,
lastModified: p.updatedAt,
}));
}Programmatic robots.ts
robots.ts generates /robots.txt at build time, keeping crawl rules version-controlled in TypeScript.
// app/robots.ts
import type { MetadataRoute } from 'next';
export default function robots(): MetadataRoute.Robots {
return {
rules: [
{ userAgent: '*', allow: '/', disallow: '/api/' },
],
sitemap: 'https://acme.com/sitemap.xml',
};
}
Discussion