Markdown for Agents Enhancements

Cloudflare's Markdown for Agents feature enables AI systems to request website content in markdown format instead of HTML, making it easier for large language models to parse and process web pages. Today's release includes significant improvements to the underlying infrastructure that powers this feature.

Key Improvements

• Increased response limit: The maximum origin response size has been doubled from 1 MB to 2 MB (2,097,152 bytes), allowing larger pages to be converted to markdown
• Flexible headers: No longer requires the origin to send the content-length header, improving compatibility with more origin servers
• Content encoding support: Now handles content-encoded responses (such as gzip-compressed content) from origins, automatically decompressing before conversion

Getting Started

If you haven't enabled this feature, navigate to the AI Crawl Control section in your Cloudflare dashboard and enable Markdown for Agents. Once enabled, AI systems requesting your pages can signal their preference for text/markdown in HTTP Accept headers, and Cloudflare's network will automatically convert HTML to markdown on the fly.

For implementation details and full documentation, refer to the Markdown for Agents developer documentation.