Markdown for Agents Enhancement
Cloudflare has released improvements to its Markdown for Agents feature, enabling AI systems to request web pages in Markdown format instead of HTML. When enabled on your zone, the feature allows AI crawlers to express a preference for text/markdown in their requests, and Cloudflare's network automatically converts HTML to Markdown on the fly.
Key Improvements
This release delivers three important enhancements:
- Origin response limit increased: The maximum origin response size has been doubled from 1 MB to 2 MB (2,097,152 bytes), enabling conversion of larger pages
- Removed content-length requirement: The origin no longer needs to send the
content-lengthheader, improving compatibility with more servers - Content encoding support: The feature now handles content-encoded (gzip, brotli, etc.) responses from the origin, expanding coverage across diverse backend configurations
Getting Started
To enable this feature, navigate to the AI Crawl Control section in your Cloudflare dashboard and enable Markdown for Agents. This allows AI systems like search engines, LLMs, and other intelligent bots to receive cleaner, more structured content that's easier to parse and process.
For detailed implementation guidance and configuration options, refer to the Markdown for Agents developer documentation.