Making APIs and documentation more accessible to AI tools
Transloadit publishes /llms.txt and /llms-full.txt files to make our documentation easier for
retrieval-capable tools and AI agents to consume. These files provide Markdown-oriented entry points
alongside the normal documentation website.
What is llms.txt?
Proposed in September 2024 by Jeremy Howard, co-founder of Answer.AI,
/llms.txt is a community convention for publishing a concise Markdown index of important pages.
It gives retrieval systems a simpler starting point than a rendered website and lets a caller select
relevant source pages before adding them to a model’s context.
llms.txt should be treated as a retrieval interface, not as an SEO or answer-ranking mechanism.
Publishing the file does not guarantee that Google, ChatGPT, or another product will discover it,
use it, rank a site differently, or produce a more accurate answer. Each crawler, assistant, and
agent decides which URLs it retrieves and how it uses the returned content.
The current convention describes a small retrieval graph:
- Path-scoped
llms.txtfiles provide concise overviews with structured links to key resources. - Documentation pages expose Markdown variants, while
rel="alternate"andrel="describedby"links help clients discover the Markdown page and the most specific index.
Transloadit also retains /llms-full.txt, an older, implementation-specific companion endpoint
that contains a monolithic documentation corpus. It is not part of the current llms.txt format and
is not the recommended starting point for retrieval.
Why does this matter to developers?
When a tool is able to retrieve these files, Transloadit’s implementation offers practical benefits:
-
A predictable entry point: tools can start from one documented index instead of discovering links in rendered navigation.
-
Selective context usage: callers can retrieve the pages relevant to a task instead of placing the whole documentation corpus in one prompt.
-
Source-oriented responses: agents can cite current pages about our Robots, parameters, and workflows. Accuracy still depends on retrieval, model behavior, and verification.
-
Lower parsing overhead: Markdown removes much of the navigation, styling, and client-side interface that is irrelevant to documentation retrieval.
What is included in our implementation?
Our /llms.txt file provides a concise overview of
Transloadit's core functionality, including:
- a clear explanation of our file uploading and processing capabilities;
- example Assembly Instructions, demonstrating our JSON-based workflow configuration;
- links to key documentation resources;
- a comprehensive list of available Robots.
The more detailed /llms-full.txt expands on this with:
- detailed Robot parameter schemas;
- code examples for various integration scenarios;
- comprehensive documentation on all aspects of the Transloadit service.
How to use llms.txt with AI tools
Start with https://transloadit.com/llms.txt, select the links
that match the task, and retrieve those pages. If your tool cannot fetch URLs, copy the relevant
Markdown into the prompt or attach the downloaded file. Avoid loading llms-full.txt by default:
the focused pages usually consume less context and are easier to verify.
Do not assume that mentioning the URL makes an assistant fetch it. Check the tool’s retrieval or browsing capabilities and ask it to identify the source pages it actually used.
Join the AI-friendly web movement
We see llms.txt as a useful, lightweight convention for documentation retrieval. Its value comes
from giving tools a stable source map and giving developers control over which pages enter a model’s
context—not from an assumed ranking boost.
We encourage other services to join this movement and implement their own llms.txt files. For more
information about the proposal, visit llmstxt.org.
Resources
- Transloadit llms.txt
- Transloadit llms-full.txt
- llms.txt proposal documentation
- Jeremy Howard's proposal
- Directory of llms.txt implementations
- llms.txt hub
We’re interested to see how developers use this retrieval interface with Transloadit. As always, we welcome your feedback and questions in our Community Forum.