Recently, I noticed something interesting in my website analytics: crawlers from companies like OpenAI and Google were visiting my site.
At first, I was worried—
- Is my content being scraped without permission?
- Will too many AI bot visits slow down my site?
But then I discovered a new opportunity: llms.txt.
This simple file can guide AI tools (like ChatGPT, Claude, or Gemini) to the content you want them to read and potentially cite. It’s like rolling out a welcome mat for AI while pointing them to your best pages.
In this guide, we’ll cover:
- What is llms.txt?
- How it’s different from robots.txt
- How to add it in WordPress (plugin & manual method)
- How to block AI bots if you want
🔍 What is an llms.txt File?
The llms.txt file is a new format designed for Large Language Models (LLMs). It tells AI crawlers which parts of your site are most relevant, trustworthy, and worth using in AI-generated answers.
Think of it as a content guide for AI. For example:
“If you’re going to use my site in answers, start with these pages.”
The file usually lives at:
It can include:
- A link to your XML sitemap
- Your most important articles, guides, or pages
- Any content you want AI to notice first
This is part of Generative Engine Optimization (GEO) — helping AI models present your site better in their responses.
🤔 llms.txt vs robots.txt: What’s the Difference?
Both files deal with bots, but they serve different purposes:
| Feature | robots.txt | llms.txt |
|---|---|---|
| Purpose | Blocks crawlers from certain pages | Guides AI to your best pages |
| Format | User-agent & Disallow rules | Markdown list of links |
| Effect on AI | Can block AI from accessing your site | Helps AI know what to reference |
| Adoption | Widely supported by search engines | New & still evolving |
💡 Tip: You can use both. Let robots.txt block unwanted bots, and let llms.txt guide the bots you welcome.
🛠 How to Add llms.txt in WordPress
You can create llms.txt in two ways:
- Method 1: Using a plugin (Easy)
- Method 2: Manually uploading the file
Method 1: Using AIOSEO Plugin (Recommended)
The All in One SEO (AIOSEO) plugin now supports llms.txt by default.
Steps:
- Install and activate AIOSEO.
- Go to All in One SEO → General Settings → Advanced Tab.
- Make sure the toggle “Generate LLMs.txt file” is turned ON.
- Click “Open LLMs.txt” to view your file.
✅ Done! Your site now has an AI-friendly llms.txt automatically updated as you publish new content.
Method 2: Creating llms.txt Manually
If you don’t want to use a plugin, you can create the file yourself.
Steps:
- Open Notepad (Windows) or TextEdit (Mac).
- Create a file named llms.txt.
- Add your content links. Example:
Upload the file to your site’s root directory (public_html) via FTP or File Manager.
Check by visiting https://yoursite.com/llms.txt.
🚫 Bonus: Blocking AI Bots (Using robots.txt)
If you don’t want certain AI bots to use your content:
- Open AIOSEO → Tools → Robots.txt Editor
- Add rules like:
You can add more bots (Claude, CCBot, etc.) as needed.
❓ FAQs
1. Does llms.txt affect SEO rankings?
No. It’s for AI models, not Google Search rankings.
2. Will llms.txt bring more AI traffic?
Not guaranteed. But it increases your chances of AI citing your site.
3. Can I use llms.txt and robots.txt together?
Yes! llms.txt guides bots you want, robots.txt blocks the ones you don’t.
🎯 Final Thoughts
AI and Generative Engine Optimization are still evolving.
Adding llms.txt today is like early SEO—it positions you ahead of others.
You can:
- Guide AI (llms.txt)
- Block unwanted bots (robots.txt)
- Review your strategy every few months
👉 Whether you welcome AI bots or block them, at least you’re in control.
