Building AI-Only Pages in WordPress (and Teaching Bots Where to Look)

For the last week, I’ve been working on a side project: creating a WordPress plugin that generates AI-optimized versions of a page. The goal was simple, give large language models (LLMs) like ChatGPT, Claude, and Perplexity a version of a page that’s structured for them, not for humans.

That meant tackling three big challenges:

1. Generation: automatically producing a machine-friendly version of each post.
2. Routing: making sure bots get the AI version, while humans still see the normal page.
3. Logging: tracking who gets what, so I know if GPTBot or PerplexityBot actually found the /ai/ pages.

Here’s what I learned along the way.

Step 1: Generating the AI Pages

The core of the plugin lives in a file I labeled Generator.php. Each time I click “Generate AI Page” in my new post editor, the plugin takes the human version of the post, sends it through OpenAI’s API, and asks the model to produce a full HTML5 document designed for RAG (retrieval-augmented generation).

If I was a real WordPress developer, I would have all sorts of ways to choose your preferred LLM and add the API keys, but this was more of a proof-of-concept. My LLM of choice for this was ChatGPT.

I started with a simple system prompt (“make this page chunkable, add JSON-LD, avoid fluff”) but quickly expanded it to include:

• JSON-LD with schema.org metadata.
• A RAG manifest that identifies the document and its entities.
• A JSONL chunk file embedded in <script> tags, so bots can slurp up structured passages with explicit semantic triples.

The first hurdle? The model sometimes wrapped its output in Markdown fences or duplicated <script> tags. That required writing a pretty gnarly postprocess_ai_html() function to clean things up. I also added guardrails: don’t save anything under 100 bytes, and always check that JSON is valid before it hits the database.

Step 2: Routing Bots to /ai/

Next, I needed a way to serve these AI versions. I didn’t want to mess with Apache rewrites, so the plugin watches for /slug/ai/ in template_redirect. If it matches, it serves the AI page directly.

But here’s the clever part: when GPTBot or PerplexityBot hit a normal page, the plugin detects the User-Agent and sends a 302 redirect to /ai/. That way, humans never see the AI pages unless they add /ai/ to the URL themselves, but bots get funneled to the optimized version automatically.

 protected static function is_ai_request(): bool {
    $ua  = $_SERVER['HTTP_USER_AGENT'] ?? '';
    $sig = strtolower($_SERVER['HTTP_SIGNATURE_AGENT'] ?? '');
    if (preg_match('/(Googlebot|bingbot)/i', $ua)) return false;
    if (preg_match('/(ChatGPT-User|GPTBot|OAI-SearchBot|Perplexity-User|PerplexityBot|Claude-User|ClaudeBot)/i', $ua)) return true;
    if ($sig && strpos($sig, 'chatgpt.com') !== false) return true;
    return false;
  }

The page used in this example is here: https://1918.me/site-taking-forever-build/ai/

Caching threw me a curveball. At first, everything worked in development, but once WP-Cache stepped in, GPTBot kept getting cached human pages instead of redirects. The fix? Flush the cache and regenerate the rewrite rules after installing the plugin. Lesson learned: bot routing + caching needs careful testing.

Step 3: Logging Everything

Finally, I added a small logging system. Every redirect or AI page view gets written into a custom table with:

• Timestamp
• Action (view or redirect)
• Status code (200 or 302)
• Post ID
• User-Agent
• IP address

$row = [
      'time'           => current_time('mysql'),
      'action'         => isset($args['action']) ? substr(sanitize_text_field((string)$args['action']), 0, 32) : '',
      'status'         => isset($args['status']) ? (string)intval($args['status']) : '',
      'is_ai_endpoint' => empty($args['is_ai_endpoint']) ? 0 : 1,
      'post_id'        => isset($args['post_id']) ? intval($args['post_id']) : 0,
      'user_agent'     => substr(sanitize_text_field($_SERVER['HTTP_USER_AGENT'] ?? ''), 0, 255),
      'ip'             => substr(sanitize_text_field($_SERVER['REMOTE_ADDR'] ?? ''), 0, 64),
    ];

    $wpdb->insert($table, $row, ['%s','%s','%s','%d','%d','%s','%s']);
  }

Inside the WordPress admin, there’s now a neat little “AI Page Logs” screen. It shows me when GPTBot hits a page, whether it got redirected to /ai/, and which post was involved. This is invaluable for verifying that the routing actually works, and it’s oddly satisfying to see bots behaving as expected.

Lessons Learned

A few takeaways from the build:

• AI pages need different rules. LLMs don’t care about your sidebar or fonts; they need structured, chunkable data with clean triples and no co-reference.
• Caching will fight you. If you’re doing conditional routing based on User-Agent, make sure your cache layer isn’t serving the same response to everyone.
• Fail loud. I added byte-length checks and detailed admin notices so I’d know when the model returned junk. Silent failures were the most frustrating part early on.
• Bots are predictable. GPTBot, Claude, and Perplexity all send identifiable User-Agents. Routing them is just a regex away.

What’s Next

Now that the basics are working, I’m thinking about:

• Adding support for more granular chunking strategies (e.g., by heading level).
• Experimenting with different schema types beyond Article/TechArticle.
• Building a front-end toggle so I can preview both the human and AI versions side-by-side.

The big picture is this: we’re entering a world where bots are primary readers of our sites. If you don’t give them something structured and usable, they’ll make do with whatever they can scrape. I’d rather hand them a clean /ai/ version than leave it to chance.

There is so much more you could add here, but as a proof of concept, I’m pretty happy. Then I see Sherin Thomas going way beyond my ideas and hooking up an API to answer!

Unfortunately, I only had my very old and neglected blog to test this on, but the promise is aimed at sites where there is lots of information, like e-commerce or dense sales pages.