> For the complete documentation index, see [llms.txt](https://docs.getlimy.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.getlimy.ai/maze-redirect/a2a/akamai.md).

# Akamai

### **Overview**

This documentation describes the implementation of an **Akamai EdgeWorker** designed to detect traffic from AI search engines and agentic web crawlers using User-Agent analysis.\
Instead of modifying content inline, the EdgeWorker **routes AI systems to machine-optimized endpoints**, improving how your website is consumed, interpreted, and referenced across the agentic web.

This approach enables:

* Clearer machine-readable signals for AI models
* Better control over how AI agents access and process your content
* Separation of human vs. AI requester behavior in your analytics
* Improved presence across AI-driven discovery platforms (ChatGPT, Perplexity, Gemini, Claude, Grok, etc.)

## **Architecture**

#### **High-Level Flow**

**Incoming Request → User-Agent Analysis → AI Bot Detection → Routing Decision → URL Rewrite**

#### **Key Components**

* **AI Bot Detection Engine**\
  Identifies modern agentic web crawlers via User-Agent rules.
* **Routing Logic**\
  Applies URL rewrites for AI-optimized content.
* **Configuration Management**\
  Centralized bot patterns and routing rules.
* **Logging & Observability**\
  Tracks how AI agents browse, fetch, and interpret your content.

## **AI Search Engine Bot Detection & Routing — Akamai**

### **Introduction**

This EdgeWorker implementation identifies traffic from **AI search engines and agentic web crawlers** such as ChatGPT, Perplexity, Gemini, Claude, Grok, DeepSeek, and more.

As AI models increasingly pull, structure, and reuse web content to answer user prompts, this routing layer allows you to:

* Serve AI-specific variants of content
* Improve clarity of machine-consumable signals
* Understand how AI systems traverse and interpret your site
* Strengthen your visibility across the agentic web

## **Supported AI Search Engines**

#### Introduction

This Netlify Edge Function implementation provides intelligent detection and routing of **AI search engine traffic**.\
The system identifies requests from major AI-powered search services and routes them to **specialized content endpoints optimized for AI consumption**.

#### Supported AI Search Engines

ChatGPT / OpenAI, Gemini / Google AI, AI Overviews, Perplexity, Claude, Grok, DeepSeek, and generic AI modes.

## **Configuration**

#### **AI Bot Pattern Configuration**

```json
{
  "aiSearchBots": {
    "chatgpt": ["chatgpt-user", "gptbot", "openai-searchbot", "chatgpt-search", "openai-imagesbot"],
    "gemini": ["gemini-bot", "google-extended", "geminibot", "bard", "googlebot-ai"],
    "aiOverviews": ["google-inspectiontool", "google-ai-overview", "google-sge"],
    "perplexity": ["perplexitybot", "perplexity-search", "perplexityai"],
    "claude": ["claude-web", "claudebot", "claude-search", "anthropic-ai"],
    "grok": ["grok-bot", "grokbot", "xai-bot", "x-ai-bot"],
    "deepseek": ["deepseekbot", "deepseek-search", "deepseek-ai"],
    "aiMode": ["ai-mode-bot", "aimode", "intelligent-agent"]
  }
}
```

#### **Routing Rules Configuration**

```json
{
  "routingRules": {
    "chatgpt": { "method": "path_prefix", "target": "/chatgpt", "preserveQuery": true },
    "gemini": { "method": "path_prefix", "target": "/gemini", "preserveQuery": true },
    "aiOverviews": { "method": "path_prefix", "target": "/ai-overview", "preserveQuery": true },
    "perplexity": { "method": "path_prefix", "target": "/perplexity", "preserveQuery": true },
    "claude": { "method": "path_prefix", "target": "/claude", "preserveQuery": true },
    "grok": { "method": "path_prefix", "target": "/grok", "preserveQuery": true },
    "deepseek": { "method": "path_prefix", "target": "/deepseek", "preserveQuery": true },
    "aiMode": { "method": "path_prefix", "target": "/ai-mode", "preserveQuery": true }
  }
}
```

## **Implementation**

#### **Core Detection Function**

```js
function detectAISearchBot(userAgent) {
    const ua = userAgent.toLowerCase();

    if (ua.includes('chatgpt') || ua.includes('gptbot') || ua.includes('openai'))
        return 'chatgpt';

    if (ua.includes('gemini') || ua.includes('bard') || ua.includes('google-extended'))
        return 'gemini';

    if (ua.includes('google-sge') || ua.includes('ai-overview'))
        return 'aiOverviews';

    if (ua.includes('perplexity'))
        return 'perplexity';

    if (ua.includes('claude') || ua.includes('anthropic'))
        return 'claude';

    if (ua.includes('grok') || ua.includes('xai-bot'))
        return 'grok';

    if (ua.includes('deepseek'))
        return 'deepseek';

    if (ua.includes('ai-mode') || ua.includes('intelligent-agent'))
        return 'aiMode';

    return null;
}
```

## **URL Routing Function**

```js
function generateAIBotUrl(originalUrl, aiBot, config) {
    const url = new URL(originalUrl);
    const rule = config.routingRules[aiBot];

    if (!rule) return originalUrl;

    switch (rule.method) {
        case 'path_prefix':
            url.pathname = rule.target + url.pathname;
            break;
        case 'subdomain':
            url.hostname = rule.target;
            break;
        case 'parameter':
            url.searchParams.set(rule.param || 'ai', rule.value || aiBot);
            break;
    }

    if (!rule.preserveQuery) {
        url.search = '';
    }

    return url.toString();
}
```

## **Complete EdgeWorker Implementation**

```js
import { aiSearchBots, routingRules } from './config.js';

export async function onClientRequest(request) {
    const userAgent = request.getHeader('User-Agent') || '';
    const originalUrl = request.url;

    // Detect AI search engine or agentic web crawler
    const aiBot = detectAISearchBot(userAgent);

    if (!aiBot) {
        // Human traffic — no special handling needed
        return;
    }

    // Generate AI-optimized URL
    const targetUrl = generateAIBotUrl(originalUrl, aiBot, { routingRules });

    // Add request metadata for analytics
    request.setHeader('X-AI-Bot', aiBot);
    request.setHeader('X-Original-URL', originalUrl);
    request.setHeader('X-AI-Optimized', 'true');

    // Route the AI agent to specialized content
    request.url = targetUrl;

    // Observability
    console.log(`Agentic Web Routing: ${aiBot} -> ${targetUrl}`);
}
```

## **Routing Methods (Agentic Web–Ready)**

#### **Path Prefix Routing**

Best for agentic clarity:

* `/article/ai-trends`\
  → `/chatgpt/article/ai-trends`

#### **Subdomain Routing**

Useful for large multi-property ecosystems:

* `www.example.com/page`\
  → `perplexity.example.com/page`

#### **Parameter Routing**

Lightweight AI classification:

* `/documentation`\
  → `/documentation?ai=claude`

## **Environment Configuration**

#### Production

Focus on stability, clarity for AI models, and observability.

#### Staging

Used for expanding bot patterns, experimenting with new AI indexing behaviors, etc.

*(all JSON examples updated to reflect agentic terminology; no SEO terms.)*

## **Monitoring & Analytics**

#### AI-Specific Headers

* `X-AI-Bot` - Identifies which AI system accessed the content
* `X-AI-Optimized` - Indicates that agentic routing occurred
* `X-Original-URL` - For reconstructing navigation flow

#### Example Log

```json
{
  "timestamp": "2025-01-01T12:00:00Z",
  "aiService": "chatgpt",
  "originalUrl": "/article/example",
  "routedUrl": "/chatgpt/article/example",
  "userAgent": "ChatGPT-User/1.0",
  "routingMethod": "path_prefix"
}
```

## **Deployment**

* Upload EdgeWorker bundle
* Apply routing rule via Property Manager
* Enable monitoring
* Deploy to staging, validate AI routing
* Roll out progressively to production

## **Performance & Security (Agentic Web Context)**

* AI-specific rate limits
* Pattern validation
* Known bot IP verification
* Agent behavior anomaly detection

That’s it! You have now successfully configured Limy router for your website. Data should begin to populate on your dashboard within an hour.