Connecting your data

​

AWS CloudFront

1. Select your AWS region in Peec.
2. Click Open in AWS Console. A CloudFormation quick-create page opens with all parameters pre-filled.
3. Deploy the stack. This usually takes 1 to 2 minutes.
4. Back in Peec, enter your CloudFront Distribution ID(s), comma-separated if you have several.

​

Google Cloud CDN

1. Select your GCP region in Peec.
2. Copy the generated setup command.
3. Run it in your terminal or Google Cloud Shell. The script asks for your Cloud Load Balancer URL map name, then completes setup and activates the integration.

​

Cloudflare

​

Create an API token

1. Navigate to your profile in Cloudflare
2. Select API Tokens from the left-hand panel
3. Select Create Token and click on Get Started (the first option)
4. Give your token a name
5. Add three permission fields by clicking on Add More under the permission section. From there, you need to add in order:
   1. Workers Scripts > Edit
   2. Zone > Zone > Read
   3. Zone > Workers Routes > Edit
6. No need to edit anything else, you can continue to the summary and click on Create Token

​

Deploy the Worker

1. Enter your Cloudflare API token in Peec and press Enter to validate it.
2. Select your Zone from the dropdown.
3. Click Deploy worker.

​

Cloudflare Worker limits

• Peec stops receiving log data: For the remainder of that day, Peec stops receiving any log data coming from your domain, resulting in gaps in your Crawl Insights.
• Errors to visitors: Cloudflare routes requests to Fail closed by default, meaning that once the limit is hit, Cloudflare serves a 1027 error page instead of your site.

• Enable minimum safeguard: In your Cloudflare dashboard, set the route mode to Fail Open. Your site will remain accessible to visitors, though data collection will still be paused for the rest of the day.
• Upgrade to a Workers Paid plan: The Workers Paid plan removes the daily request cap, ensuring uninterrupted log ingestion and continuous data in Crawl Insights. You can find Cloudflare’s pricing page for details.

​

Vercel

1. Enter your Vercel API token in Peec and press Enter to validate it.
2. Select your Team from the dropdown.
3. Select the Projects to monitor, or choose All projects.
4. Click Deploy drain.

​

WordPress

1. Click Download Plugin in Peec. A .zip file is generated with your credentials embedded.
2. In your WordPress admin panel, go to Plugins → Add New → Upload Plugin.
3. Upload the .zip file and click Activate Plugin.

​

Akamai

1. Enter your Akamai EdgeGrid credentials in Peec and press Enter to validate them.
2. Select a Group from the dropdown.
3. Select the Properties to monitor.
4. Click Deploy DataStream.

​

Generic webhook

1. Click Generate API Key in Peec.
2. Click Reserve API Key to lock it in.
3. Copy the Webhook URL and required headers.
4. Configure your system to POST log batches to the endpoint.
5. Click Confirm webhook deployment.

​

Endpoint

POST https://api.peec.ai/agent-analytics/generic-access-log

​

Required headers

Authorization: Bearer <your-api-key>
x-org-id: <your-organization-id>
Content-Type: application/json

​

Request body

[
  {
    "timestamp": "2024-01-15T12:34:56Z",
    "request_method": "GET",
    "request_url": "https://example.com/blog/my-post",
    "response_status": 200,
    "user_agent": "GPTBot/2.0",
    "country_code": "US",
    "client_ip": "1.2.3.4",
    "referer": "https://some-site.com/"
  }
]

​

File upload

1. Drag and drop your log file (.csv or CLF .log), or browse for it.
2. For CLF files, enter your domain (e.g. example.com) when prompted, so Peec can build full URLs from the request paths.
3. Peec processes the file and shows how many AI bot requests it found.
4. Click Upload to import them. A progress bar tracks completion.

​

CSV format

timestamp,request_method,response_status,user_agent,request_url
2024-01-15T12:34:56Z,GET,200,GPTBot/2.0,https://example.com/blog/post

​

Common Log Format (Apache / Nginx)

1.2.3.4 - - [15/Jan/2024:12:34:56 +0000] "GET /path HTTP/1.1" 200 1234 "https://referer" "GPTBot/2.0"

​

Supported AI bots