LLM Scraper 12 Endpoints

The LLM Scraper retrieves actual AI-generated search results directly from ChatGPT and Gemini's search interfaces — not via the model's API, but by scraping the AI search product itself. This returns what real users see when they search, including sources, citations, and follow-up suggestions.

Endpoints

ChatGPT LLM Scraper (6 endpoints)

Call Name	Method	Endpoint
Set ChatGPT Scraper Tasks	POST	/v3/ai_optimization/chatgpt/llm_scraper/task_post
Get Completed Tasks	GET	/v3/ai_optimization/chatgpt/llm_scraper/tasks_ready
Get Re-parsed Tasks	GET	/v3/ai_optimization/chatgpt/llm_scraper/tasks_fixed
Task GET	GET	/v3/ai_optimization/chatgpt/llm_scraper/task_get/{id}
Live Scraper	POST	/v3/ai_optimization/chatgpt/llm_scraper/live
Locations	GET	/v3/ai_optimization/chatgpt/llm_scraper/locations

Gemini LLM Scraper (6 endpoints)

Call Name	Method	Endpoint
Set Gemini Scraper Tasks	POST	/v3/ai_optimization/gemini/llm_scraper/task_post
Get Completed Tasks	GET	/v3/ai_optimization/gemini/llm_scraper/tasks_ready
Get Re-parsed Tasks	GET	/v3/ai_optimization/gemini/llm_scraper/tasks_fixed
Task GET	GET	/v3/ai_optimization/gemini/llm_scraper/task_get/{id}
Live Scraper	POST	/v3/ai_optimization/gemini/llm_scraper/live
Locations	GET	/v3/ai_optimization/gemini/llm_scraper/locations

Scraper vs. LLM Responses API

Feature	LLM Responses	LLM Scraper
Source	Model API	Live AI search interface
What it returns	Structured model response	What users actually see
Source citations	Sometimes	Always (Gemini, ChatGPT search)
Speed	Up to 72 hours	Live available
Cost	~$0.01/task	Varies
Best for	Brand mention tracking	Citation audits, competitive intel

Use Cases

• See exactly which websites ChatGPT Search or Gemini is citing for your target queries
• Identify which competitors dominate AI search citations for local service keywords
• Compare scraped AI responses to organic SERP results — spot gaps in your citation strategy
• Monitor whether your client's site appears as a source in AI-generated answers

ChatGPT Scraper — Live Example

import os
import requests
from requests.auth import HTTPBasicAuth

auth = HTTPBasicAuth(os.environ["DATAFORSEO_LOGIN"], os.environ["DATAFORSEO_PASSWORD"])
BASE = "https://api.dataforseo.com"

r = requests.post(
    f"{BASE}/v3/ai_optimization/chatgpt/llm_scraper/live",
    auth=auth,
    headers={"Content-Type": "application/json"},
    json=[{
        "keyword": "best roofing contractor in Dallas TX",
        "location_code": 2840,
        "language_code": "en"
    }]
)

result = r.json()["tasks"][0]["result"][0]
print("Response:", result.get("response_text", ""))
print("Sources:", result.get("urls_mentioned", []))

Gemini Scraper — Standard (Bulk)

import time

# Post batch of queries
keywords = [
    "roofing contractor Dallas",
    "roof replacement cost Texas",
    "emergency roof repair Dallas"
]

task_ids = []
for kw in keywords:
    r = requests.post(
        f"{BASE}/v3/ai_optimization/gemini/llm_scraper/task_post",
        auth=auth,
        headers={"Content-Type": "application/json"},
        json=[{"keyword": kw, "location_code": 2840, "language_code": "en"}]
    )
    task_ids.append(r.json()["tasks"][0]["id"])

# Poll for completion
time.sleep(120)

results = []
for task_id in task_ids:
    r = requests.get(
        f"{BASE}/v3/ai_optimization/gemini/llm_scraper/task_get/{task_id}",
        auth=auth
    )
    results.append(r.json()["tasks"][0]["result"][0])

# Extract citation URLs
for res in results:
    print(f"\nKeyword: {res.get('keyword')}")
    for url in res.get("urls_mentioned", []):
        print(f"  Cited: {url}")

Response Fields

Field	Type	Description
keyword	string	Query submitted
response_text	string	Full AI-generated answer
brands_mentioned	array	Brand names detected in response
urls_mentioned	array	Source URLs cited by the AI
items	array	Structured search result items (if returned)
se_type	string	"chatgpt_scraper" or "gemini_scraper"

Pricing

Live endpoint: ~$0.01–$0.02 per request. Standard: ~$0.005–$0.01. Check account dashboard for current rates.

Notes

• LLM Scraper is the most accurate representation of what users see — use it for citation audits, not just brand mention counting
• Gemini scraper is especially valuable because Google's AI search directly competes with organic results
• Always pair scraper data with organic SERP data to compare AI citation URLs vs. organic ranking URLs
• Not all queries return sources — conversational/factual queries may return only response text with no citations