# AEO Site Checker — Got Mortgages Edition > The Got Mortgages Edition (from Got Mortgages, powered by PRMG) of a free, open Lighthouse-style auditor that grades any URL on Answer Engine Optimization (AEO) — how well a page is set up to be fetched, parsed, and cited by AI search engines (ChatGPT, Claude, Perplexity, Google AI Overviews and AI Mode). Scores 0–100 across 27 weighted checks (+1 type-conditional) in 5 categories. The auditor reads server-rendered HTML directly via undici (no headless browser, no JS execution — the same view an AI crawler gets), then runs structured-data extraction, Mozilla Readability, robots.txt parsing for the live AI-bot allow/block matrix, and content heuristics from Princeton's GEO paper. If the direct fetch hits a Cloudflare/Akamai/PerimeterX challenge, the auditor falls back to BrightData's Web Unlocker so the page can still be graded — but the major `fetch_direct` credit (18 of the raw weight) is lost, because that is exactly what an LLM crawler would also fail on. The site is built with Astro 5 + island-hydrated SolidJS, prerendered for AI crawlers, persisted to SQLite via Prisma. Every audit gets a shareable permalink and a downloadable PDF report. ## Pages - [Home](https://gotmortgages.aeositechecker.com/): What the tool measures, the five categories, FAQ. - [Run audit](https://gotmortgages.aeositechecker.com/audit): URL form that runs the full audit and returns a permalink. - [Saved audit (permalink)](https://gotmortgages.aeositechecker.com/audit/{id}): Read-only view of a previously run audit with the per-check breakdown. Replace `{id}` with a real cuid. - [Whitepaper](https://gotmortgages.aeositechecker.com/whitepaper): Full reference — site-type detection, every check and weight, letter grades, pipeline, bot-block detection, API, limitations. - [Knowledge base](https://gotmortgages.aeositechecker.com/blog): All articles — primers, technical guides, the rubric, and research summaries. ## Knowledge base articles - [The AEO scoring rubric, check by check](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): All 27 checks and the type-conditional contact check, their weights, the 104/109 denominator, why fetchability carries 43 of the raw weight, and what the score cannot see. - [Cloudflare's September 15 AI crawler defaults](https://gotmortgages.aeositechecker.com/blog/cloudflare-september-15-ai-crawler-defaults): The 2026 Search / Agent / Training categories, "Verified no longer means allowed", the Googlebot multi-purpose trap, the robots.txt `use=` signal, and the three settings to check. - [Where AI answers get their sources](https://gotmortgages.aeositechecker.com/blog/where-ai-answers-get-their-sources): Synthesis of 680M+ tracked citations — Wikipedia's 47.9% of ChatGPT's top-10 share, Reddit's 46.7% of Perplexity's, 11% domain overlap, 4.5% URL match on AI Overviews, and what one site can do. - [llms.txt, one year on](https://gotmortgages.aeositechecker.com/blog/llms-txt-one-year-on): Ahrefs (137k domains, 97% of files never requested) and SE Ranking (300k domains, no citation correlation); where the file still works (coding agents) and why the audit keeps checking it. - [Google AI Overviews and AI Mode: what decides whether you're in](https://gotmortgages.aeositechecker.com/blog/google-ai-overviews-ai-mode-inclusion): Googlebot, not Google-Extended, is the control; snippet controls that apply; query fan-out; AI Mode at 1B monthly users. - [Training, index, or live fetch: the three kinds of AI crawler](https://gotmortgages.aeositechecker.com/blog/three-kinds-of-ai-crawler): Every major AI user agent by function, quoted from OpenAI, Anthropic, Perplexity, and Google documentation, with robots.txt behavior and IP verification lists. - [Cloudflare bot protection and AEO](https://gotmortgages.aeositechecker.com/blog/cloudflare-bot-protection-and-aeo): The single most common AEO failure — Cloudflare blocking AI bots — and the WAF rules to fix it. - [The Princeton GEO paper, summarized](https://gotmortgages.aeositechecker.com/blog/geo-paper-summary): Aggarwal et al. (2024) — what the +41% statistics lift and +28% quotation lift actually mean, and what gets misread. - [Front-loaded answers](https://gotmortgages.aeositechecker.com/blog/front-loaded-answers): Why direct answers in the first paragraph are the highest-leverage AEO content fix. - [Why AI crawlers don't run JavaScript](https://gotmortgages.aeositechecker.com/blog/why-ai-crawlers-dont-run-javascript): Cost, determinism, latency — why ChatGPT/Claude/Perplexity fetch raw HTML, and how to make a JS-heavy site visible. - [JSON-LD for AI search](https://gotmortgages.aeositechecker.com/blog/json-ld-for-ai-search): The seven schema.org types AI engines actually look for, with copy-pasteable JSON-LD blocks. - [How AI search engines crawl](https://gotmortgages.aeositechecker.com/blog/how-ai-search-engines-crawl): Technical breakdown of the fetch-and-cite pipeline for ChatGPT, Claude, Perplexity, and Google AI Overviews. - [robots.txt for AI crawlers](https://gotmortgages.aeositechecker.com/blog/robots-txt-for-ai-crawlers): The 12 AI crawlers we audit against, the 7 that are critical, and the exact allowlist that works for almost everyone. - [What is /llms.txt](https://gotmortgages.aeositechecker.com/blog/what-is-llms-txt): Format spec, copy-pasteable template, common gotchas, and how the AEO Site Checker scores the file. - [What is AEO?](https://gotmortgages.aeositechecker.com/blog/what-is-aeo): Plain-English primer on Answer Engine Optimization, the four pillars, and how AEO differs from SEO. - [AEO vs SEO: what's actually different](https://gotmortgages.aeositechecker.com/blog/aeo-vs-seo): Side-by-side comparison and which 2018-era SEO tactics now actively hurt you. ## API - [POST /api/audits](https://gotmortgages.aeositechecker.com/api/audits): Send `{ "url": "https://example.com" }` to run an audit. Response includes the full per-check breakdown and a permalink `id`. - [GET /api/audits/{id}](https://gotmortgages.aeositechecker.com/api/audits/): Fetch a saved audit by id. - [GET /api/audits/{id}/pdf](https://gotmortgages.aeositechecker.com/api/audits/): Download the PDF report for a saved audit. - [GET /api/health](https://gotmortgages.aeositechecker.com/api/health): Liveness probe. ## Scoring rubric 27 always-on checks in 5 categories plus 1 type-conditional check; earned and possible points are summed across the checks that ran and normalized to 0–100. - [Fetchability — 43 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): Direct fetch reachable without bot challenge (18), HTTPS (4), reasonable page size (3), robots.txt allows critical AI bots (10), content rendered server-side without JS (8). - [Core SEO — 21 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): Title 25–65 chars (4), meta description 80–175 chars (4), canonical URL (3), OpenGraph tags (3), Twitter card (2), html lang (1), sitemap.xml (4). - [Semantic HTML — 13 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): Exactly one h1 (3), heading hierarchy (3), header/nav/main/footer landmarks (4), image alt-text coverage (3). - [Answer Engine signals — 26 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): /llms.txt (5), /llms-full.txt bonus (1), JSON-LD types (6), author + Organization sameAs (3), dateModified freshness (2), Mozilla Readability extraction (5), site breadth (4). - [Content quality — 6 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): Front-loaded answer (2), question-shaped headings (1), statistics density (2), blockquotes/quotations (1). - [Type-conditional contact signals — 5 points](https://gotmortgages.aeositechecker.com/blog/aeo-scoring-rubric): Phone, address, hours, service area on local-business / real-estate / person pages. ## AI crawlers checked in robots.txt The auditor probes robots.txt for `OAI-SearchBot`, `ChatGPT-User`, `GPTBot`, `Claude-User`, `Claude-SearchBot`, `ClaudeBot`, `PerplexityBot`, `Perplexity-User`, `Google-Extended`, `Applebot-Extended`, `Meta-ExternalAgent`, and `CCBot`. The "critical" set — the search-index and live-fetch agents whose blocking removes you from AI answers (plus `Google-Extended`, which controls Gemini grounding) — is `ChatGPT-User`, `OAI-SearchBot`, `Claude-User`, `Claude-SearchBot`, `PerplexityBot`, `Perplexity-User`, and `Google-Extended`. Note that `Google-Extended` does not affect Google Search, AI Overviews, or AI Mode; those are controlled by `Googlebot`. ## References - [llms.txt specification](https://llmstxt.org/): The proposed convention this auditor checks for at site root. - [Princeton GEO paper](https://arxiv.org/abs/2311.09735): The empirical study behind the content-quality heuristics. - [OpenAI crawler documentation](https://developers.openai.com/api/docs/bots): OAI-SearchBot, ChatGPT-User, GPTBot, OAI-AdsBot and their IP lists. - [Anthropic crawler documentation](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler): ClaudeBot, Claude-User, Claude-SearchBot. - [Perplexity crawler documentation](https://docs.perplexity.ai/guides/bots): PerplexityBot and Perplexity-User. - [Google crawler documentation](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers): Googlebot, Google-Extended, and what each controls. - [Cloudflare verified bots](https://radar.cloudflare.com/bots): Reverse-DNS-verified bot operator IP ranges.