Aident AI

How to Audit URL Indexing in Search Console With Codex
To audit whether important pages are indexed with Codex, use the Google Search Console URL Inspection Action on a short, reviewed URL list. Keep the property and inspection URL explicit, then classify the returned verdict, coverage state, robots state, indexing state, fetch state, canonical pair, and last crawl time. Treat the result as Google's indexed snapshot, not a live URL test or a request for indexing.
The safest output is a diagnosis queue. It should separate pages that are indexed, pages Google does not know, pages blocked by robots or noindex, fetch failures, and canonical mismatches. It should not change a sitemap, request indexing, edit a canonical, or expose an OAuth token.
This workflow uses Aident Loadout to give Codex bounded access to the current Search Console Action. Success means every reviewed URL has a source-backed status and next check, with zero provider writes.
What URL Inspection Can Tell You
Google's URL Inspection API returns information about the version in Google's index. The most useful fields are:
Field | Question it answers | Safe interpretation |
|---|---|---|
Verdict | Did the indexed result pass the inspection category? | Use |
Coverage state | How does Google describe index coverage? | Preserve Google's returned wording instead of converting it into a binary too early. |
Robots state | Was crawling allowed by | A disallowed result needs a public robots check before any content change. |
Indexing state | Did Google observe an indexing block such as | Confirm the rendered page and headers before changing the directive. |
Fetch state | Could Google fetch the page? | Investigate DNS, response status, timeout, redirect, or access controls when unsuccessful. |
User and Google canonical | Did Google select the declared canonical? | A mismatch is a review signal, not automatic proof that Google's choice is wrong. |
Last crawl time | How fresh is the indexed snapshot? | Old crawl data can explain why a recent change is not reflected yet. |
Crawled as | Which crawler type was reported? | Keep mobile and desktop context with the result. |
An unknown URL can have several detailed fields left unspecified. Do not turn missing detail into a guessed cause.
What URL Inspection Cannot Do
The API result is not:
a live fetch of the current page;
proof that the page ranks for a query;
a guarantee that Google will keep the URL indexed;
evidence that a recent deployment caused the status;
a request for indexing; or
a substitute for opening the public URL and checking its canonical, robots directives, response, and rendered text.
Use the URL Inspection tool in the Search Console interface when you need a live test. Use Search Analytics when the question is about impressions, clicks, CTR, or position.
What You Need
Prepare:
a verified Search Console property;
a connected Google account with permission to inspect its URLs;
the installed public Aident CLI;
a bounded list of fully qualified URLs under that property; and
a written rule for what counts as indexed, unknown, blocked, failed, or mismatched.
Give Codex the canonical setup instruction:
Then confirm account and Vault state:
If Google Search Console is not connected, start with How to Connect Google Search Console to Codex With Aident Loadout.
Step 1: Build a Bounded Inspection Queue
Start with pages that matter to one decision, such as:
a new page expected to be discoverable;
a canonical that recently changed;
a page with an unexpected Search Analytics drop;
a URL omitted from reported query-page rows; or
a small set of revenue or activation paths.
Do not begin with every URL in a sitemap. Google documents per-site URL Inspection quotas of 2,000 requests per day and 600 requests per minute. A small queue is easier to review and less likely to turn an indexing audit into a quota-burning crawl.
For each row, preserve the requested URL, expected canonical, publication or change time, and reason for inspection. This makes a later result explainable.
Step 2: Discover and Inspect the Current Action
Ask Codex to search Aident's live catalog for the Google Search Console Action that inspects one URL. Inspect the current schema before execution rather than saving an internal capability identifier in a permanent prompt.
Confirm:
the exact Search Console property identifier;
the fully qualified inspection URL;
the language code; and
that the URL is under the selected property.
Preflight the exact input. In our August 28, 2026 check, the current input validated and the preflight estimate was free. Pricing and schemas can change, so the durable procedure is to preflight, not to assume.
Run one proof inspection first. Verify that the response contains an inspection result and that no credential appears in the prompt, output, or repository.
Step 3: Classify the Detailed Result
Use the detailed fields in this order:
Preserve the verdict and coverage state exactly.
Check whether robots allowed crawling.
Check whether indexing was allowed.
Check whether the page fetch succeeded.
Compare the user-declared and Google-selected canonicals.
Record the last crawl time and crawler type.
Then assign one review class:
Review class | Evidence | Next check |
|---|---|---|
Indexed and aligned | Indexed coverage, successful fetch, indexing allowed, canonical match | Protect the page and monitor Search Analytics |
Unknown to Google | Neutral verdict and unknown coverage with little detail | Verify public access, internal links, sitemap presence, and elapsed time |
Crawl blocked | Robots disallowed | Inspect the public robots rule and intended policy |
Indexing blocked | Indexing state reports a block | Inspect rendered meta robots and |
Fetch failed | Page fetch was unsuccessful | Check the exact response path, redirects, server errors, and access controls |
Canonical mismatch | Google and user canonicals differ | Compare duplicates, redirects, internal links, sitemap entries, and content intent |
Do not use one field in isolation. An overall verdict without the detailed states cannot identify the next safe check.
A Four-URL Aident Inspection Snapshot
On August 28, 2026, we ran one bounded inspection pass against four Aident blog URLs under the same domain property. This was a current product proof, not a ranking benchmark.
URL | Coverage state | Verdict | Detailed result |
|---|---|---|---|
| Submitted and indexed | PASS | Mobile crawl, robots allowed, indexing allowed, successful fetch, declared and selected canonical matched |
| Submitted and indexed | PASS | Mobile crawl, robots allowed, indexing allowed, successful fetch, declared and selected canonical matched |
| URL is unknown to Google | NEUTRAL | Other detailed states were unspecified |
| URL is unknown to Google | NEUTRAL | Other detailed states were unspecified |
Both indexed pages reported a last crawl time of August 28, 2026 at 11:55:22 UTC. The two unknown results were newer pages at the time of the check. That timing is context, not causal proof. We did not request indexing, submit a sitemap, or infer that either page had a defect.
This snapshot demonstrates why the detailed classification matters. Two PASS results supplied canonical, crawl, robots, indexing, and fetch evidence. The NEUTRAL unknown results supplied much less. The appropriate next step is to verify public retrieval and ordinary discovery signals, then recheck after a defined interval.
Step 4: Verify the Public Page Separately
For every unknown, blocked, failed, or mismatched URL, perform read-only public checks:
the exact URL returns the intended status and body;
the rendered page contains the expected title and useful text;
the canonical points to the intended public URL;
meta robots and
X-Robots-Tagallow the intended indexing behavior;robots.txtdoes not block the path;the sitemap contains the canonical when appropriate;
internal links use the canonical URL; and
CDN or WAF rules do not treat crawlers differently from ordinary retrieval.
Record observations and timestamps. A current HTTP 200 does not rewrite the older indexed snapshot, and an old successful crawl does not prove the page works now.
Step 5: Choose the Smallest Safe Action
Match the action to verified evidence:
Finding | Smallest useful action |
|---|---|
New URL is publicly healthy but unknown | Improve relevant internal discovery, confirm sitemap presence, and schedule a later inspection |
Robots unexpectedly blocks the intended path | Fix the narrow rule after owner review, then verify public retrieval |
| Remove the exact directive through the owning deployment and verify the rendered response |
Fetch fails | Repair the response, redirect, DNS, timeout, or access-control cause before rechecking |
Google selects another canonical | Reconcile duplicates, redirects, internal links, sitemap entries, and page intent |
Indexed page has no impressions | Move to Search Analytics and intent diagnosis instead of repeating inspection |
Keep sitemap submission, indexing requests, redirects, canonical changes, and deletions behind explicit review. Diagnosis does not authorize a write.
Common Mistakes
Treating the Result as a Live Test
The API reports the indexed version. A recent deployment may not appear until Google crawls and processes it.
Calling Unknown a Defect
Unknown means the API has no indexed result to describe. Verify retrieval and discovery before assigning a cause.
Comparing Canonicals Without Normalizing URLs
Scheme, host, path, query, slash, and redirect differences matter. Compare the exact declared and selected values.
Inspecting the Whole Sitemap First
Large blind runs consume quota and create a review backlog. Start with URLs tied to one business or search decision.
Mixing Inspection With Write Operations
Do not submit a sitemap or request indexing inside the diagnostic run. Keep evidence collection read-only and approvals visible.
Reporting Indexing as Product Growth
An indexed URL is eligible for discovery. It does not prove clicks, setup starts, connections, activations, users, payments, pipeline, or revenue.
Reusable Codex Prompt
Paste this after setup and replace the bracketed values:
Use Aident Loadout to confirm my Google Search Console connection and inspect the current URL Inspection schema. Ask me to select the exact property. Review this bounded list of fully qualified URLs:
[URLs]. Preflight the exact input, then inspect one proof URL before the remaining queue. For each URL, preserve the verdict, coverage state, robots state, indexing state, fetch state, user-declared canonical, Google-selected canonical, last crawl time, and crawler type. Classify each result as indexed and aligned, unknown, crawl blocked, indexing blocked, fetch failed, or canonical mismatch. Treat the result as Google's indexed snapshot, not a live test. Recommend the smallest read-only next check. Do not request indexing, submit a sitemap, change a canonical, redirect or delete a URL, expose credentials, or perform another write.
Set up Aident Loadout and run one bounded URL indexing audit.
To interpret query reporting after pages are indexed, read Why Search Console Query Rows Do Not Match Totals. To measure page-level keyword breadth, continue with How to Count Ranking Keywords in Search Console With Codex.
Sources
Refresh this guide when Google's URL Inspection fields or quotas change, the live Aident Action schema changes, or a supported age-matched cohort establishes a better inspection and activation benchmark.



The one tool
for every tool
your agent needs.
Give any AI agent real capabilities in seconds. Connect 27,000+ tools once, skip the setup headache, and let your agents execute.
