How search works, in the detail a clinic needs
You do not need to know how a search engine is built. You do need to know the five gates a page passes through, because your page is stuck at one of them.
PrerequisitesModule 01 · What a clinic sells
A page reaches a patient only after passing five gates: it must be discovered, crawled, rendered, indexed and then selected for a particular query. Most clinic pages that are not found are stuck at the first four, which are technical and fixable, rather than the fifth, which is competitive and slow. Diagnosing which gate you are stuck at comes before any attempt to improve the writing.
What you will be able to do
By the end of this module you will be able to:
- Name the five stages between publishing a page and it appearing for a query.
- Diagnose which stage a missing page is stuck at, using Search Console rather than guesswork.
- Explain in plain terms what a search engine is trying to do when it ranks, and why that makes some tactics permanently unsafe.
- Set a realistic expectation for how long a new page takes to settle.
Five gates
Publishing a page is not the same as having a page. Between the two there are five distinct stages, each of which can fail independently, and each of which fails in a way that looks identical from the outside: the page does not appear.
The practical value of this model is that it turns "we are not ranking" into a specific question with a specific answer. Four of the five gates are technical, cheap to check and quick to fix. Only the fifth is a genuine competition, and it is the only one that takes months.
Discovery
A search engine finds pages by following links and by reading sitemap files. A page that nothing links to, and that appears in no sitemap, is invisible in a very literal sense. This sounds too basic to be a real problem and it is one of the most common real problems, particularly for pages added to a site through a booking system or a page builder that does not add them to navigation.
The two fixes are to link to the page from somewhere a reader would plausibly go, which is covered properly in Module 23, Internal linking as structure, and to make sure the page appears in the sitemap file the site submits. Google's documentation on building and submitting a sitemap is short and worth reading once.
Crawling and rendering
Crawling is the fetch. It can be blocked by the site's robots file, by a server that refuses the request, or by a password. Rendering is what happens when the fetched file needs scripts to run before any content exists. If your treatment descriptions are inserted by a script that fails or takes too long, a crawler can see an empty page where a human sees a full one.
This is the single most common serious technical fault on clinic sites built with heavy page builders or single-page frameworks. The test is not "does it look right in my browser". The test is what the page contains when scripts are disabled, or what Search Console's inspection tool reports as the rendered content.
Indexing
Being crawled is not being stored. A page can be fetched and then declined, either because it carries a directive asking not to be indexed, because it names a different page as its canonical version, or because the engine judged it to add nothing that is not already indexed elsewhere.
The first two are settings and are covered in Module 20, Crawling and indexing: what actually blocks you. The third is a content judgement and is the polite way a search engine tells you that your fourteen location pages are the same page. It is not a penalty and there is nothing to appeal. The remedy is fewer, better pages.
Selection, which is the slow one
Only now does ranking happen, and it happens per query rather than per page. The same page can be first for one phrasing and absent for another. This is why a single position number is a poor measure and why Module 31, What to measure, and what to ignore recommends counting queries and entrances rather than positions.
Selection is also where patience is required. A new page on a new site is competing against pages that have existed for years and have been visited, linked and cited in that time. A quarter is a reasonable interval before drawing conclusions. A fortnight is not, and a great deal of poor decision-making in clinic marketing comes from re-writing a page every three weeks because it has not yet worked.
What the engine is trying to do
It is worth stating plainly, because it explains what is safe and what is not. A search engine is trying to predict which page will satisfy the person who typed the query, well enough that they do not come back and try again. Everything it measures is a proxy for that.
Two consequences follow. First, anything that makes a page genuinely better at satisfying the reader is permanently safe, because the engine's interest and yours point the same way. Second, anything that manipulates a proxy without improving the page is temporarily effective and permanently at risk, because the proxy will eventually be replaced. Google's spam policies are largely a list of proxies that have been abused and are now detected.
If a technique would embarrass you to explain to a patient, it is a proxy manipulation, and it has a shelf life.
Diagnosing your own page
Search Console is free, it is the only source of this information that comes from the engine itself, and a clinic that does not have it verified is guessing. Verifying it is the first step of the exercise, and if you do nothing else from this module, do that.
Once verified, the URL inspection tool answers the gate question directly for any single page: whether it is known, whether it was crawled, what was rendered, whether it is indexed and if not why not. That is a diagnosis rather than an opinion, and it takes about ninety seconds per page.
Inspect five pages
Forty five minutes, most of which is the one-off verification. You need access to your site's DNS records or the ability to upload a file to it.
- Verify your site in Google Search Console if it is not already verified. Use the domain property option if you can edit DNS, as it covers every subdomain and both protocols.
- Submit your sitemap file. If you do not have one, note that and move on; Module 20 deals with it.
- Pick five pages: your homepage, your two most important treatment pages, one page you suspect is invisible, and one page you added recently.
- Run each through URL inspection. For each, write down: known to Google, crawled, indexed, and if not indexed the stated reason.
- Sort the five into the five gates from Figure 4.1. Any page stuck at gates one to four goes on a fix list ahead of any writing work.
Keep the fix list. Part V of the course is organised so that each of these faults has a module devoted to it.
Questions readers ask about this module
How long should a new page take to be indexed?
Indexing can happen within days on a site that is crawled often, and can take considerably longer on a small site with few incoming links. What matters is whether the page is known at all. If Search Console reports the URL as unknown after a fortnight, that is a discovery problem to fix rather than a waiting game.
Do I need to submit every new page manually?
No. Manual submission is useful for a handful of important pages or after fixing a fault. Routine submission of everything is unnecessary and does not accelerate anything meaningfully; a correct sitemap and sensible internal linking do the same job continuously.
Is there any way to make ranking faster?
Not legitimately, and the shortcuts that claim to are the proxy manipulations described above. What you can compress is the technical part: a page that is discovered, crawled and indexed promptly starts its competitive clock earlier than one that sits unnoticed for two months.
What about Bing and the AI answer engines?
The same five gates apply, with different crawlers. Most answer engines draw on indexes built the same way, so a page that fails at gate two fails everywhere at once. Module 19 deals with the differences that do matter, which are about how the text is written rather than how it is crawled.
Sources and primary references
Disclosure. This module contains no commercial links of any kind. This course is published by Northbank Media, is free to read, and is funded by the workbook pack and by clearly labelled series sponsorship. We do not rank, review or recommend agencies or clinics.
Module 04 of 32