Why Google isn't indexing your site, and how to fix it

First: find out what Google knows about your site

• Search Console's URL Inspection tool tells you whether a specific URL is on Google and, if it is not, why. It also shows you the HTML Googlebot saw ("View crawled page") and lets you request indexing.
• The page indexing report does the same for the whole site: how many URLs are indexed and how many are not, grouped by reason.
• site:yourdomain.com on Google gives you a rough idea, but Google warns that it doesn't necessarily return all indexed URLs. Do not draw conclusions from it alone.

If you do not have Search Console yet, start there: add your site as a domain property (verified with a TXT record in your DNS) and submit your sitemap.

The reasons, in four families

1. You blocked it (without meaning to)

• "Excluded by 'noindex' tag". There is a <meta name="robots" content="noindex"> tag or an X-Robots-Tag: noindex header. It usually comes from a template, an SEO plugin or a staging configuration. Remove it from the pages you want on Google.
• "Blocked by robots.txt". robots.txt prevents crawling. Be careful using it to keep pages out of Google: if other sites link to a blocked URL, Google can still index it without visiting it. To deindex a page, use noindex and let Google crawl the page so it can see it.
• "Blocked due to unauthorized request (401)" or "Blocked due to access forbidden (403)". The page asks for a password or the server turns Googlebot away. Check firewalls, bot protection and password protection left switched on.
• "Alternate page with proper canonical tag". Not an error: the page points with rel="canonical" to another version, which is the one that gets indexed. It is only a problem if this page should be indexed in its own right.
• "Duplicate, Google chose different canonical than user". Your canonical says one thing and Google decided another, nearly always because it sees two pages that are too similar. Make them different or merge them.

2. It returns a technical error

• "Server error (5xx)". The server failed when Googlebot asked for the page. If it keeps happening, Google slows down crawling of the whole site.
• "Redirect error". Chains that are too long, loops, or redirects to invalid URLs.
• "Not found (404)". Normal for pages that no longer exist. If the page has moved, redirect it with a 301 to its new address.
• "Soft 404". The page answers 200 but looks empty or like an error: an internal search with no results, a category with no products. Return a real 404, or give the page content.

3. Google has not found it

If a URL does not even appear in the report, Google does not know it exists.

• Orphan pages: no other page links to them. Link them from the menu, the home page or related pages.
• Links Google cannot follow: Google follows <a href="..."> links. A button that navigates with JavaScript and has no real href is no use to it.
• A sitemap that is incomplete or was never submitted. Submit it in Search Console and check that it lists every URL you want on Google.

4. Google has seen it and decided to wait

These are the two statuses that confuse people most, and the most common ones on new sites:

• "Discovered – currently not indexed". According to Search Console Help, Google found the page but has not crawled it yet, typically because crawling it was expected to overload the site, so it rescheduled the crawl. In practice, Google has decided that crawling it is not a priority.
• "Crawled – currently not indexed". Google crawled the page and did not index it. Search Console Help says it may or may not be indexed in the future, and that there is no need to resubmit it.

Neither is fixed by pressing "Request indexing" more often. What moves the needle:

• A page that offers something of its own. Pages that are near-identical to each other, nearly empty, or repeat what a hundred other sites already say stand little chance.
• Internal links from your strongest pages, starting with the home page.
• External signals: links and mentions from other sites, which tell Google your site deserves more attention.
• Time. A new site starts with no history, and Google crawls it cautiously.

If your site is new

1. Verify a domain property in Search Console and submit the sitemap.
2. Link every important page from the home page or the menu.
3. Request indexing of the key pages with URL Inspection, once each: there is a quota, and Google says asking repeatedly does not speed anything up.
4. Add the site to Bing Webmaster Tools. You can import it straight from Search Console. Besides Bing itself, its index feeds Copilot and search engines such as DuckDuckGo.
5. If your platform supports it, turn on IndexNow to notify the search engines that use it (Bing, Yandex, Seznam, Naver and Yep, among others) the moment you publish or change something. Google does not use IndexNow, and since 2023 it no longer accepts the old sitemap "ping" either: with Google, what works is the sitemap submitted in Search Console.
6. Get your first external mentions: a company page on LinkedIn, directories in your sector, clients and partners who can link to you.
7. Be patient. Google warns that crawling can take anywhere from a few days to a few weeks, and that requesting it does not guarantee indexing.

What does not work

• Requesting indexing of the same URL over and over. It does not speed up crawling.
• Publishing dozens of near-identical pages to "have more content". It adds to the discovered-but-not-indexed pile and can fall under Google's spam policies on scaled content.
• Buying links. That is another of those policies, and the risk is not worth it.

Summary: status and what to do

For a quick first picture, our free SEO audit checks robots.txt, the sitemap, noindex, canonicals, redirects and broken links live. For the full picture, follow how to do an SEO audit, step by step.

Sources

• Search Console Help: Page indexing report
• Google Search Central: Ask Google to recrawl your URLs
• Google Search Central: Introduction to robots.txt
• Google Search Central: Block search indexing with noindex
• Google Search Central: the site: search operator
• Google Search Central: Spam policies for Google web search
• IndexNow