In this guide
- What programmatic SEO is, and when it works
- What Google actually says
- Step 1: Validate demand before you automate
- Step 2: Make sure the data is ready
- Step 3: Give every page unique value
- Step 4: Control indexation and crawling
- Step 5: Quality assurance and editorial governance
- Should this page exist? A decision test
- When to pause, prune or refresh
- Who does what
- How I start a programmatic project
- Frequently asked questions
- Sources and further reading
Key takeaways
- Google does not object to pages built from templates and data. Its spam policies target scaled content abuse and doorway abuse: many pages made mainly to manipulate rankings, however they are produced.
- The deciding factor is whether each page helps someone and contains information the neighbouring pages do not. Swapping a city or product name into the same text does not count.
- Validate demand and data quality before you automate, start with a small pilot, and give every page type required fields and a kill switch.
- Control what gets indexed, keep every page reachable through a browseable hierarchy, and prune, merge or noindex pages that do not earn their place.
- Use a written decision test for every page type. If a page fails the first three questions, do not build it.
What programmatic SEO is, and when it works
Programmatic SEO means generating many pages from a template and a dataset instead of writing each one by hand: product and category pages, branch or location pages, comparison pages, directories, listings, tool pages. It is a method of building pages, not a category Google judges. Google judges the pages that come out of it.
It works when the data is the value. A page showing a real branch's address, opening hours, services and staff, or a product with real specifications, stock and price, gives a visitor something specific, and writing thousands of those by hand would be impractical. It fails when only the keyword changes. A page that says the same thing about every city, with the city name swapped in, adds a URL and nothing else.
I get asked about this most by businesses that want to cover many locations, products or services at once. The honest answer is always the same: the template is the easy part, and the data and the decision about which pages deserve to exist are the work. This guide sits alongside the enterprise SEO overview because programmatic pages are one of the quickest ways for a site to become enterprise-sized.
What Google actually says
Before building anything, read the policies that apply. The wording matters, and it is more precise than the word "penalty" suggests.
| Policy | What Google says | What it means for programmatic work |
|---|---|---|
| Scaled content abuse | Scaled content abuse is when "many pages are generated for the primary purpose of manipulating search rankings and not helping users". Examples include using generative AI or similar tools to generate many pages without adding value for users, and creating many pages where the content makes little sense to a reader but contains search keywords | The test is purpose and value, not the tool. Hand-written or automated, many low-value pages are the problem |
| Doorway abuse | Pages created to rank for specific, similar queries. Examples include multiple domains or pages targeting specific regions or cities that funnel users to one page, and substantially similar pages that are closer to search results than to a clearly defined, browseable hierarchy | Near-identical city or service pages that lead to one destination are a classic risk. Pages should sit in a real, navigable structure |
| AI-assisted content | Generative AI can be useful for research and adding structure to original content. Using it to generate many pages without adding value may violate the scaled content abuse policy. Output should be reviewed, and Google suggests explaining how content was created where readers would wonder | AI can help fill a template, but a person must check accuracy and value, including titles, descriptions and structured data |
| People-first content | Content made primarily for people, not to manipulate rankings. Google warns against extensive automation to produce content on many topics, and against mass-producing content so that individual pages get little attention or care | Ask the "who, how and why" questions of every page type |
On consequences, Google says sites that violate its policies may rank lower or not appear in results at all, and violations can be found by automated systems or human review, which can lead to a manual action. Google's debugging guide for traffic drops points to the Manual Actions report in Search Console as one place to check. So in practice there are two risks: a manual action you can see, and quieter demotion or non-indexing that you only see in the data. Note that this guide cites policy, not a ranking formula, because Google does not publish one.
Step 1: Validate demand before you automate
I start with evidence, not a keyword list pasted into a template. Demand for a pattern is rarely uniform: a pattern that looks like 500 pages of opportunity could, for example, turn out to be 40 pages with demand and 460 without.
- Look at what you already have. In Search Console, find existing pages and queries that resemble the pattern. Do the closest pages earn impressions and clicks? If not, a thousand more like them will not either.
- Check the results. Search for a sample of the queries and see which page types Google shows. If every result is a different kind of page than the one you plan, the pattern may not fit the intent.
- Check seasonality and trend. Google Trends helps separate steady demand from a short spike.
- Expand from real phrases. The Keyword Generator turns a seed term into real search suggestions and questions, and the Keyword Extractor shows how competing pages word things. Treat both as ideas to check, not demand data.
Then map each query group to one page intent, so two templates do not compete for the same search. If you find yourself writing the same page for two phrases, you have one page.
Step 2: Make sure the data is ready
A template is only as good as the data feeding it. Before building, I define for every page type:
- Mandatory fields. The pieces of information a page must have to be useful. If a record is missing one, the page is not generated.
- Freshness rules. Who updates the data, how often, and what happens when it goes stale, for example a price or an opening time.
- Normalisation. Consistent names, units, currencies and spellings, so the same thing is not described three ways.
- Provenance. Where the data comes from and whether you have the right to publish it. Showing the source and last-updated date on the page also helps readers trust it.
- A fallback for gaps. If data is thin, do not pad the page. Either skip generating it, or point the visitor to a richer parent page.
As a hypothetical example, a clinic group with five real branches can generate a page per branch with real addresses, hours, services offered at that branch and staff. The same group generating a page for a town with no branch just because people search there would be creating the kind of page the doorway policy describes. The data tells you which pages are honest.
I also want a kill switch for every page type: a way to unpublish or noindex a whole template quickly if the data or the results turn out bad.
Step 3: Give every page unique value
Thin or duplicate pages arise when one template is reused with minimal changes. The check I apply is to compare two sibling pages side by side with the name removed. If the useful content is the same, one of them should not exist.
| Thin page | Useful page |
|---|---|
| Same paragraphs across variants with a name swapped in | Specific data, such as real stock, prices, branches, schedules or measurements |
| A list of results that mirrors a search results page | A page that helps a visitor complete a task or make a decision |
| Generic claims that apply anywhere | Context verified for this exact place, product or case |
| No author, source or date | Clear ownership, a source for the data and a last-updated date |
| Reachable only through the sitemap or search | Reachable through category and hub pages a visitor can browse |
| Markup copied from a template whatever the page contains | Structured data that matches what is visible on this page |
Where variation should come from is the data, not the prose. Add comparisons that are actually computed from your records, moderated user reviews, photos you own, and answers to questions customers really ask. I would not add generated filler to make a page longer. Length is not a goal, and Google's guidance consistently points to value for the reader. Check each template with the On-Page SEO Checker and Schema Markup Validator on a sample of real output, not only on the design mock-up.
Step 4: Control indexation and crawling
Reachable through a real structure
Because the doorway policy criticises pages that are closer to search results than a browseable hierarchy, I build hubs first: category, region or topic pages that group the generated pages and link to them in a way a visitor would actually use. Generated pages reachable only through a sitemap are less likely to be treated as important, and large sets of them often show up under statuses such as "Discovered - currently not indexed" or "Crawled - currently not indexed". The Internal Link Analyzer shows orphans and depth on a sample.
One URL per intent
Decide which URL is canonical for each intent, link to it consistently and give each variant a self-referencing canonical. For filters and sort orders, Google's faceted navigation guidance gives two paths: prevent crawling of filter URLs you do not need indexed (for example with robots.txt), or, for the ones you do want indexed, keep the URLs consistent and return a 404 when a combination has no results; the ecommerce guide covers it in context. Check the output with the Canonical URL Checker.
Sitemaps by page type
A sitemap file is limited to 50,000 URLs or 50 MB uncompressed, so large programmes need several. I split them by template as well, for a practical reason: in Search Console you can then see how many pages of each type are indexed. Keep lastmod accurate, since Google uses it only when it is consistently reliable, and ignore priority and changefreq, which Google ignores. The XML Sitemap Generator and CSV-to-Sitemap Converter split large lists.
Crawl budget, in proportion
Google says its crawl budget guide is for sites with roughly a million or more pages changing weekly, or 10,000 or more changing daily, and for sites with a large share of URLs in "Discovered - currently not indexed". Programmatic builds can reach that scale quickly, but for most the more relevant signal is the indexing status of each template. If a large share of a template is "Crawled - currently not indexed", treat that as feedback on page quality before you treat it as a crawling problem. The indexing diagnosis guide explains each status.
Release in batches
I publish in batches, and I want to be clear about why. Google has not published a rule penalising fast publishing, so this is risk control, not a Google requirement. A batch lets me check quality, confirm the pages are indexed and read how they perform before the next, and it limits the damage if a template has a flaw. How templates are released matters too, which is the subject of the deploy risk guide.
Step 5: Quality assurance and editorial governance
Quality at scale comes from a gate that every batch must pass, built from automated checks plus human review.
- Automated checks: mandatory fields present, valid structured data, unique titles, working links and no template placeholders left in the output.
- Similarity alerts: flag pages whose main content is too close to a sibling. Choose the threshold by reading real pairs of pages yourself and setting the alert just below the level where they stop feeling distinct, rather than copying a number from elsewhere.
- Human review: read a random sample from every batch, plus the riskiest variants, such as pages built from the thinnest data. If you use AI to draft any part, a person must verify facts and claims, as Google advises.
- A publish gate: a batch does not go live until it passes both layers, and someone is accountable for the decision.
Treat templates like code: versioned, with a change log, reviewed before release. That is the same governance idea covered in the governance guide. After launch, schedule reviews to refresh data and retire pages that no longer meet the bar, rather than leaving them to decay.
Should this page exist? A decision test
I use these questions for every page type before building it, and again for any page type that is struggling. Failing any of the first three means I do not build it.
| # | Question | Why it matters |
|---|---|---|
| 1 | Is there evidence of a real need for this exact page, from search data or from customer questions? | Pages without demand only add URLs |
| 2 | Would a visitor be better served by this page than by its parent or a sibling? | If not, it should be a section of another page |
| 3 | Does it contain information not found on sibling pages, beyond the name or keyword? | This is the line between a useful page and a doorway |
| 4 | Is the data accurate, current and ours to publish? | Wrong or stale data harms trust, which Google calls the most important part of E-E-A-T |
| 5 | Can a visitor reach it through normal navigation? | A browseable structure is part of what separates a hub from a doorway |
| 6 | Does it help the visitor do something: contact, buy, compare or decide? | Pages that serve a task are more likely to earn engagement and satisfy visitors than pages that only exist to rank |
| 7 | Does it have an owner and a review date? | Unowned pages are the first to go stale |
Where a proposed page passes some questions but not others, the outcome is usually not a flat yes or no. It becomes a section on a richer page, a noindexed page kept for visitors, or a smaller launch.
When to pause, prune or refresh
Review each batch against what you expected, using indexing status per template, impressions and clicks per template, and engagement for the pages that do get visits. If a template is underperforming, pause further publishing of that type until you understand why. Then choose the least destructive fix that works.
| Situation | Action | Notes from Google's guidance |
|---|---|---|
| Page is useful but thin | Refresh: add real data and context | Value is what matters, so improve rather than remove where the need is real |
| Several pages overlap | Consolidate into one page with 301 redirects | Redirects are a strong canonical signal; redirect to the closest relevant page, not the homepage |
| Needed for visitors but not for search | Keep, with noindex | The page must not be blocked in robots.txt or Google will not see the noindex. Do not use noindex to pick a canonical within your own site |
| No longer needed | Remove and return 404 or 410 | Google advises 404 or 410 for permanently removed pages |
Before removing pages, check whether any have backlinks or real traffic, and document every decision in the change log. Pausing is almost always better than accumulating more pages while the signals are unclear. If the index is filling with weak pages across a site, the 12 errors that matter shows how to separate real problems from noise.
Who does what
Programmatic SEO works when three groups share the work. SEO defines the intent map, the quality thresholds and the decision test. Content or editorial owns the differentiated value and the review samples. Engineering owns the data pipelines, the template releases and the automated checks. The accountable owner for launching a new page type should be named in advance, using something like the RACI in the governance guide.
If you are also changing platforms, remember that programmatic sections are the part of a migration most likely to be forgotten. The migration audit checklist covers the inventory and redirect work.
How I start a programmatic project
I start with a pilot: one page type, a few dozen pages built from real data and checked by hand, published in one place. Only when those pages are indexed and doing their job do I widen the pattern. It is slower on day one and much cheaper than unpicking thousands of weak pages later. It is also how I would approach this as part of enterprise SEO, with content strategy deciding which topics deserve a system and local SEO shaping branch and location pages. In my Advanced SEO course, programmatic SEO appears among the types of SEO that learners compare.
Frequently asked questions
Is programmatic SEO against Google's guidelines?
No, building pages from templates and data is not itself a violation. Google's policies target scaled content abuse and doorway abuse, meaning many pages created mainly to manipulate rankings or funnel visitors. Pages that give people real, distinct value are different.
What makes programmatic SEO risky?
Volume without value. Shallow or stale data, repeated templates, pages targeting near-identical queries and no real navigation to them are what turn scale into a liability.
How many pages should I launch at once?
Google gives no number. I launch a small pilot first, and widen only after those pages are indexed and performing. The right batch size is whatever lets you review quality before the next release.
How do I measure whether scaled pages are working?
Track the indexing status of each template, impressions and clicks per template, and engagement or conversions on pages that receive visits. Splitting sitemaps by template makes the indexing picture easier to read.
What data problems break these programmes most?
Stale information, missing fields, duplicated records and inconsistent naming. A mandatory-field rule and a freshness owner for every dataset prevent most of them.
Can I use AI to write the pages?
AI can help structure and draft, and Google says generative AI can be useful for research and adding structure. Using it to produce many pages without adding value may violate the scaled content abuse policy. Have a person verify facts and consider explaining how the content was produced where readers would expect it.
Is "Crawled - currently not indexed" a penalty?
No. It is a status meaning Google crawled the URL and has not indexed it. On a programmatic template it often signals that the pages are not distinct or useful enough, so I read it as quality feedback.
Sources and further reading
Statements about Google in this guide are based on the pages below, checked on 22 October 2026. The five steps, the decision test and the pilot approach are my own working methods.
- Spam policies for Google web search: scaled content abuse, doorway abuse and consequences
- Google Search's guidance on using generative AI content: acceptable use, review and disclosure
- Creating helpful, reliable, people-first content: people-first content, E-E-A-T and the who, how and why questions
- Managing crawling of faceted navigation URLs: robots.txt, canonical and 404 for empty filters
- Build and submit a sitemap: size limits, priority, changefreq and lastmod
- How to specify a canonical URL: canonical signals, redirects and noindex
- Block Search indexing with noindex: noindex and robots.txt interaction
- Crawl budget management for large sites: who needs it, 404 and 410 for removed pages
- General structured data guidelines: markup must match visible content
- Page indexing report: status names such as Crawled - currently not indexed
- Debugging drops in Google Search traffic: Manual Actions report and spam checks