A shop sells work shoes. Its menu offers size, colour, safety class, brand and delivery time. Even a few values per field produce a great many possible combinations. For customers, that is helpful; for search, a different question arises. Should “black work shoes in size 44, sorted by price” really be an entry page of its own? Often the answer is no. “S3 safety shoes”, on the other hand, can be a permanent category with its own range and clear demand.
Which shop filter pages have a job of their own
Start with the products and the buyers’ questions, not with the number of existing parameters. An indexable selection should show a permanent range, be reachable via internal links and explain with an understandable title what it offers. It needs enough matching products or an honest explanation if stock fluctuates. A one-off random filter, an empty combination or sorting by price rarely solves a new search task.
This is not a rule based on search volume alone. A small but specific selection can make sense for a specialist shop if it meets a real need. What matters is that its content and product list are not just an almost identical copy of the parent category. Give such pages readable, stable addresses and link them from the navigation or from matching categories. A page that only comes into existence after five clicks in a JavaScript filter is harder to maintain as an editorial entry point.
S3 safety shoes: a standalone, maintained category is possible.
S3 + size 44 + price ascending: a useful view in the shop, but usually not a search entry point of its own.
The example is illustrative; actual demand and range have to be checked case by case.Why filter URLs bloat crawling
Google describes faceted navigation as a common source of practically unlimited URL spaces. If combinations are offered via parameters in links, crawlers can discover and fetch a great many addresses before it is clear that they add no value. That uses server resources and can slow down the discovery of important new pages. Interchangeable parameter orders, calendar values without limits and filters that do not sensibly exclude each other are particularly problematic.
An inventory quickly provides concrete figures: which parameters exist? How many URLs does a typical path generate? Which variants appear in the crawl report or server logs? Which are visible in search results? Separate product pages, important categories, individual approved filter pages and pure UI views. Only then can you decide which patterns should stay open.
Canonical, robots.txt and noindex: three different jobs
If a filter view is very similar to the parent category, a rel="canonical" can signal the preferred version. It is a hint, not a command; Google may decide otherwise. In addition, canonical does not prevent the first fetch of the variants. Google writes that it can at most reduce crawling of such URLs over time. Anyone who mainly wants to prevent many unnecessary fetches needs a different control.
For filter URLs that are not needed, Google names robots.txt as a way to limit crawling. This has to match your own URL patterns precisely so that important categories and products stay reachable. However, Google can know a blocked URL without its content and may still show it under certain circumstances. noindex in turn requires that the crawler can fetch the page. Anyone who blocks the same URL via robots.txt cannot rely on its noindex being read. These mechanisms are therefore not interchangeable labels.
When making a change, also check the linking. If internal links offer thousands of unnecessary variants, the URL space stays large even if individual signals are set. A manageable selection of static categories and specifically permitted filters is easier for editors and developers to control than a combination network repaired after the fact.
A pilot with one product group
Take a category with heavy filter use. Mark three types of URL: permanent search entry points, helpful shop views without a search claim of their own, and technically pointless combinations. Define a behaviour for each type. Then check status code, internal links, canonical, robots rules and visible content on example URLs. Afterwards, observe whether important product pages are still found and whether unnecessary variants decrease in the crawl.
The goal is not the smallest possible index at any price. It is a catalogue in which buyers can filter freely and search engines find a few meaningful entry points. Anyone who separates these two jobs can keep the shop function and still limit the flood of URLs.
