Field guide · technical workflow
Audit the URL cost of Magento layered navigation
Layered navigation helps customers narrow a catalogue, but every layered combination can create another crawlable address. A large Magento estate needs a deliberate decision about which combinations are useful search destinations and which simply support browsing.
Who this is for: Give an Adobe Commerce technical director a source-backed method for classifying filter URLs without undermining product discovery.
Editorial preview — facts, worked examples and publication approval are still being checked.
The practical problem
A practical audit the url cost of magento layered navigation exercise: Group actual attribute/filter combinations using real crawled URL samples. The outcome is a documented decision backed by assign a separate crawl and indexing policy to each url class. The final check is to retest meaningful product discovery and mobile filters after changes.
What you will be able to check
- Group actual attribute/filter combinations using real crawled URL samples.
- Assign a separate crawl and indexing policy to each URL class.
- Retest meaningful product discovery and mobile filters after changes.
Reproducible scenario
Worked example: Audit the URL cost of Magento layered navigation
An illustrative Adobe Commerce PaaS retailer sells lighting, with a large ceiling-light category offering colour temperature, fixture type, price range and sort controls. A crawler discovers parameter combinations that repeat the same product selection in slightly different orders. The engineering team considers blocking all filters, but some curated fixture-type collections genuinely answer a specific buying need. The task is to distinguish persistent product families from temporary navigation states without claiming any actual crawl logs were analysed.
Starting assumptions
- The default store view exposes filter URLs and linked products through a known category template.
- The team can obtain a sample HTTP crawl and final HTML for both curated and arbitrary layered navigation combinations.
Decisions and checks
Identify which layered combinations exist beyond theoretical possibility
Export actual filter paths and parameters from the storefront and group them by product subset. Record whether a link in HTML or a script generated each address. A large theoretical space is not proof that every combination is being fetched by search crawlers.
What to verify: A URL-class inventory with real response codes, link origins and distinguishable product assortments for sample combinations.
Decide which attributes merit a lasting destination
Fixture types with distinctive assortment and guidance may deserve a curated category, whereas price-sorted states usually do not. The team records actual buyer purpose before assigning indexation, canonical or robots controls. A disallow and a noindex are not interchangeable mechanisms.
What to verify: An approved class-by-class search-intent map explaining what is indexable, what can be crawled and what remains browsing-only.
Test category and product reachability after the rule change
A technical release should not solve a crawler trap by cutting off product discovery. Run a representative category crawl, inspect pagination and access a few products through real HTML links. Include the non-default store view if it uses different attributes.
What to verify: The after-state crawl confirms the chosen filter policy while retaining accessible product links and truthful category content.
Expected output: The deliverable is a measured URL-class inventory and an implementable layered-navigation policy rather than a blanket ruleset. Each approved category remains useful to shoppers; transient variants should not accidentally acquire the authority of a full product collection. A retained crawl sample allows the change owner to verify what improved technically.
The magento layered navigation crawl policy example is illustrative only and has no observed performance or client-result claim.
Inventory layered filter routes
Take a category with multiple filters and record the actual path or query parameter pattern for each combination. Include sorting, availability, price and product attributes. Measure what the system exposes in internal links before discussing crawl controls; an assumed URL pattern is not a real crawl diagnosis.
Reference: official · 2026-11-07
Differentiate index destinations from navigation
Google explains how faceted navigation can create enormous URL spaces. Document the intention per class: a stable curated category may deserve a useful landing page, while arbitrary combinations can remain shopper controls. A canonical hint does not prevent crawling on its own, and robots disallow and noindex answer different questions.
Reference: official · 2026-11-07
Validate the policy against product discovery
Run the selected rule on representative categories and verify that product links still surface through crawlable routes. Include mobile navigation, product pagination and the storefront's real canonical output. Record exceptions and a release owner before a layered navigation configuration is applied broadly.
Reference: official · 2026-11-07
Decision aid
Decision table: Magento layered navigation: control crawl traps
| Condition | Practical decision | Proof to collect |
|---|---|---|
| Fixture-type filter is a durable curated collection | Consider a distinct landing destination only if it adds specific inventory and buyer guidance. | Check assortment, title, owned search evidence, and stable links. |
| Price-order and stock-only parameter combinations | Treat as transient controls requiring a scoped crawl/index decision. | Verify actual links, canonical output and response under changed inventory. |
| Several filters produce an empty product result | Choose a clear useful response and avoid publishing an empty category as commercial content. | Request empty views and test the fallback navigation and status. |
What a passing test looks like
These checks describe the evidence that should exist after a real implementation. They are not reports of tests already completed on a customer website.
Identify which layered combinations exist beyond theoretical possibility
Expected result: A URL-class inventory with real response codes, link origins and distinguishable product assortments for sample combinations.
Keep as evidence: Archive exact before/after request and rendered HTML evidence with a named reviewer for identify which layered combinations exist beyond theoretical possibility.
Decide which attributes merit a lasting destination
Expected result: An approved class-by-class search-intent map explaining what is indexable, what can be crawled and what remains browsing-only.
Keep as evidence: Archive exact before/after request and rendered HTML evidence with a named reviewer for decide which attributes merit a lasting destination.
Test category and product reachability after the rule change
Expected result: The after-state crawl confirms the chosen filter policy while retaining accessible product links and truthful category content.
Keep as evidence: Archive exact before/after request and rendered HTML evidence with a named reviewer for test category and product reachability after the rule change.
Scope and limitations
The fixture type, catalogue size and filter behaviour are examples only. Adobe Commerce PaaS, Cloud Service and headless storefronts can expose different routes. Neither a theoretical combination count nor a category configuration screenshot establishes actual crawl cost or Google indexing; live HTML, links and source evidence are required.
Sources and verification
These references explain documented technical behaviour. The worked decisions are illustrative until checked on your own installation.
- official reference — Google's faceted-navigation guidance documents the crawl cost of URL combinations and appropriate controls for useless filtered routes. Recorded: 2026-11-07