SEO - IndexNow

General Information

Search engines used to discover updates by crawling. That meant a price change on Friday afternoon might not be reflected on Bing until Monday — by which time the customer who was about to convert had moved on. Microsoft, Yandex and Cloudflare addressed this in 2021 by launching IndexNow, an open protocol where the site notifies the search engine of changed URLs in real time. Bing, Yandex, Naver and Seznam now process every notification within minutes.

The Qoliber SEO IndexNow extension wires Magento's product, category and CMS save events into the IndexNow protocol. Save a product in admin and the URL is pushed to Bing within seconds. Bulk-import 5,000 products from your ERP and every URL is announced. No cron lag, no waiting for the next sitemap regeneration, no third-party services.

What Does This Extension Do?

• Real-time notifications to IndexNow-compatible search engines on every product, category and CMS save event.
• Multi-engine coverage — Bing, Yandex, Naver, Seznam. Use the generic api.indexnow.org endpoint (which forwards to all participating engines) or pin to a specific engine.
• Per-trigger toggles — notify_on_product_save, notify_on_category_save, notify_on_cms_save. Turn each one on or off independently to match your operational pattern.
• Ownership-verification key file served via a built-in controller — no manual file upload to your web root, no nginx config to deploy. Magento serves the proof-of-ownership file at the standard path automatically.
• Best-effort, non-blocking — submissions are sent fire-and-forget with a 5-second timeout. The merchant's save operation never blocks on a slow IndexNow endpoint.
• URL deduplication — repeated saves of the same entity within a request collapse to one outbound notification.
• Automatic URL key extraction — the product/category url_key becomes the canonical path segment in the IndexNow payload.

Why Is This a Game Changer?

• Stock changes that actually matter, when they matter. A product going out of stock on Bing within minutes is the difference between a refunded order and a satisfied customer.
• Bulk imports announce themselves. ERP-driven catalogue imports propagate to Bing at the speed of the import, not at the speed of the next sitemap cron.
• Zero-config ownership proof. The verification key file is served by a Magento route. No SFTP'ing files into your webroot, no nginx aliases.
• Production-safe defaults. Submissions are non-blocking with a 5-second timeout; if IndexNow goes down for an afternoon, your save flow doesn't.

How Does It Work?

Three observers — ProductSave, CategorySave, CmsPageSave — fire when an entity is saved in admin. Each observer:

1. Reads the per-trigger toggle (skip if disabled).
2. Resolves the entity's URL key into a canonical storefront URL.
3. Hands the URL to a Notifier service which dedups within the request.
4. Sends a fire-and-forget POST to the configured IndexNow endpoint with a 5-second timeout.

A separate controller serves the ownership-verification key file at the path the IndexNow operator polls. The key itself is admin-configured.

Configuration at a Glance

Find the module under Stores → Configuration → Qoliber → SEO: IndexNow. Set your IndexNow API key, choose your endpoint (generic or per-engine), and toggle the three event triggers. The key-file route is wired automatically.

What's New in 2.1

First public release of the module. Ships with:

• Observer-based triggers on product, category and CMS page save — each independently toggle-able.
• 16 integration tests across the observer chain and the Notifier (URL dedup, missing-key handling, disabled-module no-op).
• Ownership-key file route served by a custom controller — no filesystem touchpoint required.
• Generic api.indexnow.org endpoint as the default, with admin override to pin to specific engines.

Roadmap

A queue-backed pipeline is planned for 2.2 — replacing the current observer-fires-sync-HTTP pattern with a dedup-aware queue and cron-driven flush. This unlocks ERP-scale stock-sync flows that touch hundreds of products in seconds without per-save HTTP latency. Until then, the synchronous best-effort pattern handles typical merchant edit traffic comfortably.