IndexNow submission

Search engines no longer automatically crawl websites as frequently as they used to. Nowadays, it's important to proactively notify them when new or updated content is available. You can do this by submitting URLs in a specific format with IndexNow.

Submitting URLs to IndexNow

For reference when submitting an url: 

https://<searchengine>/indexnow?url=https://www.example.com/product.html&key=<your host key>

Posting multiple url:s with curl through terminal: 

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  <search engine URL with /indexnow>

Return codes when submitting with curl

HTTP Code                Reasons
200 OK                   URL submitted successfully
202 Accepted             URL received. IndexNow key validation pending.
400 Bad request          Invalid format
403 Forbidden            In case of key not valid (e.g. key not found,file found but key not in the file)
422 Unprocessable Entity In case of URLs which don’t belong to the host or the key is not matching the schema in the protocol
429 Too Many Requests    Too Many Requests (potential Spam)

Host key

IndexNow requires a host key in order to be able to verify the submission.

In Cradle CMS the host key is the site-id (found on the Site settings page in admin) without hyphens.

For example if the site id is 17327f63-e498-4754-bcaa-d3f20efac2f8 the host key is 17327f63e4984754bcaad3f20efac2f8 and is located at https://yourdomain.name.com/17327f63e4984754bcaad3f20efac2f8.txt

Supported search engines

Currently supported search engines:

IndexNow 

https://api.indexnow.org/indexnow?url=url-changed&key=your-key

Multiple URLs

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://api.indexnow.org/indexnow

Microsoft Bing 

https://www.bing.com/indexnow?url=url-changed&key=your-key

Multiple URLs

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://www.bing.com/indexnow

Naver 

https://searchadvisor.naver.com/indexnow?url=url-changed&key=your-key

Multiple urls

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://searchadvisor.naver.com/indexnow

Seznam.cz 

https://search.seznam.cz/indexnow?url=url-changed&key=your-key

Multiple URLs

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://search.seznam.cz/indexnow

Yandex 

https://yandex.com/indexnow?url=url-changed&key=your-key

Multiple URLs

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://yandex.com/indexnow

Yep 

https://indexnow.yep.com/indexnow?url=url-changed&key=your-key

Multiple URLs

curl -v --header "Content-Type: application/json" \
  --request POST \
  --data '{"host": "www.example.com","key": "<your host key>","urlList": ["https://www.example.com/url1","https://www.example.com/folder/url2","https://www.example.com/url3"]}' \
  https://indexnow.yep.com/indexnow

References