hexo-submit-urls-to-search-engine Documentation

Preface

After using the hexo-submit-urls-to-search-engine plugin for Hexo, you can actively push new links from your Hexo blog to Google, Bing, and Baidu search engine webmaster platforms to improve website indexing quality and speed. This plugin allows you to send all-natural, organically-fed indexing requests to major search engines, once and for all.

For example, one of my articles can now sometimes be found on Bing within 5 minutes of being published (this is the best-case scenario, and long-term stability is not guaranteed). See, that’s the power of hexo-submit-urls-to-search-engine. Baidu and Google are slightly slower.

These three major search engines already account for 97% of the global search engine market share (including meta-search engines like Dogpile, Duckduckgo, etc.; and Yahoo and Ecosia, which use the Bing index).

Warning

Non-professional use of other similar push codes may lead to dangerous side effects, including but not limited to: security flaw syndrome, redundant code syndrome, reinventing the wheel syndrome, documentation-gnawing syndrome, existential doubt, depression, headaches, and even death, soul scattering, and annihilation.

Github address

The official QQ group and Telegram group have been discontinued. Please go to the Github issue page to participate in discussions.

Operation steps

Quick Start:

Obtain the keys for each webmaster platform
Install and configure the hexo-submit-urls-to-search-engine plugin
hexo clean && hexo g && hexo d, and check the push results
If the push is successful, go to the Github address and click the Star button to support it
Watch the Github project to get plugin upgrade reminders
You can also click here to donate

Obtain webmaster platform keys

The keys “noted” in this section should all be filled into hexo’s _config.yml, and the format is given later.

Regarding website verification: Some Hexo themes support adding webmaster platform HTML verification tags. After filling in the relevant settings, hexo g && hexo d can be used for verification. For general detailed operation steps, please Google: hexo webmaster platform verification.

Baidu

Baidu is a Chinese search engine. Unless complicated website filing procedures are completed with the Chinese authorities, it is difficult for a website to be indexed.

If your website server is located outside of China, it is basically impossible to complete website filing procedures.

Please see the old documentation to learn how to submit links to Baidu

Bing

The Bing Webmaster Tools platform has new and old versions.

Register and log in to the new Bing Webmaster Tools platform Bing Webmaster Tools
Add website
Enter the website management page, Settings, API access, API key, and note down the API key

Get the API key from the new Bing Webmaster Tools platform

~~(Recently, the Bing Webmaster Tools platform has been upgraded, and the server and layout are unstable. If the token cannot be displayed, you can try again after a while)~~ Now it has been upgraded to the new Webmaster Tools platform.

This is a schematic diagram of obtaining it from the old Bing Webmaster Tools platform:

Get the Bing Webmaster API key from the old Bing Webmaster Tools platform

Google

If your server is located in China, please note that your server needs to be able to access the international Internet to be pushed to Google.

The Google Webmaster Tools platform also has new and old versions, but the relationship between the two sites is not as well handled as Bing. In addition, the Google Indexing API we use is not provided by the Webmaster Tools platform, but belongs to the Google Developers platform. During the process, you can refer to the official documentation (accessible in mainland China) for configuration.

Steps for use:

Open official documentation, and follow the prompts to create a project and create a service account. You only need to perform configuration operations on the website as prompted. You do not need to complete the subsequent operations of obtaining access tokens, etc.
You will still be prompted with “no permission” if you only add the permission email address in Google’s new “Search Console” according to the official Google documentation. Please go to the old Webmaster Tools, click on your website, and add the email address to it. Now you can submit it directly on the new Webmaster Tools platform.
Place the obtained JSON key file in the hexo root directory (same location as the hexo _config.yml file)

Old Google Webmaster Tools:

In addition, there is the following prompt in its official documentation. Please decide whether to enable the function of submitting links to Google according to the actual situation:

Currently, the Indexing API can only be used to crawl web pages containing JobPosting or BroadcastEvent (nested in VideoObject). For websites containing many short-lived pages (such as recruitment information or live videos), the Indexing API will keep the content in the search results up to date by pushing updates for different content separately.

Indexing API Quick Start | Google Search Central | Google for Developers

Anyway, I submitted it. If Google doesn’t crawl your website, how would it know if your webpage doesn’t contain recruitment information or live videos? Since it did crawl it, even if it didn’t find recruitment information or live videos, what’s the reason for wasting resources and not using the crawled information? Since it’s being used, our goal is achieved. In actual use, Google doesn’t seem to have demoted the ranking for this. Maybe Google forgot to change the documentation? I have currently found two serious errors in this document that have not been corrected for many years.

Configure hexo

Install this plugin

Please run in the hexo root directory:

Of course, using cnpm in the Greater China region is faster

npm install --save hexo-submit-urls-to-search-engine

This plugin also supports yarn installation:

yarn add hexo-submit-urls-to-search-engine

Edit hexo’s _config.yml

hexo-submit-urls-to-search-engine

Configure the hexo-submit-urls-to-search-engine plugin, and paste the following into hexo’s _config.yml.

Hint

You can of course use environment variables to set the record keys, so that even if the source code is placed in a public repository, the keys will not be leaked

hexo_submit_urls_to_search_engine:
  submit_condition: count #The condition for submitting links, optional values: count | period Only count is currently supported
  count: 10 # Submit the latest 10 links
  period: 900 # Submit links with modification times within 900 seconds
  google: 0 # Whether to submit to Google, optional values: 1 | 0 (0: No; 1: Yes)
  bing: 1 # Whether to submit to Bing, optional values: 1 | 0 (0: No; 1: Yes)
  baidu: 1 # Whether to submit to Baidu, optional values: 1 | 0 (0: No; 1: Yes)
  txt_path: submit_urls.txt ## The name of the text document, the links that need to be pushed will be saved in this text document
  baidu_host: https://cjh0613.github.io ## The domain name registered in the Baidu Webmaster Tools platform
  baidu_token: Please obtain according to the documentation instructions ## Please note that this is your key, so please do not publish it directly in a public repository!
  bing_host: https://cjh0613.github.io ## The domain name registered in the Bing Webmaster Tools platform
  bing_enable_indexnow: false # Whether to use indexNow to submit links to Bing: true (Yes) | false (No). This function can only be enabled in version 2.1.1 and later.
  bing_token: Please obtain according to the documentation instructions ## Please note that this is your key, so please do not publish it directly in a public repository!
  google_host: https://cjh0613.github.io ## The domain name registered in the Google Webmaster Tools platform
  google_key_file: Project.json #The JSON file for storing the Google key, place it in the website root directory (same location as the hexo _config.yml file), please do not publish the contents of the json file directly in a public repository!
  google_proxy: http://127.0.0.1:8080 # The system http proxy used to submit URLs to Google, fill in 0 to not use
  replace: 0  # Whether to replace part of the string in the link, optional values: 1 | 0 (0: No; 1: Yes)
  find_what: http://cjh0613.github.io/blog
  replace_with: https://cjh0613.com

Consequences of exposing the key

Your “nemesis”, sentient artificial intelligence, or terrorists may use your key to push non-existent page links to search engines. This would both consume your quota so you cannot push normally, and cause the website’s weight to decrease due to dead links being pushed.

Comments

submit_condition: Supports two ways of judging whether to submit a link, namely count | period. The corresponding count | period configuration items below are applied respectively
period: A period of time, in seconds. If the difference between the article update time and the plugin runtime is less than this value, its link will be submitted. For example: there are two files a.html and b.html, corresponding to https://yoursite.com/a.html and https://yoursite.com/b.html respectively. The differences between their update times and the plugin runtime are 800s and 1000s respectively (the period field is set to 900), then https://yoursite.com/a.html will be submitted, and https://yoursite.com/b.html will not.
google_proxy: The system http proxy used to submit URLs to Google, format: http://proxyhost:port, modify proxyhost and port; If Hexo is deployed overseas, you can fill in 0 to not use. If you use a scientific internet access software, proxyhost is generally 127.0.0.1, and port can be viewed and set in the software, generally 8080. Win10 can be viewed in Settings->Proxy
replace: Some links may be generated incorrectly (such as Chinese domain names, the homepage is located at http://cjh0613.github.io/blog, etc.). This is a simple and crude replacement function that replaces the specified string in all generated links. The example replaces http://cjh0613.github.io/blog with https://cjh0613.com. You can also replace Chinese domain names with the converted domain names.

July 12, 2020 Update to this section, thanks to liuyib for the suggestion May 12, 2021 Added google_proxy

After completing the above configuration, when you run the hexo generate command in the Hexo root directory, a .txt file will be generated to store the links to be pushed.

You can open this file to see if the links are correct. If they are not correct, please visit hexo’s _config.yml to configure it. You can also manually modify this .txt file before pushing links.

deploy

If you have not added a deploy: configuration item before, paste the following directly into hexo’s _config.yml to overwrite the default deploy: configuration item.

deploy:
- type: cjh_google_url_submitter
- type: cjh_bing_url_submitter
- type: cjh_baidu_url_submitter

After completing this step, you can automatically push links to search engines when you run the hexo deploy command.

You must run the hexo deploy or hexo d command to trigger the push, regardless of whether you have used this command for deployment before.

Of course, you need to run hexo generate before to generate the latest .txt file to store the links to be pushed.

If the push is successful, please go to the Github repository and give it a Star to support us, thank you!

If you have already set the deploy: configuration item, directly pasting the above code block will prompt configuration conflict. You can directly add it under the existing deploy: configuration item. such as:

deploy:
- type: git
  repo: 
    coding: git@xxx
  branch: master 
  
  #Add the configuration items of this plugin:
- type: cjh_google_url_submitter
- type: cjh_bing_url_submitter
- type: cjh_baidu_url_submitter

Note: There is a - before each type: above

Continuous Integration Support

This section provides a reference for deployment methods where hexo is stored in a public repository and compiled and published using CI (Continuous Integration, such as Github action, travis, coding-ci, netlify, etc.).

Baidu and Bing can use environment variables to record keys. Google has other solutions, see below

Please note that if you use CI for automatic deployment, you must add updated: to the Front-matter of each page (.md file at the top area separated by ---, used to specify variables for individual files) to inform Hexo to correct the “last modified time of the article”. For example:

---
title: Hello World
date: 2013/7/13 20:46:25
updated: 2020-08-25 9:36:00
---

This is because, when the file is placed on the local computer, if there is no updated:, Hexo will read the file properties and use the “last modified time of the file” as the “last modified time of the article”, which can be used to correctly judge and submit the latest 10 links; and for CI automatic deployment, since the Hexo resources are all cloned to the server, if updated: is not added, Hexo will also read the file properties, but at this time the last modified time in the properties is the completely identical git clone time. Hexo will get an incorrect “last modified time of the article” and will not be able to correctly judge and submit the latest links.

First fill in 0 at the configuration items baidu_token: and bing_token: in hexo_submit_urls_to_search_engine:
Set the environment variables BAIDU_TOKEN, BING_TOKEN, and the value is the obtained token
If you use Github Action, you also need to set environment variables in the yml file. There is an example below
hexo clean && hexo generate && hexo deploy and enjoy life

Reference:

hexo_submit_urls_to_search_engine:
  submit_condition: count #The condition for submitting links, optional values: count | period Only count is currently supported
  count: 10 # Submit the latest 10 links
  period: 900 # Submit links with modification times within 900 seconds
  google: 1 # Whether to submit to Google, optional values: 1 | 0 (0: No; 1: Yes)
  bing: 1 # Whether to submit to Bing, optional values: 1 | 0 (0: No; 1: Yes)
  baidu: 1 # Whether to submit to Baidu, optional values: 1 | 0 (0: No; 1: Yes)
  txt_path: submit_urls.txt ## The name of the text document, the links that need to be pushed will be saved in this text document
  baidu_host: https://en.cjh0613.com ## The domain name registered in the Baidu Webmaster Tools platform
  baidu_token: 0 ## Please note that this is your key, so please do not publish it directly in a public repository!
  bing_host: https://en.cjh0613.com ## The domain name registered in the Bing Webmaster Tools platform
  bing_token: 0 ## Please note that this is your key, so please do not publish it directly in a public repository!
  google_host: https://en.cjh0613.com ## The domain name registered in the Google Webmaster Tools platform
  google_key_file: Project.json #The JSON file for storing the Google key, place it in the website root directory (same location as the hexo _config.yml file), please do not publish the contents of the json file directly in a public repository!
  google_proxy: 0 # The system http proxy used to submit URLs to Google, fill in 0 to not use
  replace: 0  # Whether to replace part of the string in the link, optional values: 1 | 0 (0: No; 1: Yes)
  find_what: http://cjh0613.github.io/blog
  replace_with: https://cjh0613.com

Github Action, you also need to set environment variables in the yml file:

- name: xxxx #Fill in anything
  env:
    BAIDU_TOKEN: ${{ secrets.BAIDU_TOKEN }} # Note here, the others are irrelevant
    BING_TOKEN: ${{ secrets.BING_TOKEN }} # Note here, the others are irrelevant
  run: |
    npm i -g hexo-cli # Install hexo
    npm i
    ……
    hexo deploy

As for pushing links to Google, there is so much information in the json key file, and copying and pasting to set environment variables is too troublesome. I put the json key file into a private repository, then clone it, and copy it to the root directory to solve it.

Plugin Upgrade

Plugin upgrade reminder

After you install this plugin, please watch this project from Github to get plugin upgrade prompts (select Releases only to only get upgrade prompts, if you select Watching, Issues, Pull requests, etc. will also be pushed, sent via email)

Upgrade Operation

Please run in the hexo root directory:

Of course, using cnpm in the Greater China region is faster

npm install --save hexo-submit-urls-to-search-engine@version

I previously gave another command: npm update --save hexo-submit-urls-to-search-engine, but I encountered problems during my actual testing.

yarn upgrade:

yarn add hexo-submit-urls-to-search-engine@version

Incompatible Upgrade

Warning

Sometimes, plugin upgrades are not backward compatible and may require you to modify the configuration items related to this plugin in hexo’s _config.yml, or make other modifications. This situation generally occurs when the number before the first decimal point in the version number increases. For example, ^1.0.0 is upgraded to ^2.0.0.

Upgrade from 1.x.x to 2.x.x:

You need to add the following three configuration items to the hexo_submit_urls_to_search_engine: configuration item:

  replace: 0  # Whether to replace part of the string in the link, optional values: 1 | 0 (0: No; 1: Yes)
  find_what: http://cjh0613.github.io/blog
  replace_with: https://cjh0613.com

Well, I found that this upgrade is backward compatible, and it won’t report an error even if there are no these three configuration items…

Return Information

Successful push information

Bing

Bing response:  { d: null }

Google

Google response:  { urlNotificationMetadata:
   { url:
      'https://cjh0613.github.io',
     latestUpdate:
      { url:
         'https://cjh0613.github.io',
        type: 'URL_UPDATED',
        notifyTime: '2020-06-12T05:37:25.701779228Z' } } }

Other

You can refer to their official documentation for other returns, but generally you will know the reason by translating the return.

Sometimes it is because search engines only allow a certain number of links to be submitted per day. You can log in to their official websites to query. After my site domain name cjh0613.github.io was changed to cjh0613.com, the Bing quota was 10, and after 10 consecutive days of pushing, the quota suddenly changed from 10 to 10000.

If some links fail to be pushed, please handle them manually: modify the txt that saves the submitted links according to the feedback (generally remove the successfully pushed links), and then only run hexo deploy to push again without regenerating the website.

If you want to push links to search engines every day, just run hexo d every day. You can also set Github Action to run hexo g && hexo d every day without reinventing the wheel.

Query Results

Query Index

Taking querying the index of this site as an example, open any search engine and enter site:domain, here is:

site:cjh0613.github.io

Search.

The time it takes for a new site’s index volume to go from 0→1 may be very long, maybe a month… please be patient

The actual inclusion speed is also related to other factors. There is a lot of SEO information online, but it is all centered on the official instructions.

Query Push Record

Bing

You can log in to the new Bing Webmaster Tools platform to find a detailed list of pushed links (updated in real time):

https://www.bing.com/webmasters/submiturl?siteUrl=Your website link

Bing push record and remaining quota query

Google

You can only find the cumulative number of pushed links and the percentage of failed pushes over a period of time (up to about 30 days) in the “Errors” table at the following address (updated in real time). Although a chart of daily submitted data is given, it is equivalent to not giving it…

https://console.developers.google.com/apis/api/indexing.googleapis.com/metrics?project=

Reference materials for my development of this plugin

Please see the old documentation

Preface#

Operation steps#

Obtain webmaster platform keys#

Baidu#

Bing#

Google#

Configure hexo#

Install this plugin#

Edit hexo’s _config.yml#

hexo-submit-urls-to-search-engine#

deploy#

Continuous Integration Support#

Plugin Upgrade#

Plugin upgrade reminder#

Upgrade Operation#

Incompatible Upgrade#

Return Information#

Successful push information#

Bing#

Google#

Other#

Query Results#

Query Index#

Query Push Record#

Bing#

Google#

Reference materials for my development of this plugin#

Preface

Operation steps

Obtain webmaster platform keys

Baidu

Bing

Google

Configure hexo

Install this plugin

Edit hexo’s _config.yml

hexo-submit-urls-to-search-engine

deploy

Continuous Integration Support

Plugin Upgrade

Plugin upgrade reminder

Upgrade Operation

Incompatible Upgrade

Return Information

Successful push information

Bing

Google

Other

Query Results

Query Index

Query Push Record

Bing

Google

Reference materials for my development of this plugin