Use Fetch as Google to Index Pages Faster
Today I want to show you how you can leverage the power of Google's Search Console (formerly Webmaster Tools) to help the pages on your website index faster using the "Fetch as Google" feature.
Search Console is like the command center of your website. Perhaps a more apt analogy would be engineering on a starship like the Enterprise. Search Console is where you'll diagnose most issues, learn what Google has to say about your site, and find unique information regarding how your website is crawled.
How to use the Fetch as Google tool
1. Log in the Search Console and verify your website
If you have not verified your website, it's really quite simple. There are even plugins like Yoast SEO that will help you verify your site quickly. Google has also created a very helpful section of Search Console support to help you claim your website.
2. Navigate on the left hand side to the "Crawl" section and expand the navigation
3. Click on "Fetch as Google"
This option will open up the Fetch as Google tool.
4. Enter your URL into the tool or leave the field blank to fetch your homepage
When submitting a URL, all you want to include if everything after the forward slash "/" that follows your .com, .net, .org, etc.
For example, to submit this blog post I would enter in (without quotes) "fetch-as-google/" which would end up fetching: "https://marketerdoug.com/fetch-as-google/"
5. Submit your fetched page to Google's index
Depending on how busy Google's bots are and how large your website/page are to render, you will soon be granted the option to submit that page to Google's index.
Simply click the "Request Indexing" button.
6. Prove you're not a robot and choose your submission type
You are allowed up to 500 fetches per week according to Google Support.
"Crawl only this URL" vs "Crawl this URL and its direct links"
This option is fairly straight forward. The "Crawl only this URL" will tell Google to only look at this page when submitting to the index. The "Crawl this URL and its direct links" option will
Fetch vs. Fetch and Render
You may have noticed the option to "fetch" and the option to "fetch and render." These are similar and merely there for your benefit.
Fetch will simply crawl the page without simulating any of the resources on the page. In other words, this is going to crawl the simple text of your page and report back any security or network issues.
Fetch and Render
The Fetch and Render option will trigger resources on the page so that Google's bot shows you how the bot sees the page. This can be useful in identifying whether or not your page is rendering different for bots than it is a user.
Request Fetch Status Codes
The Fetch as Google tool will store your last 100 fetches for you to review. These fetches come with status codes indicating any issues that may have arisen from the fetch.
These codes are explained when you click on them and will provide more details to diagnose any problems.
Complete - The fetch was successful and all resources on the patch could be reached. This is your ideal result.
Partial - Some of the resources on the page would not be reached. As you can see in the image below, sometimes these resources are not on your server and the severity of the issue will be noted by Google.
Redirected - This usually means you entered a URL that was redirected in some way. The usual cause is forgetting your trailing forward slash at the end of your URL, but can sometimes be indicated by a page redirect as well. Whenever possible, make sure your URL is not redirecting.
Unreachable - This is not a valid URL, or your site/page is having some issues. Note any problem Google indicated in the status screen and fix the problem before trying again.
When to Use the Fetch as Google Tool
I always use the Fetch as Google tool whenever I make major revisions to a post or page. I will also use Fetch as Google when I write a new post (like this one) and wish to see it appear in Google's index quicker than waiting for a natural crawl.
The Fetch as Google tool does not give you any direct benefit other than a quicker index. In other words, you won't rank higher because you used a Fetch. However, this can be viewed as an indirect benefit in a niche where information is timely and you want to rank faster.
Now get out there and play fetch with Google!