News, Views and More...
Get the latest digital marketing news, views and
more direct to your inbox
Nov 18th, 2017
Almost every web developer in the world has to deal with Google’s crawl errors, and it’s better to remedy problems quickly to prevent being penalised by Google’s ranking algorithm. Crawl errors let Google know that certain elements of your website aren’t working as they should, such as broken links, and, fortunately, most issues are easier to resolve than you might expect.
Some problems, such as DNS and server errors, usually require professional assistance, though there are some things you can try first to avoid calling your web developer. However, most common problems can be fixed by just about any amateur in a matter of minutes.
If you’re trying to master the skills of a professional web developer, you’ll benefit from learning how to deal with common problems without help. First, you’ll need to familiarise yourself with Google Search Console, previously called Google Webmaster Tools, as this is usually the key to eliminating the vast majority of crawl errors.
What is Google Search Console?
Google Search Console gives you insights into your website’s performance and highlights problematic areas that could affect your search ranking, and it’s also the platform that’s most widely used to identify crawl errors. Even though Google Search Console is intuitive and easy-to-use for most amateur developers, professional developers and SEO companies also use it almost constantly.
For this article, I’ll be referring to the Crawl Errors view within Google’s Search console, which is split into two sections: URL errors and site errors. Site errors may be complex issues that require a professional’s healing touch, but URL errors are usually relatively straightforward to fix.
Before we start talking about how to refine your website maintenance skills, let’s take a look at the most common crawl errors you’re likely to come across as you optimise your website for the search engines.
List of Common Crawl Errors
How to Eliminate URL Crawl Errors
If you’ve never checked your website for crawl errors, you’ll likely be shocked to find hundreds listed in the Google Search Console. However, although it may sound counterproductive, you need to start the process of eliminating the crawl errors by marking them all as fixed before getting to work. That way, you can keep checking for crawl errors weekly and fix them in a structured way. Plus, with any luck, the irrelevant errors won’t crop up again.
Fortunately, the most common error status code is the easiest to fix, usually only requiring a few minutes of your time. The Googlebot reports a 404 error when it detects faulty or broken links, whether there’s an issue with a link to your URL on somebody else’s website or there are broken internal links. Here’s how you fix it:
Remember to check your Google Search Console weekly to see whether or not the errors have been dealt with for good.
The Googlebot may return a 404 error code if it finds an empty page that it assumes is a faulty link, which means although the link led to an active URL, there was no or very little content on the page. Yet again, this issue is easily fixed by setting up 301 redirects, provided you no longer want the faulty link to lead to the URL it currently takes users to. However, if the “soft” 404 page is your target URL, you might have to add some valuable content to the page to make the Googlebot ignore it the next time it goes crawling through your website.
Unfortunately, other status codes, such as 400, 500 and 503, are not usually as straightforward to eliminate as the more common 404 errors, but it’s still vital to deal with them as quickly as possible so that Google knows your website is in good health. You can read Google’s advice to try and get rid of these errors without help, and if that fails, you might have to call your web developer.
How to Deal with Site Errors
Site errors can be difficult to eliminate, meaning your only option might be to get in touch with your web developer or hosting platform, but it’s vital to deal with site errors quickly because they prevent the Googlebot from accessing any web page on your entire site. When you check the Google Search Console, you’ll see a list of all site errors reported in the previous 90 days.
Site errors can be broken down into three categories:
Server issues are among the most serious of crawl errors because if they persist, Google will think there are problems with your entire site and consequently penalise you in its rankings. If you have any DNS, server or website errors, it’s probably best to call your web developer to remedy the issue promptly. However, you can find out how to eliminate site crawl errors if you’d rather put your web DIY skills to the test.
You’ll undoubtedly feel frustrated the first time you attempt to deal with crawl errors without professional help, but with time, practice and patience, the task becomes much simpler. Keep these tips in mind for when things become extra challenging:
Contact Smart Traffic for More Guidance on Crawl Errors
Here at Smart Traffic, we know that fixing crawl errors can be both challenging and annoying, especially if you have hundreds of errors that need dealing with sooner rather than later. Fortunately, when you hire our SEO specialists, you don’t have to worry about the prospect of crawl errors affecting your search rankings. Hopefully, the information above should help you get rid of most problems without help, but if you ever need assistance, we’re only a phone call away. Contact us today on 02 8205 3133 to speak with a leading SEO company in Sydney, Australia.
Simply call 02 8205 3133 or complete your details below and a Smart Traffic consultant will be in touch shortly.
Thank you for your enquiry!
We will be in touch very shortly