It is common to have stale or outdated pages on the website. Removing these web pages or content can be routine maintenance, but it leaves a gap on your site, requiring you to inform Google and other search engines to know if the content is really removed, or it has just moved to another page. Many of us already know that a 404 error is displayed when the page is not found. But it sometimes shows Soft 404 Error that is not good for your website.
That’s why it is important to learn the difference between regular or Hard 404 error and Soft 404 error.
SO, WHAT IS A REGULAR (HARD) 404 ERROR?
When someone tries to open a page on your website that has been removed, your site generally responded by showing a generic “Page Not Found” message and by showing 404 that indicates that the page doesn’t exist anymore.
404 errors also occur when the visitor types in a wrong web address.
While the visitor might be satisfied with such messages, the crawlers from Google and other search engines look for the code returned from your server to determine if there is a page content to be indexed.
How to Deal with 404 Errors
The first step is to look for the URLs present on the website, which are showing the 404 error. This can be done with the help of two tools—Google Search Console’s Coverage report and using other tools like Screaming Frog.
Once you get a list of such URLs, try to find out why the page is showing an error. Maybe there is an error in the URL or the page was removed/taken down. This is a crucial step as it will give you an idea of the issues to be resolved. Here are some common things to address to fix 404 errors.
Fixing the Incorrect URL:
Sometimes, your web pages are linking to an incorrect URL. But identifying such broken links across the website is a stressful job. That’s why you can use tools like Screaming Frog or DeepCrawl to find identify broken links. Once you get the list, you can fix the linking errors easily.
Fixing the Linking Errors:
The 404 error often results from the pages being removed from the website. They can be fixed by….
- Restoring the Pages
If a page was removed by mistake, you should restore the page and submit it in Google Search Console to get it re-indexed. Or, the sitemap could be updated. Once this is done, validate the 404 URLs in the Search Console.
- Redirecting to the Most Relevant Page
If the 404 error pages are outdated, then you should redirect them to the relevant page on the website. For example, you run an e-commerce site. If a product is not available any longer and the page has been removed, then it should be redirected to the category page of the product.
For example, the 404 product URL (www.abc.com/category/XYZ item) can be redirected to the category URL (www.abc.com/category/), but not to the main or home page.
WHAT IS A SOFT 404 ERROR?
Your website server should display a 404 error code for a removed page, right?
But sometimes it can show that everything is okay by showing a wrong code like 200 OK Code. But a search engine crawler expects your website to send a 404 code not a 200-ok status code for a removed page.
Your website is telling the crawlers that the URL or page is fine and not broken or missing when a soft 404 error happens. This way, it does nothing but confuses the crawlers.
Even if the website shows an error page to the reader, it is still required to return the right error code to signal the search crawlers to ignore the page. Or, you have removed an old page and you redirected it to a new page that is not relevant to what the original content was on the deleted page.
When your website server doesn’t send the 404 code, the crawlers get a message that the content is on the page, and spend time to index that content.
As a result, Google bots crawl those totally useless pages. And it will just eat up your crawl budget that could be spent better elsewhere. Above all, it will kill user experience and minimize the visibility of your important pages. Such mischaracterization of your deleted pages by a search engine can impact your SEO and site reputation as well.
Soft 404 errors happen because of poor server configuration, pages with very less or no content, and issues with page rendering.
How to Fix Soft 404 Errors?
First of all, extract all the URLs from the Soft 404 section in Search Console’s coverage report. Identify the URLs being labeled the 404. Fix these URLs using the tips mentioned above. Now, you can move to fix the soft 404’s with the steps given below:
Check the Correct Status Code for All URLs:
Make sure that the servers serve the correct status code for all URLs. A valid page should have a status code 200, a missing page should show 404 and redirected pages should be 301 or 302.
Address the Pages with Duplicate or Thin Content:
You can use the Screaming Frog tool to run the soft 404 URLs to find the word count of the content present on each page. This will let you know if the pages have thin content.
Also, use the Near Duplicate or Exact Duplicate feature to check the duplicity of the content.
Apart from this, check the technical issues that cause duplication, such as non-trailing or do trailing slash URLs, www or non-www version of the site URLs, HTTP or HTTPS version of URLs, URLs with or without “.html”. Also, make sure to check if the canonicals are defined for duplicate URLs.
Check If Google can Render Your Pages:
If the pages are still labeled soft 404 despite having sufficient content, then there might be a possibility that the Google bots don’t crawl or render your pages efficiently.
To address such URLs, check the rendered screenshot and HTML in Search Console. If the screenshot is blank or nearly blank, then the pages have a rendering issue. Analyze the rendered HTML to find which resources are leading to such problems. Don’t block these issues for crawlers and make sure that they are not overly large.
Conclusion:
A 404 error and a soft 404 error generally occur for the webpage that doesn’t exist. The only difference is that a 404 error tells search engine crawlers that the page is not there anymore a soft 404 error doesn’t. Rather, a soft 404 error will confuse the search engine and make it crawl the meaningless pages if not fixed.
That’s why it is important to check your site for both a 404 error and a soft 404 error. Otherwise, they can impact the ranking of your website and user experience as well if they grow over time.
Have you ever experienced such problems? How did you remove them? Share with me in the comment box given below!