Having your website indexed in Google is the most important feature of Search Engine Optimization. In order for your site to get any sort of organic traffic, it must be indexed with the almighty Google. If not, it will be impossible for anyone to find your content organically, as it will not be part of the Google search index. In order to diagnose and rectify this issue, there are a few things users can do.
Some things a user can do include:
1.) Make sure your website is indexed though a www domain or a non-WWW domain. As WWW is my technicality a sub-domain, it may be necessary to set a preferred domain, making sure to verify ownership of both, of course.
2.) If your site is newer, it may not have been discovered by Google quite yet. However, this can be solved by making sure your sitemap has been loaded, then request a Google crawl, and that it “fetch your site.” This can simply be done through Webmaster tools.
3.) Your site may be blocked by robots.txt – which is a simple fix by removing the entry from the robots.txt. This will enable your site to reappear in the index.
4.) You may be missing a sitemap.xml all together. This is an essentially a specific list of directions Google should follow when indexing your website. For those experiencing index problems, it is advised to look over your sitemap.xml and resubmit in Webmaster tools.
5.) Often times, Google will fail to index pages because it is unable to crawl them; however, even though these pages may not be crawled, Google can still see the pages. To identify these crawl errors, we can go to Webmaster tools and click on “Crawl errors.” Here you can correct the issue and enable your pages to be crawled.
6.) Duplicate content is a huge issue with Google today. This is because Google will see duplicate content on multiple URLs, and won’t know which one to index; therefore, not indexing any at all. This can be fixed by choosing one of the pages, then submitting a 301 re-direct for the rest of the duplicate pages.
7.) While WordPress is wonderful, we must double check our privacy settings, as they may be on without your knowledge. This can be found through Admin, Settings, and Privacy.
8.) Check, and make sure, your site does not have a NO INDEX in the Meta tag. This will prevent robots from crawling your site, and as it says, Google will not index your page, or entire site.
9.) The website may be blocked by an .htaccess - which is a piece of your websites existence on the server. This feature allows your website to be available to users around the world through the internet. While useful, I can also be used to blow Google from crawling your site, again not indexing it.
10.)If your website takes a long time to load, it may also prevent your site from being indexed. While Google may make multiple attempts to access the page, it will not index the page if it cannot access it, eventually skipping the page entirely.
While there are a number of other reasons your website may not be indexed by Google, these are a few of the most common reasons, and solutions to getting your site crawled and indexed by Google. If you require further assistance with this issue, our team at THAT Agency is here to help!