Under normal circumstances, you would want search engines like Google, Bing and Yahoo to list your site in their search results. In fact, most individual website owners and businesses devote so much time and resources to making sure their website is SEO (Search Engine Optimization) optimized.
However, there are some cases where you would like to keep search engine crawlers away from your website. For example, you typically wouldn’t want search engines under the following circumstances:
- When a site is still under development. At this point, the site is not ready for use by the general public
- When the site is meant to be private and accessible only to select people, such as your family or work colleagues.
In this article, I’ll walk you through the processes to prevent search engines from finding and indexing your WordPress website.
Turn off the visibility of search engines in the admin panel
The easiest way to prevent search engines from indexing your site is to disable “search engine visibility” in your WordPress admin dashboard. To do this, follow the instructions below:
- Login to your WordPress account.
- In the sidebar, go to Settings > Reading
- Scroll down to Visibility on search engines.
- Once there, select the option that says, Discourage search engines from indexing this site.
- Click on Save your changes when done.
Note that this method is unreliable – it’s up to the search engines to honor your request.
To forcibly prevent search engines from finding your website, you’ll need to protect it with a password. We will discuss what this means and how to do it in the following section.
Protect your entire website with a password
When you password protect a WordPress website, anyone who wants to access the site will first be prompted to provide the correct password. As a result, search engine bots will be prevented from accessing the site to crawl it.
There are several methods to password protect your site. If the WordPress site in question is on a hosting provider, such as Bluehost, you can enable a password within the cPanel of your hosting account.
While the interface will be different for different providers, the process should be similar between them. Basically, you’ll need to search for the “password protection” directory, choose the website you want to protect (if you have multiple websites), and then create the password for the entire site.
Contact your hosting provider if you need more information on the process.
Use a free plugin
You can use a plugin to password protect your entire website and thereby prevent search engine crawlers from finding it. While there are many plugins to do this, I’m going to walk you through how to use a free plugin called Password Protect.
In the sidebar:
- Navigate to Plugins > Add New
- Use the search bar in the upper right corner of the page to search for “Password protected“.
- Click on Install nowthen click Activate.
At this point the plugin is active.
To enable password protection and create your own password, log in to Settings > Password protected and fill out the form accordingly.
Tick the Qualified option (next to Password protected status) to enable password protection and the Allow administrators option (next to Permissions protected) to allow administrators access to the site. (You can also allow logged in users, RSS feeds, and the REST API to access if you like.)
After that, set the password for the site and confirm it. If you want to allow specific users to access the site, add their IP addresses to the file Allow IP address box.
Finally, tick the Allow Remember me option (this ensures that next login does not prompt for the password) and add the number of days you want to remember an authenticated user.
When you’re done, click Save your changes to activate password protection on the site. From then on, anyone who visits your site will be required to provide their password, and search engine crawlers will not be able to access the website.
Use a paid plugin to include advanced features
Free plugins like the one we discussed in the last section allow you to password protect your site, no question. But if you are looking for more advanced features, which I will list soon, you can choose a paid plugin.
For example, there is a plugin called SeedProd. In addition to password protection, this plugin offers several descriptive splash screens that you can display on your site to inform the user that maintenance is in progress.
The plugin also includes “Coming Soon” themes which, when used on the site, will require the user to subscribe and be notified when the site goes live (or some other update occurs) via email.
Conclusion
In this article we have seen how to prevent search engines from showing our site in search engine results. This is useful if you want only certain people to access your site rather than the general public.
Now that you know how to prevent search engines from indexing your site should the need arise, check out some of our other WordPress articles.
