A Fan of CMS platform like you and me always keep an eye on new development updates, features added or removed and security level optimization.
WordPress is one of the most popular website solution and cost effective CMS portal which is has a lot of privileges like:-
– Easiest use.
– Less Code experience requirement.
– Large developer community.
– Large market place including a massive number of free plugins.
WordPress 5.3 which scheduled to finally launch next Nov 12th 2019 will carry on critical update which is “block search engines from indexing website through Robot.txt file”
WordPress relied on Robots.txt for blocking the indexing of a website because that’s how everybody kept pages from showing in Google’s search results. That was the standard way of doing it. This is the Robots Meta Tag that WordPress will use:
There is two ways to discourage search engines from crawling your site:-
1- Direct from WP dashboard – Setting – Reading – Select “Discourage search engines from indexing this site” on the page bottom
2- You can do it through Robot.txt file
Now I know most of you has vital question running in your mind. It is!
“Why should I discourage search engines from indexing my site or specific pages?”
Actually it’s a useful feature in a few and very specific cases. Like:-
1- Your Website is under construction
2- Pages like “Thank you page” it’s not healthy for the business to appear on search results
3- Pages with duplicate content. “Some content may share with same content category”
4- Spammy community profile pages, like (FQA pages). Britney Mullerof Moz recently deindexed 75% of Moz’s website and found huge success.