What is Robots.txt in Blogger or how to use it?

                   Hello blogger, did you know that each site has a different robots.Txt file that tells search engines what to index and what not to index for search results. In this post, we will know about it, what is it or what is its function.

What is Robots.txt?

Robots.txt is a text file that is present in every site, automatically generated in Blogger. Through this file, search engines such as Google, Yahoo, Bing are told about the site whether to index or not to index. For example, suppose that on your site blog, which page or label you want to show in the search engine and which page or label is not to be shown, the search engines get information from the Robots.txt file itself.

Every site on the Internet has a link to the robots.txt file -

www.website.com/robots.txt

What is the function of robots.Txt?

Whenever a search engine's bot comes to our site, search the site, then most of all he sees robots.Txt only, he knows whether to show me the search by indexing or not.

In simple words, as the name suggests, it is a robot that whenever the search engines crawl a site, it tells them where the permission is to be seen.

As you must have known, i am telling about blogger.Com. So this article is only for blogger.

How to Create Blogger Robots.txt File?

For this, you simply go to the site which I am telling you.

Now the site that will open, enter the full address of your blog

Jesse https://www.technokhurwal.com/

• Enter your blog's address with http://
• Click on Generate Sitemap

Now a code will be generated as shown below, copy it.

So your code is ready, now let's see how or where to add it to blogger?

[ alert-note] if you do this setting wrong, then your site may be ignored by the search engine or your site may not be searched. Do it carefully or if you do not understand then do not do it, do it with someone's advice [ alert-note]

How to Add Custom Robots.txt File in Blogger?

1. First login to Blogger's account.
2. Go to Settings >> Search Preferences >> Custom robots.txt me Click on Edit and paste the code then save .

Now it happened, how to make Robots.txt or how to add Robots.txt. Let us now know if we want to disable any page, post, labels in search then how to do it..

How to Allow/Disallow robots.txt?

Important Note: Change it only if you understand it well, otherwise the file you have pasted is enough, no need to edit it much.

If you do not want a page post or label to appear in search results, then you can DisAllow it.

For example, if there is a page which you do not want to be indexed then -
DisAllow: DisAllow: /p/about-us.html

If there is a label -
DisAllow: /search/label/yourlabel

Disallow the post -
Disclaimer: /2018/08/my-blog-posts.html

So then if suppose you have to reject the pure page then
DisAllow: /p

or if to do something like allow
Allow: /p/About-us.html

In this way you can set what to index and what not to index.

Keep sharing the wealth of knowledge and keep blogging......!