A Little Backstory on the Robots Meta Tag
A long time ago, as websites grew, there was a need to guide search engines on what they should check out on a website. The Robots Meta Tag was like a friendly guide, helping search engines know where to go and where not to.
What is robots in meta tag?
Example Scenario:
Imagine you have a special note you place on your bedroom door, telling friends what they can or can’t do inside. The Robots Meta Tag is like that note, but for search engines. It’s a small piece of information on a website page that gives directions.
Here’s what the code would look like:
<meta name="robots" content="noindex">
Where is it placed?
This tag would be placed in the <head> section of your HTML document. So, the structure would look something like this:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Your Webpage Title</title>
<!-- Here's the Robots Meta Tag -->
<meta name="robots" content="noindex">
<!-- Other meta tags, stylesheets, or scripts can follow -->
</head>
<body>
<!-- Your webpage content goes here -->
</body>
</html>
In this example, the <meta name=”robots” content=”noindex”> tag tells search engines not to index this particular page. So, when search engines like Google see this tag, they’ll understand that this page shouldn’t appear in their search results.
Remember, this is just one use of the Robots Meta Tag. It can have other values like nofollow, which tells search engines not to follow the links on the page, or a combination like noindex, nofollow to apply both rules.
What is the difference between robots.txt and robots meta tag?
Think of robots.txt as the main rules of your house, like “no shoes inside.” It gives general rules for the whole website.
Example for robots.txt:
User-agent: *
Disallow: /private/
This example tells all search engines not to access or index the “private” directory of the website.
The Robots Meta Tag is more specific. It’s like saying, “don’t touch the vase in the living room.” It gives rules for specific pages.
Example for Robots Meta Tag:
<meta name="robots" content="noindex">
This tag tells search engines not to index this particular page. So, when search engines like Google see this tag, they’ll understand that this page shouldn’t appear in their search results.
Can you have multiple robots meta tags?
Yes! It’s like having several notes on your door, each telling a different thing. Maybe one says “don’t open the closet” and another says “feel free to use the desk.” Each Robots Meta Tag gives a different instruction for the same page.
Example for Multiple Robots Meta Tags:
<meta name="robots" content="noindex"><meta name="robots" content="nofollow">
The first tag tells search engines not to index this page, while the second one says not to follow the links on this page. Both instructions are for the same page, but they give different directions to search engines.
Where should the meta robots tag be placed?
<head>
<meta name=”robots” content=”noindex, nofollow”>
</head>
<body>
…
</body>
</html>
Add Your Heading Text Here
<head>
<meta name=”robots” content=”noindex”>
</head>
<body>
…
</body>
</html>
