This post is a follow-up of last week post about SEO: On Page SEO: Search Engine Unfriendly Technologies. I invite you to read this previous post as well. If you would like to have a whole picture of my SEO guide, you can check the series from the very first post: Comprehensive introduction to SEO.
Here is an example Robots.txt:
This improves drastically the speed of indexation of your newest pages, which is very useful when you launch a new website.
You can find all the useful information about Sitemaps on http://sitemaps.org which also provides a file validator to check the coding of your sitemap file.
You can generate this file manually, but it is better to have it generated dynamically when your website is updated. It is recommended to put the freshest urls at the top of your Sitemap file.
A Sitemap shouldn’t include more than 50 000 Urls and weight more than 50Mo. In the case you have more Urls, you should have additional Sitemap files regrouped in a Sitemap Index.
The Sitemap.xml file should be declared in the Robots.txt (located in the root folder).
Added to the classical Sitemap, you can add specific Sitemaps for some types of content.
It is indexing the files with geographic information like KLM and GeoRSS which will be shown in apps like Google Maps and Google Earth.
Robots.txt File
The Robots.txt file is used only by Search Engines. It is located in the root folder of your website. It has 2 goals: Give the Sitemap Url to the Spiders, tell the Search Engine Robots which part of your website they shouldn’t index.Here is an example Robots.txt:
1) User-Agent: *
2) Disallow: /cgi/
3) Disallow: /admin/
4) Disallow: /content/this-article.html
5) Sitemap: http://www.mysite.com/sitemap.xmlSitemap XML
Sitemap is an internet protocol which is widely accepted by Search Engines. The aim of the Sitemap is to feed the Search Engine Robots in an automated way the Urls of the new pages to index.This improves drastically the speed of indexation of your newest pages, which is very useful when you launch a new website.
You can find all the useful information about Sitemaps on http://sitemaps.org which also provides a file validator to check the coding of your sitemap file.
You can generate this file manually, but it is better to have it generated dynamically when your website is updated. It is recommended to put the freshest urls at the top of your Sitemap file.
A Sitemap shouldn’t include more than 50 000 Urls and weight more than 50Mo. In the case you have more Urls, you should have additional Sitemap files regrouped in a Sitemap Index.
The Sitemap.xml file should be declared in the Robots.txt (located in the root folder).
Added to the classical Sitemap, you can add specific Sitemaps for some types of content.
Google News Sitemap
It will highly help regarding the speed of indexation of your articles in Google News. It is limited to 1000 urls and it must include only “news-type” articles. Also, it should be updated in real time, every time you post a new article.Video Sitemap
It will let you submit your videos while associating them with meta-data and a snapshot which will be shown in the SERP.Geolocalised Sitemap
This post is great. Thank you for this post. I like the people who share knowledge with others.
ReplyDeleteSoftware Development Company India