A Robots.txt File
Whenever troubles occur, solutions come, it is a natural thought and it works in all fields. If we talk about the page index in search engine’s database we can realize that problem may occur if our website has hidden text, repeated text and flash images, now if you submit this site then it might be spam, so what to do ?
Now solution comes, and that is “robot.txt file”, no matter how much hidden text or repeated keywords your site has, but the thing that matters, is indexing the page , However by using this “ robot.txt file” you can easily submit such kind of web sites to search engines.
Actually, “robot.txt file” prevents such kind of pages to get indexed; those have rich hidden text and repeated keywords.