Art-Online.com 0.9(Beta)

Who?

Spider name : Art-Online.com 0.9(Beta). It crawls the internet everyday searching for art related websites, following the links.

Rules

To avoid any trouble on your servers, we set our spider at a frequency range of 5-9 seconds and below 500 queries every 24 hours.

Art-online crawlers abide by the Robots Exclusion Standard

If you do not wish the Art-online crawlers to crawl and/or index your Web site, please read the Robots Exclusion Standard and add a robots.txt file to your Web pages. Art-online crawler also recognizes a metatag exclusion options:

• NOINDEX the crawler will not index the page
• NOFOLLOW : the crawler will not follow the links