CaravelShieldBot
CaravelShieldBot is the robot of Caravel Store Shield. It reads public pages of an online store and writes down what they show. If you found it in your logs, this page explains what it did.
How it identifies itself
Every request sends this User-Agent:
CaravelShieldBot/0.1 (+https://caravelsuite.com/shield/bot/)
What it reads
- Public pages of the store: the home page, categories, the empty cart page, 1 legal page and a sample of up to 20 products (from a sale collection and the sitemap).
- The visible text, the price data published in the page and the buttons of the first layer of the cookie banner (without clicking them).
- The requests the page tries to make, and the cookies and localStorage in its own browser, which is emptied after each visit.
- Screenshots of the pages, for the dossier.
A page is first opened as a normal visit. On 2 pages at most it is opened again with the Global Privacy Control signal (Sec-GPC: 1), as a complementary technical test. It waits at least 5 seconds between pages of the same site, or longer if your robots.txt asks for it with Crawl-delay. In stores whose owners ordered the service and proved control, it also returns every day to record prices.
What it never does
- It never clicks, never accepts or refuses cookies and never closes banners.
- It never submits forms, signs in, adds products to the cart or buys.
- It never sends known collection requests to services on our list: they are answered locally by the robot. Their script libraries, unknown paths and review widgets load like any other file of the page.
- It never reads pages that your robots.txt does not allow it to read, and it never follows links to other sites.
- It never changes anything on your site.
- It never tries to get around a block: at the first captcha, 403, 429, 503 or challenge it records it and stops asking that store for pages.
robots.txt
It follows the group for CaravelShieldBot in your robots.txt, or the group for all robots (*) if there is none. To exclude your whole site:
User-agent: CaravelShieldBot
Disallow: /Ask to be excluded
You can also write to caravel@caravelsuite.com with your domain. We add it to our exclusion list and the robot will not visit it again.
The robot is run by Jardim Idílico Unipessoal Lda (Caravel Suite), Portugal.