> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fingerprint.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Search bot request filtering

> Allow basic search bots filtering through a single switch in the Request Filtering Dashboard section

Search bots generate legitimate internet traffic allowing search engines to index your website. But you might not want search bots to trigger Fingerprint Identification or Smart Signals processing.

Although you can [filter requests based on the `User-Agent` header](/docs/request-filtering-for-websites#filter-by-http-header), you would need to manage your list of all different search bot `User-Agent` strings. Instead, we keep and update our own list of search bot filtering rules that **can be turned on or off through a switch in the Dashboard**.

Search bots that are filtered through this setting then behave in the same way as other request filtering rules and you don't get billed for them. Keep in mind that we won't return any information except `requestId` in those cases.

<Note>
  Search bots filtering based on `User-Agent` is a basic mechanism we provide for free to filter out commonly encountered and generally well behaved search bots.
  Because of its simplicity, it might introduce both **false positive and false negative** cases. If you need **better protection against sophisticated bots**, check out our [Smart Signals that contain a paid Bot Detection product](/docs/smart-signals-reference#browser-bot-detection).
</Note>

## Configuration

Search Bots Filtering can be turned on or off in our Dashboard.

1. Navigate to **Dashboard** > **[Security](https://dashboard.fingerprint.com/traffic-rules)** and switch to the **Bots** tab.
2. Enable **Exclude Search Bots** to start filtering search bots from your billable API calls.

<img src="https://mintcdn.com/fingerprint/JTUbc3rQcwtp3hbV/images/3e8a6f350c6ce3af02a681325e63ccf3cd07d23e6619ff0fa71813549415c3f4-CleanShot_2024-11-06_at_13.33.192x.png?fit=max&auto=format&n=JTUbc3rQcwtp3hbV&q=85&s=33f85234f145421d56d3736dea53b0f4" alt="Dashboard - Search Bots Filtering" width="1688" height="902" data-path="images/3e8a6f350c6ce3af02a681325e63ccf3cd07d23e6619ff0fa71813549415c3f4-CleanShot_2024-11-06_at_13.33.192x.png" />

## List of supported search bots `User-Agent` strings

The list of search bots`User-Agent` strings we support is public and we keep it updated based on the currently supported `User-Agent` strings. The following table contains all supported `User-Agent` strings with their respective information sources.

| Owner      | User-Agent (Substrings)                                                                                                                                                                                                                                                                                                    | Public doc                                                                                     |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Mail.ru    | `mail.ru`                                                                                                                                                                                                                                                                                                                  | [source](https://help.mail.ru/webmaster/indexing/robots/robot_log)                             |
| Microsoft  | `bingbot`<br />`AdIdxBot`<br />`MicrosoftPreview`                                                                                                                                                                                                                                                                          | [source](https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0)           |
| Apple      | `applebot`                                                                                                                                                                                                                                                                                                                 | [source](https://support.apple.com/en-us/HT204683)                                             |
| Huawei     | `petalbot`                                                                                                                                                                                                                                                                                                                 | [source](https://aspiegel.com/petalbot)                                                        |
| Naver      | `naver.me/spd`                                                                                                                                                                                                                                                                                                             | [source](https://help.naver.com/service/5626/contents/8026)                                    |
| Yahoo      | `slurp`                                                                                                                                                                                                                                                                                                                    | [source](https://help.yahoo.com/kb/SLN22600.html)                                              |
| Baidu      | `baiduspider`                                                                                                                                                                                                                                                                                                              | [source](https://www.baidu.com/search/robots_english.html)                                     |
| Sogou      | `sogou web spider`<br />`sogou wap spider`<br />`sogou pic spider`                                                                                                                                                                                                                                                         | [source](https://help.sogou.com/spider.html)                                                   |
| Exalead    | `exabot`                                                                                                                                                                                                                                                                                                                   | [source](https://www.exalead.com/search/webmasterguide)                                        |
| Coccoc     | `coccocbot`                                                                                                                                                                                                                                                                                                                | [source](https://help.coccoc.com/en/search-engine/coccoc-robots)                               |
| Seznam     | `seznamBot`                                                                                                                                                                                                                                                                                                                | [source](https://napoveda.seznam.cz/cz/fulltext-hledani-v-internetu/seznambot)                 |
| Google     | `googlebot`<br />`AdsBot-Google`<br />`Storebot-Google`<br />`APIs-Google`<br />`AdsBot-Google-Mobile`<br />`Mediapartners-Google`<br />`Googlebot-Image`<br />`Googlebot-News`<br />`Googlebot-Video`<br />`AdsBot-Google-Mobile-Apps`<br />`FeedFetcher-Google`<br />`Google-Read-Aloud`<br />`Google-Site-Verification` | [source](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers) |
| Yandex     | `yandex.com/bots`                                                                                                                                                                                                                                                                                                          | [source](https://yandex.com/support/webmaster/robot-workings/check-yandex-robots.html)         |
| Bytedance  | `bytespider`                                                                                                                                                                                                                                                                                                               | [source](https://zhanzhang.toutiao.com/docs/intro/26899)                                       |
| Duckduckgo | `duckduckbot`                                                                                                                                                                                                                                                                                                              | [source](https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/)                    |
| Ahrefs     | `AhrefsBot`<br />`AhrefsSiteAudit`                                                                                                                                                                                                                                                                                         | [source](https://ahrefs.com/robot)                                                             |
| Facebook   | `facebookexternalhit`<br />`facebookcatalog`                                                                                                                                                                                                                                                                               | [source](https://developers.facebook.com/docs/sharing/webmasters/crawler/)                     |
| Pinterest  | `Pinterestbot`                                                                                                                                                                                                                                                                                                             | [source](https://help.pinterest.com/en/business/article/pinterest-crawler)                     |
| Cincraw    | `Cincraw`                                                                                                                                                                                                                                                                                                                  | [source](https://cincrawdata.net/bot/)                                                         |
| Semrush    | `http://www.semrush.com/bot.html`                                                                                                                                                                                                                                                                                          |                                                                                                |
| Seekport   | `SeekportBot`                                                                                                                                                                                                                                                                                                              | [source](https://bot.seekport.com/)                                                            |
| Amazon     | `Amazonbot/0.1`<br />`amazonproductbot`                                                                                                                                                                                                                                                                                    | [source](https://developer.amazon.com/support/amazonbot)                                       |
| DataDog    | `datadog`                                                                                                                                                                                                                                                                                                                  | [source](https://docs.datadoghq.com/synthetics/guide/identify_synthetics_bots)                 |
