Page MenuHomeDevCentral
Files F24947019

botPolicies.yaml
No One
Actions

Authored By
dereckson
Fri, Mar 20, 20:02
Size
8 KB
Referenced Files
None
Subscribers
None

botPolicies.yaml
View Options

## Anubis has the ability to let you import snippets of configuration into the main
## configuration file. This allows you to break up your config into smaller parts
## that get logically assembled into one big file.
##
## Of note, a bot rule can either have inline bot configuration or import a
## bot config snippet. You cannot do both in a single bot rule.
##
## Import paths can either be prefixed with (data) to import from the common/shared
## rules in the data folder in the Anubis source tree or will point to absolute/relative
## paths in your filesystem. If you don't have access to the Anubis source tree, check
## /usr/share/docs/anubis/data or in the tarball you extracted Anubis from.
bots:
# You can import the entire default config with this macro:
# - import: (data)/meta/default-config.yaml
# Pathological bots to deny
- # This correlates to data/bots/_deny-pathological.yaml in the source tree
# https://github.com/TecharoHQ/anubis/blob/main/data/bots/_deny-pathological.yaml
import: (data)/bots/_deny-pathological.yaml
- import: (data)/bots/aggressive-brazilian-scrapers.yaml
# Aggressively block AI/LLM related bots/agents by default
- import: (data)/meta/ai-block-aggressive.yaml
# Consider replacing the aggressive AI policy with more selective policies:
# - import: (data)/meta/ai-block-moderate.yaml
# - import: (data)/meta/ai-block-permissive.yaml
# Search engine crawlers to allow, defaults to:
# - Google (so they don't try to bypass Anubis)
# - Apple
# - Bing
# - DuckDuckGo
# - Qwant
# - The Internet Archive
# - Kagi
# - Marginalia
# - Mojeek
- import: (data)/crawlers/_allow-good.yaml
# Challenge Firefox AI previews
- import: (data)/clients/x-firefox-ai.yaml
# Allow common "keeping the internet working" routes (well-known, favicon, robots.txt)
- import: (data)/common/keep-internet-working.yaml
## If your backend service is running on the same operating system as Anubis,
## you can uncomment this rule to make the challenge easier when the system is
## under low load.
##
## If it is not, remove weight.
- name: low-load-average
action: WEIGH
expression: load_15m <= 2.0
weight:
adjust: -10
# Assert behaviour that only genuine browsers display. This ensures that Chrome
# or Firefox versions
- name: realistic-browser-catchall
expression:
all:
- '"User-Agent" in headers'
- '( userAgent.contains("Firefox") ) || ( userAgent.contains("Chrome") ) || ( userAgent.contains("Safari") )'
- '"Accept" in headers'
- '"Sec-Fetch-Dest" in headers'
- '"Sec-Fetch-Mode" in headers'
- '"Sec-Fetch-Site" in headers'
- '"Accept-Encoding" in headers'
- '( headers["Accept-Encoding"].contains("zstd") || headers["Accept-Encoding"].contains("br") )'
- '"Accept-Language" in headers'
action: WEIGH
weight:
adjust: -10
# The Upgrade-Insecure-Requests header is typically sent by browsers, but not always
- name: upgrade-insecure-requests
expression: '"Upgrade-Insecure-Requests" in headers'
action: WEIGH
weight:
adjust: -2
# Chrome should behave like Chrome
- name: chrome-is-proper
expression:
all:
- userAgent.contains("Chrome")
- '"Sec-Ch-Ua" in headers'
- 'headers["Sec-Ch-Ua"].contains("Chromium")'
- '"Sec-Ch-Ua-Mobile" in headers'
- '"Sec-Ch-Ua-Platform" in headers'
action: WEIGH
weight:
adjust: -5
- name: should-have-accept
expression: '!("Accept" in headers)'
action: WEIGH
weight:
adjust: 5
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: WEIGH
weight:
adjust: 10
dnsbl: false
# #
# impressum:
# # Displayed at the bottom of every page rendered by Anubis.
# footer: >-
# This website is hosted by Zombocom. If you have any complaints or notes
# about the service, please contact
# <a href="mailto:contact@domainhere.example">contact@domainhere.example</a>
# and we will assist you as soon as possible.
# # The imprint page that will be linked to at the footer of every Anubis page.
# page:
# # The HTML <title> of the page
# title: Imprint and Privacy Policy
# # The HTML contents of the page. The exact contents of this page can
# # and will vary by locale. Please consult with a lawyer if you are not
# # sure what to put here
# body: >-
# <p>Last updated: June 2025</p>
# <h2>Information that is gathered from visitors</h2>
# <p>In common with other websites, log files are stored on the web server saving details such as the visitor's IP address, browser type, referring page and time of visit.</p>
# <p>Cookies may be used to remember visitor preferences when interacting with the website.</p>
# <p>Where registration is required, the visitor's email and a username will be stored on the server.</p>
# <!-- ... -->
# Open Graph passthrough configuration, see here for more information:
# https://anubis.techaro.lol/docs/admin/configuration/open-graph/
openGraph:
# Enables Open Graph passthrough
enabled: false
# Enables the use of the HTTP host in the cache key, this enables
# caching metadata for multiple http hosts at once.
considerHost: false
# How long cached OpenGraph metadata should last in memory
ttl: 24h
# # If set, return these opengraph values instead of looking them up with
# # the target service.
# #
# # Correlates to properties in https://ogp.me/
# override:
# # og:title is required, it is the title of the website
# "og:title": "Techaro Anubis"
# "og:description": >-
# Anubis is a Web AI Firewall Utility that helps you fight the bots
# away so that you can maintain uptime at work!
# "description": >-
# Anubis is a Web AI Firewall Utility that helps you fight the bots
# away so that you can maintain uptime at work!
# By default, send HTTP 200 back to clients that either get issued a challenge
# or a denial. This seems weird, but this is load-bearing due to the fact that
# the most aggressive scraper bots seem to really, really, want an HTTP 200 and
# will stop sending requests once they get it.
status_codes:
CHALLENGE: 200
DENY: 200
# Anubis can store temporary data in one of a few backends. See the storage
# backends section of the docs for more information:
#
# https://anubis.techaro.lol/docs/admin/policies#storage-backends
store:
backend: memory
parameters: {}
# The weight thresholds for when to trigger individual challenges. Any
# CHALLENGE will take precedence over this.
#
# A threshold has four configuration options:
#
# - name: the name that is reported down the stack and used for metrics
# - expression: A CEL expression with the request weight in the variable
# weight
# - action: the Anubis action to apply, similar to in a bot policy
# - challenge: which challenge to send to the user, similar to in a bot policy
#
# See https://anubis.techaro.lol/docs/admin/configuration/thresholds for more
# information.
thresholds:
# By default Anubis ships with the following thresholds:
- name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
expression: weight <= 0 # a feather weighs zero units
#x-action: ALLOW # Allow the traffic through
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
# For clients that had some weight reduced through custom rules, give them a
# lightweight challenge.
- name: mild-suspicion
expression:
all:
- weight > 0
- weight < 10
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
algorithm: metarefresh
difficulty: 1
# For clients that are browser-like but have either gained points from custom rules or
# report as a standard browser.
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 2 # two leading zeros, very fast for most clients
- name: mild-proof-of-work
expression:
all:
- weight >= 20
- weight < 30
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
# For clients that are browser like and have gained many points from custom rules
- name: extreme-suspicion
expression: weight >= 30
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 6

File Metadata

Mime Type
text/plain; charset=utf-8
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
3550236
Default Alt Text
botPolicies.yaml (8 KB)

Event Timeline

Nasqueron DevCentral · If it had been much bigger the moon would have had a core of ice. · Powered by Phabricator