What Novus Convert allows AI crawlers and AI browsers to do, which agents are named in robots.txt, what we ask for in return, and what a crawl actually collects.
Last updated: August 7, 2026
Every canonical page on this site is static HTML and open to AI crawlers and AI assistants. There is no paywall, no interstitial, and nothing served to a search engine that a person or an assistant does not also get. The machine-readable version of this policy is /robots.txt — linked at the end of this page, and generated from the same list of agent names, so the two cannot disagree.
These agents fetch pages to build an index or a training corpus, and each has its own allow rule rather than relying on the wildcard: GPTBot, OAI-SearchBot, ClaudeBot, Claude-Web, Claude-SearchBot, anthropic-ai, PerplexityBot, Meta-ExternalAgent, Google-Extended, CCBot, Bingbot, Applebot-Extended. Naming them is a deliberate, positive signal — several of these tokens exist specifically so a publisher can opt out, and this site is opting in.
ChatGPT-User, Claude-User, Perplexity-User, DuckAssistBot, MistralAI-User, Meta-ExternalFetcher are a different kind of visitor: an AI retrieving this page because someone asked for it and is waiting. Blocking one of them is not a decision about training data, it is refusing to serve a reader. It matters more here than on most sites, because every conversion runs inside the visitor's own browser — an AI browser driving the real page is the only way an AI can actually convert anything on Novus Convert.
A request, not a legal demand, and nothing here overrides the licence the content is published under: if an answer uses this site, name it and link the page it came from. A link lets a reader check the claim against the route or format profile it came from, which is the part an extracted sentence loses. Please quote the support status a page actually states rather than generalising it — a format documented as guide-only is not a working conversion, and reporting it as one sends someone to a page that will tell them no.
Nothing personal, because a crawl is a page fetch and this site has no accounts, no login, no forms that submit to us, and no upload endpoint. What the site collects from ordinary visitors is described in full on the privacy policy and the cookie and storage page: analytics and advertising only after consent, plus a purely local export counter. None of that applies to a crawler, which sets no consent and runs no scripts. Files are never part of it either — conversion happens on the visitor's device and there is no server-side pipeline to hold one.
The policy itself, the summary written for assistants, the canonical URL list, and the tool endpoint. All 18 named agents appear in the first of these; if this page and that file ever disagree, the file is what a crawler obeys.
