Q1: What is a robots.txt file?
A robots.txt file is a small text file placed at the root of a website that tells search engines (like Google, Bing, or DuckDuckGo) which parts of your site they can or cannot crawl.
It doesn't block bots per se, it indicates them what is off limits.
More info here.
Q2: Why would I add a custom robots.txt to my Qualifio campaign?
By default, search engines might try to index your campaigns, even if they’re embedded, or not meant to appear publicly.
Adding a custom robots.txt lets you define exactly how crawlers interact with your campaigns.
Preventing temporary or private campaigns from being indexed
Many Qualifio campaigns are time-limited (e.g., contests, quizzes, seasonal games). You may not want them showing up in Google search.
Example:
User-agent: *
Disallow: /This tells all crawlers not to index or crawl any part of your campaign.
Directing crawlers to your sitemap
If your brand hosts multiple Qualifio campaigns and you want crawlers to understand your campaign structure better, you can point them to a specific sitemap.
Example:
User-agent: *
Allow: /
Sitemap: https://www.yourbrand.com/sitemap.xmlThis helps ensure only the right campaign URLs are indexed.
Controlling specific bots
Some companies prefer to allow or block specific crawlers (e.g., Googlebot allowed, AI scrapers blocked).
Example:
# Block GPTBot
User-agent: GPTBot
Disallow: /
# Block Google Extended
User-agent: Google-Extended
Disallow: /
# Allow X shares
User-agent: Twitterbot
Allow: /This keeps your content visible on Google but not scraped for AI training.
Q3: Are robots.txt active per campaign or per domain?
A custom robots.txt is active for all campaigns on a domain, not per campaign. It is matched on the exact hostname: www.yourbrand.com and yourbrand.com are two different hosts, so each one needs its own file.
A domain without a custom file uses Qualifio's default robots.txt, which allows all crawlers.
Q4: Is it mandatory to add a custom robots.txt?
No — it’s optional.
Q5: How do I create one?
You can create a robots.txt file in any text editor — just plain text, no formatting.
Q6: How do I add my robots.txt to my Qualifio campaigns?
Open a ticket, add the robots.txt and tell us the exact hostname of each domain where you would like us to use it (for example form.yourbrand.com).
To check it, open https://your-domain/robots.txt in your browser.
That's it — it's that simple!
Q7: Can I also add an assetlinks.json file?
Yes. Qualifio can also serve a custom /.well-known/assetlinks.json file on your campaign domains. It is currently used to declare related sites with a lookalikes/allowlist relation.
Example:
[
{
"relation": ["lookalikes/allowlist"],
"target": {
"namespace": "web",
"site": "https://www.yourbrand.com"
}
}
]Like the robots.txt, it applies to a whole domain and is matched on the exact hostname. A domain without a custom file has none, so the address returns "not found".
To add one, open a ticket, attach the file and tell us on which of your domains you would like us to use it.
Other useful links: