Q1: What is a robots.txt file?
A robots.txt file is a small text file placed at the root of a website that tells search engines (like Google, Bing, or DuckDuckGo) which parts of your site they can or cannot crawl.
It doesn't block bots per se, it indicates them what is off limits.
More info here.
Q2: Why would I add a custom robots.txt to my Qualifio campaign?
By default, search engines might try to index your campaigns, even if they’re embedded, or not meant to appear publicly.
Uploading a custom robots.txt lets you define exactly how crawlers interact with your campaigns.
Preventing temporary or private campaigns from being indexed
Many Qualifio campaigns are time-limited (e.g., contests, quizzes, seasonal games). You may not want them showing up in Google search.
Example:
User-agent: *
Disallow: /This tells all crawlers not to index or crawl any part of your campaign.
Directing crawlers to your sitemap
If your brand hosts multiple Qualifio campaigns and you want crawlers to understand your campaign structure better, you can point them to a specific sitemap.
Example:
User-agent: *
Allow: /
Sitemap: https://www.yourbrand.com/sitemap.xmlThis helps ensure only the right campaign URLs are indexed.
Controlling specific bots
Some companies prefer to allow or block specific crawlers (e.g., Googlebot allowed, AI scrapers blocked).
Example:
# Block GPTBot
User-agent: GPTBot
Disallow: /
# Block Google Extended
User-agent: Google-Extended
Disallow: /
# Allow X shares
User-agent: Twitterbot
Allow: /This keeps your content visible on Google but not scraped for AI training.
Q3 : Are robots.txt active per campaign or per domain?
A robots.txt will be active for all campaigns for a defined domain(s), not per campaign.
Q4: Is it mandatory to add a custom robots.txt?
No — it’s optional.
Q5: How do I create one?
You can create a robots.txt file in any text editor — just plain text, no formatting.
Q6: How do I add my robots.txt to my Qualifio campaigns?
Open a ticket, add the robots.txt and let us know on which of your domains you would like us to use it.
That's it — it's that simple!
Other useful links: