Skip to main content
All CollectionsSEO Support
Add a Robots.txt to your store
Add a Robots.txt to your store

If you need to link to a robots.txt to your webmaster tools or if you need your SEO company to upload a unique robots file.

Niall Diamond avatar
Written by Niall Diamond
Updated over a week ago

The robots.txt file is one of the ways search engines use to determine what pages they can crawl on your website. All the top search engines support robots.txt. 

⚠️ Important: Don't go any further without advice from your SEO professionals. Otherwise you might do more harm than good.  

Location of your robots.txt file 

By default you will have a robots.txt file on your site found at
https://yourDomain.com/robots.txt

By default this file has the recommended defaults to encourage the search engines to crawl your website. 

👍 ProTip: Your website is SEO friendly and encourages Google and other search engines to crawl your website. You don't need to do anything, that happens by default. 

Uploading a new file

 If your SEO company provides you with a new custom robots.txt file, then you will need to compare it with the default file and then only add the new items that aren't included in the default. The store merges the default robots with your custom uploaded robots file.  

⚠️ Important: Only add the new additions that you need included in the file. The store will automatically merge your file with the existing default robots.txt file.

⚠️ Important: This file is case sensitive, so make sure your upload the correct txt file with all lowercase "robots.txt"  

 Upload to the Media Manager

Upload the "robots.txt" to the root of your media manager (e.g. not inside of any folder).

That's it now check it's working by visiting https://yourDomain.com/robots.txt

Submit to Google

Google Robots Tester Tool

  • Click to "see live robots.txt" and review it is correct

  • Then click submit and choose to submit to Google


Dangerzone setting 

You might be wondering about the "Dangerzone" setting and what relation it has to your robots.txt. The meta tags are controlled by the dangerzone setting. 

The search engine will first look at your robots.txt file and then determine what pages can be crawled. Then when it lands on a page it will read the meta tags set by the dangerzone toggle. 

Thus, the Meta Tags and the Robots file are slightly different but related. One of them doesn't override the other one.  

These can come in conflict which is why you should probably just stick to the defaults, or ask your SEO professionals. 

Checkout Yoast.com for more great articles on robots.txt
Want more on meta robots tags checkout Yoast.com

Did this answer your question?