Shopify Store Robots.txt can be edited now

Editing robots.txt.liquid

Search engines, such as Google, Bing, normally will constantly crawl the internet in search of new data as a source for their search results. The robots.txt file tells search engine bots(known as crawlers), which pages to request to view from your online store. Actually, your Shopify store has a default robots.txt file that’s optimal for Search Engine Optimization (SEO).

Your sitemap is used by search engines to place your online store in the search engine results.

On this page

Overview

The default robots.txt file works for most stores, but you can edit the file through the robots.txt.liquid theme template. You can make the following edits:

  • allow or disallow certain URLs from being crawled
  • add crawl-delay rules for certain crawlers
  • add extra sitemap URLs
  • block certain crawlers

The robots.txt file is located at the root directory of your Shopify store’s primary domain name. For example: www.mojoin.com/robots.txt.

CAUTION

This is an unsupported customization. Shopify Support can’t help with edits to the robots.txt.liquid file. If you need to edit your robots.txt.liquid file, then you can hire a Shopify expert to edit the code for you. Incorrect use of the feature can result in loss of all traffic.

Edit robots.txt.liquid

If you want to edit the robots.txt.liquid file, then you should work with a Shopify Expert or have expertise in code edits and SEO.

You can use Liquid to add or remove directives from the robots.txt.liquid template. This method preserves Shopify’s ability to keep the file updated automatically in the future, and is recommended.

Remove any previous customizations or workarounds, such as using a third-party service such as Cloudflare, before you edit the robots.txt.liquid file.

Steps:

  1. From your Shopify admin, go to Online Store > Themes.
  2. Click Actions, and then click Edit Code.
  3. Click Add a new template, and then select robots.
  4. Click Create template.
  5. Make the changes that you want to make to the default template. For more information on liquid variables and common use cases, refer to Shopify’s Developer page Customize robots.txt.liquid.
  6. Save changes to the robots.txt.liquid file in your published theme.

Changes are instant, but crawlers don’t always react immediately. You can test changes using Google’s robots.txt Tester.

You can also delete the contents of the template and replace it with plain text rules. This method is strongly not recommended, as rules may become out of date. If you choose this method, then Shopify can’t ensure that SEO best practices are applied to your robots.txt over time, or make changes to the file with future updates.

NOTE

ThemeKit or command line changes will preserve the robots.txt.liquid file. Uploading a theme from within the Themes section of the Shopify admin will not import robots.txt.liquid.

Delete robots.txt.liquid customizations

If you want to use the default robots.txt file again, then save a copy of your robots.txt.liquid template customizations as you need to delete the template, which can’t be undone.

Steps:

  1. From your Shopify admin, go to Online Store > Themes.
  2. Click Actions, and then click Edit Code.
  3. Click robots.liquid, and then click Delete file.
  4. If you still want to delete robots.txt.liquid, then click Delete file.

You can always edit the robots.txt.liquid file in the future, or if you saved your past customizations, then you can redo your changes by copying over your saved previous customizations.

Please follow and like us:

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *