You can edit your robots.txt file with the help of the Rank Math SEO plugin from Rank Math > General Settings > Edit robots.txt.
But there might be cases wherein you are not able to edit your file. In this knowledgebase tutorial, we’ll look at why you may be unable to edit your website’s robots.txt file.
Common Reasons Your Site’s Robots.txt Can’t Be Edited
Let us look at some of the common reasons:
1 Presence of a Physical File
One of the common reasons for this issue is that a physical robots.txt file is present in the root folder of your website. Rank Math creates a virtual robots.txt file and lets you handle it through the WordPress dashboard. If you have an actual robots.txt file in your site’s root folder, then you may not be able to edit it with Rank Math and you’ll see a notice as shown below.
To fix this, you can remove this file via FTP or cPanel File Manager and then try to edit the robots.txt file via Rank Math.
2 Other Plugin Creates Robots.txt
If another plugin is creating the robots.txt file then you will not be able to edit it with Rank Math. You can simply find and disable the plugin that is creating the file to resolve the issue.
3 Permission Issue on the Web Server
If there are some permission issues on the Web Server, you need to contact the web host. They might help you to solve the problem further.
Robots.txt File Is Not Writeable
You may notice this error message when Rank Math is unable to make any edits to the robots.txt file of your website. This is possible when the file edits are disabled through the wp-config.php file.
To enable Rank Math to edit the file, include the
DISALLOW_FILE_EDIT constant in your wp-config.php file and set it to false. If this constant is already added, ensure that it is set to false as shown below.
define ('DISALLOW_FILE_EDIT', false);
Editing the Robots.txt file is a case of “with great power, comes greater responsibility”. The power to guide the Googlebot with a well-written robots.txt file is a tempting one. Nevertheless, the advantages of having a well-written robots.txt file are enormous, better crawl speed, no useless content for crawlers, etc. However, keep in mind that one little mistake can cause you a lot of harm.
As always, if you still have any questions, doubts, or queries, we’re here to help. Please feel free to reach our support team, who are available 24×7 and will help you out as soon as possible.