This guide answers the question, “where is robots.txt in WordPress?”
It also explains how to access robots.txt in WordPress using four different methods so you can modify the file for your website.
Consider this your ultimate—and quick—guide to finding the robots.txt in WordPress and accessing it with ease to control how search engine spiders crawl your website’s files, pages, and directories.
Where Is Robots.txt In WordPress?
Robots.txt is located in the site’s root folder of the WordPress directory. You can access the robots.txt file by navigating to yourdomain.com/robots.txt in a web browser and manage the file on the server using an FTP client, plugin, or cPanel.
How to Access and Modify WordPress Robots.txt File
1. Access Robots.txt Via cPanel (or Your Hosting Company’s Alternative)
The first way to get to your WordPress robots.txt file is to use the File Manager feature offered by your hosting company.
How you access this file manager varies from host to host. With a hosting provider like Bluehost, which utilizes the popular cPanel interface, you simply select Advanced options followed by File Manager.
With hosting services like Hostinger, you’ll find the WordPress file manage as soon as you log into your dashboard as seen in the image belo.
Once you’re in the File Manager, you’ll find the website’s robots.txt file in the root folder of the WordPress installation which is usually labeled public_html.
You can then download the WordPress robots.txt file, modify it in your text editor, and re-upload it to the website.
2. Use an SEO Plugin
If you don’t want to log into your hosting account or manage files directly on the root directory, then you can access the robots.txt from directly within WordPress by using an SEO plugin you may already have installed on your site.
Popular SEO plugins like Yoast, AIOSEO, and Rank Math let you edit your website’s robots.txt file from the WordPress administrator dashboard with ease.
Here’s an example walkthrough of this method using the Yoast SEO plugin. After accessing, the WordPress dashboard menu, select Yoast SEO > Tools as seen in the image below.
From the Yoast SEO tools screen, select File Editor.
This will bring up your WordPress site’s robots.txt file. You can then modify it and save your changes within WordPress.
If you’re using an alternative WordPress SEO plugin such as Rank Math or AIOSEO, you’ll also be able to locate the robots.txt file under General Settings in Rank Math and in the Tools tab for AIOSEO.
3. Use a Dedicated Robots.txt Plugin
Another way to access and modify robots.txt in WordPress is to use a plugin that’s built specifically for this purpose.
The free Virtual Robots plugin from Infolific is a popular choice and incredibly easy to use.
After installing and activating the plugin, navigate to Settings > Virtual Robots.txt.
This opens up your WordPress robots.txt file so you can make any changes. Just remember to hit the save button when you’re done for the updates to take effect.
If you’re new to WordPress and you’re not confident in how to configure the different commands and directives in your robots.txt file, you may prefer a more beginner-friendly plugin like Better Robots.Txt, which is listed as “WordPress Robots.txt optimization” in the plugins directory. See the image below.
Activate and install this plugin like you would any other. Once installed, it automatically loads up on your WordPress dashboard. You can also return to it at any time by selecting Better Robots.txt from the administration menu.
Inside the plugin, you’ll have an easy-to-use visual interface to configure various robots.txt settings, including which search engines can crawl your site, which spam and data protection features to enable, and any custom rules you want to add. See the images below for a detailed look at all of these WordPress options.
4. Use an FTP Client for WordPress
Another way to access and modify a WordPress robots.txt file is by using an FTP client such as FileZilla. If you already have an FTP client like this set up for your website, then you can quickly locate the WordPress robots.txt.
Begin by accessing your FTP account information from your hosting provider. Copy this information, open up Filezilla, and paste the details into the relevant boxes located just beneath the software’s menu bar.
Next, click Quick Connect, and this will connect the FTP client to your hosting server, giving you access to all of your WordPress files.
From here, navigate to your root directory, where you should find your WordPress robots.txt file. To access this file so you can open and modify it, right-click and tap Download. Alternatively, you can select the file and drag it across to the local server pannel within Filezilla. See screenshots below.
Open the robots.txt file in a text editor and make your modifications. When you’re done, navigate to the file’s location in the local server panel, then click and drag the file across to the remote site server for it to upload to WordPress.
A pop up window should appear informing you that the target file already exists on the server. Choose the overwrite action, then click OK to successfully update the robots.txt file for your WordPress website.
Robots.txt In WordPress Summary
I hope you enjoyed this guide answering the question about where is robots.txt in WordPress.
As you discovered, the robots.txt file is located in the site’s root folder of the WordPress directory, and the fastest way for how to access robots.txt in WordPress is to navigate to yourdomain.com/robots.txt in a web browser. To manage and modify the file, you’ll need to access it on the server using cPanel, a WordPress plugin, or FTP client.
