How to Edit Your Robots.txt with Rank Math SEO? - Rank Math SEO

Robots.txt is a simple text file that can be used to control the way search engine bots crawl and index your website. Editing your robots.txt with Rank Math is definitely what you can vouch for!
In this video, we walk you through how to edit your robots.txt
rankmath.com/kb/how-to-edit-r...
Testing Tool:
www.google.com/webmasters/too...
Rank Math, the Swiss Army Knife of WordPress SEO can take your SEO game to the next level. Rank Math already goes above and beyond other top-rated SEO plugins. Do you want to see the magic unfold for your website?
►Visit Rank Math today: rankmath.com.
And if you enjoyed this video, don’t forget to hit like & subscribe for more…
Connect with us:
► / rankmathseo
► / rankmath
► / rankmath

[@kandaman304]

Thanks, Kate! This is was very helpful:)

[@RankMath]

Hello, and thanks for your comment. Glad that helped. Feel free to get in touch if you have questions.

[@AxelMetayer]

Thanks for the robots.txt tutorial, I wanted to see where I can edit it in RankMath, could you update the Wordpress default-blocked part? It seems that changed in the last 3 years

[@RankMath]

Hello, thank you for your comment,
To edit your robots.txt file in Rank Math, you can follow these updated steps in this tutorial:
rankmath.com/kb/cant-edit-robots-txt/
Don’t hesitate to get in touch if you have any other questions.

[@MohamedHamdy-zc2bp]

You are Great explan the method without take more time .. i hope you explan course from beginning about seo

[@RankMath]

Hello, thank you for your comment.
You can check the following lessons from our blog page:
SEO Basics: rankmath.com/blog/category/seo-basics/
Technical SEO: rankmath.com/blog/category/technical-seo/
If you encounter any issues with Rank Math, you can always raise your question here rankmath.com/support/ and our support team would be happy to assist you.

[@amrahqureshi2428]

Should I not leave an asterik * and instead add "googlebot" as I want it to crawl and index my website. I just don't understand why would i leave "user-agent" with an asterik when all I want is my website to be crawled and index? Please clarify this.

[@RankMath]

Hello, Thank you for your comment.
The "User-agent" directive specifies the user-agent (search engine bot) the rule applies to. When using an asterisk (*) as the user-agent, it serves as a wildcard, meaning the rule covers all user-agents, including Googlebot.
If you specifically want to target only Googlebot, you can mention "Googlebot" instead of using the asterisk (*).
If you encounter any issues with Rank Math, you can always raise your question here rankmath.com/support/ and our support team would be happy to assist you.

[@amrahqureshi2428]

@RankMath Thanks, one more thing, please. Does it become necessary to add this code to the website if you get the error message from the G Search console that "indexing blocked by robots.txt"?

[@RankMath]

Hello, thank you for your comment.
If you're noticing useful pages on your website are being marked as blocked by robots.txt then we would suggest you to follow this tutorial to identify the specific rule that is blocking them
rankmath.com/kb/fix-submitted-url-blocked-by-robots-txt-error/
Once identified you can adjust the rule to resolve the issue.
If you encounter any issues with Rank Math, you can always raise your question here rankmath.com/support/ and our support team would be happy to assist you.

[@News_Headline1]

Hello big brother
I've been trying to work on the Yandex webmaster, but it keeps telling me Rughts has not been verified. Meta tag not found, what should I do?

[@RankMath]

Hello, and thank you for your comment.
We have a detailed tutorial that walks through site verification at Yandex: ruclips.net/video/ePrKAHn3t9U/видео.html
If you face issue even after you follow this tutorial, then please open a support ticket here rankmath.com/support/ and our support team will closely into that closely.
Looking foward to helping you.

[@rishinegi527]

but my promblem is (robots.txt file is not writable) how I edit this

[@RankMath]

Hello, for the Rank Math’s robots.txt to work, you should delete the robots.txt that is in your website’s root folder. Once you delete that, robots.txt created using Rank Math will start taking effect and then it won’t matter if your physical robots.txt file is writeable or not.

[@rishinegi527]

@@RankMath thanx for help but where I can access root folder

[@RankMath]

Hello, You can use FTP or your web hosting file manager to access the root folder of your website. It is the same folder that houses wp-content, wp-includes, wp-admin folder. If you are not sure, please ask your web host or open a ticket on our support forum at rankmath.com/support/
Hope that helps.

[@rishinegi527]

@@RankMath thanx for the help

[@LuniInTheMoon]

i can't edit file robots ! even though it has been opened advened mode

[@RankMath]

Hello, thank you for your comment.
Please refer to this tutorial to troubleshoot the issue: rankmath.com/kb/cant-edit-robots-txt/
Hope that helps and please do not hesitate to let us know if you need our assistance with anything else.

[@Sam-gh6ij]

Okey this is not working anymore with rankmath. you guys need to make a update video on how to edit robots txt file with rankmath.

[@RankMath]

Hello, thank you for your comment.
Here is a more recent video from us on using robots.txt: ruclips.net/video/qRlQ965pGCA/видео.html
Hope this help, and please do not hesitate to let us know if you need our assistance with anything else.

[@garhwali_traveller_]

Contents are locked because a robots.txt file is present in the root folder ... how to fix this

[@RankMath]

Hello, thank you for your comment.
To fix the issue where the contents are locked because a robots.txt file is present in the root folder, you can follow these steps:
1. Identify the Physical Robots.txt File:
Log into your WordPress root directory using cPanel or an FTP application.
Check if there is a file named robots.txt in the root directory.
2. Delete the Physical Robots.txt File:
If a robots.txt file is found, select it and click Delete.
This action will remove the conflicting physical robots.txt file from the root directory.
3. Clear Cache and Test:
After deleting the file, clear your website and server-level cache.
Test your robots.txt URL again to ensure that the issue is resolved.
If you need visual guidance, you can refer to the images provided in the knowledgebase tutorial on fixing common robots.txt issues with Rank Math: rankmath.com/kb/cant-edit-robots-txt/
Hope that helps, and please do not hesitate to let us know if you need our assistance with anything else.

[@zeynow_pubg]

really helpfull

[@RankMath]

Hello and thanks for your comment. You are always welcome. Feel free to get in touch if you have questions.

[@esmusssosein5033]

i want that my Impressum with all my date will be not Crawled by all the bots. Can you give me some a code pls i can pass in Rank math?

[@esmusssosein5033]

or is tis way better?
User-agent: *
Disallow: /impressum/

[@RankMath]

Hello, thank you for your comment.
The robots.txt rule you've provided will block search engine bots from accessing the /impressum/ directory on your website. If this is your intended action, then the rule is correct.
If you encounter any issues with Rank Math, you can always raise your question here rankmath.com/support/ and our support team would be happy to assist you.

[@DigitalInfoHub]

thanks helpfull

[@RankMath]

Hello, and thanks for your comment. Glad that helped. Feel free to get in touch if you have questions.

[@PRATIKSHHA..235]

Thanks mam

[@RankMath]

Hello, and thanks for your comment. Glad that helped. Feel free to get in touch if you have questions.