Custom Robots.txt for Fast Google Indexing in 2024
HTML-код
- Опубликовано: 4 июн 2024
- Learn how to create a perfect robots.txt for any kind of website. Solve errors of robots.txt not found and search engines are blocked by robots.txt.
A robots.txt file is a set of instructions for web crawlers, often used by search engines like Google. It acts like a guide for these crawlers, telling them which parts of your website they can and cannot access.
Download for Wordpress - letscreateyourwebsite.com/rob...
Download for Woocommerce - letscreateyourwebsite.com/rob...
Download for Blogger - letscreateyourwebsite.com/rob...
Download for wix- letscreateyourwebsite.com/rob...
Download for Shopify - letscreateyourwebsite.com/rob...
A correct robots.txt file helps your site to index better and faster on google and your site will have less errors in search console
-------------------------------------------------------------
More details about links and downloading files are coming soon
---------------------------------------------------------------
Today we gonna learn about following things ;
1. What is robots.txt and why should you use a robots.txt file
2. How to create a perfect robots.txt and we will see how to write instructions for search engine to follow.
3. We will talk about the best robots.txt for different platforms like wordpress,blogger,shopify, wix webflow and custom website.
4. How to upload it to your site and submit to google
5. Common robots.Txt errors like
Robots.txt not found, search engines are blocked
Wordpress, wix, shopify, woocommerce, webflow. etc
Join this channel to get access to perks:
/ @letscreateyourwebsite
Superb and Informational. Respect from Pakistan.
Thanks for the comment dear ❤
I really liked the way you instruct. Many things are clear to me now.
That a really nice compliment.. Happy to teach you something!
Thank you friend. Very helpful. New info unlocked !
You're welcome! happy that i was able to teach you something new and useful, thanks for the comment friend ❤
I watched one of your videos, which explains how to install all the plugins without the ftp method something like define "bla bla". You are absolutely a genius buddy. Am not yet watching this video 😂
thanks for your comment, you are free to choose which video you want to watch and which not to watch. Have a great day bud! ❤
its really good thank
I'm glad you like it, thanks for your comment ❤
Sir, i am a big fan of yours . But I am facing a lot of problems related to Google indexing. My no posts is not indexing in google.
Hi, thanks for your comment. why they are not getting indexed ? did you checked the reports in the google search console account. Let me know whats the issue you can see there.
Sir I don't know what is happening to me ? Can you help me ?
thanks brother
Any time brother ❤
Tag and category page index is good or bad ?
I think its goot becouse my all products user can see here and there is more options to choose product which one he like.
Categories are good, not the tags. it will me GSC reports full of duplicate content error
@@LetsCreateYourWEBSITE ok sir thanks 👍
Dear, Thanks a lot. I used 2 domain in 1 hosting as a primary and sub domain. for two sites can i should use two robots.txt file? pls apply
Hello, yes for subdomain you can create another robots.txt file. No issues !
@@LetsCreateYourWEBSITE Thanks a lot bro.
@@Moh-NafEnigma You're Most Welcome :)
How about news website fast indexing? Any spl Robots txt?
hello sir I'm watch your video from INDIA and I'm not completely understand your video because my english is week , can you makes videos in HINDI Language. thank you
Hello dear, I can understand that it is hard to understand another language sometimes. I have an Hindi channel too where I upload videos www.youtube.com/@FreeWebDevAcademy
but I would encourage you to learn English as if you are working in IT then English is important and hey it is just an language, the more you practice is the better it will become by time.
@@LetsCreateYourWEBSITE tamil ?
@@user-xf9oh4be5u english for now, use the tamil subtitles please
need some advice if this URL path authorize?target_origin= can be disallow?
Thanks in advance
Yes, absolutely you can disallow any path use * in the end for wildcard
Hello,
our adsense asking
Your organization's registration document
Accepted documents include:
- Certificate of registration or incorporation
- Extract from commercial register
- Tax registration certificate
Do we have to fulfill all these 3 requirements or any of it will be sufficient?
please guide
Hello, any of these is fine to get approved. 😇
please make a video for the blogger ads.txt file...
Very soon brother, Thanks for suggesting the topic. Do you want me to cover anything specific on that video ?
@@LetsCreateYourWEBSITE i need for blogger custom domain to my brother, please make the tutorial
Hello sir. I am your regular video watcher. I have watched you page optimisation with lite speed plugin.
I am using Woodmart Theme
While I enable Load JS Deferred, it hides the header menu in desktop and after I refresh many times then it shows header menu.
In mobile the hamburger menu is not working. I tried your tuning JS but still I feel lost.
Hello dear, happy to meet with the regular viewer of the channel. hope you like the videos :)
okay so ls cache is really complicated one, i created whole playlist addressing the ls cache issues, even recently i made a video on the mobile menu and js issues. here you go ruclips.net/p/PLNMMOUlW_zxtSgcq51GI0sTOfWF_ojcY2
you will find solution of all of issues. Let me know how it went and were you able to find the issues on your site by these videos. Bets wishes dear!
@@LetsCreateYourWEBSITE thank you for the playlist. 50% Videos are already watched. I will watch all videos of the playlist. This may help me a lot.
@@designerden This will definitely help dear, good luck!
In GSC I see 1k 404 not found. All are deleted pages. Should i use removal tool to remove it? Or can I ignore this 404 not found
hello, ignore is 1K too much to set up redirect and then work on them. if they are removed from the site then they will go away soon from the GSC. but make sure that they are not in the sitemap and not linked on some page.
Hello sir, I'm your subscriber from Mumbai, India . I need tool for real time scan Top 10 competitor in my Niche and suggest us important SERPs & NLP keyboard so we can write SEO point of view!
Hello how do i do so for godaddy? I have a .txt file but I dont know how to access it to add some of these to disallow
go to the file manage in go daddy to access / manage the robots.txt file.
when i enter robot. txt infront of url it doesn't show any robot. txt. what is the readon can you explain
reason *
hello, the reason is that you don't have robots.txt on your site. please add a robots;txt as explained in the tutorial as it is very important
I want this robot.txt file in pdf please arrange ❤
Done, check the description :)
13:46 " ?m=1" disallow mat karo
why not, can you please explain ?
nice but it not a full detail
Thanks for your feedback dear, What did I miss ?
I was thinking that I covered so many things in 20 minutes 😅
Link
Coming soon. 🔜❤️
Seems like you also need a speech coach. Its seems really difficult for you to say robots.txt and you struggle with many other words in your speech.
thanks for your suggestion!
Bro all good, but talk slowly and clearly, you talk like drunk man, that irritated, talk properly and pronounce each word clearly, dont eat words
that was rude feedback, you need to learn how to give the feedback. usually, when people have some feedback I take it very positively, but yours really disappoints me.
@@LetsCreateYourWEBSITE Agreed. Manners matter to be civil. I love learning from you! When I was a child I watched that movie Short Circuit and you sound like Ben. He was brilliant too!
How can i contact with you
send me an email at programwarrior@gmail.com