Custom Robots.txt for Fast Google Indexing in 2024

Поделиться
HTML-код
  • Опубликовано: 27 дек 2024

Комментарии •

  • @HassanNaseerSheikh
    @HassanNaseerSheikh 6 месяцев назад +6

    Superb and Informational. Respect from Pakistan.

  • @bigdude6513
    @bigdude6513 Месяц назад

    I found your channel while searching for a solution to my ‘crawled but not indexed’ issue, and I just have to say-you are an absolute legend! Your explanations are so much clearer and more insightful than even the creators of the SEO plugins themselves. I’ve subscribed, and from now on, you’re my go-to for anything GSC/SEO-related. I am actually thinking of dropping RM in favour of Slimseo, which is a lot more light weight. I would like to hear your thoughts on it if you have any.
    Keep up the fantastic work, and I truly hope your dedication brings you great success!

  • @rekhadebnath3161
    @rekhadebnath3161 4 месяца назад +2

    Thank you sir. Nicely explained. I changed my robots.txt for blogger. I think after viewing your video I am able to understand a bit of the things. Let's hope I can learn more from your videos. Thank you.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  4 месяца назад

      hello Mam, thank you for your comment. I am happy that you are getting good at SEO through my videos. please continue watching more of them as different video shows different concepts but they are all related. If you finish the SEO series, I am sure you will have good knowledge about these tech SEO. Thank you again for taking the time to comment and interact with me . 🙏💐

    • @rekhadebnath3161
      @rekhadebnath3161 4 месяца назад

      @@LetsCreateYourWEBSITE Thankyou beta. I always appreciate good videos and comment on them to show my gratitude but such a long reply ! Thanks again beta. I have already subscribed to your channel as your way of putting things is good and helpful.

  • @afsinsiam2350
    @afsinsiam2350 Месяц назад +1

    I learned a lot today. Thank you for such a great video, brother. Sending you love!

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  18 дней назад

      so happy to hear that, I really like when people use social media for learning. Much love to you too dear ❤❤

  • @socialsparrow8337
    @socialsparrow8337 6 месяцев назад +2

    I watched one of your videos, which explains how to install all the plugins without the ftp method something like define "bla bla". You are absolutely a genius buddy. Am not yet watching this video 😂

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      thanks for your comment, you are free to choose which video you want to watch and which not to watch. Have a great day bud! ❤

  • @nahneensultana7735
    @nahneensultana7735 5 месяцев назад +3

    I really liked the way you instruct. Many things are clear to me now.

  • @DominikosKozakos
    @DominikosKozakos Месяц назад +1

    Regards from Poland :) Nice material

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  Месяц назад

      thanks for your comment dear, I appreciate the wishes from Poland 👊

  • @TRBAHADURPUR
    @TRBAHADURPUR 5 месяцев назад +3

    Tag and category page index is good or bad ?
    I think its goot becouse my all products user can see here and there is more options to choose product which one he like.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  5 месяцев назад +1

      Categories are good, not the tags. it will me GSC reports full of duplicate content error

    • @TRBAHADURPUR
      @TRBAHADURPUR 5 месяцев назад

      @@LetsCreateYourWEBSITE ok sir thanks 👍

  • @AncientDziShop
    @AncientDziShop 4 месяца назад +3

    Very clear and helpful. Thank you for creating this video.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  4 месяца назад

      Thank you for taking your time to comment and interacting with me dear ❤

  • @TravelBoxKannada
    @TravelBoxKannada 6 месяцев назад +2

    How about news website fast indexing? Any spl Robots txt?

  • @Moh-NafEnigma
    @Moh-NafEnigma 6 месяцев назад +4

    Dear, Thanks a lot. I used 2 domain in 1 hosting as a primary and sub domain. for two sites can i should use two robots.txt file? pls apply

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +2

      Hello, yes for subdomain you can create another robots.txt file. No issues !

    • @Moh-NafEnigma
      @Moh-NafEnigma 6 месяцев назад +2

      @@LetsCreateYourWEBSITE Thanks a lot bro.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +1

      @@Moh-NafEnigma You're Most Welcome :)

  • @NeiDroidHD
    @NeiDroidHD 3 месяца назад +2

    Hello, I need urgent help please, I have a problem with Google Adsense and there is an error when tracking a link on my website and it is the search page since Adsense wants to track it but according to sources there is a robot.txt file that blocks it, then if I copy this robot file from your video to my website will the problem be solved?

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  3 месяца назад

      Hello, You must remove the rule from the robots.txt file blocking AdSense file. You can copy the robots.txt if it is a WordPress site. rest I have detailed links in the description to copy robots.txt for different sites. if you want a quick call with me you can book it here letscreateyourwebsite.com/product/technical-seo-consultation/

  • @AreaOfficeFurniture-au
    @AreaOfficeFurniture-au 6 месяцев назад +3

    need some advice if this URL path authorize?target_origin= can be disallow?
    Thanks in advance

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      Yes, absolutely you can disallow any path use * in the end for wildcard

  • @seematalwar9440
    @seematalwar9440 3 месяца назад +3

    Do you suggest I disallow /tag on robots.txt as they show as "Crawled-currently not indexed"

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  3 месяца назад +1

      Hello, yes tags should not be indexed. Watch here the full explanation why ruclips.net/video/_Jzn4ov7b2g/видео.html

  • @sabamanzoor4228
    @sabamanzoor4228 4 месяца назад +2

    very informative like always

  • @TheAiStoryblocks
    @TheAiStoryblocks 3 месяца назад +1

    For last Three years Mine is blogging website but for last three days google search console showing store pages there as well.. any recommendations please how to remove those pages

  • @Moviebeacon_net
    @Moviebeacon_net 4 месяца назад +3

    I have multiple domains in my hosting account, how do i specify individual public_html ?

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  4 месяца назад +1

      Hello, So for all websites there will be a home directory. You need to add individual robots.txt for each 3 website

  • @razvanbarbu1006
    @razvanbarbu1006 4 месяца назад +3

    Hello. I have an issue with indexing pages in Google Search Console.
    Can you help me? I’m working on a recently launched online store, but most
    of the pages are not indexed. This has been going on for a while. I requested reindexing, but
    it says it’s pending. A month has passed, and it still says it’s pending.
    I tried to figure out what the problem is, but I haven’t been able to speed up
    the indexing process. In Google Search Console, it shows that the issue is under the section
    'Discovered - not indexed.' Can you give me some advice? Thank you.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  4 месяца назад

      hello dear, i know it can be frustrating with such search console issues.
      So I already create a video on this topic here ruclips.net/video/tGoBfN5RB_g/видео.html that will surely help you. If you need a personal help over a call, then you can book a call with me here and we can see all teh issues on your website letscreateyourwebsite.com/product/technical-seo-consultation/

  • @kyserares4566
    @kyserares4566 6 месяцев назад +1

    Thank you friend. Very helpful. New info unlocked !

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +1

      You're welcome! happy that i was able to teach you something new and useful, thanks for the comment friend ❤

  • @internetworkss
    @internetworkss 6 месяцев назад +3

    please make a video for the blogger ads.txt file...

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      Very soon brother, Thanks for suggesting the topic. Do you want me to cover anything specific on that video ?

    • @pilusmeow2924
      @pilusmeow2924 6 месяцев назад

      @@LetsCreateYourWEBSITE i need for blogger custom domain to my brother, please make the tutorial

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  2 месяца назад

      Sorry for the delay dear, but the video is here about ads.txt - ruclips.net/video/kU9PnR0uHd8/видео.html

  • @Bzbuyer
    @Bzbuyer 4 дня назад

    Firstly, I would like to thank you for this video. It was incredibly helpful in guiding me to create the robots.txt file. I truly appreciate your effort! Now, I have a few questions:
    I have completely copied the content from my old domain to a new domain. Now both domains are live. What issues could arise from this?
    Is there a way to prevent duplicate content issues between the two domains?

  • @4starofficial20
    @4starofficial20 6 месяцев назад +3

    its really good thank

  • @basitqureshivlogs
    @basitqureshivlogs Месяц назад +1

    How can I add robots txt file in godaddy website builder

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  Месяц назад

      Via file manager.. Login to go daddy account then avvess the files and then in root folder, create the robots.Txt as explained in the video

  • @krishnendumandal3291
    @krishnendumandal3291 6 месяцев назад +6

    Sir, i am a big fan of yours . But I am facing a lot of problems related to Google indexing. My no posts is not indexing in google.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      Hi, thanks for your comment. why they are not getting indexed ? did you checked the reports in the google search console account. Let me know whats the issue you can see there.

    • @krishnendumandal3291
      @krishnendumandal3291 6 месяцев назад

      Sir I don't know what is happening to me ? Can you help me ?

  • @TanyaNusrat
    @TanyaNusrat 2 месяца назад +1

    I have WordPress website, created for blogging purposes. which type of robot.txt file should I add?

  • @mghem2012
    @mghem2012 10 дней назад +1

    I can not type or edit rankmath like you mentioned just can click only and shows cursor but can not type. How I can fix the problem

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  10 дней назад

      Possibly your robots.txt is locked , login via cpanel and then edit as I explained in the video at 14:48 ruclips.net/video/v4gxB3ldfmg/видео.htmlsi=8DWTzmfijRLJ_jY-&t=888

  • @TheLuxuryEmbark
    @TheLuxuryEmbark 5 месяцев назад +1

    Hello how do i do so for godaddy? I have a .txt file but I dont know how to access it to add some of these to disallow

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  5 месяцев назад

      go to the file manage in go daddy to access / manage the robots.txt file.

  • @levelupwithemmanuel
    @levelupwithemmanuel 2 месяца назад +1

    Much value 🥰🥰🥰🥰

  • @We_Satisfying
    @We_Satisfying 2 месяца назад +1

    Can I Disallow/feed, Pages, Search, tag, author, nad more suggestions

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  2 месяца назад +1

      yes of course, check the details in the description to link to the prefect robots.txt files for different platforms

    • @We_Satisfying
      @We_Satisfying 2 месяца назад

      ​@@LetsCreateYourWEBSITE I have added the robots.txt file of WordPress, but you haven't disallowed pages and tags, can you guide me need to disallow page and tag

  • @surjeetshyamal
    @surjeetshyamal Месяц назад +1

    How to NoIndex Paginated Pages?

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  26 дней назад +1

      which plugin for SEO do you use ?

    • @surjeetshyamal
      @surjeetshyamal 25 дней назад

      @@LetsCreateYourWEBSITE Yoest but now i migrate into Rank Math...already all these pages 404 but not excluded from last 7 days.

  • @MunafaMarketing-v1n
    @MunafaMarketing-v1n 6 месяцев назад +3

    Hello,
    our adsense asking
    Your organization's registration document
    Accepted documents include:
    - Certificate of registration or incorporation
    - Extract from commercial register
    - Tax registration certificate
    Do we have to fulfill all these 3 requirements or any of it will be sufficient?
    please guide

  • @5win5win
    @5win5win 3 месяца назад +1

    Can i ask we need to fix all disalow to allow with ROBOT FILE tool? Can we do it without tool? Pls help

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  2 месяца назад

      hello, i did not show any tool, i exaplained the robots.txt method, watch the full video dear

  • @swagnexttuber-boombeach7092
    @swagnexttuber-boombeach7092 6 месяцев назад +2

    In GSC I see 1k 404 not found. All are deleted pages. Should i use removal tool to remove it? Or can I ignore this 404 not found

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      hello, ignore is 1K too much to set up redirect and then work on them. if they are removed from the site then they will go away soon from the GSC. but make sure that they are not in the sitemap and not linked on some page.

  • @designerden
    @designerden 6 месяцев назад +2

    Hello sir. I am your regular video watcher. I have watched you page optimisation with lite speed plugin.
    I am using Woodmart Theme
    While I enable Load JS Deferred, it hides the header menu in desktop and after I refresh many times then it shows header menu.
    In mobile the hamburger menu is not working. I tried your tuning JS but still I feel lost.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +1

      Hello dear, happy to meet with the regular viewer of the channel. hope you like the videos :)
      okay so ls cache is really complicated one, i created whole playlist addressing the ls cache issues, even recently i made a video on the mobile menu and js issues. here you go ruclips.net/p/PLNMMOUlW_zxtSgcq51GI0sTOfWF_ojcY2
      you will find solution of all of issues. Let me know how it went and were you able to find the issues on your site by these videos. Bets wishes dear!

    • @designerden
      @designerden 6 месяцев назад +1

      @@LetsCreateYourWEBSITE thank you for the playlist. 50% Videos are already watched. I will watch all videos of the playlist. This may help me a lot.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +1

      ​@@designerden This will definitely help dear, good luck!

  • @rajanagrasen7811
    @rajanagrasen7811 3 месяца назад +2

    सर कस्टम न्यूज वेबसाइट के लिए क्या robot txt होगा सर ?

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  3 месяца назад +1

      hello dear, default wordpress website wala hi hoga, description me link diya hai use copy kar lijiye

    • @rajanagrasen7811
      @rajanagrasen7811 3 месяца назад +1

      @@LetsCreateYourWEBSITE thanks Bhaiya, Aap Reply Diye, Bahut Log Reply tk nhi dete ❤️❤️

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  3 месяца назад

      @@rajanagrasen7811 bhai logo ki help karne ke liye hi ye channel hai, agar reply nahi karenge to help kaise hogi. kabhi der ho sakti hai par reply to karta hi hu.

  • @TechnoAkm1
    @TechnoAkm1 6 месяцев назад +3

    I want this robot.txt file in pdf please arrange ❤

  • @hallofentertainment8370
    @hallofentertainment8370 2 месяца назад

    My blog post image isn’t displaying on my Facebook and twitter post, how can I solve this issue?

  • @faizaanniaxi1988
    @faizaanniaxi1988 5 месяцев назад +1

    when i enter robot. txt infront of url it doesn't show any robot. txt. what is the readon can you explain

    • @faizaanniaxi1988
      @faizaanniaxi1988 5 месяцев назад

      reason *

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  5 месяцев назад

      hello, the reason is that you don't have robots.txt on your site. please add a robots;txt as explained in the tutorial as it is very important

  • @samansiddique6393
    @samansiddique6393 2 месяца назад

    Very infrmative❤

  • @tnvee
    @tnvee 6 месяцев назад +3

    13:46 " ?m=1" disallow mat karo

  • @BharatAkhabar
    @BharatAkhabar 6 месяцев назад +3

    hello sir I'm watch your video from INDIA and I'm not completely understand your video because my english is week , can you makes videos in HINDI Language. thank you

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      Hello dear, I can understand that it is hard to understand another language sometimes. I have an Hindi channel too where I upload videos www.youtube.com/@FreeWebDevAcademy
      but I would encourage you to learn English as if you are working in IT then English is important and hey it is just an language, the more you practice is the better it will become by time.

    • @Jeyaprakash-s6l
      @Jeyaprakash-s6l 6 месяцев назад

      @@LetsCreateYourWEBSITE tamil ?

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад

      @@Jeyaprakash-s6l english for now, use the tamil subtitles please

  • @hungson7017
    @hungson7017 4 месяца назад

    Thanks

  • @izal9135
    @izal9135 5 месяцев назад

    thanks brother

  • @extraonetzs8381
    @extraonetzs8381 6 месяцев назад +1

    Link

  • @vidhyadharsansare7278
    @vidhyadharsansare7278 6 месяцев назад

    Hello sir, I'm your subscriber from Mumbai, India . I need tool for real time scan Top 10 competitor in my Niche and suggest us important SERPs & NLP keyboard so we can write SEO point of view!

  • @B4Blogging
    @B4Blogging 6 месяцев назад +1

    nice but it not a full detail

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +1

      Thanks for your feedback dear, What did I miss ?
      I was thinking that I covered so many things in 20 minutes 😅

  • @dracarysssx
    @dracarysssx 2 месяца назад

    Are u on fiverr?

  • @SohailAmeen-j7t
    @SohailAmeen-j7t Месяц назад +1

    sir plz explain in hindi/urdu

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  Месяц назад

      hindi channel par ye video aayege jaldi hi, ye hai doosra hindi/urdu channel bhai www.youtube.com/@FreeWebDevAcademy

  • @OrcaMart
    @OrcaMart 5 месяцев назад +1

    Seems like you also need a speech coach. Its seems really difficult for you to say robots.txt and you struggle with many other words in your speech.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  5 месяцев назад

      thanks for your suggestion!

    • @seematalwar9440
      @seematalwar9440 3 месяца назад

      @@LetsCreateYourWEBSITE You are doing great. Just speak a little slower.

  • @almaromero8775
    @almaromero8775 Месяц назад

    The title looks useful but your pronunciation is not good, It is complicated to understand you.

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  Месяц назад +2

      I appreciate your feedback! Many people find my pronunciation clear, but I understand it might not be for everyone. If you need, subtitles are available to help you follow along more easily.

  • @rudrakumar9641
    @rudrakumar9641 6 месяцев назад

    Bro all good, but talk slowly and clearly, you talk like drunk man, that irritated, talk properly and pronounce each word clearly, dont eat words

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  6 месяцев назад +2

      that was rude feedback, you need to learn how to give the feedback. usually, when people have some feedback I take it very positively, but yours really disappoints me.

    • @MarkedMoneyTech
      @MarkedMoneyTech 6 месяцев назад

      @@LetsCreateYourWEBSITE Agreed. Manners matter to be civil. I love learning from you! When I was a child I watched that movie Short Circuit and you sound like Ben. He was brilliant too!

  • @Muhammad_Asad_7
    @Muhammad_Asad_7 6 месяцев назад +1

    How can i contact with you

  • @5win5win
    @5win5win 3 месяца назад +2

    Can i ask we need to fix all disalow to allow with ROBOT FILE tool? Can we do it without tool? Pls help

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  2 месяца назад

      hello, i did not show any tool, i exaplained the robots.txt method, watch the full video dear

  • @5win5win
    @5win5win 3 месяца назад

    Can i ask we need to fix all disalow to allow with ROBOT FILE tool? Can we do it without tool? Pls help

    • @LetsCreateYourWEBSITE
      @LetsCreateYourWEBSITE  2 месяца назад

      hello, i did not show any tool, i exaplained the robots.txt method, watch the full video dear