Alors la "politesse du crawl" c'est une traduction littérale, on pourrait parler de "gentillesse" ou de "modération" aussi : "Crawlers consume resources on visited systems and often visit sites unprompted. Issues of schedule, load, and "politeness" come into play when large collections of pages are accessed. Mechanisms exist for public sites not wishing to be crawled to make this known to the crawling agent. For example, including a robots.txt file can request bots to index only parts of a website, or nothing at all. " [wikipedia Web_crawler] . En gros il s'agit d'éviter de tabasser des petits sites ou bien des serveurs qui servent plein de web hosts.
Alors la "politesse du crawl" c'est une traduction littérale, on pourrait parler de "gentillesse" ou de "modération" aussi : "Crawlers consume resources on visited systems and often visit sites unprompted. Issues of schedule, load, and "politeness" come into play when large collections of pages are accessed. Mechanisms exist for public sites not wishing to be crawled to make this known to the crawling agent. For example, including a robots.txt file can request bots to index only parts of a website, or nothing at all. " [wikipedia Web_crawler] . En gros il s'agit d'éviter de tabasser des petits sites ou bien des serveurs qui servent plein de web hosts.
Intéressant, merci pour ton retour sur ce sujet @Guillaume 😁
tu as pu comprendre pourquoi les liens des interfaces sont pondérés négativement?
𝓅𝓇o𝓂o𝓈𝓂 🌟
😎