Crawl Budget
What is the Crawl budget? – Definition
Crawl budget determines how much resources – time and computing power – Google robots spend on indexing a given site. This reflects the number of pages added to the search engine index and the indexing frequency. The crawl budget is influenced, among others, by:
- crawl rate limit – the number of pages crawled by search engine robots in a short time,
- crawl health – low server response time, correct response codes, loading speed,
- crawl demand – the demand for (re)crawling, which depends, among others, on the frequency of content updates, web traffic,
- page size, use of JavaScript code – pages with JS take more indexing budget.
Where Can I Check the Crawl Budget of My Website?
In Google Search Console, go to the “Legacy tools and reports” section and choose “Crawl stats”. You will find there 3 charts:
- Pages crawled per day,
- Kilobytes downloaded per day,
- Time spent downloading a page (in milliseconds).
These are data from the last 90 days. Based on them, you can specify how the crawl budget has changed during this period.
