.Google's Gary Illyes uncovers Googlebot collects web links as opposed to "observing" them, opposing formal records.Googlebot picks up links instead of following all of them in real-time.Gary Illyes discussed this info in Google.com's 'Explore Off The Report' podcast.Google.com's documentation contradicts the real spider habits.