HardDesign: Web Crawler

How does a web crawler handle redirect chains (301, 302)?

Tests your understanding of this concept.

Answer Options

AFollow all redirects infinitely
BFollow up to N redirects (e.g., 5), store mapping from original URL to final URL
CNever follow redirects
DTreat redirected URL as new URL, add to frontier

Want to see the correct answer?

Get the answer with a detailed explanation, plus practice 22+ more Design: Web Crawler questions with adaptive quizzes and timed interviews.

See the Answer on Guru Sishya →

This question is from the Design: Web Crawler topic (System Design Cases).

More Design: Web Crawler Questions