Implementing a multithreaded crawler