Adding cache support to the link crawler