Search:

Another thing which can be a MAJOR factor is the DNS lookup but from a programming/code perspective there's probably not much you can do without getting into a lower level.

// Json

In what way is your crawl slow, where exactly does the code stop for a while?

Another reason for the url fetch to be slow could be the fact that the server you are connecting to is very slow in...