this code generally works (accesses thousands of urls successfully), but once in a while i get a null pointer exception on particular HttpURLConnections, and its always the same urls
for conciseness...