Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 8 of 8

Thread: url input stream returning blank

  1. #1
    Junior Member
    Join Date
    May 2010
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default url input stream returning blank

    hi all,

    i'm trying to extract a particular website into a html text file but i've tried many ways and so far i'm getting a blank website. i can open the website normally on a browser and its source code is viewable within the browser, but when i do a inputstream on it, i get nothing but nulls.

    it's basically a linkbee redirection website e.g. http:// linkbee.com/FTHNN. i realize it's a redirection website but i should still be able to extract the html code of this website. the redirection occurs only after clicking a button, there's no timer, and i don't require actually following the redirection, i simply need to the url that it redirects to. in this case it can be found in line 81 input type='hidden' id='urlholder' value='http://www.megaupload.com/?d=VBS1H7ZK'.

    my source code is -
    	URL u = new URL(str);
    	URLConnection uconn = u.openConnection();
    	uconn.connect();
    	InputStream is = uconn.getInputStream();
    	br3 = new BufferedReader(new InputStreamReader(is));
    	while(true)   br3.readLine();
    it works for every website i've tried on but this.

    thanks for any help
    Last edited by Json; June 2nd, 2010 at 02:38 AM. Reason: Please use code tags.


  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    Eastern Florida
    Posts
    18,560
    Thanks
    45
    Thanked 1,954 Times in 1,928 Posts

    Default Re: url input stream returning blank

    it works for every website i've tried on but this.
    Which website is "this"??
    What do you read from that website?

    When i tried reading from http:// linkbee.com/FTHNN (space after //) I get 1 byte of 0xFF.
    Last edited by Norm; June 1st, 2010 at 11:46 AM.

  3. #3
    Junior Member
    Join Date
    May 2010
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: url input stream returning blank

    Quote Originally Posted by Norm View Post
    Which website is "this"??
    What do you read from that website?

    When i tried reading from http:// linkbee.com/FTHNN (space after //) I get 1 byte of 0xFF.
    i'm sorry, 'this' website is http:// linkbee.com/FTHNN without the space after //. I can't type the full url because it turns into a hyperlink like this Linkbee redirection, please wait...

  4. #4
    Senile Half-Wit Freaky Chris's Avatar
    Join Date
    Mar 2009
    Location
    Wales, Bangor & England, Warwickshire
    Posts
    820
    My Mood
    Cynical
    Thanks
    7
    Thanked 104 Times in 90 Posts

    Default Re: url input stream returning blank

    Are you after the HTML of the redirect site, or the HTML of the site you are redirected to?
    chris[at]javaprogrammingforums[dot]com

    Prifysgol Bangor University, North Wales

  5. #5
    Super Moderator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,099
    Thanks
    169
    Thanked 779 Times in 725 Posts
    Blog Entries
    5

    Default Re: url input stream returning blank

    Sometimes websites require a user agent or it will return unexpected results. Try setting it with something like the following
    URLConnection conn = url.openConnection();
    conn.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)");
    There is usually a reason for a server to be configured in such a way. As a result I'd recommend checking to be sure you are not violating any policies in doing so.
    Last edited by copeg; June 2nd, 2010 at 08:49 AM.

  6. #6
    Junior Member
    Join Date
    May 2010
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: url input stream returning blank

    Quote Originally Posted by Freaky Chris
    Are you after the HTML of the redirect site, or the HTML of the site you are redirected to?
    i'm after the HTML of the site i am redirected to

    Quote Originally Posted by copeg
    There is usually a reason for a server to be configured in such a way. As a result I'd recommend checking to be sure you are not violating any policies in doing so.
    hmm true, i should figure that out first, thanks.

  7. #7
    Super Moderator Json's Avatar
    Join Date
    Jul 2009
    Location
    Warrington, United Kingdom
    Posts
    1,274
    My Mood
    Happy
    Thanks
    70
    Thanked 156 Times in 152 Posts

    Default Re: url input stream returning blank

    Btw, the default User-Agent when using Java URL.openConnection() is Java/version

    Oh and setting the user agent as suggested by copeg works just dandy for the above mentioned url.

    // Json
    Last edited by Json; June 2nd, 2010 at 02:45 PM.

  8. #8
    Junior Member
    Join Date
    May 2010
    Posts
    4
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: url input stream returning blank

    Quote Originally Posted by Json View Post
    Btw, the default User-Agent when using Java URL.openConnection() is Java/version

    Oh and setting the user agent as suggested by copeg works just dandy for the above mentioned url.

    // Json
    interesting, this explains everything! thank you very much

Similar Threads

  1. Making a little Lua script 'generator', JFrame showing up blank.
    By scribbleno1 in forum What's Wrong With My Code?
    Replies: 5
    Last Post: May 2nd, 2010, 07:00 PM
  2. [SOLVED] allow a new input, dicarding the last mismatch input without terminating the program
    By voltaire in forum What's Wrong With My Code?
    Replies: 2
    Last Post: April 9th, 2010, 04:44 AM
  3. returning a 2D array
    By straw in forum What's Wrong With My Code?
    Replies: 3
    Last Post: March 11th, 2010, 03:30 AM
  4. Client input Stream
    By gisler in forum File I/O & Other I/O Streams
    Replies: 1
    Last Post: December 19th, 2009, 08:30 PM
  5. throwing and returning
    By chronoz13 in forum Exceptions
    Replies: 6
    Last Post: October 25th, 2009, 12:00 AM