Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 11 of 11

Thread: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

  1. #1
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Hi, I am trying to use this code which works fine in Eclipse but when I try to execute the program from the command line it just hangs forever and can never accesses the URL.

    I am trying to run this program from my home computer on a localhost and I have no idea why this isn't working. I read online and some articles suggested using a proxy but I am not behind a proxy or anything. And again this code works fine when I run it in Eclipse so I guess Eclipse is auto-configuring settings to get this code to work.



    URL link = new URL("http://www.yahoo.com");
    System.out.println("got to this line 1");
    BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
    System.out.println("got to this line 2");
    //InputStream in = link.openStream();
    String inputLine = "";
    int count = 0;
    while ((inputLine = in.readLine()) != null)
    {
    System.out.println("line: " + inputLine);
    site = site + "\n" + inputLine;
    }
    in.close();
    System.out.println("got to this line 3");



    It gets to the print statement inside of the while loop then all of a sudden the connection gets reset and it stops printing the file. It always stops at the same place.
    Last edited by rosco544; September 16th, 2011 at 05:49 PM.


  2. #2
    Administrator copeg's Avatar
    Join Date
    Oct 2009
    Location
    US
    Posts
    5,320
    Thanks
    181
    Thanked 833 Times in 772 Posts
    Blog Entries
    5

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    'it just hangs forever
    Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

    This thread has been cross posted here:

    http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

    Although cross posting is allowed, for everyone's benefit, please read:

    Java Programming Forums Cross Posting Rules

    The Problems With Cross Posting


  3. #3
    Super Moderator Sean4u's Avatar
    Join Date
    Jul 2011
    Location
    Tavistock, UK
    Posts
    637
    Thanks
    5
    Thanked 103 Times in 93 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Post your full source, cut-down so that it demonstrates the problem, but can still be copy-paste-compile-run if someone feels like it. Change the body of your while loop to print inputLine so you can see progress of the stream. You know you can specify a file: URL and test your code against a file on your PC? Like:
    URL link = new URL("file:///home/rosco/dearsanta.text");

  4. #4
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Quote Originally Posted by copeg View Post
    Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

    This thread has been cross posted here:

    http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

    Although cross posting is allowed, for everyone's benefit, please read:

    Java Programming Forums Cross Posting Rules

    The Problems With Cross Posting

    Sorry about the cross-posting I didn't realize the two forums had the same viewers. I edited the original post to include the full source and if a solution is found I will post it and let everyone know in both threads.
    Last edited by rosco544; September 16th, 2011 at 04:39 PM.

  5. #5
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Quote Originally Posted by copeg View Post
    Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

    This thread has been cross posted here:

    http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

    Although cross posting is allowed, for everyone's benefit, please read:

    Java Programming Forums Cross Posting Rules

    The Problems With Cross Posting

    I edited the code to look like this in my program and re-ran it from the command line.

    URL link = new URL("http://www.yahoo.com");
    System.out.println("got to this line 1");
    BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
    System.out.println("got to this line 2");
    //InputStream in = link.openStream();
    String inputLine = "";
    int count = 0;
    while ((inputLine = in.readLine()) != null)
    {
    site = site + "\n" + inputLine;
    }
    in.close();
    System.out.println("got to this line 3");

    It gets to "got to this line 2" but not "got to this line 3".
    Last edited by rosco544; September 16th, 2011 at 04:46 PM.

  6. #6
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Quote Originally Posted by copeg View Post
    Where does it hang? Do you get exceptions? Did you add print lines to see where it might hang?

    This thread has been cross posted here:

    http://www.java-forums.org/networking/48616-open-url-read-line-line-works-eclipse-but-not-command-line.html

    Although cross posting is allowed, for everyone's benefit, please read:

    Java Programming Forums Cross Posting Rules

    The Problems With Cross Posting

    I edited the code to look like this in my program and re-ran it from the command line.

    URL link = new URL("http://www.yahoo.com");
    System.out.println("got to this line 1");
    BufferedReader in = new BufferedReader(new InputStreamReader(link.openStream()));
    System.out.println("got to this line 2");
    //InputStream in = link.openStream();
    String inputLine = "";
    int count = 0;
    while ((inputLine = in.readLine()) != null)
    {
    System.out.println("line: " + inputLine);
    site = site + "\n" + inputLine;
    }
    in.close();
    System.out.println("got to this line 3");

    It starts printing the website but never gets to "got to this line 3".

  7. #7
    Super Moderator Sean4u's Avatar
    Join Date
    Jul 2011
    Location
    Tavistock, UK
    Posts
    637
    Thanks
    5
    Thanked 103 Times in 93 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    There's nothing wrong with the fragment of code you've posted. Create a small, compilable... (Kevin? I must make one of them fancy footnote things with all the handy links in). Write the smallest possible java program with a main method so that it can be run from the command line that demonstrates the problem. While you're doing that, you'll probably spot for yourself where you've gone wrong.

  8. #8
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Quote Originally Posted by Sean4u View Post
    There's nothing wrong with the fragment of code you've posted. Create a small, compilable... (Kevin? I must make one of them fancy footnote things with all the handy links in). Write the smallest possible java program with a main method so that it can be run from the command line that demonstrates the problem. While you're doing that, you'll probably spot for yourself where you've gone wrong.
    Well I don't think I've done anything wrong because the entire project/program works flawlessly in Eclipse. I think it has something do with the Proxy settings that the JVM uses. I just don't know anything about proxies to be able to configure that.

  9. #9
    Super Moderator Sean4u's Avatar
    Join Date
    Jul 2011
    Location
    Tavistock, UK
    Posts
    637
    Thanks
    5
    Thanked 103 Times in 93 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    the entire project/program works flawlessly in Eclipse
    Great - all you need to do is to distribute your application with the Eclipse Virtual Machine.

    You'd know if your JVM was using proxy settings - you'd have specified them on the command line. As far as I know, the JVM won't silently go looking for a web proxy if you don't tell it to. Are you behind a web proxy on your network?

  10. #10
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    No I'm just on a standard home network. I do have an apache server running on the localhost if that matters, so I can use mysql and php and stuff. But I don't think that has anything to do with it.

    And is it possible to distribute a console application using a jar. I thought jar files couldn't start the console in Windows? The reason why I want to be able to run it from the command is because I want to then create a script to be able to run this automatically at intervals, and eventually I want it to be able to run on a Web server at intervals.
    Last edited by rosco544; September 16th, 2011 at 05:29 PM.

  11. #11
    Junior Member
    Join Date
    Sep 2011
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Open a URL and read it line by line (Works in Eclipse but not from Command Line)

    Finally solved it with this code found here: Java HttpURLConnection (how to open and read a url with Java) | devdaily.com

    public String websiteToString(String desiredUrl) throws Exception
    {
    URL url = null;
    BufferedReader reader = null;
    StringBuilder stringBuilder;

    try
    {
    // create the HttpURLConnection
    url = new URL(desiredUrl);
    HttpURLConnection connection = (HttpURLConnection) url.openConnection();

    // just want to do an HTTP GET here
    connection.setRequestMethod("GET");

    // uncomment this if you want to write output to this url
    //connection.setDoOutput(true);

    // give it 15 seconds to respond
    connection.setReadTimeout(15*1000);
    connection.connect();

    // read the output from the server
    reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
    stringBuilder = new StringBuilder();

    String line = null;
    while ((line = reader.readLine()) != null)
    {
    stringBuilder.append(line + "\n");
    }

    String site = stringBuilder.toString();
    site = site.replaceAll("<html([^<]*)>", "<html>"); // replace all <html xml:lang="en" lang="en" xmlnsml="xml"> with <html>
    // this is important because xpath can't parse an expression with <html> tags with junk in it

    return site;
    }
    catch (Exception e)
    {
    e.printStackTrace();
    throw e;
    }
    finally
    {
    // close the reader; this can throw an exception too, so
    // wrap it in another try/catch block.
    if (reader != null)
    {
    try
    {
    reader.close();
    }
    catch (IOException ioe)
    {
    ioe.printStackTrace();
    }
    }
    }
    }

Similar Threads

  1. How to Send command line arguments in Eclipse?
    By JavaPF in forum Java JDK & IDE Tutorials
    Replies: 0
    Last Post: April 23rd, 2009, 11:37 AM
  2. Reading a file line by line using the Scanner class
    By JavaPF in forum File Input/Output Tutorials
    Replies: 0
    Last Post: April 17th, 2009, 07:34 AM
  3. Reading a file line by line using the Scanner class
    By JavaPF in forum Java Code Snippets and Tutorials
    Replies: 0
    Last Post: April 17th, 2009, 07:34 AM
  4. How to Read a file line by line using BufferedReader?
    By JavaPF in forum File Input/Output Tutorials
    Replies: 0
    Last Post: May 19th, 2008, 06:32 AM
  5. How to Read a file line by line using BufferedReader?
    By JavaPF in forum Java Code Snippets and Tutorials
    Replies: 0
    Last Post: May 19th, 2008, 06:32 AM

Tags for this Thread