Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 2 of 2

Thread: How to download protected web page using JAVA

  1. #1
    Junior Member
    Join Date
    Sep 2012
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default How to download protected web page using JAVA

    We have a task, to design a class which can download source of any web page. But when I try to test my code and fetch page like http://anidb.net/perl-bin/animedb.pl?show=main nothing is working.
    Standard code like this fails:
    import java.net.*;
    import java.io.*;
     
    public class URLReader {
        public static void main(String[] args) throws Exception {
            URL link = new URL("http://www.anidb.net/");
            BufferedReader in = new BufferedReader(
            new InputStreamReader(link.openStream()));
     
            String inputLine;
            while ((inputLine = in.readLine()) != null)
                System.out.println(inputLine);
            in.close();
        }
    }
    Here is the result I got: wq>"5U=?k}~bd`?lz>"?j׉Ry}K\ c_DL٪_ Mm__0C_asS >dC0s_yA%A@,4xɃ?
    I have tried everything: cookies, header files but nothing seems to work. If you have some hint for me, I will appreciate it. Ive been thinking about this problem for to weeks. Thanks.


  2. #2
    Super Moderator Norm's Avatar
    Join Date
    May 2010
    Location
    SW Missouri
    Posts
    20,394
    Thanks
    49
    Thanked 2,212 Times in 2,185 Posts

    Default Re: How to download protected web page using JAVA

    Try using a a different class that will return the HTTP header etc instead of just the contents.
    Try your code with another website that returns an html page. Its possible that the site you are going ti does not return an html page.

    When I read from that site I get this:
    hdr> Date: Sat, 22 Sep 2012 12:22:41 GMT
    hdr> Server: Apache
    hdr> Cache-control: no-cache
    hdr> Pragma: no-cache
    hdr> Content-Type: text/html; charset=UTF-8
    hdr> Expires: Sat, 22 Sep 2012 12:22:41 GMT
    hdr> Set-Cookie: adbuin=1348316562-wmNq; path=/; expires=Tue, 20-Sep-2022 12:22:42 GMT
    hdr> Vary: Accept-Encoding
    hdr> Content-Encoding: gzip
    hdr> Keep-Alive: timeout=4, max=50
    hdr> Connection: Keep-Alive
    hdr> Transfer-Encoding: chunked



    Also posted at http://www.java-forums.org/advanced-...sing-java.html
    Last edited by Norm; September 22nd, 2012 at 07:30 AM.
    If you don't understand my answer, don't ignore it, ask a question.

Similar Threads

  1. JAVA DOWNLOAD
    By sephskie in forum Java Theory & Questions
    Replies: 1
    Last Post: January 9th, 2012, 07:38 PM
  2. need help with java API download from a website
    By alpha.rome in forum Java Theory & Questions
    Replies: 4
    Last Post: September 2nd, 2011, 04:11 AM
  3. Using Protected Methods, in a Java Library
    By WACman in forum Object Oriented Programming
    Replies: 2
    Last Post: March 10th, 2011, 04:42 PM
  4. Download a file by Bluetooth in Java SE
    By danielpereira in forum Java Theory & Questions
    Replies: 0
    Last Post: October 6th, 2010, 02:37 PM
  5. Replies: 2
    Last Post: June 13th, 2009, 01:44 AM