Welcome to the Java Programming Forums


The professional, friendly Java community. 21,500 members and growing!


The Java Programming Forums are a community of Java programmers from all around the World. Our members have a wide range of skills and they all have one thing in common: A passion to learn and code Java. We invite beginner Java programmers right through to Java professionals to post here and share your knowledge. Become a part of the community, help others, expand your knowledge of Java and enjoy talking with like minded people. Registration is quick and best of all free. We look forward to meeting you.


>> REGISTER NOW TO START POSTING


Members have full access to the forums. Advertisements are removed for registered users.

Results 1 to 6 of 6

Thread: How do we navigate to a page with the anchor link in Web Crawler program?

  1. #1
    Junior Member
    Join Date
    Jul 2009
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default How do we navigate to a page with the anchor link in Web Crawler program?

    Hi,
    I am writing a web crawler and I am stuck again with a problem.
    How do we navigate to a page with the anchor link:

     <a id="dlstDepartments__ctl2_DepartmentsLink" onclick="return UpdateFrames('5');" 
              href="javascript:__doPostBack('dlstDepartments$_ctl2$DepartmentsLink','')">Beverages</a>
     
     
    and here is the Javascript code being called:
     
     
    function UpdateFrames(DID)
        {
            top.navbar.location.href="Aisles.aspx?ID="+DID;
            location.href="IconAisles.aspx?ID="+DID;
            return false;
        }
     
     
        function __doPostBack(eventTarget, eventArgument) {
            var theform;
            if (window.navigator.appName.toLowerCase().indexOf("microsoft") > -1) {
                theform = document.frmPageForm;
            }
            else {
                theform = document.forms["frmPageForm"];
            }
            theform.__EVENTTARGET.value = eventTarget.split("$").join(":");
            theform.__EVENTARGUMENT.value = eventArgument;
            theform.submit();
        }

    Any help would be highly appreciated
    Thanking you


  2. #2
    Super Moderator Json's Avatar
    Join Date
    Jul 2009
    Location
    Warrington, United Kingdom
    Posts
    1,274
    My Mood
    Happy
    Thanks
    70
    Thanked 156 Times in 152 Posts

    Default Re: Web Crawler

    This is where you need to run the javascript I guess or try to figure out a way to parse out the correct url with the information you have there, but if this is a general webcrawler I'd just leave it and focus on proper links.

    // Json

  3. #3
    Junior Member
    Join Date
    Jul 2009
    Posts
    7
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Web Crawler

    Thanks for your response.
    The website that I want to crawl has most of the links of this sort. So I can't ignore these links.
    How to run JavaScript inside the java program? Any idea? Or how can I achieve the same that is being done in the JavaScript code inside the java program?


    Thanking you

  4. #4
    Super Moderator Json's Avatar
    Join Date
    Jul 2009
    Location
    Warrington, United Kingdom
    Posts
    1,274
    My Mood
    Happy
    Thanks
    70
    Thanked 156 Times in 152 Posts

    Default Re: Web Crawler

    Perhaps by using Rhino Javascript engine. Sorry I can't give you any examples though because I've never tried it.

    // Json

  5. #5
    Junior Member
    Join Date
    Sep 2009
    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Web Crawler

    I have never used and tried it.. Sorry

  6. #6
    Senile Half-Wit Freaky Chris's Avatar
    Join Date
    Mar 2009
    Posts
    834
    My Mood
    Cynical
    Thanks
    7
    Thanked 105 Times in 90 Posts

    Default Re: Web Crawler

    Which frame do you want, you can always just parse that info out....and then go to it as normal pretending its just a web URL with aspx?... with some variables on the end which you already know.

    Chris