[prev in list] [next in list] [prev in thread] [next in thread] 

List:       htmlunit-user
Subject:    Re: [Htmlunit-user] htmlunit to scrape angularjs
From:       "matt () raibledesigns ! com" <matt () raibledesigns ! com>
Date:       2013-02-05 18:21:30
Message-ID: 1360088489790-30075.post () n7 ! nabble ! com
[Download RAW message or body]

The following seems to work. It does throw some exception, but it seems that
Angular does process the page.

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.SilentCssErrorHandler;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

import java.io.IOException;
import java.lang.String;

public class JavaScriptProcessor {

    public static void main(String[] args) {
        String url = "http://angularjs.org/";
        WebClient webClient = new WebClient(BrowserVersion.FIREFOX_10);
        webClient.getOptions().setCssEnabled(true);
        webClient.setCssErrorHandler(new SilentCssErrorHandler());
        webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setRedirectEnabled(false);
        webClient.getOptions().setAppletEnabled(false);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setPopupBlockerEnabled(true);
        webClient.getOptions().setTimeout(5000);
        webClient.getOptions().setPrintContentOnFailingStatusCode(false);

        HtmlPage myPage = null;
        try {
             myPage = (HtmlPage) webClient.getPage(url);
            // important!  Give the headless browser enough time to execute
JavaScript
               // The exact time to wait may depend on your application.
            webClient.waitForBackgroundJavaScript(2000);
        } catch (IOException e) {
            e.printStackTrace();
        }

        System.out.println(myPage.asXml());
    }
}





--
View this message in context: \
http://htmlunit.10904.n7.nabble.com/htmlunit-to-scrape-angularjs-tp29931p30075.html \
Sent from the HtmlUnit - General mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/htmlunit-user


[prev in list] [next in list] [prev in thread] [next in thread]