Implementing Crawler4j with Selenium in Java doesn`t work -

-

i'm trying use crawler4j simultaneous selenium website testing. after webpage crawled, selenium should start simultaneous test parameters got crawler. url should open or id's of search fields example. if use crawler4j alone works fine , can extract information need. if start test selenium predefined parameters url , id's works should. when put same code selenium crawler code exception. guess it`s maybe threat problem? nice if give me hint or me:

exception in thread "crawler 1" java.lang.noclassdeffounderror: com/google/common/base/function     @ mypackage.com.selenium.init(selenium.java:21)     @ mypackage.com.mycrawler.visit(mycrawler.java:57)     @ edu.uci.ics.crawler4j.crawler.webcrawler.processpage(webcrawler.java:351)     @ edu.uci.ics.crawler4j.crawler.webcrawler.run(webcrawler.java:220)     @ java.lang.thread.run(thread.java:722) caused by: java.lang.classnotfoundexception: com.google.common.base.function     @ java.net.urlclassloader$1.run(urlclassloader.java:366)     @ java.net.urlclassloader$1.run(urlclassloader.java:355)     @ java.security.accesscontroller.doprivileged(native method)     @ java.net.urlclassloader.findclass(urlclassloader.java:354)     @ java.lang.classloader.loadclass(classloader.java:423)     @ sun.misc.launcher$appclassloader.loadclass(launcher.java:308)     @ java.lang.classloader.loadclass(classloader.java:356)     ... 5 more

here code crawler:

public class mycrawler extends webcrawler {       private final static pattern filters = pattern.compile(".*(\\.(css|js|bmp|gif|jpe?g" + "|png|tiff?|mid|mp2|mp3|mp4"             + "|wav|avi|mov|mpeg|ram|m4v|pdf" + "|rm|smil|wmv|swf|wma|zip|rar|gz))$");  /** * should implement function specify whether given url * should crawled or not (based on crawling logic). */     @override     public boolean shouldvisit(weburl url) {         string href = url.geturl().tolowercase();         return !filters.matcher(href).matches() && href.startswith("http://");                                             }        /**      * function called when page fetched , ready processed      * program.      */     @override     public void visit(page page) {             int docid = page.getweburl().getdocid();             string url = page.getweburl().geturl();             string domain = page.getweburl().getdomain();             string path = page.getweburl().getpath();             string subdomain = page.getweburl().getsubdomain();             string parenturl = page.getweburl().getparenturl();             string anchor = page.getweburl().getanchor();              system.out.println("docid: " + docid);             system.out.println("url: " + url);             system.out.println("domain: '" + domain + "'");             system.out.println("sub-domain: '" + subdomain + "'");             system.out.println("path: '" + path + "'");             system.out.println("parent page: " + parenturl);             system.out.println("anchor text: " + anchor);             // here webbrowser should open don`t work             webdriver driver = new firefoxdriver();        driver.get(url);                  if (page.getparsedata() instanceof htmlparsedata) {                     htmlparsedata htmlparsedata = (htmlparsedata) page.getparsedata();                     string text = htmlparsedata.gettext();                     string html = htmlparsedata.gethtml();                     list<weburl> links = htmlparsedata.getoutgoingurls();                      system.out.println("text length: " + text.length());                     system.out.println("html length: " + html.length());                     system.out.println("number of outgoing links: " + links.size());                 }                  header[] responseheaders = page.getfetchresponseheaders();             if (responseheaders != null) {                     system.out.println("response headers:");                     (header header : responseheaders) {                             system.out.println("\t" + header.getname() + ": " + header.getvalue());                     }             }              system.out.println("=============");      }

i used httpcore-4.2.2 , httpclient-4.2.3. there must have been problem using selenium. after updated them latest release worked should. lost 1 week finding this.


Comments

Post a Comment

Popular posts from this blog

database - VFP Grid + SQL server 2008 - grid not showing correctly -

-
i having problem, have grid called "gridbasica" , this: thisform.gridbase.recordsource = "sgviemp" sgviemp cursor made executing select sqlserver command sqlexec. when want modify registry, go form , the update there. so, when come form grid, grid isnt grid anymore, big white rectangle. reading around said: delete recordsource , asingt again. thisform.gridbase.recordsource = "" *--i sqlexe here thisform.gridbase.recordsource = "sgviemp" i , works, but, when click button modify again cursos updated modify form doesnt load fields did first time. could me out that? thanks in advance. --------------- edit ------------------ i have been doing recommended: lnselect = select(0) n = sqlexec(thisform.conexion,"select emp_ccodigo codigo, emp_cnombre 'nombre o razon social', emp_cnumrif 'r.i.f' sgviempr","sgviemp1") set echo on suspend select sgviemp zap in sgviemp *-- browse *--zaps correct
Read more

jquery - Set jPicker field to empty value -

-
im using jpicker color selector in form. have clear value set window/icon 'no color selected' state. i've tried following: $(':input[name="color"]').val(''); //access value directly. (does not affect jpicker) 2: $.jpicker.list[0].color.active.val('hex', '', this); //set color black , input value 000000, instead of empty string 3: $.jpicker.list[0].color.active.val(null, '', this); //does nothing none of 3 methods seems behave properly. in advance. this works me :) $.jpicker.list[0].color.active.val('v', null); here example: http://jsfiddle.net/3up7n/
Read more

.htaccess - htaccess convert request to clean url and add slash at the end of the url -

-
ok , searched more 4 days , more 20 posts here in stach or in websites without kind of success first here htaccess file rewriteengine on rewriterule ^([a-za-z0-9_]+)/?$ index.php?page=$1 [nc] rewriterule ^([a-za-z0-9_]+)\/([a-za-z0-9_]+)\/?$ index.php?page=$1&action=$2 [nc] i tried many codes 1 worked me can call request , works me www.example.com/pagename => www.example.com/index.php?page=pagename here questions , please provide me information "in detail" understand code well 1- how convert request automatically www.example.com/index.php?page=pagename www.example.com/pagename 2- if tried add slash @ end shows me page without images or styling mean www.example.com/pagename => works www.example.com/pagename/ => doesn't work that's :) for first part need 301 rule this: rewritecond %{the_request} /index\.php\?page=([^\s&]+) [nc] rewriterule ^ /%1? [r=301,l] for second part use absolute path in css, js, images files r
Read more