Cassandra as input to Hadoop job exception -


for reason i'm getting following exception while trying use cassandra input hadoop

exception in thread "main" java.lang.noclassdeffounderror: com/datastax/driver/core/policies/loadbalancingpolicy 

here code

public class cdriver extends configured implements tool{  public static void main(string[] args) throws ioexception, interruptedexception, classnotfoundexception, exception {     configuration conf = new configuration();      toolrunner.run(new cdriver(), args); }  @override public int run(string[] args) throws exception {      string output = args[0];      configuration conf = super.getconf();      job job = new job(conf);      job.setjarbyclass(cdriver.class);     job.setjobname("cassandra input");      confighelper.setinputinitialaddress(conf, "127.0.0.1");     confighelper.setinputcolumnfamily(conf, "basketball", "nba");     confighelper.setinputpartitioner(conf, "murmur3partitioner");     cqlconfighelper.setinputcqlpagerowsize(conf, "3");     job.setinputformatclass(cqlinputformat.class);      fileoutputformat.setoutputpath(job, new path(output));      job.setmapperclass(cmapper.class);     job.setreducerclass(creducer.class);      job.setmapoutputkeyclass(text.class);     job.setmapoutputvalueclass(intwritable.class);      job.setoutputkeyclass(text.class);     job.setoutputvalueclass(intwritable.class);      job.waitforcompletion(true);      return 0; } 

}

it goes off on following line

cqlconfighelper.setinputcqlpagerowsize(conf, "3"); 

and here maven dependencies:

    <?xml version="1.0" encoding="utf-8"?> <project xmlns="http://maven.apache.org/pom/4.0.0" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://maven.apache.org/pom/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <modelversion>4.0.0</modelversion>     <groupid>com.nissatech</groupid>     <artifactid>testingcassandra</artifactid>     <version>1.0-snapshot</version>     <packaging>jar</packaging>     <properties>         <project.build.sourceencoding>utf-8</project.build.sourceencoding>         <maven.compiler.source>1.7</maven.compiler.source>         <maven.compiler.target>1.7</maven.compiler.target>     </properties>     <dependencies>         <dependency>             <groupid>org.apache.hadoop</groupid>             <artifactid>hadoop-core</artifactid>             <version>1.2.1</version>         </dependency>         <dependency>             <groupid>org.apache.cassandra</groupid>             <artifactid>cassandra-all</artifactid>             <version>2.1.5</version>         </dependency>     </dependencies>     <build>         <plugins>             <plugin>                 <artifactid>maven-assembly-plugin</artifactid>                 <configuration>                     <archive>                         <manifest>                             <mainclass>com.nissatech.testingcassandra.cdriver</mainclass>                         </manifest>                     </archive>                     <descriptorrefs>                         <descriptorref>                             jar-with-dependencies                         </descriptorref>                     </descriptorrefs>                 </configuration>                 <executions>                     <execution>                         <id>make-assembly</id>                         <phase>package</phase>                         <goals>                             <goal>single</goal>                         </goals>                     </execution>                 </executions>             </plugin>         </plugins>     </build> </project> 

can explain problem? have cassandra running on localhost.

you need include datastax java driver third party dependency in pom.xml, too. see https://github.com/datastax/java-driver#maven. if have transitive dependency, make sure using right version of it.


Comments

Popular posts from this blog

Magento/PHP - Get phones on all members in a customer group -

php - .htaccess mod_rewrite for dynamic url which has domain names -

Website Login Issue developed in magento -