java - Storm is having trouble connecting to nimbus -
i'm trying set storm cluster having trouble running topologies on cluster. can run them locally on each machine not together. using following command storm starter kit try , make work:
$ storm jar storm-starter-*.jar storm.starter.rollingtopwords production-topology remote i getting 2 different similar errors when run topology on 1 of supervisor machines vs nimbus machine. have instance of zookeeper supervisor nodes should able connect not happening. honestly, don't understand how zookeeper manages cluster.
error thrown in supervisor:
scix3@m5:~/apache/storm/examples/storm-starter/target$ storm jar storm-starter-0.9.4.jar storm.starter.rollingtopwords production-topology remote running: /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -client - dstorm.options= -dstorm.home=/home/scix3/apache/storm -dstorm.log.dir=/home/scix3/apache/storm/logs -djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -dstorm.conf.file= -cp /home/scix3/apache/storm/lib/kryo- 2.21.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/ring-servlet- 0.3.11.jar:/home/scix3/apache/storm/lib/clj-stacktrace-0.2.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/clojure- 1.5.1.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/compojure- 1.1.3.jar:/home/scix3/apache/storm/lib/minlog-1.2.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/tools.macro- 0.1.0.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/commons-io- 2.4.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:storm-starter-0.9.4.jar:/home/scix3/apache/storm/conf:/home/scix3/apache/storm/bin - dstorm.jar=storm-starter-0.9.4.jar storm.starter.rollingtopwords production- topology remote 200 [main] info storm.starter.rollingtopwords - topology name: production- topology exception in thread "main" java.lang.noclassdeffounderror: com/google/common/collect/lists @ storm.starter.tools.rankings.<init>(rankings.java:33) @ storm.starter.bolt.abstractrankerbolt.<init> (abstractrankerbolt.java:69) @ storm.starter.bolt.abstractrankerbolt.<init> (abstractrankerbolt.java:56) @ storm.starter.bolt.intermediaterankingsbolt.<init> (intermediaterankingsbolt.java:41) @ storm.starter.rollingtopwords.wiretopology(rollingtopwords.java:68) @ storm.starter.rollingtopwords.<init>(rollingtopwords.java:52) @ storm.starter.rollingtopwords.main(rollingtopwords.java:122) caused by: java.lang.classnotfoundexception: com.google.common.collect.lists @ java.net.urlclassloader$1.run(urlclassloader.java:366) @ java.net.urlclassloader$1.run(urlclassloader.java:355) @ java.security.accesscontroller.doprivileged(native method) @ java.net.urlclassloader.findclass(urlclassloader.java:354) @ java.lang.classloader.loadclass(classloader.java:425) @ sun.misc.launcher$appclassloader.loadclass(launcher.java:308) @ java.lang.classloader.loadclass(classloader.java:358) ... 7 more scix3@m5:~/apache/storm/examples/storm-starter/target$ error thrown nimbus:
scix3@m6:~/apache/storm/examples/storm-starter/targetstorm jar storm-starter-0.9.4.jar storm.starter.rollingtopwords production-topology remote running: java -client -dstorm.options= -dstorm.home=/home/scix3/apache/storm -dstorm.log.dir=/home/scix3/apache/storm/logs - djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -dstorm.conf.file= -cp /home/scix3/apache/storm/storm-starter-0.9.4-jar-with- dependencies.jar:/home/scix3/apache/storm/storm-starter-0.9.4.jar:/home/scix3/apache/storm/lib/clj-stacktrace- 0.2.2.jar:/home/scix3/apache/storm/lib/clojure-1.5.1.jar:/home/scix3/apache/storm/lib/minlog- 1.2.jar:/home/scix3/apache/storm/lib/tools.logging-0.2.3.jar:/home/scix3/apache/storm/lib/commons-codec-1.6.jar:/home/scix3/apache/storm/lib/commons-io-2.4.jar:/home/scix3/apache/storm/lib/json-simple-1.1.jar:/home/scix3/apache/storm/lib/asm-4.0.jar:/home/scix3/apache/storm/lib/commons-fileupload-1.2.1.jar:/home/scix3/apache/storm/lib/commons-lang-2.5.jar:/home/scix3/apache/storm/lib/ring-servlet-0.3.11.jar:/home/scix3/apache/storm/lib/carbonite-1.4.0.jar:/home/scix3/apache/storm/lib/jetty-util-6.1.26.jar:/home/scix3/apache/storm/lib/tools.macro-0.1.0.jar:/home/scix3/apache/storm/lib/jgrapht-core-0.9.0.jar:/home/scix3/apache/storm/lib/kryo-2.21.jar:/home/scix3/apache/storm/lib/logback-core-1.0.13.jar:/home/scix3/apache/storm/lib/ring-devel-0.3.11.jar:/home/scix3/apache/storm/lib/ring-jetty-adapter-0.3.11.jar:/home/scix3/apache/storm/lib/logback-classic-1.0.13.jar:/home/scix3/apache/storm/lib/commons-logging-1.1.3.jar:/home/scix3/apache/storm/lib/tools.cli-0.2.4.jar:/home/scix3/apache/storm/lib/reflectasm-1.07-shaded.jar:/home/scix3/apache/storm/lib/math.numeric-tower-0.0.1.jar:/home/scix3/apache/storm/lib/chill-java-0.3.5.jar:/home/scix3/apache/storm/lib/clout-1.0.1.jar:/home/scix3/apache/storm/lib/slf4j-api-1.7.5.jar:/home/scix3/apache/storm/lib/disruptor-2.10.1.jar:/home/scix3/apache/storm/lib/joda-time-2.0.jar:/home/scix3/apache/storm/lib/log4j-over-slf4j-1.6.6.jar:/home/scix3/apache/storm/lib/compojure-1.1.3.jar:/home/scix3/apache/storm/lib/hiccup-0.3.6.jar:/home/scix3/apache/storm/lib/servlet-api-2.5.jar:/home/scix3/apache/storm/lib/commons-exec-1.1.jar:/home/scix3/apache/storm/lib/core.incubator-0.1.0.jar:/home/scix3/apache/storm/lib/jetty-6.1.26.jar:/home/scix3/apache/storm/lib/storm-core-0.9.4.jar:/home/scix3/apache/storm/lib/ring-core-1.1.5.jar:/home/scix3/apache/storm/lib/objenesis-1.2.jar:/home/scix3/apache/storm/lib/jline-2.11.jar:/home/scix3/apache/storm/lib/snakeyaml-1.11.jar:/home/scix3/apache/storm/lib/clj-time-0.4.1.jar:storm-starter-0.9.4.jar:/home/scix3/.storm:/home/scix3/apache/storm/bin -dstorm.jar=storm-starter-0.9.4.jar storm.starter.rollingtopwords production-topology remote 202 [main] info storm.starter.rollingtopwords - topology name: production-topology 237 [main] info storm.starter.rollingtopwords - running in remote (cluster) mode exception in thread "main" java.lang.runtimeexception: org.apache.thrift7.transport.ttransportexception: java.net.connectexception: connection refused @ backtype.storm.utils.nimbusclient.getconfiguredclient(nimbusclient.java:38) @ backtype.storm.stormsubmitter.submittopology(stormsubmitter.java:116) @ backtype.storm.stormsubmitter.submittopology(stormsubmitter.java:70) @ storm.starter.util.stormrunner.runtopologyremotely(stormrunner.java:45) @ storm.starter.rollingtopwords.runremotely(rollingtopwords.java:78) @ storm.starter.rollingtopwords.main(rollingtopwords.java:129) caused by: org.apache.thrift7.transport.ttransportexception: java.net.connectexception: connection refused @ org.apache.thrift7.transport.tsocket.open(tsocket.java:183) @ org.apache.thrift7.transport.tframedtransport.open(tframedtransport.java:81) @ backtype.storm.security.auth.simpletransportplugin.connect(simpletransportplugin.java:83) @ backtype.storm.security.auth.thriftclient.<init>(thriftclient.java:63) @ backtype.storm.utils.nimbusclient.<init>(nimbusclient.java:47) @ backtype.storm.utils.nimbusclient.<init>(nimbusclient.java:43) @ backtype.storm.utils.nimbusclient.getconfiguredclient(nimbusclient.java:36) ... 5 more caused by: java.net.connectexception: connection refused @ java.net.plainsocketimpl.socketconnect(native method) @ java.net.abstractplainsocketimpl.doconnect(abstractplainsocketimpl.java:339) @ java.net.abstractplainsocketimpl.connecttoaddress(abstractplainsocketimpl.java:200) @ java.net.abstractplainsocketimpl.connect(abstractplainsocketimpl.java:182) @ java.net.sockssocketimpl.connect(sockssocketimpl.java:392) @ java.net.socket.connect(socket.java:579) @ org.apache.thrift7.transport.tsocket.open(tsocket.java:178) ... 11 more scix3@m6:~/apache/storm/examples/storm-starter/target$ although i've come far setting storm , zookeeper don't understand how work , why have issues. tips, regarding issue or not, more welcome!
Comments
Post a Comment