hadoop + Flume + HDFS IO error + ConnectException -


i'm working cloudera manager cdh 5.4.2, installed flume, can not save information twitter

when run flume agent, starts okay ends in error when attempts writing new event data hdfs. got following error:

 info org.apache.flume.sink.hdfs.bucketwriter: creating hdfs://192.168.109.6:8020/user/flume/tweets/2015/06/03/06//flumedata.1433311217583.tmp  warn org.apache.flume.sink.hdfs.hdfseventsink: hdfs io error java.net.connectexception: call cluster-05.xxxx.com/192.168.109.6 cluster-05.xxxx.com:8020 failed on connection exception: java.net.connectexception: connection refused; more details see: http://wiki.apache.org/hadoop/connectionrefused

the configuration did :

flume-conf.property:

twitteragent.sinks.hdfs.channel = memchannel twitteragent.sinks.hdfs.type = hdfs twitteragent.sinks.hdfs.hdfs.path = hdfs://192.168.109.6:8020/user/flume/tweets/%y/%m/%d/%h/ twitteragent.sinks.hdfs.hdfs.filetype = datastream twitteragent.sinks.hdfs.hdfs.writeformat = text twitteragent.sinks.hdfs.hdfs.batchsize = 1000 twitteragent.sinks.hdfs.hdfs.rollsize = 0 twitteragent.sinks.hdfs.hdfs.rollcount = 10000

i using follown pluggins:

flume-sources-1.0-snapshot.jar twitter4j-core-2.2.6.jar twitter4j-media-support-2.2.6.jar twitter4j-stream-2.2.6.jar

(i replace version of twitter4j--3.0.3.jar twitter4j--2.2.6.jar)

also directory using hdfs user

hadoop fs -ls /user/flume :  drwxrwxrwx - flume flume  /user/flume/tweets

core-site.xml ( @ /hadoop/conf ) added:

<property> <name>fs.default.name</name> <value>hdfs://localhost:8020</value> </property>

i run hadoop dfsadmin -safemode leave on host left flume agent hdfs user

i appreciate help, on issue.


Comments

Post a Comment

Popular posts from this blog

android - Sent Blob results empty -

in cordova/phonegap app i'm testing on android 4.0 need send file server multipart/form-data through ajax. i have file content in arraybuffer , put in formdata , firstly creating blob it. the problem file sent appears empty. this console session (performed on android platform through weinre ) can see that: the file content loaded in mybuf a blob created mybuf (and size non-zero) a formdata object created sent ( fd ) (i'm using webkitblobbuilder because blob constructor raises typeerror on platform) ❯ mybuf ▼ arraybuffer bytelength: 23673 ▶ __proto__: arraybuffer ❯ var bb = new webkitblobbuilder() undefined ❯ bb.append(mybuf) undefined ❯ myblob = bb.getblob("image/jpeg") ▼ blob size: 23673 type: "image/jpeg" ▶ __proto__: blob ❯ fd = new formdata() ▶ formdata ❯ fd.append("pics[]", myblob, "1433412118197.jpg") undefined when perform ajax request passing fd object data, see...
Read more

javascript - Bootstrap Popover: iOS Safari strange behaviour -

i looking through twitter bootstrap documentation, , i'm interested in implementing dismissible popover when button clicked. here example bootstrap's docs: http://getbootstrap.com/javascript/#dismiss-on-next-click when press "dismissible popover" button os x safari browser works fine. when access same link above ios safari, button not work. is, popover not dismissed on click. why button not working ios safari? the example linked using data-trigger="focus" attribute. try changing data-trigger="click" . <a tabindex="0" class="btn btn-lg btn-danger" role="button" data-toggle="popover" data-trigger="click" title="dismissible popover" data-content="and here's amazing content. it's engaging. right?">dismissible popover</a>
Read more