What does this debug log of Storm mean? -


i learned storm, , tested wordcount example code here. uses setdebug(true) enable debug logging.
confused these debug information:

13867 [storm.starter.wordcounttopology.main()] info  backtype.storm.daemon.nimbus - setting new assignment topology id word-count-1-1403745226:  #backtype.storm.daemon.common.assignment{:master-code-dir "/tmp/4ef1358b-92ce-4051-a4ce- b33a25ace66d/nimbus/stormdist/word-count-1-1403745226", :node->host {"47fdd74b-65e0-4650- a000-f1f0418dd7c7" "ms-virtualbox"}, :executor->node+port {[2 2] ["47fdd74b-65e0-4650-a000- f1f0418dd7c7" 1027], [3 3] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027], [4 4] ["47fdd74b- 65e0-4650-a000-f1f0418dd7c7" 1027], [5 5] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027], [6  6] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027], [7 7] ["47fdd74b-65e0-4650-a000- f1f0418dd7c7" 1027], [8 8] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027], [9 9] ["47fdd74b- 65e0-4650-a000-f1f0418dd7c7" 1027], [10 10] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027],  [1 1] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027]}, :executor->start-time-secs {[2 2]  1403745226, [3 3] 1403745226, [4 4] 1403745226, [5 5] 1403745226, [6 6] 1403745226, [7 7]  1403745226, [8 8] 1403745226, [9 9] 1403745226, [10 10] 1403745226, [1 1] 1403745226}}   ...  14547 [thread-6] info  backtype.storm.daemon.supervisor - launching worker  assignment #backtype.storm.daemon.supervisor.localassignment{:storm-id "word-count-1- 1403745226", :executors ([2 2] [3 3] [4 4] [5 5] [6 6] [7 7] [8 8] [9 9] [10 10] [1 1])}  supervisor 47fdd74b-65e0-4650-a000-f1f0418dd7c7 on port 1027 id 80043f34-85af- 4862-840e-99a58b1cf1f5  ...      16140 [thread-6] info  backtype.storm.daemon.executor - loading executor count:[2 2]     16153 [thread-6] info  backtype.storm.daemon.task - emitting: count __system ["startup"]     16155 [thread-6] info  backtype.storm.daemon.executor - loaded executor tasks count:[2  2]     16182 [thread-6] info  backtype.storm.daemon.executor - finished loading executor count:[2 2]     16186 [thread-8-count] info  backtype.storm.daemon.executor - preparing bolt count:(2)     16198 [thread-6] info  backtype.storm.daemon.executor - loading executor count:[3 3]     16198 [thread-6] info  backtype.storm.daemon.task - emitting: count __system ["startup"]     16198 [thread-6] info  backtype.storm.daemon.executor - loaded executor tasks count:[3  3]     16206 [thread-8-count] info  backtype.storm.daemon.executor - prepared bolt count:(2)     16213 [thread-6] info  backtype.storm.daemon.executor - finished loading executor count: [3 3]     16217 [thread-10-count] info  backtype.storm.daemon.executor - preparing bolt count:(3)     16221 [thread-10-count] info  backtype.storm.daemon.executor - prepared bolt count:(3) 

what [1 1],[2 2] ... mean? can give me detailed explanation. much.

thanks comment, found useful there.
after taking deep @ concepts of component, node, worker, executor, task, know [1 1] means.

firstly

:node->host {"47fdd74b-65e0-4650-a000-f1f0418dd7c7" "ms-virtualbox"} 

this means there's 1 node here. node id 47fdd74b-65e0-4650-a000-f1f0418dd7c7, responding host machine ms-virtualbox.

secondly

:executor->node+port {[2 2] ["47fdd74b-65e0-4650-a000-f1f0418dd7c7" 1027], ... 

the executor thread runs spout/bolt instance, above sentence shows executor is. [2 2] executor id, responds node 47fdd74b-65e0-4650-a000-f1f0418dd7c7's 1027 port.

since wordcounttopology has code snippet conf.setmaxtaskparallelism(3);, , there 3 spout/bolt, maximum number of executors spout/bolt 9 (from [2 2] [10 10]). (1 task responds 1 executor in case) in addition, executor ([1 1]) spawned called acker, track tuple trees , detect when spout tuple has been processed. so, there 10 executors totally, [1 1] [10 10].


Comments

Popular posts from this blog

google api - Incomplete response from Gmail API threads.list -

Installing Android SQLite Asset Helper -

Qt Creator - Searching files with Locator including folder -