zookeeperStarted.html 19.4 KB
Newer Older
1
2
3
4
5
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta content="Apache Forrest" name="Generator">
6
<meta name="Forrest-version" content="0.9">
7
<meta name="Forrest-skin-name" content="pelt">
8
<title>ZooKeeper Getting Started Guide</title>
9
10
11
12
13
14
15
16
17
18
19
20
21
22
<link type="text/css" href="skin/basic.css" rel="stylesheet">
<link media="screen" type="text/css" href="skin/screen.css" rel="stylesheet">
<link media="print" type="text/css" href="skin/print.css" rel="stylesheet">
<link type="text/css" href="skin/profile.css" rel="stylesheet">
<script src="skin/getBlank.js" language="javascript" type="text/javascript"></script><script src="skin/getMenu.js" language="javascript" type="text/javascript"></script><script src="skin/fontsize.js" language="javascript" type="text/javascript"></script>
<link rel="shortcut icon" href="images/favicon.ico">
</head>
<body onload="init()">
<script type="text/javascript">ndeSetTextSize();</script>
<div id="top">
<!--+
    |breadtrail
    +-->
<div class="breadtrail">
23
<a href="http://www.apache.org/">Apache</a> &gt; <a href="http://zookeeper.apache.org/">ZooKeeper</a> &gt; <a href="http://zookeeper.apache.org/">ZooKeeper</a><script src="skin/breadcrumbs.js" language="JavaScript" type="text/javascript"></script>
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
</div>
<!--+
    |header
    +-->
<div class="header">
<!--+
    |start group logo
    +-->
<div class="grouplogo">
<a href="http://hadoop.apache.org/"><img class="logoImage" alt="Hadoop" src="images/hadoop-logo.jpg" title="Apache Hadoop"></a>
</div>
<!--+
    |end group logo
    +-->
<!--+
    |start Project Logo
    +-->
<div class="projectlogo">
42
<a href="http://zookeeper.apache.org/"><img class="logoImage" alt="ZooKeeper" src="images/zookeeper_small.gif" title="ZooKeeper: distributed coordination"></a>
43
44
45
46
47
48
49
50
51
</div>
<!--+
    |end Project Logo
    +-->
<!--+
    |start Search
    +-->
<div class="searchbox">
<form action="http://www.google.com/search" method="get" class="roundtopsmall">
52
<input value="zookeeper.apache.org" name="sitesearch" type="hidden"><input onFocus="getBlank (this, 'Search the site with google');" size="25" name="q" id="query" type="text" value="Search the site with google">&nbsp; 
53
54
55
56
57
58
59
60
61
62
63
                    <input name="Search" value="Search" type="submit">
</form>
</div>
<!--+
    |end search
    +-->
<!--+
    |start Tabs
    +-->
<ul id="tabs">
<li>
64
<a class="unselected" href="http://zookeeper.apache.org/">Project</a>
65
66
</li>
<li>
67
<a class="unselected" href="https://cwiki.apache.org/confluence/display/ZOOKEEPER/">Wiki</a>
68
69
</li>
<li class="current">
Patrick D. Hunt's avatar
Patrick D. Hunt committed
70
<a class="selected" href="index.html">ZooKeeper 3.5 Documentation</a>
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
</li>
</ul>
<!--+
    |end Tabs
    +-->
</div>
</div>
<div id="main">
<div id="publishedStrip">
<!--+
    |start Subtabs
    +-->
<div id="level2tabs"></div>
<!--+
    |end Endtabs
    +-->
<script type="text/javascript"><!--
document.write("Last Published: " + document.lastModified);
//  --></script>
</div>
<!--+
    |breadtrail
    +-->
<div class="breadtrail">

             &nbsp;
           </div>
<!--+
    |start Menu, mainarea
    +-->
<!--+
    |start Menu
    +-->
<div id="menu">
105
<div onclick="SwitchMenu('menu_selected_1.1', 'skin/')" id="menu_selected_1.1Title" class="menutitle" style="background-image: url('skin/images/chapter_open.gif');">Overview</div>
106
107
108
109
110
<div id="menu_selected_1.1" class="selectedmenuitemgroup" style="display: block;">
<div class="menuitem">
<a href="index.html">Welcome</a>
</div>
<div class="menuitem">
111
<a href="zookeeperOver.html">Overview</a>
112
113
114
115
</div>
<div class="menupage">
<div class="menupagetitle">Getting Started</div>
</div>
116
117
118
<div class="menuitem">
<a href="releasenotes.html">Release Notes</a>
</div>
119
120
121
122
123
124
</div>
<div onclick="SwitchMenu('menu_1.2', 'skin/')" id="menu_1.2Title" class="menutitle">Developer</div>
<div id="menu_1.2" class="menuitemgroup">
<div class="menuitem">
<a href="api/index.html">API Docs</a>
</div>
125
126
127
128
<div class="menuitem">
<a href="zookeeperProgrammers.html">Programmer's Guide</a>
</div>
<div class="menuitem">
129
<a href="javaExample.html">Java Example</a>
130
131
</div>
<div class="menuitem">
132
<a href="zookeeperTutorial.html">Barrier and Queue Tutorial</a>
133
134
</div>
<div class="menuitem">
135
<a href="recipes.html">Recipes</a>
136
</div>
137
</div>
138
<div onclick="SwitchMenu('menu_1.3', 'skin/')" id="menu_1.3Title" class="menutitle">Admin &amp; Ops</div>
139
<div id="menu_1.3" class="menuitemgroup">
140
<div class="menuitem">
141
<a href="zookeeperAdmin.html">Administrator's Guide</a>
142
</div>
143
<div class="menuitem">
144
145
146
<a href="zookeeperQuotas.html">Quota Guide</a>
</div>
<div class="menuitem">
147
148
<a href="zookeeperJMX.html">JMX</a>
</div>
149
150
151
<div class="menuitem">
<a href="zookeeperObservers.html">Observers Guide</a>
</div>
152
</div>
153
154
<div onclick="SwitchMenu('menu_1.4', 'skin/')" id="menu_1.4Title" class="menutitle">Contributor</div>
<div id="menu_1.4" class="menuitemgroup">
155
156
157
<div class="menuitem">
<a href="zookeeperInternals.html">ZooKeeper Internals</a>
</div>
158
</div>
159
160
<div onclick="SwitchMenu('menu_1.5', 'skin/')" id="menu_1.5Title" class="menutitle">Miscellaneous</div>
<div id="menu_1.5" class="menuitemgroup">
161
<div class="menuitem">
162
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER">Wiki</a>
163
164
</div>
<div class="menuitem">
165
<a href="https://cwiki.apache.org/confluence/display/ZOOKEEPER/FAQ">FAQ</a>
166
167
</div>
<div class="menuitem">
168
<a href="http://zookeeper.apache.org/mailing_lists.html">Mailing Lists</a>
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
</div>
</div>
<div id="credit"></div>
<div id="roundbottom">
<img style="display: none" class="corner" height="15" width="15" alt="" src="skin/images/rc-b-l-15-1body-2menu-3menu.png"></div>
<!--+
  |alternative credits
  +-->
<div id="credit2"></div>
</div>
<!--+
    |end Menu
    +-->
<!--+
    |start content
    +-->
<div id="content">
<div title="Portable Document Format" class="pdflink">
<a class="dida" href="zookeeperStarted.pdf"><img alt="PDF -icon" src="skin/images/pdfdoc.gif" class="skin"><br>
        PDF</a>
</div>
190
<h1>ZooKeeper Getting Started Guide</h1>
191
<div id="front-matter">
192
193
194
<div id="minitoc-area">
<ul class="minitoc">
<li>
195
<a href="#ch_GettingStarted">Getting Started: Coordinating Distributed Applications with
196
197
198
      ZooKeeper</a>
<ul class="minitoc">
<li>
199
200
201
202
203
204
205
<a href="#sc_Prerequisites">Pre-requisites</a>
</li>
<li>
<a href="#sc_Download">Download</a>
</li>
<li>
<a href="#sc_InstallingSingleMode">Standalone Operation</a>
206
207
</li>
<li>
208
209
210
<a href="#sc_FileManagement">Managing ZooKeeper Storage</a>
</li>
<li>
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
<a href="#sc_ConnectingToZooKeeper">Connecting to ZooKeeper</a>
</li>
<li>
<a href="#sc_ProgrammingToZooKeeper">Programming to ZooKeeper</a>
</li>
<li>
<a href="#sc_RunningReplicatedZooKeeper">Running Replicated ZooKeeper</a>
</li>
<li>
<a href="#Other+Optimizations">Other Optimizations</a>
</li>
</ul>
</li>
</ul>
</div>
226
</div>
227
228
229
230
231
  

  

  
232
<a name="ch_GettingStarted"></a>
233
234
235
236
<h2 class="h3">Getting Started: Coordinating Distributed Applications with
      ZooKeeper</h2>
<div class="section">
<p>This document contains information to get you started quickly with
237
    ZooKeeper. It is aimed primarily at developers hoping to try it out, and
238
239
240
241
242
    contains simple installation instructions for a single ZooKeeper server, a
    few commands to verify that it is running, and a simple programming
    example. Finally, as a convenience, there are a few sections regarding
    more complicated installations, for example running replicated
    deployments, and optimizing the transaction log. However for the complete
243
    instructions for commercial deployments, please refer to the <a href="zookeeperAdmin.html">ZooKeeper
244
    Administrator's Guide</a>.</p>
245
<a name="sc_Prerequisites"></a>
246
247
248
<h3 class="h4">Pre-requisites</h3>
<p>See <a href="zookeeperAdmin.html#sc_systemReq">
          System Requirements</a> in the Admin guide.</p>
249
<a name="sc_Download"></a>
250
251
<h3 class="h4">Download</h3>
<p>To get a ZooKeeper distribution, download a recent
252
        <a href="http://zookeeper.apache.org/releases.html">
253
254
          stable</a> release from one of the Apache Download
        Mirrors.</p>
255
<a name="sc_InstallingSingleMode"></a>
256
<h3 class="h4">Standalone Operation</h3>
257
<p>Setting up a ZooKeeper server in standalone mode is
258
259
260
261
262
263
      straightforward. The server is contained in a single JAR file,
      so installation consists of creating a configuration.</p>
<p>Once you've downloaded a stable ZooKeeper release unpack
      it and cd to the root</p>
<p>To start ZooKeeper you need a configuration file. Here is a sample,
      create it in <strong>conf/zoo.cfg</strong>:</p>
264
265
<pre class="code">
tickTime=2000
266
dataDir=/var/lib/zookeeper
267
268
269
clientPort=2181
</pre>
<p>This file can be called anything, but for the sake of this
270
271
272
273
274
      discussion call
      it <strong>conf/zoo.cfg</strong>. Change the
      value of <strong>dataDir</strong> to specify an
      existing (empty to start with) directory.  Here are the meanings
      for each of the fields:</p>
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
<dl>
        
<dt>
<term>
<strong>tickTime</strong>
</term>
</dt>
<dd>
<p>the basic time unit in milliseconds used by ZooKeeper. It is
            used to do heartbeats and the minimum session timeout will be
            twice the tickTime.</p>
</dd>
      
</dl>
<dl>
        
<dt>
<term>
<strong>dataDir</strong>
</term>
</dt>
<dd>
<p>the location to store the in-memory database snapshots and,
            unless specified otherwise, the transaction log of updates to the
            database.</p>
</dd>

        
<dt>
<term>
<strong>clientPort</strong>
</term>
</dt>
<dd>
<p>the port to listen for client connections</p>
</dd>
      
</dl>
<p>Now that you created the configuration file, you can start
      ZooKeeper:</p>
315
<pre class="code">bin/zkServer.sh start</pre>
316
317
318
319
<p>ZooKeeper logs messages using log4j -- more detail
      available in the
      <a href="zookeeperProgrammers.html#Logging">Logging</a>
      section of the Programmer's Guide. You will see log messages
320
321
      coming to the console (default) and/or a log file depending on
      the log4j configuration.</p>
322
<p>The steps outlined here run ZooKeeper in standalone mode. There is
323
324
      no replication, so if ZooKeeper process fails, the service will go down.
      This is fine for most development situations, but to run ZooKeeper in
325
      replicated mode, please see <a href="#sc_RunningReplicatedZooKeeper">Running Replicated
326
      ZooKeeper</a>.</p>
327
<a name="sc_FileManagement"></a>
328
329
330
331
332
<h3 class="h4">Managing ZooKeeper Storage</h3>
<p>For long running production systems ZooKeeper storage must
      be managed externally (dataDir and logs). See the section on
      <a href="zookeeperAdmin.html#sc_maintenance">maintenance</a> for
      more details.</p>
333
<a name="sc_ConnectingToZooKeeper"></a>
334
<h3 class="h4">Connecting to ZooKeeper</h3>
335
<p>Once ZooKeeper is running, you have several options for connection
336
337
338
339
340
341
      to it:</p>
<ul>
        
<li>
          
<p>
342
343
344
<strong>Java</strong>: Use</p>

          
345
<pre class="code">bin/zkCli.sh -server 127.0.0.1:2181</pre>
346
347
348
349
350
351
352
353
354
355
356
357

          
<p>This lets you perform simple, file-like operations.</p>
        
</li>

        
<li>
          
<p>
<strong>C</strong>: compile cli_mt
          (multi-threaded) or cli_st (single-threaded) by running
358
359
360
361
362
363
364
365
366
          <span class="codefrag command">make cli_mt</span> or <span class="codefrag command">make
          cli_st</span> in
          the <strong>src/c</strong> subdirectory in
          the ZooKeeper sources. See the README contained within
          <strong>src/c</strong> for full details.</p>

          
<p>You can run the program
          from <strong>src/c</strong> using:</p>
367
368

          
369
370
371
372
373
374
375
376
377
378
<pre class="code">LD_LIBRARY_PATH=. cli_mt 127.0.0.1:2181</pre>

          
<p>or</p>

          
<pre class="code">LD_LIBRARY_PATH=. cli_st 127.0.0.1:2181</pre>
          
<p>This will give you a simple shell to execute file
          system like operations on ZooKeeper.</p>
379
380
381
382
        
</li>
      
</ul>
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
<p>Once you have connected, you should see something like:
        </p>
<pre class="code">

Connecting to localhost:2181
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
Welcome to ZooKeeper!
JLine support is enabled
[zkshell: 0]
        </pre>
<p>
        From the shell, type <span class="codefrag command">help</span> to get a listing of commands that can be executed from the client, as in:
      </p>
<pre class="code">

[zkshell: 0] help
ZooKeeper host:port cmd args
        get path [watch]
        ls path [watch]
        set path data [version]
        delquota [-n|-b] path
        quit
        printwatches on|off
407
        create path data acl
408
409
410
411
412
413
414
415
416
        stat path [watch]
        listquota path
        history
        setAcl path acl
        getAcl path
        sync path
        redo cmdno
        addauth scheme auth
        delete path [version]
417
        deleteall path
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
        setquota -n|-b val path

        </pre>
<p>From here, you can try a few simple commands to get a feel for this simple command line interface.  First, start by issuing the list command, as
      in <span class="codefrag command">ls</span>, yielding:
      </p>
<pre class="code">

[zkshell: 8] ls /
[zookeeper]
        </pre>
<p>Next, create a new znode by running <span class="codefrag command">create /zk_test my_data</span>. This creates a new znode and associates the string "my_data" with the node.
      You should see:</p>
<pre class="code">

[zkshell: 9] create /zk_test my_data
Created /zk_test
      </pre>
<p>  Issue another <span class="codefrag command">ls /</span> command to see what the directory looks like:
        </p>
<pre class="code">

[zkshell: 11] ls /
[zookeeper, zk_test]

        </pre>
<p>
      Notice that the zk_test directory has now been created.
      </p>
<p>Next, verify that the data was associated with the znode by running the <span class="codefrag command">get</span> command, as in:
      </p>
<pre class="code">

[zkshell: 12] get /zk_test
my_data
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 5
mtime = Fri Jun 05 13:57:06 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0
dataLength = 7
numChildren = 0
        </pre>
<p>We can change the data associated with zk_test by issuing the <span class="codefrag command">set</span> command, as in:
        </p>
<pre class="code">

[zkshell: 14] set /zk_test junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0
[zkshell: 15] get /zk_test
junk
cZxid = 5
ctime = Fri Jun 05 13:57:06 PDT 2009
mZxid = 6
mtime = Fri Jun 05 14:01:52 PDT 2009
pZxid = 5
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0
dataLength = 4
numChildren = 0
      </pre>
<p>
       (Notice we did a <span class="codefrag command">get</span> after setting the data and it did, indeed, change.</p>
<p>Finally, let's <span class="codefrag command">delete</span> the node by issuing:
      </p>
<pre class="code">

[zkshell: 16] delete /zk_test
[zkshell: 17] ls /
[zookeeper]
[zkshell: 18]
</pre>
<p>That's it for now.  To explore more, continue with the rest of this document and see the <a href="zookeeperProgrammers.html">Programmer's Guide</a>. </p>
507
<a name="sc_ProgrammingToZooKeeper"></a>
508
509
510
511
<h3 class="h4">Programming to ZooKeeper</h3>
<p>ZooKeeper has a Java bindings and C bindings. They are
      functionally equivalent. The C bindings exist in two variants: single
      threaded and multi-threaded. These differ only in how the messaging loop
512
      is done. For more information, see the <a href="zookeeperProgrammers.html#ch_programStructureWithExample">Programming
513
      Examples in the ZooKeeper Programmer's Guide</a> for
514
      sample code using of the different APIs.</p>
515
<a name="sc_RunningReplicatedZooKeeper"></a>
516
517
518
519
520
521
522
523
<h3 class="h4">Running Replicated ZooKeeper</h3>
<p>Running ZooKeeper in standalone mode is convenient for evaluation,
      some development, and testing. But in production, you should run
      ZooKeeper in replicated mode. A replicated group of servers in the same
      application is called a <em>quorum</em>, and in replicated
      mode, all servers in the quorum have copies of the same configuration
      file. The file is similar to the one used in standalone mode, but with a
      few differences. Here is an example:</p>
524
<pre class="code">
525
tickTime=2000
526
dataDir=/var/lib/zookeeper
527
528
529
530
531
532
clientPort=2181
initLimit=5
syncLimit=2
server.1=zoo1:2888:3888
server.2=zoo2:2888:3888
server.3=zoo3:2888:3888
533
</pre>
534
<p>The new entry, <strong>initLimit</strong> is
535
      timeouts ZooKeeper uses to limit the length of time the ZooKeeper
536
      servers in quorum have to connect to a leader. The entry <strong>syncLimit</strong> limits how far out of date a server can
537
      be from a leader.</p>
538
539
540
541
542
543
<p>With both of these timeouts, you specify the unit of time using
      <strong>tickTime</strong>. In this example, the timeout
      for initLimit is 5 ticks at 2000 milleseconds a tick, or 10
      seconds.</p>
<p>The entries of the form <em>server.X</em> list the
      servers that make up the ZooKeeper service. When the server starts up,
544
545
546
547
548
549
550
551
552
553
554
555
556
557
      it knows which server it is by looking for the file
      <em>myid</em> in the data directory. That file has the 
      contains the server number, in ASCII.</p>
<p>Finally, note the two port numbers after each server
       name: " 2888" and "3888". Peers use the former port to connect
       to other peers. Such a connection is necessary so that peers
       can communicate, for example, to agree upon the order of
       updates. More specifically, a ZooKeeper server uses this port
       to connect followers to the leader. When a new leader arises, a
       follower opens a TCP connection to the leader using this
       port. Because the default leader election also uses TCP, we
       currently require another port for leader election. This is the
       second port in the server entry.
       </p>
558
559
560
561
<div class="note">
<div class="label">Note</div>
<div class="content">
        
562
563
564
565
566
567
568
569
570
571
<p>If you want to test multiple servers on a single
        machine, specify the servername
        as <em>localhost</em> with unique quorum &amp;
        leader election ports (i.e. 2888:3888, 2889:3889, 2890:3890 in
        the example above) for each server.X in that server's config
        file. Of course separate <em>dataDir</em>s and
        distinct <em>clientPort</em>s are also necessary
        (in the above replicated example, running on a
        single <em>localhost</em>, you would still have
        three config files).</p>
572
573
574
      
</div>
</div>
575
<a name="Other+Optimizations"></a>
576
577
578
579
580
581
582
<h3 class="h4">Other Optimizations</h3>
<p>There are a couple of other configuration parameters that can
      greatly increase performance:</p>
<ul>
        
<li>
          
583
584
585
586
587
<p>To get low latencies on updates it is important to
          have a dedicated transaction log directory. By default
          transaction logs are put in the same directory as the data
          snapshots and <em>myid</em> file. The dataLogDir
          parameters indicates a different directory to use for the
588
589
590
591
592
593
594
595
          transaction logs.</p>
        
</li>

        
<li>
          
<p>
596
<em>[tbd: what is the other config param?]</em>
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
</p>
        
</li>
      
</ul>
</div>

<p align="right">
<font size="-2"></font>
</p>
</div>
<!--+
    |end content
    +-->
<div class="clearboth">&nbsp;</div>
</div>
<div id="footer">
<!--+
    |start bottomstrip
    +-->
<div class="lastmodified">
<script type="text/javascript"><!--
document.write("Last Published: " + document.lastModified);
//  --></script>
</div>
<div class="copyright">
        Copyright &copy;
624
         2008-2013 <a href="http://www.apache.org/licenses/">The Apache Software Foundation.</a>
625
626
627
628
629
630
631
</div>
<!--+
    |end bottomstrip
    +-->
</div>
</body>
</html>