临夏做网站,网络广告策划的内容,你有网站 我做房东 只收佣金的网,172分销系统zookeeper是一个挺好玩的东西 有着独特的选举机制#xff0c;一般在中小型集群中#xff0c;zookeeper一般装在三个节点 其中只有一个节点对外提供服务#xff0c;处于leader状态#xff0c;另外两台未follower状态 这得益于zookeeper独特的选举机制#xff0c;可以保证le…zookeeper是一个挺好玩的东西 有着独特的选举机制一般在中小型集群中zookeeper一般装在三个节点 其中只有一个节点对外提供服务处于leader状态另外两台未follower状态 这得益于zookeeper独特的选举机制可以保证leader节点的主机挂掉后 从节点可以通过选举机制很快成为leader节点并对外提供服务。
zookeeper的结构为树状结构其每个树状节点中存储着其他组件的元数据。 本篇内容主讲znode节点的一些功能和使用以及ACL的一些问题。
#找到zookeeper的运行目录
cd /opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/运行脚本zkCli.sh [rootcloud01 bin]# ./zkCli.sh Connecting to localhost:2181 2024-04-07 11:29:08,782 [myid:] - INFO [main:Environment100] - Client environment:zookeeper.version3.4.5-cdh6.3.2–1, built on 02/01/2022 16:17 GMT 2024-04-07 11:29:08,786 [myid:] - INFO [main:Environment100] - Client environment:host.namecloud01 2024-04-07 11:29:08,786 [myid:] - INFO [main:Environment100] - Client environment:java.version1.8.0_161 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.vendorOracle Corporation 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.home/usr/local/jdk8/jre 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.class.path/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/build/classes:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/build/lib/.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/slf4j-log4j12.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/slf4j-log4j12-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/slf4j-api-1.7.25.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/netty-3.10.6.Final.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/jline-2.11.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/lib/audience-annotations-0.5.0.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/zookeeper-3.4.5-cdh6.3.2.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/src/java/lib/.jar:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin/…/conf:.:/usr/local/jdk8/lib:/usr/local/jdk8/jre/lib: 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.library.path/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.io.tmpdir/tmp 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:java.compiler 2024-04-07 11:29:08,790 [myid:] - INFO [main:Environment100] - Client environment:os.nameLinux 2024-04-07 11:29:08,791 [myid:] - INFO [main:Environment100] - Client environment:os.archamd64 2024-04-07 11:29:08,791 [myid:] - INFO [main:Environment100] - Client environment:os.version3.10.0-1127.13.1.el7.x86_64 2024-04-07 11:29:08,791 [myid:] - INFO [main:Environment100] - Client environment:user.nameroot 2024-04-07 11:29:08,791 [myid:] - INFO [main:Environment100] - Client environment:user.home/root 2024-04-07 11:29:08,791 [myid:] - INFO [main:Environment100] - Client environment:user.dir/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p5253.21605619/lib/zookeeper/bin 2024-04-07 11:29:08,791 [myid:] - INFO [main:ZooKeeper619] - Initiating client connection, connectStringlocalhost:2181 sessionTimeout30000 watcherorg.apache.zookeeper.ZooKeeperMain M y W a t c h e r 7506 e 922 W e l c o m e t o Z o o K e e p e r ! 2024 − 04 − 0711 : 29 : 08 , 895 [ m y i d : ] − I N F O [ m a i n − S e n d T h r e a d ( l o c a l h o s t : 2181 ) : C l i e n t C n x n MyWatcher7506e922 Welcome to ZooKeeper! 2024-04-07 11:29:08,895 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn MyWatcher7506e922WelcometoZooKeeper!2024−04−0711:29:08,895[myid:]−INFO[main−SendThread(localhost:2181):ClientCnxnSendThread1118] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2024-04-07 11:29:09,008 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn S e n d T h r e a d 962 ] − S o c k e t c o n n e c t i o n e s t a b l i s h e d , i n i t i a t i n g s e s s i o n , c l i e n t : / 127.0.0.1 : 33928 , s e r v e r : l o c a l h o s t / 127.0.0.1 : 21812024 − 04 − 0711 : 29 : 09 , 068 [ m y i d : ] − I N F O [ m a i n − S e n d T h r e a d ( l o c a l h o s t : 2181 ) : C l i e n t C n x n SendThread962] - Socket connection established, initiating session, client: /127.0.0.1:33928, server: localhost/127.0.0.1:2181 2024-04-07 11:29:09,068 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn SendThread962]−Socketconnectionestablished,initiatingsession,client:/127.0.0.1:33928,server:localhost/127.0.0.1:21812024−04−0711:29:09,068[myid:]−INFO[main−SendThread(localhost:2181):ClientCnxnSendThread1378] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid 0xff8ea1e469371d73, negotiated timeout 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] 查看zookeeper下面的节点
[zk: localhost:2181(CONNECTED) 2] ls /
[cluster, controller, brokers, zookeeper, yarn-leader-election, hadoop-ha, admin, isr_change_notification, log_dir_event_notification, controller_epoch, rmstore, consumers, hive_zookeeper_namespace_hive, latest_producer_id_block, config, hbase, sentry]zookeeper在cdh集群中又扮演什么角色呢 这是在hbase配置中的设置 ZooKeeper 是 HBase 集群的协调服务它存储了 HBase 集群的状态信息、表结构信息、RegionServer 的信息。一般存储在zookeeper中的hbase节点下
[zk: localhost:2181(CONNECTED) 3] ls /hbase
[meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, acl, switch, master, running, balancer, tokenauth, draining, namespace, hbaseid, table]这是hdfs组件中的配置 hdfs要依赖于zookeeper来实现自身的高可用性zkfc机制
[zk: localhost:2181(CONNECTED) 4] ls /
[cluster, controller, brokers, zookeeper, yarn-leader-election, hadoop-ha, admin, isr_change_notification, log_dir_event_notification, controller_epoch, rmstore, consumers, hive_zookeeper_namespace_hive, latest_producer_id_block, config, hbase, sentry]
[zk: localhost:2181(CONNECTED) 5] ls /hadoop-ha
[nameservice1]
[zk: localhost:2181(CONNECTED) 6] 这是hive中的配置 Hive 的 HA 配置在配置 Hive 的高可用性High AvailabilityHA环境时ZooKeeper 可以用于管理和协调 Hive 的主从节点确保 Hive 元数据服务的可用性和一致性。
这是impala中的配置 ZooKeeper 可以用作元数据存储帮助 Impala 管理元数据信息的分布和一致性。
这是在kafka上的配置 做元数据存储还有就是协助选举leader
这是在Sentry中的配置 元数据存储和管理。
这是在yarn中的配置 高可用元数据存储。
关于ACL权限开启Kerberos认证后这个就不开了。 而且这个权限每个目录节点的权限都是独立的。
