Top

NSD ARCHITECTURE DAY05

  1. 案例1：组建zookeeper集群
  2. 案例2：测试集群的远程管理和高可用
  3. 案例3：在node节点上搭建3台kafka
  4. 案例4：准备实验环境
  5. 案例5：配置namenode与resourcemanager高可用
  6. 案例6：启动服务，验证高可用

1 案例1：组建zookeeper集群

1.1 问题

本案例要求：

1.2 步骤

实现此案例需要按照如下步骤进行。

步骤一：安装Zookeeper

1）编辑/etc/hosts ,所有集群主机可以相互 ping 通（在hadoop1上面配置，同步到node-0001，node-0002，node-0003）

  1. [root@hadoop1 hadoop]# vim /etc/hosts
  2. 192.168.1.50  hadoop1
  3. 192.168.1.51  node-0001
  4. 192.168.1.52  node-0002
  5. 192.168.1.53  node-0003
  6. 192.168.1.56  newnode
  7.  
  8. [root@nn01 hadoop]# for i in {52..54}  \
  9. do    \
  10. scp /etc/hosts 192.168.1.$i:/etc/    \
  11. done        //同步配置
  12. hosts       100%  253   639.2KB/s   00:00    
  13. hosts       100%  253   497.7KB/s   00:00    
  14. hosts       100%  253   662.2KB/s   00:00    

2）安装 java-1.8.0-openjdk-devel,由于之前的hadoop上面已经安装过，这里不再安装，若是新机器要安装

3）zookeeper 解压拷贝到 /usr/local/zookeeper

  1. [root@hadoop1 ~]# tar -xf zookeeper-3.4.13.tar.gz 
  2. [root@hadoop1 ~]# mv zookeeper-3.4.13 /usr/local/zookeeper

4）配置文件改名，并在最后添加配置

  1. [root@hadoop1 ~]# cd /usr/local/zookeeper/conf/
  2. [root@hadoop1 conf]# ls
  3. configuration.xsl  log4j.properties  zoo_sample.cfg
  4. [root@hadoop1 conf]# mv zoo_sample.cfg  zoo.cfg
  5. [root@hadoop1 conf]# chown root.root zoo.cfg
  6. [root@hadoop1 conf]# vim zoo.cfg
  7. server.1=node-0001:2888:3888
  8. server.2=node-0002:2888:3888
  9. server.3=node-0003:2888:3888
  10. server.4=hadoop1:2888:3888:observer

5）拷贝 /usr/local/zookeeper 到其他集群主机

  1. [root@hadoop1 conf]# for i in {52..54}; do rsync -aSH --delete /usr/local/zookeeper/ 192.168.1.$i:/usr/local/zookeeper  -e 'ssh' & done
  2. [4] 4956
  3. [5] 4957
  4. [6] 4958

6）创建 mkdir /tmp/zookeeper，每一台都要

  1. [root@hadoop1 conf]# mkdir /tmp/zookeeper
  2. [root@hadoop1 conf]# ssh node-0001 mkdir /tmp/zookeeper
  3. [root@hadoop1 conf]# ssh node-0002 mkdir /tmp/zookeeper
  4. [root@hadoop1 conf]# ssh node-0003 mkdir /tmp/zookeeper

7）创建 myid 文件，id 必须与配置文件里主机名对应的 server.(id) 一致

  1. [root@hadoop1 conf]# echo 4 >/tmp/zookeeper/myid
  2. [root@hadoop1 conf]# ssh node-0001 'echo 1 >/tmp/zookeeper/myid'
  3. [root@hadoop1 conf]# ssh node-0002 'echo 2 >/tmp/zookeeper/myid'
  4. [root@hadoop1 conf]# ssh node-0003 'echo 3 >/tmp/zookeeper/myid'

8）启动服务，单启动一台无法查看状态，需要启动全部集群以后才能查看状态，每一台上面都要手工启动（以hadoop1为例子）

  1. [root@hadoop1 conf]# /usr/local/zookeeper/bin/zkServer.sh start
  2. ZooKeeper JMX enabled by default
  3. Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
  4. Starting zookeeper ... STARTED

注意：刚启动zookeeper查看状态的时候报错，启动的数量要保证半数以上，这时再去看就成功了

9）查看状态

  1. [root@hadoop1 conf]# /usr/local/zookeeper/bin/zkServer.sh status
  2. ZooKeeper JMX enabled by default
  3. Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
  4. Mode: observe
  5. [root@hadoop1 conf]# /usr/local/zookeeper/bin/zkServer.sh stop  
  6. //关闭之后查看状态其他服务器的角色
  7. ZooKeeper JMX enabled by default
  8. Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
  9. Stopping zookeeper ... STOPPED

2 案例2：测试集群的远程管理和高可用

2.1 问题

本案例要求：

2.2 步骤

实现此案例需要按照如下步骤进行。

步骤一： 测试集群的远程管理和高可用

  1. [root@hadoop1 conf]# socat - TCP:node1:2181
  2. stat
  3. ... ...
  4. Outstanding: 0
  5. Zxid: 0x0
  6. Mode: follower
  7. Node count: 4
  8. [root@hadoop1 conf]# vim api.sh
  9. #!/bin/bash
  10. function getstatus(){
  11.     exec 9<>/dev/tcp/$1/2181 2>/dev/null
  12.     echo stat >&9
  13.     MODE=$(cat <&9 |grep -Po "(?<=Mode:).*")
  14.     exec 9<&-
  15.     echo ${MODE:-NULL}
  16. }
  17. for i in node{1..3} hadoop1;do
  18.     echo -ne "${i}\t"
  19.     getstatus ${i}
  20. done
  21. [root@hadoop1 conf]# chmod 755 api.sh
  22. [root@hadoop1 conf]# api.sh 
  23. node-0001        follower
  24. node-0002        leader
  25. node-0003        follower 
  26. hadoop1        observer

3 案例3：在node节点上搭建3台kafka

3.1 问题

本案例要求：

3.2 步骤

实现此案例需要按照如下步骤进行。

步骤一：搭建Kafka集群

1）解压 kafka 压缩包

Kafka在node-0001，node-0002，node-0003上面操作即可

  1. [root@node-0001 hadoop]# tar -xf kafka_2.12-2.1.0.tgz

2）把 kafka 拷贝到 /usr/local/kafka 下面

  1. [root@node-0001 ~]# mv kafka_2.12-2.1.0 /usr/local/kafka

3）修改配置文件 /usr/local/kafka/config/server.properties

  1. [root@node-0001 ~]# cd /usr/local/kafka/config
  2. [root@node-0001 config]# vim server.properties
  3. broker.id=22
  4. zookeeper.connect=node-0001:2181,node-0002:2181,node-0003:2181

4）拷贝 kafka 到其他主机，并修改 broker.id ,不能重复

  1. [root@node-0001 config]# for i in 53 54; do rsync -aSH --delete /usr/local/kafka 192.168.1.$i:/usr/local/; done
  2. [1] 27072
  3. [2] 27073
  4.  
  5. [root@node-0002 ~]# vim /usr/local/kafka/config/server.properties        
  6. //node-0002主机修改
  7. broker.id=23
  8. [root@node-0003 ~]# vim /usr/local/kafka/config/server.properties        
  9. //node-0003主机修改
  10. broker.id=24

5）启动 kafka 集群（node-0001，node-0002，node-0003启动）

  1. [root@node-0001 local]# /usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties 
  2. [root@node-0001 local]# jps        //出现kafka
  3. 26483 DataNode
  4. 27859 Jps
  5. 27833 Kafka
  6. 26895 QuorumPeerMain

6）验证配置，创建一个 topic

  1. [root@node-0001 local]# /usr/local/kafka/bin/kafka-topics.sh --create --partitions 1 --replication-factor 1 --zookeeper localhost:2181 --topic mymsg
  2.     
  3. Created topic "mymsg".

7) 模拟生产者，发布消息

  1. [root@node-0002 ~]# /usr/local/kafka/bin/kafka-console-producer.sh --broker-list  localhost:9092 --topic mymsg
  2.         //写一个数据
  3. ccc
  4. ddd

9）模拟消费者，接收消息

  1. [root@node-0003 ~]# /usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mymsg
  2.         //这边会直接同步
  3. ccc
  4. ddd

注意：kafka比较吃内存，做完这个kafka的实验可以把它停了

4 案例4：准备实验环境

4.1 问题

本案例要求：

4.2 方案

配置Hadoop的高可用，解决NameNode单点故障问题，使用之前搭建好的hadoop集群，新添加一台hadoop2，ip为192.168.1.56，具体要求如图-1所示：

图-1

4.3 步骤

实现此案例需要按照如下步骤进行。

步骤一：hadoop的高可用

1）停止所有服务（由于 kafka的实验做完之后就已经停止，这里不在重复）

  1. [root@hadoop1 ~]# cd /usr/local/hadoop/
  2. [root@hadoop1 hadoop]# ./sbin/stop-all.sh  //停止所有服务

2）启动zookeeper（需要一台一台的启动）这里以hadoop1为例子

  1. [root@hadoop1 hadoop]# /usr/local/zookeeper/bin/zkServer.sh start
  2. [root@hadoop1 hadoop]# sh /usr/local/zookeeper/conf/api.sh //利用之前写好的脚本查看
  3. node-0001        follower
  4. node-0002        leader
  5. node-0003        follower
  6. hadoop1        observer

3）新加一台机器hadoop2，这里之前有一台newnode，可以用这个作为hadoop2

  1. [root@newnode ~]# echo hadoop2 > /etc/hostname 
  2. [root@newnode ~]# hostname hadoop2

4）修改vim /etc/hosts

  1. [root@hadoop1 hadoop]# vim /etc/hosts
  2. 192.168.1.50  hadoop1
  3. 192.168.1.56  hadoop2
  4. 192.168.1.51  node-0001
  5. 192.168.1.52  node-0002
  6. 192.168.1.53  node-0003

5）同步到hadoop2，node-0001，node-0002，node-0003

  1. [root@hadoop1 hadoop]# for i in {51..53} 56; do rsync -aSH --delete /etc/hosts 192.168.1.$i:/etc/hosts  -e 'ssh' & done
  2. [1] 14355
  3. [2] 14356
  4. [3] 14357
  5. [4] 14358

6）配置SSH信任关系

注意：hadoop1和hadoop2互相连接不需要密码，hadoop2连接自己和node-0001，node-0002，node-0003同样不需要密码

  1. [root@hadoop2 ~]# vim /etc/ssh/ssh_config
  2. Host *
  3.         GSSAPIAuthentication yes
  4.         StrictHostKeyChecking no
  5. [root@hadoop1 hadoop]# cd /root/.ssh/
  6. [root@hadoop1 .ssh]# scp id_rsa id_rsa.pub  hadoop2:/root/.ssh/    
  7. //把hadoop1的公钥私钥考给hadoop2

7）所有的主机删除/var/hadoop/*

  1. [root@hadoop1 .ssh]# rm -rf /var/hadoop/*

5 案例5：配置namenode与resourcemanager高可用

5.1 问题

本案例要求：

5.2 步骤

实现此案例需要按照如下步骤进行。

步骤一：hadoop的高可用

1）配置 core-site

  1. [root@hadoop1 .ssh]# vim /usr/local/hadoop/etc/hadoop/core-site.xml
  2. <configuration>
  3. <property>
  4.         <name>fs.defaultFS</name>
  5.         <value>hdfs://nsdcluster</value>    
  6. //nsdcluster是随便起的名。相当于一个组，访问的时候访问这个组
  7.     </property>
  8.     <property>
  9.         <name>hadoop.tmp.dir</name>
  10.         <value>/var/hadoop</value>
  11.     </property>
  12.     <property>
  13.         <name>ha.zookeeper.quorum</name>
  14.         <value>node-0001:2181,node-0002:2181,node-0003:2181</value>    //zookeepe的地址
  15.     </property>
  16.     <property>
  17.         <name>hadoop.proxyuser.nfs.groups</name>
  18.         <value>*</value>
  19.     </property>
  20.     <property>
  21.         <name>hadoop.proxyuser.nfs.hosts</name>
  22.         <value>*</value>
  23.     </property>
  24. </configuration>

2）配置 hdfs-site

  1. [root@hadoop1 ~]# vim /usr/local/hadoop/etc/hadoop/hdfs-site.xml
  2. <configuration>
  3.     <property>
  4.         <name>dfs.replication</name>
  5.         <value>2</value>
  6.     </property>
  7.     <property>
  8.         <name>dfs.nameservices</name>
  9.         <value>nsdcluster</value>
  10.     </property>
  11.     <property>
  12.         <name>dfs.ha.namenodes.nsdcluster</name>                
  13. //nn1,nn2名称固定，是内置的变量，nsdcluster里面有nn1，nn2
  14.         <value>nn1,nn2</value>
  15.     </property>
  16.     <property>
  17.         <name>dfs.namenode.rpc-address.nsdcluster.nn1</name>        
  18. //声明nn1 8020为通讯端口，是hadoop1的rpc通讯端口
  19.         <value>hadoop1:8020</value>
  20.     </property>
  21.     <property>
  22.         <name>dfs.namenode.rpc-address.nsdcluster.nn2</name>        
  23. //声明nn2是谁，hadoop2的rpc通讯端口
  24.         <value>hadoop2:8020</value>
  25.     </property>
  26.     <property>
  27.         <name>dfs.namenode.http-address.nsdcluster.nn1</name>    
  28. //hadoop1的http通讯端口
  29.         <value>hadoop1:50070</value>
  30.     </property>
  31.     <property>
  32.         <name>dfs.namenode.http-address.nsdcluster.nn2</name>     
  33. //hadoop1和hadoop2的http通讯端口
  34.         <value>hadoop2:50070</value>
  35.     </property>
  36.     <property>
  37.         <name>dfs.namenode.shared.edits.dir</name>        
  38. //指定namenode元数据存储在journalnode中的路径
  39.         <value>qjournal://node-0001:8485;node-0002:8485;node-0003:8485/nsdcluster</value>
  40.     </property>
  41.     <property>
  42.         <name>dfs.journalnode.edits.dir</name>            
  43. //指定journalnode日志文件存储的路径
  44.         <value>/var/hadoop/journal</value>
  45.     </property>
  46.     <property>
  47.         <name>dfs.client.failover.proxy.provider.nsdcluster</name>    
  48. //指定HDFS客户端连接active namenode的java类
  49.         <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
  50.     </property>
  51.     <property>
  52.         <name>dfs.ha.fencing.methods</name>                    //配置隔离机制为ssh
  53.         <value>sshfence</value>
  54.     </property>
  55.     <property>
  56.         <name>dfs.ha.fencing.ssh.private-key-files</name>    //指定密钥的位置
  57.         <value>/root/.ssh/id_rsa</value>
  58.     </property>
  59.     <property>
  60.         <name>dfs.ha.automatic-failover.enabled</name>        //开启自动故障转移
  61.         <value>true</value>                
  62.     </property>
  63. </configuration>

3）配置yarn-site

  1. [root@hadoop1 ~]# vim /usr/local/hadoop/etc/hadoop/yarn-site.xml
  2. <configuration>
  3.  
  4. <!-- Site specific YARN configuration properties -->
  5.     <property>
  6.         <name>yarn.nodemanager.aux-services</name>
  7.         <value>mapreduce_shuffle</value>
  8.     </property>
  9.     <property>
  10.         <name>yarn.resourcemanager.ha.enabled</name>
  11.         <value>true</value>
  12.     </property> 
  13.     <property>
  14.         <name>yarn.resourcemanager.ha.rm-ids</name>        //rm1,rm2代表hadoop1和hadoop2
  15.         <value>rm1,rm2</value>
  16.     </property>
  17.     <property>
  18.         <name>yarn.resourcemanager.recovery.enabled</name>
  19.         <value>true</value>
  20.     </property>
  21.     <property>
  22.         <name>yarn.resourcemanager.store.class</name>
  23.         <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
  24.     </property>
  25.     <property>
  26.         <name>yarn.resourcemanager.zk-address</name>
  27.         <value>node-0001:2181,node-0002:2181,node-0003:2181</value>
  28.     </property>
  29.     <property>
  30.         <name>yarn.resourcemanager.cluster-id</name>
  31.         <value>yarn-ha</value>
  32.     </property>
  33.     <property>
  34.         <name>yarn.resourcemanager.hostname.rm1</name>
  35.         <value>hadoop1</value>
  36.     </property>
  37.     <property>
  38.         <name>yarn.resourcemanager.hostname.rm2</name>
  39.         <value>hadoop2</value>
  40.     </property>
  41. </configuration>

6 案例6：启动服务，验证高可用

6.1 问题

本案例要求：

6.2 步骤

实现此案例需要按照如下步骤进行。

步骤一：hadoop高可用的验证

1）同步到hadoop2，node-0001，node-0002，node-0003

  1. [root@hadoop1 ~]# for i in {51..53} 56; do rsync -aSH --delete /usr/local/hadoop/ 192.168.1.$i:/usr/local/hadoop  -e 'ssh' & done
  2. [1] 25411
  3. [2] 25412
  4. [3] 25413
  5. [4] 25414

2）删除所有机器上面的/user/local/hadoop/logs，方便排错

  1. [root@hadoop1 ~]# for i in {50..53} 56; do ssh 192.168.1.$i rm -rf /usr/local/hadoop/logs ; done

3）同步配置

  1. [root@hadoop1 ~]# for i in {51..53} 56; do rsync -aSH --delete /usr/local/hadoop 192.168.1.$i:/usr/local/hadoop -e 'ssh' & done
  2.  
  3. [1] 28235
  4. [2] 28236
  5. [3] 28237
  6. [4] 28238

4）初始化ZK集群

  1. [root@hadoop1 ~]# /usr/local/hadoop/bin/hdfs zkfc -formatZK 
  2. ...
  3. 18/09/11 15:43:35 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/nsdcluster in ZK    //出现Successfully即为成功
  4. ...

5）在node-0001，node-0002，node-0003上面启动journalnode服务（以node-0001为例子）

  1. [root@node-0001 ~]# /usr/local/hadoop/sbin/hadoop-daemon.sh start journalnode 
  2. starting journalnode, logging to /usr/local/hadoop/logs/hadoop-root-journalnode-node-0001.out
  3. [root@node-0001 ~]# jps
  4. 29262 JournalNode
  5. 26895 QuorumPeerMain
  6. 29311 Jps

6）格式化，先在node-0001，node-0002，node-0003上面启动journalnode才能格式化

  1. [root@hadoop1 ~]# /usr/local/hadoop//bin/hdfs  namenode  -format   
  2. //出现Successfully即为成功
  3. [root@hadoop1 hadoop]# ls /var/hadoop/
  4. dfs

7）hadoop2数据同步到本地 /var/hadoop/dfs

  1. [root@hadoop2 ~]# cd /var/hadoop/
  2. [root@hadoop2 hadoop]# ls
  3. [root@hadoop2 hadoop]# rsync -aSH  hadoop1:/var/hadoop/  /var/hadoop/
  4. [root@hadoop2 hadoop]# ls
  5. dfs

8）初始化 JNS

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs namenode -initializeSharedEdits
  2. 18/09/11 16:26:15 INFO client.QuorumJournalManager: Successfully started new epoch 1        //出现Successfully，成功开启一个节点

9）停止 journalnode 服务（node-0001，node-0002，node-0003）

  1. [root@node-0001 hadoop]# /usr/local/hadoop/sbin/hadoop-daemon.sh stop journalnode
  2. stopping journalnode
  3. [root@node-0001 hadoop]# jps
  4. 29346 Jps
  5. 26895 QuorumPeerMain

步骤二：启动集群

1）hadoop1上面操作

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/sbin/start-all.sh  //启动所有集群
  2. This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
  3. Starting namenodes on [hadoop1 hadoop2]
  4. hadoop1: starting namenode, logging to /usr/local/hadoop/logs/hadoop-root-namenode-hadoop1.out
  5. hadoop2: starting namenode, logging to /usr/local/hadoop/logs/hadoop-root-namenode-hadoop2.out
  6. node-0002: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-node-0002.out
  7. node-0003: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-node-0003.out
  8. node-0001: starting datanode, logging to /usr/local/hadoop/logs/hadoop-root-datanode-node-0001.out
  9. Starting journal nodes [node-0001 node-0002 node-0003]
  10. node-0001: starting journalnode, logging to /usr/local/hadoop/logs/hadoop-root-journalnode-node-0001.out
  11. node-0003: starting journalnode, logging to /usr/local/hadoop/logs/hadoop-root-journalnode-node-0003.out
  12. node-0002: starting journalnode, logging to /usr/local/hadoop/logs/hadoop-root-journalnode-node-0002.out
  13. Starting ZK Failover Controllers on NN hosts [hadoop1 hadoop2]
  14. hadoop1: starting zkfc, logging to /usr/local/hadoop/logs/hadoop-root-zkfc-hadoop1.out
  15. hadoop2: starting zkfc, logging to /usr/local/hadoop/logs/hadoop-root-zkfc-hadoop2.out
  16. starting yarn daemons
  17. starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-hadoop1.out
  18. node-0002: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node-0002.out
  19. node-0001: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node-0001.out
  20. node-0003: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-root-nodemanager-node-0003.out

2）hadoop2上面操作

  1. [root@hadoop2 hadoop]# /usr/local/hadoop/sbin/yarn-daemon.sh start resourcemanager
  2. starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-root-resourcemanager-hadoop2.out

3）查看集群状态

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn1
  2. active
  3. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn2
  4. standby
  5. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn rmadmin -getServiceState rm1
  6. active
  7. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn rmadmin -getServiceState rm2
  8. standby

4）查看节点是否加入

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs dfsadmin -report
  2. ...
  3. Live datanodes (3):    //会有三个节点
  4. ...
  5. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn  node  -list
  6. Total Nodes:3
  7.          Node-Id         Node-State    Node-Http-Address    Number-of-Running-Containers
  8.      node-0002:43307            RUNNING           node-0002:8042                               0
  9.      node-0001:34606            RUNNING           node-0001:8042                               0
  10.      node-0003:36749            RUNNING           node-0003:8042                               0

步骤三：访问集群

1）查看并创建

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hadoop  fs -ls  /
  2. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hadoop  fs -mkdir /aa //创建aa
  3. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hadoop  fs -ls  /        //再次查看
  4.  
  5. Found 1 items
  6. drwxr-xr-x   - root supergroup          0 2018-09-11 16:54 /aa
  7.  
  8. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hadoop  fs -put *.txt /aa
  9. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hadoop  fs -ls hdfs://nsdcluster/aa  
  10. //也可以这样查看
  11. Found 3 items
  12. -rw-r--r--  2 root supergroup 86424 2018-09-11 17:00 hdfs://nsdcluster/aa/LICENSE.txt
  13. -rw-r--r--  2 root supergroup 14978 2018-09-11 17:00 hdfs://nsdcluster/aa/NOTICE.txt
  14. -rw-r--r--  2 root supergroup 1366 2018-09-11 17:00 hdfs://nsdcluster/aa/README.txt

2）验证高可用，关闭 active namenode

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn1
  2. active
  3. [root@hadoop1 hadoop]# /usr/local/hadoop/sbin/hadoop-daemon.sh stop namenode
  4. stopping namenode
  5. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn1      
  6. //再次查看会报错
  7. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn2  
  8. //hadoop2由之前的standby变为active
  9. active
  10.  
  11. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn rmadmin -getServiceState rm1
  12. active
  13. [root@hadoop1 hadoop]# /usr/local/hadoop/sbin/yarn-daemon.sh stop resourcemanager  
  14. //停止resourcemanager 
  15. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn rmadmin -getServiceState rm2
  16. active

3） 恢复节点

  1. [root@hadoop1 hadoop]# /usr/local/hadoop/sbin/hadoop-daemon.sh start namenode       
  2. //启动namenode
  3. [root@hadoop1 hadoop]# /usr/local/hadoop/sbin/yarn-daemon.sh start resourcemanager 
  4. //启动resourcemanager
  5. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/hdfs haadmin -getServiceState nn1      
  6. //查看
  7. [root@hadoop1 hadoop]# /usr/local/hadoop/bin/yarn rmadmin -getServiceState rm1      
  8. //查看