建设银行激活网站,2021年4月重大新闻事件摘抄,网站登记备案 个人,免费网站空间注册目录 一、实验
1.环境
2.Yarn 节点扩容
3.Yarn 节点缩容
二、问题
1.yarn启动服务报错 一、实验
1.环境
#xff08;1#xff09;主机
表1 主机
主机架构软件版本IP备注hadoop NameNode #xff08;已部署#xff09; SecondaryNameNode #xff08;已部署…目录 一、实验
1.环境
2.Yarn 节点扩容
3.Yarn 节点缩容
二、问题
1.yarn启动服务报错 一、实验
1.环境
1主机
表1 主机
主机架构软件版本IP备注hadoop NameNode 已部署 SecondaryNameNode 已部署 ResourceManager已部署 hadoop 2.7.7192.168.204.50 node01 DataNode已部署 NodeManager已部署 hadoop 2.7.7192.168.204.51node02 DataNode已部署 NodeManager已部署 hadoop 2.7.7192.168.204.52node03 DataNode已部署 NodeManager已部署 hadoop 2.7.7192.168.204.53node05NodeManager hadoop 2.7.7192.168.204.55
2查看jps
hadoop节点
[roothadoop hadoop]# jpsnode01节点 node02节点 node03节点 3 查看节点
[roothadoop hadoop]# ./bin/yarn node -list
24/03/14 13:40:21 INFO client.RMProxy: Connecting to ResourceManager at hadoop/192.168.204.50:8032
Total Nodes:3Node-Id Node-State Node-Http-Address Number-of-Running-Containersnode01:40551 RUNNING node01:8042 0node02:46073 RUNNING node02:8042 0node03:40601 RUNNING node03:8042 02.Yarn 节点扩容
1查看IP
地址为192.168.204.55
[rootlocalhost ~]# ip addr2安全机制
查看
[rootlocalhost ~]# sestatus关闭
[rootlocalhost ~]# vim /etc/selinux/config
……
SELINUXdisabled
…… 再次查看需要reboot重启
[rootlocalhost ~]# sestatus3防火墙
关闭
[rootlocalhost ~]# systemctl stop firewalld
[rootlocalhost ~]# systemctl mask firewalld4安装java
[rootlocalhost ~]# yum install -y java-1.8.0-openjdk-devel.x86_64查看
[rootlocalhost ~]# jps5修改主机名
[rootlocalhost ~]# hostnamectl set-hostname node05
[rootlocalhost ~]# bash6添加免密登录
[roothadoop ~]# cd /root/.ssh/
[roothadoop .ssh]# ls
authorized_keys id_rsa id_rsa.pub known_hosts
[roothadoop .ssh]# ssh-copy-id -i id_rsa.pub 192.168.204.55验证
[roothadoop .ssh]# ssh 192.168.204.557域名主机名(hadoop节点)
[roothadoop ~]# vim /etc/hosts
……
192.168.205.50 hadoop
192.168.205.51 node01
192.168.205.52 node02
192.168.205.53 node03
192.168.204.54 node04
192.168.204.55 node05 8同步域名配置文件 [roothadoop ~]# rsync -av /etc/hosts node01:/etc/
sending incremental file list
hostssent 382 bytes received 41 bytes 282.00 bytes/sec
total size is 291 speedup is 0.69
[roothadoop ~]# rsync -av /etc/hosts node02:/etc/
sending incremental file list
hostssent 382 bytes received 41 bytes 282.00 bytes/sec
total size is 291 speedup is 0.69
[roothadoop ~]# rsync -av /etc/hosts node03:/etc/
sending incremental file list
hostssent 382 bytes received 41 bytes 846.00 bytes/sec
total size is 291 speedup is 0.69
[roothadoop ~]# rsync -av /etc/hosts node05:/etc/
Warning: Permanently added node05 (ECDSA) to the list of known hosts.
sending incremental file list
hostssent 382 bytes received 41 bytes 846.00 bytes/sec
total size is 291 speedup is 0.699同步Hadoop文件
[roothadoop ~]# rsync -aXSH --delete /usr/local/hadoop node05:/usr/local/(10) 清除日志node05节点
[rootnode05 ~]# cd /usr/local/hadoop/
[rootnode05 hadoop]# ls
bin etc include lib libexec LICENSE.txt logs NOTICE.txt README.txt sbin share
[rootnode05 hadoop]# cd logs/
[rootnode05 logs]# ls
hadoop-root-balancer-hadoop.log hadoop-root-namenode-hadoop.out hadoop-root-secondarynamenode-hadoop.out yarn-root-resourcemanager-hadoop.log
hadoop-root-balancer-hadoop.out hadoop-root-namenode-hadoop.out.1 hadoop-root-secondarynamenode-hadoop.out.1 yarn-root-resourcemanager-hadoop.out
hadoop-root-namenode-hadoop.log hadoop-root-secondarynamenode-hadoop.log SecurityAuth-root.audit
[rootnode05 logs]# rm -f *
[rootnode05 logs]# ls
[rootnode05 logs]#11启动服务 node05节点
[rootnode05 hadoop]# ./sbin/yarn-daemon.sh start nodemanager查看jps 出现新角色 (15) 验证 hadoop节点
服务已互通
[roothadoop hadoop]# ./bin/yarn node -list
24/03/14 18:07:06 INFO client.RMProxy: Connecting to ResourceManager at hadoop/192.168.204.50:8032
Total Nodes:4Node-Id Node-State Node-Http-Address Number-of-Running-Containersnode01:40551 RUNNING node01:8042 0node05:39920 RUNNING node05:8042 0node02:46073 RUNNING node02:8042 0node03:40601 RUNNING node03:8042 03.Yarn 节点缩容
1删除节点 node05节点
[rootnode05 hadoop]# ./sbin/yarn-daemon.sh stop nodemanager查看jps 2验证 hadoop节点
这时node05节点还在(因为存在检测超时)
[roothadoop hadoop]# ./bin/yarn node -list(3) 停止服务hadoop节点
[roothadoop hadoop]# ./sbin/yarn-daemon.sh stop resourcemanager4启动服务hadoop节点
[roothadoop hadoop]# ./sbin/yarn-daemon.sh start resourcemanager5验证 hadoop节点
这时暂无节点需要等待30秒-1分钟
[roothadoop hadoop]# ./bin/yarn node -list6再次验证 hadoop节点
node05节点已移除
[roothadoop hadoop]# ./bin/yarn node -list二、问题
1.yarn启动服务报错
1报错
错误: 找不到或无法加载主类 nodemanager2原因分析
命令错误。
3解决方法
修改命令hadoop-daemon.sh 改为 yarn-daemon.sh
[rootnode05 hadoop]# ./sbin/yarn-daemon.sh start nodemanager
