淘宝加盟网站建设,dw手机网站怎么做,专业网页设计,网上教育培训机构排名目录
效果图
环境准备
安装 influxdb
安装我们刚刚下载 influxdb rpm文件
查看默认配置
修改参数
启动 influxdb
查看启动状态
设置基本配置 influxdb 其他命令扩展
安装 jmxtrans
可能遇到的异常
验证jmxtrans是否成功运行
安装 Grafana 安装
influxDB 与 Grafa…
目录
效果图
环境准备
安装 influxdb
安装我们刚刚下载 influxdb rpm文件
查看默认配置
修改参数
启动 influxdb
查看启动状态
设置基本配置 influxdb 其他命令扩展
安装 jmxtrans
可能遇到的异常
验证jmxtrans是否成功运行
安装 Grafana 安装
influxDB 与 Grafana 监控模板 效果图
先看下 环境准备
JDK1.8
jmxtrans 安装包
# 我们系统是CentOS这里选择rpm
https://github.com/downloads/jmxtrans/jmxtrans/jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm
# 其他系统在下面链接中可以找到
https://github.com/jmxtrans/jmxtrans/downloads# 下载源码后面编译使用
https://github.com/jmxtrans/jmxtrans/releases
# 我这里使用的271版本
https://github.com/jmxtrans/jmxtrans/archive/jmxtrans-parent-271.tar.gz
influxdb 安装包
# 下载安装包
https://dl.influxdata.com/influxdb/releases/influxdb-1.8.0.x86_64.rpm
# 官网下载页面
https://portal.influxdata.com/downloads/
Grafana 安装包
https://dl.grafana.com/oss/release/grafana-6.7.3-1.x86_64.rpm
安装 influxdb
安装我们刚刚下载 influxdb rpm文件
rpm -ivh influxdb-1.8.0.x86_64.rpm
查看默认配置 influxd configMerging with configuration at: /etc/influxdb/influxdb.conf
reporting-disabled false
bind-address 127.0.0.1:8088[meta]dir /var/lib/influxdb/metaretention-autocreate truelogging-enabled true[data]dir /var/lib/influxdb/dataindex-version inmemwal-dir /var/lib/influxdb/walwal-fsync-delay 0svalidate-keys falsequery-log-enabled truecache-max-memory-size 1073741824cache-snapshot-memory-size 26214400cache-snapshot-write-cold-duration 10m0scompact-full-write-cold-duration 4h0m0scompact-throughput 50331648compact-throughput-burst 50331648max-series-per-database 1000000max-values-per-tag 100000max-concurrent-compactions 0max-index-log-file-size 1048576series-id-set-cache-size 100series-file-max-concurrent-snapshot-compactions 0trace-logging-enabled falsetsm-use-madv-willneed false[coordinator]write-timeout 10smax-concurrent-queries 0query-timeout 0slog-queries-after 0smax-select-point 0max-select-series 0max-select-buckets 0[retention]enabled truecheck-interval 30m0s[shard-precreation]enabled truecheck-interval 10m0sadvance-period 30m0s[monitor]store-enabled truestore-database _internalstore-interval 10s[subscriber]enabled truehttp-timeout 30sinsecure-skip-verify falseca-certs write-concurrency 40write-buffer-size 1000[http]enabled truebind-address :8086auth-enabled falselog-enabled truesuppress-write-log falsewrite-tracing falseflux-enabled falseflux-log-enabled falsepprof-enabled truepprof-auth-enabled falsedebug-pprof-enabled falseping-auth-enabled falsehttps-enabled falsehttps-certificate /etc/ssl/influxdb.pemhttps-private-key max-row-limit 0max-connection-limit 0shared-secret realm InfluxDBunix-socket-enabled falseunix-socket-permissions 0777bind-socket /var/run/influxdb.sockmax-body-size 25000000access-log-path max-concurrent-write-limit 0max-enqueued-write-limit 0enqueued-write-timeout 30000000000[logging]format autolevel infosuppress-logo false[[graphite]]enabled falsebind-address :2003database graphiteretention-policy protocol tcpbatch-size 5000batch-pending 10batch-timeout 1sconsistency-level oneseparator .udp-read-buffer 0[[collectd]]enabled falsebind-address :25826database collectdretention-policy batch-size 5000batch-pending 10batch-timeout 10sread-buffer 0typesdb /usr/share/collectd/types.dbsecurity-level noneauth-file /etc/collectd/auth_fileparse-multivalue-plugin split[[opentsdb]]enabled falsebind-address :4242database opentsdbretention-policy consistency-level onetls-enabled falsecertificate /etc/ssl/influxdb.pembatch-size 1000batch-pending 5batch-timeout 1slog-point-errors true[[udp]]enabled falsebind-address :8089database udpretention-policy batch-size 5000batch-pending 10read-buffer 0batch-timeout 1sprecision [continuous_queries]log-enabled trueenabled truequery-stats-enabled falserun-interval 1s[tls]min-version max-version
修改参数
默认influxDB使用以下端口
8086: 用于客户端和服务端交互的HTTP API8088: 用于提供备份和恢复的RPC服务
我这里修改配置文件,使用8087端口我这里8088和其他服务冲突了
同时修改了数据保存的路径 vim /etc/influxdb/influxdb.confbind-address 127.0.0.1:8087
# metadata 保存路径
dir /root/jast/influxdb/meta
#数据保存路径
dir /root/jast/influxdb/data
#write-ahead-log(WAL)保存路径
wal-dir /root/jast/influxdb/wal 注意选择的路径要有权限否则会启动失败
启动 influxdb
systemctl start influxdb
查看启动状态
systemctl status influxdb 此时influxdb已经启动成功
我们也可以指定配置文件启动在/etc/influxdb/influxdb.conf ,是默认的目录也可以不指定
influxd -config /etc/influxdb/influxdb.conf
设置基本配置
在服务器 输入 influx 进入交互页面
[rootecs-t-001-0001 influx]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0创建用户
CREATE USER admin WITH PASSWORD 123456 WITH ALL PRIVILEGES 创建数据库后面保存数据用
create database jmxDB
查看是否创建成功
[rootecs-t-001-0001 jmxtrans]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0show databases
name: databases
name
----
_internal
jmxDBinfluxdb 其他命令扩展
#创建数据库
create database db_name
#显示所有的数据库
show databases
#删除数据库
drop database db_name
#使用数据库
use db_name
#显示该数据库中所有的表
show measurements
#创建表直接在插入数据的时候指定表名
insert test,host127.0.0.1,monitor_nametest count1
#删除表
drop measurement measurement_name
#退出
quit 安装 jmxtrans
安装我们刚刚下载jmxtrans rpm文件
rpm -ivh jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm
安装完成后默认安装目录在
[rootecs-t-001-0001 jmxtrans]# whereis jmxtrans
jmxtrans: /usr/share/jmxtrans这里我们先简单配置Kafka 的Memory 监控其他配置在文末统一整理
我们创建json文件供jmxtrans读取json文件名称自己根据业务取名即可
{servers : [ {port : 9393,host : 172.11.0.1,queries : [ {obj : java.lang:typeMemory,attr : [ HeapMemoryUsage, NonHeapMemoryUsage ],resultAlias:jvmMemory,outputWriters : [ {class : com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory,url : http://172.11.0.1:8086/,username : admin,password : 123456,database : jmxDB,tags : {application : kafka_server}} ]} ]} ]
}
简单解释一下上面的说明
port: 我们要监控的Kafka JMX端口
host我们要监控的Kafka host
resultAlias自定义表名收集到的数据会存入influxdb的定义的表中自动创建
outputWriters为连接influxdb的配置
class不需要修改
urlinfluxdb的机器端口默认端口8086
username和passwordinfluxdb的用户和密码
databaseinfluxdb数据库我们刚刚创建的
启动之前我们把 /usr/share/jmxtrans 目录下的所有 .json 文件换个名因为它会默认会读取 /usr/share/jmxtrans 目录下的所有json文件
在 /usr/share/jmxtrans 目录下启动 jmxtrans.sh
jmxtrans.sh start
到这里正常来说就是要启动成功了我们先说下可能遇到的异常
可能遇到的异常
异常1
[rootecs-t-001-0001 jmxtrans]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize384m; support was removed in 8.0
MaxTenuringThreshold of 16 is invalid; must be between 0 and 15
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
MaxTenuringThreshold 这个参数用于控制对象能经历多少次Minor GC才晋升到旧生代
提示设置的是16但是范围在0-15我们直接修改一下启动脚本 jmxtrans.sh vim jmxtrans.shGC_OPTS${GC_OPTS:--Xms${HEAP_SIZE}M -Xmx${HEAP_SIZE}M -XX:UseConcMarkSweepGC -XX:NewRatio${NEW_RATIO} -XX:NewSize${NEW_SIZE}m -XX:MaxNewSize${NEW_SIZE}m -XX:MaxTenuringThreshold15 -XX:GCTimeRatio9 -XX:PermSize${PERM_SIZE}m -XX:MaxPermSize${MAX_PERM_SIZE}m -XX:UseTLAB -XX:CMSInitiatingOccupancyFraction${IO_FRACTION} -XX:CMSIncrementalMode -XX:CMSIncrementalPacing -XX:ParallelGCThreads${CPU_CORES} -Dsun.rmi.dgc.server.gcInterval28800000 -Dsun.rmi.dgc.client.gcInterval28800000}异常2
[rootecs-t-001-0001 jmxtrans]# Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize384m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
Exception in thread main com.googlecode.jmxtrans.util.LifecycleException: com.googlecode.jmxtrans.util.LifecycleException: Error parsing json: /var/lib/jmxtrans/kafka.jsonat com.googlecode.jmxtrans.JmxTransformer.start(JmxTransformer.java:146)at com.googlecode.jmxtrans.JmxTransformer.doMain(JmxTransformer.java:107)at com.googlecode.jmxtrans.JmxTransformer.main(JmxTransformer.java:92)
Caused by: com.googlecode.jmxtrans.util.LifecycleException: Error parsing json: /var/lib/jmxtrans/kafka.jsonat com.googlecode.jmxtrans.JmxTransformer.processFilesIntoServers(JmxTransformer.java:358)at com.googlecode.jmxtrans.JmxTransformer.startupSystem(JmxTransformer.java:301)at com.googlecode.jmxtrans.JmxTransformer.start(JmxTransformer.java:142)... 2 more
Caused by: java.lang.IllegalArgumentException: Invalid type id com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory (for id type Id.class): no such class foundat org.codehaus.jackson.map.jsontype.impl.ClassNameIdResolver.typeFromId(ClassNameIdResolver.java:89)at org.codehaus.jackson.map.jsontype.impl.TypeDeserializerBase._findDeserializer(TypeDeserializerBase.java:73)at org.codehaus.jackson.map.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:65)at org.codehaus.jackson.map.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:81)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:118)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:116)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:116)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:93)at org.codehaus.jackson.map.deser.CollectionDeserializer.deserialize(CollectionDeserializer.java:25)at org.codehaus.jackson.map.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:149)at org.codehaus.jackson.map.deser.SettableBeanProperty$MethodProperty.deserializeAndSet(SettableBeanProperty.java:237)at org.codehaus.jackson.map.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:496)at org.codehaus.jackson.map.deser.BeanDeserializer.deserialize(BeanDeserializer.java:350)at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:1980)at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1225)at com.googlecode.jmxtrans.util.JmxUtils.getJmxProcess(JmxUtils.java:494)at com.googlecode.jmxtrans.JmxTransformer.processFilesIntoServers(JmxTransformer.java:352)... 4 more说是解析 com.googlecode.jmxtrans.util.LifecycleException 异常这里需要我们自己编译一下jar包在文章开头我们下载过jmxtrans源码
在 项目目录下进行编译文末有我编译好的jar包 mvn clean package -Dmaven.test.skiptrue -DskipTeststrue;
编译完成我们需要的jar包在 jmxtrans-jmxtrans-parent-271\jmxtrans\target 目录下
jmxtrans-271-all.jar 就是我们需要用的jar包 将jar包传到jmxtrans目录下 我们对比一下发现我们编译的包是有这个类的而他自带的那个没有
[rootecs-t-001-0001 jmxtrans]# grep com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory ./jmxtrans-271-all.jar
Binary file ./jmxtrans-271-all.jar matches
[rootecs-t-001-0001 jmxtrans]# grep com.googlecode.jmxtrans.model.output.InfluxDbWriterFactory ./jmxtrans-all.jar
[rootecs-t-001-0001 jmxtrans]#
替换 jmxtrans.sh 中应用的 jmxtrans jar包名称
#JAR_FILE${JAR_FILE:-jmxtrans-all.jar}
JAR_FILE${JAR_FILE:-jmxtrans-271-all.jar}
再次启动即可 验证jmxtrans是否成功运行
进入 influx jmxDB数据库(我们之前创建的)查看表 show MEASUREMENTS我们发现自动创了jvmMemory
[rootecs-t-001-0001 jmxtrans]# influx
Connected to http://localhost:8086 version 1.8.0
InfluxDB shell version: 1.8.0show databases
name: databases
name
----
_internal
jmxDBuse jmxDB
Using database jmxDBshow
ERR: error parsing query: found EOF, expected CONTINUOUS, DATABASES, DIAGNOSTICS, FIELD, GRANTS, MEASUREMENT, MEASUREMENTS, QUERIES, RETENTION, SERIES, SHARD, SHARDS, STATS, SUBSCRIPTIONS, TAG, USERS at line 1, char 6show MEASUREMENTS
name: measurements
name
----
jvmMemory 具体查看数据发现有数据写入 至此 jmxtrans 已成功监控 Kafka JMX端口离成功更近了 安装 Grafana 安装
yum install grafana-6.7.3-1.x86_64.rpm
配置文件默认路径 /etc/grafana/grafana.ini 修改下web端口 vim /etc/grafana/grafana.ini# web页面端口默认3000
http_port 9099 启动服务并设置为开机启动
systemctl start grafana-server
systemctl enable grafana-server
查看启动状态
systemctl status grafana-server
访问 web页面 第一次登陆账号密码 是 admin/admin 登陆完成后会提示你设置密码 开始配置Grafana显示模板
点击DataSource 选择添加 InfluxDB 并填写基本信息 填写 influxDB 数据库信息 点击保存 此时InfluxDB配置完成 我们几区创建仪表盘 选择 添加一个查询 选择我们上面设置的KafkaMonitor因为我们只有一个默认的也是这个 简单修改一下sql因为我们上面只监控了JMX中的内存信息 注意如果上面使用tag进行划分这里就不要设置了否则都叫Jvm内存使用了 简单配置完成我们对比一下监控与jconsole监控对比 influxDB 与 Grafana 监控模板 模板下载
链接https://pan.baidu.com/s/1ld-Yhv7wVutRxslV084GoQ
提取码0pzr
复制这段内容后打开百度网盘手机App操作更方便哦
