ElasticSearch 集群监控
最近在做 ElasticSearch 的信息（集群和节点）监控，特此稍微整理下学到的东西。这篇文章主要介绍集群的监控。
要监控哪些 ElasticSearch metrics
Elasticsearch 提供了大量的 Metric，可以帮助您检测到问题的迹象，在遇到节点不可用、out-of-memory、long garbage collection times 的时候采取相应措施。但是指标太多了，有时我们并不需要这么多，这就需要我们进行筛选。
集群健康
一个 Elasticsearch 集群至少包括一个节点和一个索引。或者它 可能有一百个数据节点、三个单独的主节点，以及一小打客户端节点——这些共同操作一千个索引（以及上万个分片）。
不管集群扩展到多大规模，你都会想要一个快速获取集群状态的途径。 ClusterHealth API 充当的就是这个角色。你可以把它想象成是在一万英尺的高度鸟瞰集群。它可以告诉你安心吧一切都好，或者警告你集群某个地方有问题。
让我们执行一下 cluster-health API 然后看看响应体是什么样子的：
GET _cluster/health
和 Elasticsearch 里其他 API 一样， cluster-health 会返回一个 JSON 响应。这对自动化和告警系统来说，非常便于解析。响应中包含了和你集群有关的一些关键信息：
{"cluster_name": "elasticsearch_zach","status": "green","timed_out": false,"number_of_nodes": 1,"number_of_data_nodes": 1,"active_primary_shards": 10,"active_shards": 10,"relocating_shards": 0,"initializing_shards": 0,"unassigned_shards": 0}
响应信息中最重要的一块就是 status 字段。状态可能是三个值。
集群统计信息包含 集群的分片数，文档数，存储空间，缓存信息，内存作用率，插件内容，文件系统内容，JVM 作用状况，系统 CPU，OS 信息，段信息。
查看全部统计信息命令：
curl -XGET 'http://localhost:9200/_cluster/stats?human&pretty'
返回 JSON 结果：
{"timestamp": 1459427693515,"cluster_name": "elasticsearch","status": "green","indices": {"count": 2,"shards": {"total": 10,"primaries": 10,"replication": 0,"index": {"shards": {"min": 5,"max": 5,"avg": 5},"primaries": {"min": 5,"max": 5,"avg": 5},"replication": {"min": 0,"max": 0,"avg": 0}}},"docs": {"count": 10,"deleted": 0},"store": {"size": "16.2kb","size_in_bytes": 16684,"throttle_time": "0s","throttle_time_in_millis": 0},"fielddata": {"memory_size": "0b","memory_size_in_bytes": 0,"evictions": 0},"query_cache": {"memory_size": "0b","memory_size_in_bytes": 0,"total_count": 0,"hit_count": 0,"miss_count": 0,"cache_size": 0,"cache_count": 0,"evictions": 0},"completion": {"size": "0b","size_in_bytes": 0},"segments": {"count": 4,"memory": "8.6kb","memory_in_bytes": 8898,"terms_memory": "6.3kb","terms_memory_in_bytes": 6522,"stored_fields_memory": "1.2kb","stored_fields_memory_in_bytes": 1248,"term_vectors_memory": "0b","term_vectors_memory_in_bytes": 0,"norms_memory": "384b","norms_memory_in_bytes": 384,"doc_values_memory": "744b","doc_values_memory_in_bytes": 744,"index_writer_memory": "0b","index_writer_memory_in_bytes": 0,"version_map_memory": "0b","version_map_memory_in_bytes": 0,"fixed_bit_set": "0b","fixed_bit_set_memory_in_bytes": 0,"file_sizes": {}},"percolator": {"num_queries": 0}},"nodes": {"count": {"total": 1,"data": 1,"coordinating_only": 0,"master": 1,"ingest": 1},"versions": ["5.6.3"],"os": {"available_processors": 8,"allocated_processors": 8,"names": [{"name": "Mac OS X","count": 1}],"mem" : {"total" : "16gb","total_in_bytes" : 17179869184,"free" : "78.1mb","free_in_bytes" : 81960960,"used" : "15.9gb","used_in_bytes" : 17097908224,"free_percent" : 0,"used_percent" : 100}},"process": {"cpu": {"percent": 9},"open_file_descriptors": {"min": 268,"max": 268,"avg": 268}},"jvm": {"max_uptime": "13.7s","max_uptime_in_millis": 13737,"versions": [{"version": "1.8.0_74","vm_name": "Java HotSpot(TM) 64-Bit Server VM","vm_version": "25.74-b02","vm_vendor": "Oracle Corporation","count": 1}],"mem": {"heap_used": "57.5mb","heap_used_in_bytes": 60312664,"heap_max": "989.8mb","heap_max_in_bytes": 1037959168},"threads": 90},"fs": {"total": "200.6gb","total_in_bytes": 215429193728,"free": "32.6gb","free_in_bytes": 35064553472,"available": "32.4gb","available_in_bytes": 34802409472},"plugins": [{"name": "analysis-icu","version": "5.6.3","description": "The ICU Analysis plugin integrates Lucene ICU module into elasticsearch, adding ICU relates analysis components.","classname": "org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin","has_native_controller": false},{"name": "ingest-geoip","version": "5.6.3","description": "Ingest processor that uses looksup geo data based on ip adresses using the Maxmind geo database","classname": "org.elasticsearch.ingest.geoip.IngestGeoIpPlugin","has_native_controller": false},{"name": "ingest-user-agent","version": "5.6.3","description": "Ingest processor that extracts information from a user agent","classname": "org.elasticsearch.ingest.useragent.IngestUserAgentPlugin","has_native_controller": false}]}}
内存使用和 GC 指标
在运行 Elasticsearch 时，内存是您要密切监控的关键资源之一。 Elasticsearch 和 Lucene 以两种方式利用节点上的所有可用 RAM：JVM heap 和文件系统缓存。 Elasticsearch 运行在Java虚拟机（JVM）中，这意味着JVM垃圾回收的持续时间和频率将成为其他重要的监控领域。
上面返回的 JSON监控的指标有我个人觉得有这些：
nodes.successful
nodes.failed
nodes.total
nodes.mem.used_percent
nodes.process.cpu.percent
nodes.jvm.mem.heap_used
可以看到 JSON 文件是很复杂的，如果从这复杂的 JSON 中获取到对应的指标（key）的值呢，这里请看文章 ：JsonPath —— JSON 解析神器
最后
这里主要讲下 ES 集群的一些监控信息，有些监控指标是个人觉得需要监控的，但是具体情况还是得看需求了。下篇文章主要讲节点的监控信息。转载请注明地址：http://www.54tianzhisheng.cn/2017/10/15/ElasticSearch-cluster-health-metrics/
参考资料
1、How to monitor Elasticsearch performance
2、ElasticSearch 性能监控
3、cluster-health
4、cluster-stats
相关阅读
1、Elasticsearch 默认分词器和中分分词器之间的比较及使用方法
2、全文搜索引擎 Elasticsearch 集群搭建入门教程