接上一篇搭建文章中,nginx.conf 定义的log格式是:

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                     '"$http_user_agent" "$http_x_forwarded_for"';

实际的nginx日志实例如下:

192.168.1.137 - - [23/May/2022:14:33:18 +0800] "GET /index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1 HTTP/1.1" 200 5 "http://192.168.1.240:10003/index.php?m=bug&f=browse&productID=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50"

所以Logstash中filter的grok表达式是:

%{IPORHOST:remote_ip} - %{DATA:user_name} \[%{HTTPDATE:time}\] \"%{WORD:method} %{DATA:url} HTTP/%{NUMBER:http_version}\" %{NUMBER:response_code} %{NUMBER:body_sent:bytes} \"%{DATA:referrer}\" \"%{DATA:user_agent}\"

ELK的版本是7.16.1

1.Elasticsearch的RESTful API

1.1 RESTful API介绍

es的RESTful API提供了众多的api和丰富的功能;常用的API分为如下几类

  • Document APIs :es的文档的CRUD操作相关API
  • Search APIs:查询检索相关的API
  • Indices APIs:索引管理相关API
  • cat APIs:集群健康状态、索引信息、分片信息等等,输出的是在命令行界面下更友好的制表信息
  • Cluster APIs:es集群查看和管理配置相关API

想要系统性的了解请参考:
www.elastic.co/guide/en/elasticsearch/reference/7.16/index.html

1.2 常用RESTful API使用

让我们借助Kibana的 Dev Tools工具来了解一下常用的API使用。

GET /

响应结果如下:

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
{
  "name" : "node-1",
  "cluster_name" : "es-cluster",
  "cluster_uuid" : "fx1Mm_OlQ6KCA41PQdLtHw",
  "version" : {
    "number" : "7.16.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "5b38441b16b1ebb16a27c107a4c3865776e20c53",
    "build_date" : "2021-12-11T00:29:38.865893768Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

可以查看es的简单信息包括版本号、集群名称、lucene版本号等。

GET nginx-access-log*

可以查看到es对应的index的aliases,mappings,settings信息。其中重要的是mappings,mapping是es中定义文档和字段是如何被存储和索引的,例如定义哪些字符串字段应被视为全文字段;哪些字段包含数字、日期、地理位置;日期字段的日期格式以及对动态增加的字段的一些自定义规则。

mappings 返回数据片段举例

{
  "nginx-access-log-2022.05.23" : {
    "aliases" : { },
    "mappings" : {                    // mappings 都有哪些属性
      "properties" : {
        "@timestamp" : {              // @timestamp字段是日期类型
          "type" : "date"
        },
        "@version" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "agent" : {
          "properties" : {
            "ephemeral_id" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "hostname" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
            "id" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
        ......
        }
    }
}

1.3 Search APIs的常见用法

GET nginx-access-log*/_search

查询nginx-access-log为前缀的索引全部数据(默认只返回前10条数据),查询结果如下

{
  "took" : 620,             //查询所花费的毫秒数
  "timed_out" : false,
  "_shards" : {
    "total" : 2,            //查询了几个分片
    "successful" : 2,       //查询执行成功的分片数
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {                //查询命中的数据
    "total" : {
      "value" : 10000,      //indexpattern下的查询命中的全部数据条数
      "relation" : "gte"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "nginx-access-log-2022.05.23",       //表示数据来自那个索引
        "_type" : "_doc",
        "_id" : "f-FB8IABH99_gnUyqP4c",
        "_score" : 1.0,
        "_source" : {       //表示原始数据
          "log" : {
            "offset" : 6699146,
            "file" : {
              "path" : "/var/log/nginx/zentaopms.access.log"
            }
          },
          "http_version" : "1.1",
          "body_sent" : "5",
          "input" : {
            "type" : "log"
          },
          "remote_ip" : "192.168.1.35",
          "agent" : {
            "id" : "dbfdd21f-edf6-4e04-beca-78d20dbea316",
            "type" : "filebeat",
            "version" : "7.16.1",
            "ephemeral_id" : "35990523-3475-4235-bbae-a577c583b46e",
            "name" : "69760ea94693",
            "hostname" : "69760ea94693"
          },
          "host" : {
            "name" : "69760ea94693"
          },
          "user_name" : "-",
          "response_code" : "200",
          "referrer" : "http://192.168.1.240:10003/index.php?m=bug&f=browse&root=1&branch=&type=byModule&param=1",
          "method" : "GET",
          "ecs" : {
            "version" : "1.12.0"
          },
          "@timestamp" : "2022-05-23T09:31:14.322Z",
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "@version" : "1",
          "tags" : [
            "_jsonparsefailure",
            "beats_input_codec_json_applied"
          ],
          "fields" : {
            "log_source" : "nginx"
          },
          "time" : "09/Nov/2021:14:43:24 +0800",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
        }
      },
    ]
  }
}

如果想要查询status=200的最近1条数据应该怎么写呢?我们可以分别使用URI search和Request body两种查询方式。

GET nginx-access-log*/_search?q=response_code:200&size=1&sort=@timestamp:desc

加上参数q=status:200 表示查询status字段值为200的数据,size=1表示只返回前1条数据,sort=@timestamp:desc表示使用@timestamp字段倒序排序
使用URI search的查询方式,更多内容请参考www.elastic.co/guide/en/elasticsearch/reference/7.16/search.html

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
{
  "took" : 9,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "z-JI9YABH99_gnUyL-CT",
        "_score" : null,
        "_source" : {
          "log" : {
            "offset" : 25277051,
            "file" : {
              "path" : "/var/log/nginx/zentaopms.access.log"
            }
          },
          "http_version" : "1.1",
          "body_sent" : "5",
          "input" : {
            "type" : "log"
          },
          "remote_ip" : "192.168.1.100",
          "agent" : {
            "type" : "filebeat",
            "id" : "dbfdd21f-edf6-4e04-beca-78d20dbea316",
            "version" : "7.16.1",
            "ephemeral_id" : "35990523-3475-4235-bbae-a577c583b46e",
            "name" : "69760ea94693",
            "hostname" : "69760ea94693"
          },
          "host" : {
            "name" : "69760ea94693"
          },
          "user_name" : "-",
          "response_code" : "200",
          "referrer" : "http://192.168.1.240:10003/index.php?m=bug&f=view&bugID=431",
          "method" : "GET",
          "ecs" : {
            "version" : "1.12.0"
          },
          "@timestamp" : "2022-05-24T08:56:27.847Z",
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "@version" : "1",
          "tags" : [
            "_jsonparsefailure",
            "beats_input_codec_json_applied"
          ],
          "fields" : {
            "log_source" : "nginx"
          },
          "time" : "24/May/2022:16:56:25 +0800",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36"
        },
        "sort" : [
          1653382587847
        ]
      }
    ]
  }
}

如果我们只想查询_source中的某些字段,而不想要全部字段时,可以加上_source:[“字段1”,”字段2”]进行过滤

GET nginx-access-log*/_search?q=response_code:200&size=2&sort=@timestamp:desc
{
  "_source": ["url", "user_agent"]
}

或者

GET nginx-access-log*/_search
{
  "size":2,
  "_source": ["url", "user_agent"],
    "query":{
    "term":{
      "response_code":200
    }
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

返回结果如下:

{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "z-JI9YABH99_gnUyL-CT",
        "_score" : null,
        "_source" : {
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36"
        },
        "sort" : [
          1653382587847
        ]
      },
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "zuJD9YABH99_gnUyruDP",
        "_score" : null,
        "_source" : {
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36"
        },
        "sort" : [
          1653382292775
        ]
      }
    ]
  }
}

_source还可以有如下用法

"_source": {
        "includes": [ "http*", "geoip.*" ], //包含的字段,http开头的字段,geoip的嵌套类型字段
        "excludes": [ "xxx" ] //排除哪些字段
}

"_source": false, //不显示_source内容

如果我们只想要统计数量而不需要查询数据,可以使用 Count API

GET nginx-access-log*/_count
{
  "query":{
    "term":{
      "response_code":200
    }
  }
}

上面的查询会返回如下内容,count表示查询命中的数量

{
  "count" : 76381,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  }
}

如果我们想要查询的数据比较多,我们可以利用scroll来进行游标查询;scroll=5m表示游标查询窗口会保持5分钟

GET nginx-access-log*/_search?scroll=5m
{
  "size":2,
  "_source": ["url", "user_agent"],
    "query":{
    "term":{
      "response_code":200
    }
  },
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

上面的查询会得到如下结果

{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDnF1ZXJ5VGhlbkZldGNoAhZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlQWRTNaTGY5RFVRT1doMVUyVzFNWng1URZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlMWRTNaTGY5RFVRT1doMVUyVzFNWng1UQ==",
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 76384,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "3uJd9YABH99_gnUyIuDL",
        "_score" : null,
        "_source" : {
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36"
        },
        "sort" : [
          1653383961174
        ]
      },
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "3eJY9YABH99_gnUyouCP",
        "_score" : null,
        "_source" : {
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "user_agent" : "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36"
        },
        "sort" : [
          1653383666111
        ]
      }
    ]
  }
}

上面的返回结果中有一个 _scroll_id 字段,要基于这个游标继续遍历数据只需要像下面这样,调用 /_search/scroll 接口,将前面返回结果的_scroll_id作为scroll_id参数值;scroll:5m表示将当前的scroll_id查询窗口再次延长5分钟。这样就可以一直查询,直到数据为空

GET /_search/scroll
{
  "scroll":"5m",
  "scroll_id":"FGluY2x1ZGVfY29udGV4dF91dWlkDnF1ZXJ5VGhlbkZldGNoAhZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlQWRTNaTGY5RFVRT1doMVUyVzFNWng1URZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlMWRTNaTGY5RFVRT1doMVUyVzFNWng1UQ=="
}

游标超过时间窗口会自动清理,也可以通过 DELETE /_search/scroll 来清理一个游标

DELETE /_search/scroll
{
    "scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDnF1ZXJ5VGhlbkZldGNoAhZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlQWRTNaTGY5RFVRT1doMVUyVzFNWng1URZqMFRDR09mcVFnLWk5cjM5WGpRVlNBAAAAAAABUlMWRTNaTGY5RFVRT1doMVUyVzFNWng1UQ=="
}

返回结果如下:

{
  "succeeded" : true,
  "num_freed" : 2
}

查询结果高亮,更多内容请参考 www.elastic.co/guide/en/elasticsearch/reference/7.16/highlighting.html

GET nginx-access-log*/_search
{
  "size":15,
  "_source": [ "url", "remote_ip","response_code" ],
  "query":{
    "term":{
      "response_code":200
    }
  },
  "highlight": {
    "fields": {"response_code":{}}
  }, 
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ]
}

运行结果如下:

#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/security-minimal-setup.html to enable security.
{
  "took" : 13,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "4uJv9YABH99_gnUyduBw",
        "_score" : null,
        "_source" : {
          "response_code" : "200",
          "remote_ip" : "192.168.1.100",
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1"
        },
        "highlight" : {
          "response_code" : [
            "<em>200</em>"
          ]
        },
        "sort" : [
          1653385161394
        ]
      },
      {
        "_index" : "nginx-access-log-2022.05.24",
        "_type" : "_doc",
        "_id" : "4eJq9YABH99_gnUyzuA5",
        "_score" : null,
        "_source" : {
          "response_code" : "200",
          "remote_ip" : "192.168.1.100",
          "url" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1"
        },
        "highlight" : {
          "response_code" : [
            "<em>200</em>"
          ]
        },
        "sort" : [
          1653384856313
        ]
      }
    ]
  }
}

其中:

"highlight" : {
          "response_code" : [
            "<em>200</em>"
          ]
        },

为高亮内容。

1.4 Aggregation聚合查询

如果想要统计某索引每天(每小时)的数量、按照某个字段计数等类似的查询就要用到aggregation聚合查询

例如我们想要知道每天的请求数量,可像下面这样写:

  • aggs表示聚合查询;
  • “day_count”是我们自定义的一个聚合的名称(aggs可以有多个和多层所以需要指定一个名称);
  • date_histogram表示是一个日期分布器,是按照“@timestamp”这个日期字段按照1d(1天)的时间间隔进行分布的;
  • size:0 表示不返回具体的记录,hits部分是空数组
GET nginx-access-log*/_search
{
  "aggs": {
    "day_count": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "1d"
      }
    }
  },
  "size": 0
}

返回数据会出现一个aggreations部分;day_count就是我们自定义的聚合名称,buckets表示按照日期分布到了那几个“桶”里。

{
  "took" : 22,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "day_count" : {
      "buckets" : [
        {
          "key_as_string" : "2022-05-23T00:00:00.000Z",
          "key" : 1653264000000,
          "doc_count" : 78953
        },
        {
          "key_as_string" : "2022-05-24T00:00:00.000Z",
          "key" : 1653350400000,
          "doc_count" : 303
        }
      ]
    }
  }
}

如果我们想要固定查询的日期范围,并且只查询status=404的数量,可以在查询增加query部分,如下:

GET nginx-access-log*/_search
{
  "aggs": {
    "day_count": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "1d"
      }
    }
  },
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {"range": {         //一个range查询
            "@timestamp": { //查询1月20到1月25时间范围
              "gte": "2022-05-23",
              "lte": "2022-05-24",
              "format": "yyyy-MM-dd"
            }
          }
        },{
          "match": {        //只匹配status=200的记录
            "response_code": 404
          }
        }
      ]
    }
  }
}

查询结果如下:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 28,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "day_count" : {
      "buckets" : [
        {
          "key_as_string" : "2022-05-23T00:00:00.000Z",
          "key" : 1653264000000,
          "doc_count" : 28
        }
      ]
    }
  }
}

我们也可以按照terms进行聚合,比如统计一天内接口调用次数,只需要将aggs部分修改成如下,interface.keyword 就是interface的全文字段,size:5 表示只展示前面5个(默认是按照doc_count倒序排序)

GET nginx-access-log*/_search
{
  "aggs": {
    "interface_count": {
      "terms": {
        "field": "url.keyword",
        "size": 5
      }
    }
  },
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {"range": {
            "@timestamp": {
              "gte": "2022-05-23",
              "lte": "2022-05-24",
              "format": "yyyy-MM-dd"
            }
          }
        },{
          "match": {
            "response_code": 200
          }
        }
      ]
    }
  }
}

统计结果如下:

{
  "took" : 33,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "interface_count" : {
      "doc_count_error_upper_bound" : 373,
      "sum_other_doc_count" : 30492,
      "buckets" : [
        {
          "key" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "doc_count" : 39031
        },
        {
          "key" : "/index.php?m=project&f=index&locate=no",
          "doc_count" : 1826
        },
        {
          "key" : "/index.php?m=product&f=index&locate=no",
          "doc_count" : 1748
        },
        {
          "key" : "/index.php?m=my&f=index",
          "doc_count" : 1695
        },
        {
          "key" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=0",
          "doc_count" : 1603
        }
      ]
    }
  }
}

那么,如果我们想要看每个接口一天内每个小时的调用数量改怎么办呢?

GET nginx-access-log*/_search
{
  "aggs": {
    "interface_count": {
      "terms": {
        "field": "url.keyword",
        "size": 5
      },
      "aggs" :{
        "hours" : {
          "date_histogram": {
            "field": "@timestamp",
            "interval": "1h" //按照1小时间隔分布
          }
        }
      }
    }
  },
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {"range": {
            "@timestamp": {
              "gte": "2022-05-23",
              "lte": "2022-05-24",
              "format": "yyyy-MM-dd"
            }
          }
        },{
          "match": {
            "response_code": 200
          }
        }
      ]
    }
  }
}

只需要在上面的interface_count aggs基础上再嵌套一个date_histogram类型的聚合
返回结果中在interface_count里面有增加了一层“hours”的聚合,展示了“key”(即接口名)按照每个小时的分布计数结果

{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "interface_count" : {
      "doc_count_error_upper_bound" : 373,
      "sum_other_doc_count" : 30492,
      "buckets" : [
        {
          "key" : "/index.php?m=message&f=ajaxGetMessage&t=html&windowBlur=1",
          "doc_count" : 39032,
          "hours" : {
            "buckets" : [
              {
                "key_as_string" : "2022-05-23T09:00:00.000Z",
                "key" : 1653296400000,
                "doc_count" : 38672
              },
              {
                "key_as_string" : "2022-05-23T10:00:00.000Z",
                "key" : 1653300000000,
                "doc_count" : 22
              },
              {
                "key_as_string" : "2022-05-23T11:00:00.000Z",
                "key" : 1653303600000,
                "doc_count" : 11
              },
......

2.利用kibana进行日志查询与分析

2.1 基础过滤查询

对于在上面通过grok匹配存储在Elasticsearch中的字段都是可以使用Kibana的 Discover菜单的中 “Add a filter” 进行查询。

“字段名” 和 “字段名.keyword” 两种过滤方式

在查询里,有“字段名” 和 “字段名.keyword” 两种过滤方式。两种过滤方式,这两者的区别是:

  • 使用“字段名”可以进行分词查询,比如:
    66fb2a0274c44dc9b93e6626a2897c08
  • 而interface.keyword意思是使用interface整体进行查询不支持分词,使用时Kibana时也会弹出下拉列表。
    ded17786185241808e3b737d055fc239

单字段多词查询

当想要查询一个字段的多个值可以使用 “is one of” 或者 “is not one of” ,用来表示要查询的分词在其中或者不在其中:
7fcb129f5b674bf399863462a865c5c0

多过滤条件查询

可以进行多个字段查询,它们之间的关系是 and 关系,例如查询的是“url包含index.php关键字的并且response_code是200的”
1e05401f069e41d48ad50225963deeeb

2.2 在Kibana中使用kuery-query和Lucene进行高阶查询

Kibana使用的查询语法既包括Lucene的查询语法,也包括其kibana自身的查询语言增强kuery-query,这样在应用在相对复杂查询时,在kibana查询框内的自定义内容查询。

下面了解下简单的Lucene查询语法,通过几个不同的示例,帮助我们在日常工作中更好的使用Kibana的进行日志和数据查询。

参考文档:
www.elastic.co/guide/en/elasticsearch/reference/7.16/query-dsl-query-string-query.html#query-string-syntax
www.elastic.co/guide/en/kibana/7.16/lucene-query.html
www.elastic.co/guide/en/kibana/7.16/kuery-query.html

全文搜索:

直接在搜索框输入想查询的内容:

${content} 或者 “${content}”

引号在这里起到的作用是对搜索内容的限制,如果不写引号,那么搜索内容将按照分词处理,不区分内容顺序,比如你可能得到"quick fox brown"这样的结果
55f5ac945e3249758256be8995e967f9

字段搜索:

field:value 或者field:“value”
89b391fbbeca44d0aa52bb0a967e74b6

通配符:

? 匹配单个字符,如app?d
* 匹配0到多个字符,如searc*h

注意通配符都不可用作表示第一个字符,如*test,?test
e57ab4756f754bfab5e92543eca37896

正则搜索:

支持简单的正则搜索,但效率很低,不推荐使用,使用时用"/"引用
[name:/joh?n(ath[oa]n)/](http://name/joh?n(ath[oa]n)/)

模糊搜索:

quikc~ brwn~ foks~
~:在一个单词后面加上~启用模糊搜索,可以搜到一些拼写错误的单词
first~ 这种也能匹配到 frist
还可以设置编辑距离(整数),指定需要多少相似度
cromm~1 会匹配到 from 和 chrome
默认2,越大越接近搜索的原始值,设置为1基本能搜到80%拼写错误的单词

近似搜索:

在短语后加上~,可以搜到被隔开或顺序不同的单词
“fox quick”~5,表示fox和quick中间可以隔5个单词,可以搜索到"quick brown fox"

范围搜索:

All days in 2012:
date:[2012-01-01 TO 2012-12-31]
Numbers 1…5
count:[1 TO 5]
Tags between alpha and omega, excluding alpha and omega:
tag:{alpha TO omega}
Numbers from 10 upwards
count:[10 TO *]
Dates before 2012
date:{* TO 2012-01-01}
1~5,但不包括5
[1,5}
可以简化成以下写法:
age:>10
age:<=10
age:(>=10 AND <20)
age:(+>=10 +<20)

优先级:

使用^表示,使一个词语比另一个搜索优先级更高,默认为1,可以为0~1之间的浮点数,来降低优先级
quick^2 fox

逻辑操作:

AND OR + -
+:搜索结果中必须包含此项
-:不能含有此项
+appid -s-ppid aaa bbb ccc:结果中必须存在appid,不能有s-ppid,剩余部分尽量都匹配到
逻辑操作符kibana6也支持写为 && ,|| ,!,但不推荐使用
逻辑组合示例:((quick AND fox) OR (brown AND fox) OR fox) AND NOT news

转义特殊字符:

+ - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ /
以上字符当作值搜索的时候需要用\转义,其中< 和 > 根本无法规避,如果包含这两个字符只能另寻他法
(1+1)=2用来查询(1+1)=2

2.3数据统计

比如想要知道某个接口最近1小时都有哪些用户和他们的ip都来自哪些城市。这里就用到了kibana Visualize,点击Visualize + 按钮,新增一个视图。

如果想要统计一些字段并有可能需要在别的地方使用这些字段,应该使用 Data Table数据表视图,它可以下载成csv格式的文件
53e9215dd71c44d9b61f24bdd04eb750
在这个基础之上我们在增加一个rows
43f065049b41452e97168a9d74f38c41
在此基础上,添加一列method.keyword 请求方式。
b6b2464b2bc541308aa003916983a5f9
可以给每个metric和rows起别名,然后下载成csv文件
237dae1012794c23ae8d119504acb669
打开的csv文件如下:
44f29461ce8f43d2a5847b82f730ce76

参考:https://zhuanlan.zhihu.com/p/159029039