6.1、elasticsearch的输出配置
参数 | 类型 | 是否必选 | 默认值 | 描述 |
---|---|---|---|---|
action | string 可选项:["index","delete","create","update"] | No | "index" | |
cacert | a valid filesystem path | No | ||
codec | codec | No | "plain" | |
doc_as_upsert | boolean | No | false | |
document_id | string | No | ||
document_type | string | No | ||
flush_size | number | No | 500 | |
hosts | array | No | ["127.0.0.1"] | |
idel_flush_time | number | No | 1 | |
index | string | No | "logstash-%{+YYYY.MM.dd}" | |
keystore | a valid filesystem path | No | ||
keystore_password | password | No | ||
manage_tomplate | password | No | ||
max_retries | number | No | 3 | |
parent | string | No | nil | |
password | password | No | ||
path | string | No | "/" | |
proxy | <<,>> | No | ||
retey_max_interval | number | No | 2 | |
routing | string | No | ||
script | string | No | "" | |
script_lan | string | No | "" | |
script_type | string,可选值有["inline", "indexed", "file"] | No | ["inline"] | |
script_var_name | string | No | "event" | |
scripted_upsert | boolean | No | false | |
sniffing | boolean | No | false | |
sniffing_delay | number | No | 5 | |
ssl | boolean | No | ||
ssl_certificate_verification | boolean | No | true | |
template | a valid filesystem path | No | ||
template_name | string | No | "logstash" | |
template_overwrite | boolean | No | false | |
timeout | number | No | ||
truststore | a valid filesystem path | No | ||
truststore_password | password | No | ||
upsert | string | No | "" | |
user | string | No | ||
workers | number | No | 1 |
action
Elasticsearch所能执行的action,有效的action有:
- index 给一个文档建立索引(一起来自logstash的事件)
- delete 通过id值删除一个文档(这个action需要指定一个id值)
- create 插入一条文档信息,如果这条文档信息在索引中已经存在,那么本次插入工作失败。
- update 通过id值更新一个文档。更新有个特殊的案例upsert,如果被更新的文档还不存在,那么就会用到upsert。详见upsert选项。
- A sprintf style string to change the action based on the content of the event. The value %{[foo]} would use the foo field for the action
cacert
- cacert参数的值是一个路径
- 这个参数没有默认值。 *.cer 或*.pem文件用来验证服务器的证书
codec
- codec类型的值
- 默认值是“plain”
codec在output数据中使用,output codecs一个很方便的方法,在数据离开output之前对数据进行编码,省去了在Logstash 管道中的指定分隔符。
doc_as_upsert
设置一种更新的模式。更新时如果document_id不存在,则创建一个新的文档。
document_id
设置用于索引的document ID,在覆盖老的文档数据时会用到。
document_type
The document type to write events to. Generally you should try to write only similar events to the same type. String expansion %{foo} works here. Unless you set document_type, the event type will be used if it exists otherwise the document type will be assigned the value of logs
flush_size
This plugin uses the bulk index API for improved indexing performance. In Logstashes >= 2.2 this setting defines the maximum sized bulk request Logstash will make You you may want to increase this to be in line with your pipeline’s batch size. If you specify a number larger than the batch size of your pipeline it will have no effect, save for the case where a filter increases the size of an inflight batch by outputting events.
In Logstashes ⇐ 2.1 this plugin uses its own internal buffer of events. This config option sets that size. In these older logstashes this size may have a significant impact on heap usage, whereas in 2.2+ it will never increase it. To make efficient bulk API calls, we will buffer a certain number of events before flushing that out to Elasticsearch. This setting controls how many events will be buffered before sending a batch of events. Increasing the flush_size has an effect on Logstash’s heap size. Remember to also increase the heap size using LS_HEAP_SIZE if you are sending big documents or have increased the flush_size to a higher value.
hosts
- 是一个数组类型的值。
用来设置远程主机的host信息,如果参数是一个数据,它会将请求平衡的分配给各主机,注意http协议使用的是http地址(如:9200,而不是9300)。“127.0.0.1” ["127.0.0.1:9200", "127.0.0.2:9200"] ["https://127.0.0.1:9200"]["https://127.0.0.1:9200/mypath"] (如果在子路径上使用代理)。这里需要注意的是不能只将master nodes(主节点)添加到hosts 列表中,这回妨碍LS发送bulk 请求到主节点。所以这个参数只能引用Elasticsearch中的数据节点或客户节点。
idle_flush_timeedit
从上一次触发flash到又被强制flash之间间隔的时间。 这项设置确保了低速事件在Logstash中不会被卡住,例如,如果你设置的flush_size是100,这时你接收到了10个时间,而且从上次触发flush到现在的时间间隔已经超过了idle_flush_time的值,Logstash将自动触发flush将这10个事件提交上去。
这一机制确保无论是快速的还是慢速的日志流,都能以准实时的速度传送。
index
写入事件所用的索引。可以动态的使用%{foo}语法。它的默认值可以以天为单位分割的索引,使你可以很容易的删除老的数据或者搜索指定时间范围内的数据。索引不能包含大写字母。推荐使用以周为索引的ISO 8601格式,例如logstash-%{+xxxx.ww}
keystore
用于向服务器展现证书,它可以是.jks或.p12。
keystore_password
Value type is password
用于设置truststore的密码
manage_template
从Logstash 1.3开始(除非你将manage_template选项设置为false),Elasticsearc启用了一个默认的映射模板。如果你还没有设置匹配定义的索引模式(模式 logstash-%{+YYYY.MM.dd}),减去任意变量。例如,这种情况下模板将对所有以logstash开头的索引生效。
如果你有一个动态模板,(如基于field名创建索引),需要将manage_template设置为false并使用REST API 手动上传你的模板。
max_retries
最大重试次数 弃用:这个设置将不再起任何作用。在未来的版本中将被标记为过期。
parent
对于子文档,用于关联父文档的ID。可以动态的使用%{foo}表达式。
password
password参数用于认证一个节点的安全性。
path
HTTP Path at which the Elasticsearch server lives. 如果你需要在网络代理服务器后边部署Elasticsearch,需要使用这个设置,它可以把根路径映射到 Elasticsearch HTTP API lives
proxy
设置http代理服务器的地址,该值可以是一个字符串如http://localhost:123,也可以是key-value的形式如:{host:"proxy.org", port:"80", scheme:"http"}。注意这里不是一个SOCKS代理,而是一个plain HTTP代理。
retry_max_interval
设置两次bulk重试之间的最大时间间隔。
retry_max_items(DEPRECATED)
弃用警告:这个配置项已经被弃用了,在未来的版本中可能就不可用了。
routing
这个routing可以覆盖所有被处理的时间,它可以动态的使用%{foo}表达式
script
设置脚本名称,用于设置脚本更新模式
script_lang
设置脚本使用的语言
script_type
Define the type of script referenced by "script" variable inline : "script" contains inline script indexed : "script" contains the name of script directly indexed in elasticsearch file : "script" contains the name of script stored in elasticseach’s config directory
script_var_name
Set variable name passed to script (scripted update) 设置传递给脚本的变量名
script_upsert
if enabled, script is in charge of creating non-existent document (scripted update) 如果被设置为true,脚本负责创建不存在的文档(脚本更新)
sniffing
this setting asks Elasticsearch for the list of all cluster nodes and adds them to the hosts list. Note: This will return ALL nodes with HTTP enabled (including master nodes!). If you use this with master nodes, you probably want to disable HTTP on them by setting http.enabled to false in their elasticsearch.yml. You can either use the sniffing option or manually enter multiple Elasticsearch hosts using the hosts parameter.
这个设置向Elasticsearch请求获取所有的集群节点(cluster nodes)列表,并将其添加到host列表中。注意,它会返回所有HTTP能访问的节点。
sniffing_delay
两次嗅探之间间隔的时间
ssl
Enable SSL/TLS secured communication to Elasticsearch cluster. Leaving this unspecified will use whatever scheme is specified in the URLs listed in hosts. If no explicit protocol is specified plain HTTP will be used. If SSL is explicitly disabled here the plugin will refuse to start if an HTTPS URL is given in hosts
激活 Elasticsearch 集群中各节点间的SSL/TLS加密通讯。如果没有使用这个参数项,那么url中指定使用什么协议,就会使用什么协议。对于没有明确指定使用协议的则使用http协议。如果指定不适用SSL协议而host的url中又出现了https,那么本插件会拒绝启动
template
You can set the path to your own template here, if you so desire. If not set, the included template will be used. 如果你愿意,你可以设置指向你自己模板的路径。如果没有设置,那么包含的模板会被使用
template_name
这个配置项用来定义在Elasticsearch中模板的命名问题,注意你使用了”template management features“ 而之后又做了修改,那么你将需要手工删除旧的模板。
curl -XDELETE <http://localhost:9200/_template/OldTemplateName?pretty>
其中OldTemplateName
是之前设置的。
template_overwrite
The template_overwrite option will always overwrite the indicated template in Elasticsearch with either the one indicated by template or the included one. This option is set to false by default. If you always want to stay up to date with the template provided by Logstash, this option could be very useful to you. Likewise, if you have your own template file managed by puppet, for example, and you wanted to be able to update it regularly, this option could help there as well.
Please note that if you are using your own customized version of the Logstash template (logstash), setting this to true will make Logstash to overwrite the "logstash" template (i.e. removing all customized settings)
timeout
Set the timeout for network operations and requests sent Elasticsearch. If a timeout occurs, the request will be retried. 处置超时时间,如果超时发生,会重新尝试发起请求。
truststore
The JKS truststore to validate the server’s certificate. Use either :truststore or :cacert JKS truststore 用来验证服务器的证书。可以使用:truststore or :cacert
truststore_password
设置 truststore 密码
upsert
Set upsert content for update mode. Create a new document with this parameter as json string if document_id doesn’t exists 为更新模式设置upsert参数内容。如果创建一个新文档时,文档id已经存在了,那么使用这个参数作为json字符串。
user
Username to authenticate to a secure Elasticsearch cluster 用户名用于认证一个安全的Elasticsearch集群。
workers
The number of workers to use for this output. Note that this setting may not be useful for all outputs.