修改1、生成filter插件工程
[root@bogon logstash_plugin]# /opt/logstash/bin/logstash-plugin generate --type filter --name lynis --path /opt/l
ogstash_plugin
Creating /opt/logstash_plugin/logstash-filter-lynis
create logstash-filter-lynis/CHANGELOG.md
create logstash-filter-lynis/CONTRIBUTORS
create logstash-filter-lynis/DEVELOPER.md
create logstash-filter-lynis/Gemfile
create logstash-filter-lynis/LICENSE
create logstash-filter-lynis/README.md
create logstash-filter-lynis/Rakefile
create logstash-filter-lynis/lib/logstash/filters/lynis.rb
create logstash-filter-lynis/logstash-filter-lynis.gemspec
create logstash-filter-lynis/spec/filters/lynis_spec.rb
create logstash-filter-lynis/spec/spec_helper.rb
[root@bogon logstash_plugin]# ll
total 8
drwxr-xr-x. 4 root root 4096 9月 26 10:48 logstash-filter-lynis
drwxr-xr-x. 4 root root 4096 9月 26 10:47 logstash-filter-test
[root@bogon logstash_plugin]#
2、修改代码
文件：/opt/logstash_plugin/logstash-filter-lynis/lib/logstash/filters/lynis.rb
- 增加类成员变量source：
class LogStash::Filters::Lynis < LogStash::Filters::Base
# Setting the config_name here is required. This is how you
# configure this filter from your Logstash config.
#
# filter {
# {
# message => "My message..."
# }
# }
#
config_name "lynis"
# Replace the message with this value.
#config :message, :validate => :string, :default => "Hello World!"
config :source, :validate => :string, :default => "message"
- 修改filter函数：
public
def filter(event)
source_value = event.get(@source)
index = source_value.index("=")
source_len = source_value.length
if index!=nil && index!=0
dest_key = source_value[0, index]
dest_value = source_value[index+1, source_len]
event.set(dest_key, dest_value)
end
# filter_matched should go in the last line of our successful code
filter_matched(event)
end
3、修改配置信息
root@bogon logstash-filter-lynis]# cat logstash-filter-lynis.gemspec
Gem::Specification.new do |s|
s.name = 'logstash-filter-lynis'
s.version = '1.1.0'
s.licenses = ['Apache License (2.0)']
s.summary = 'this plugin is used process lynis report data'
s.description = 'Write a longer description or delete this line.'
s.homepage = 'https://github.com/anbc'
s.authors = ['anbc']
s.email = '[email protected]'
s.require_paths = ['lib']
# Files
s.files = Dir['lib/**/*','spec/**/*','vendor/**/*','*.gemspec','*.md','CONTRIBUTORS','Gemfile','LICENSE','NOTICE.TXT']
# Tests
s.test_files = s.files.grep(%r{^(test|spec|features)/})
# Special flag to let us know this is actually a logstash plugin
s.metadata = { "logstash_plugin" => "true", "logstash_group" => "filter" }
# Gem dependencies
s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
s.add_development_dependency 'logstash-devutils'
end
4、安装Logstash plugin需要的依赖
bundle install
5、使用spec进行单元测试
通过编写单元测试代码完成单元测试工作，具体单元测试代码位于
/opt/logstash_plugin/logstash-filter-lynis/spec/filters/lynis_spec.rb
# encoding: utf-8
require_relative '../spec_helper'
require "logstash/filters/lynis"
describe LogStash::Filters::Lynis do
describe "Set to Hello World" do
#设置logstash的配置文件
let(:config) do <<-CONFIG
filter { #具体的filter
lynis { #filter插件的名称
source => "message" #将message字段的信息传入source中
}
}
CONFIG
end
#传入一条输入数据
sample("message" => "hello=world") do #输入数据的key是message，value是hello=world。
#sample("hello=world") do #忽略key的表达方式
expect(subject).to include("hello") #判断解析后的output数据中，是否包含key为hello的数据
expect(subject.get('hello')).to eq('world')#判断以hello为key的数据的值是否是world
end
end
end
5、测试插件
运行命令：
bundle exec rspec
运行结果：
[root@bogon logstash-filter-lynis]# bundle exec rspec
--- jar coordinate com.fasterxml.jackson.core:jackson-databind already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-annotations already loaded with version 2.7.0 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.module:jackson-module-afterburner already loaded with version 2.7.4 - omit version 2.9.1
--- jar coordinate com.fasterxml.jackson.core:jackson-core already loaded with version 2.7.4 - omit version 2.9.1
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
Sending Logstash's logs to which is now configured via log4j2.properties
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :couchdb=>true, :elasticsearch=>true, :elasticsearch_secure=>true, :export_cypher=>true, :integration=>true, :windows=>true}
Randomized with seed 32702
.
Finished in 0.42843 seconds (files took 8.8 seconds to load)
1 example, 0 failures
Randomized with seed 32702
6、生成gem包
[root@bogon logstash-filter-lynis]# gem build logstash-filter-lynis.gemspec
Successfully built RubyGem
Name: logstash-filter-lynis
Version: 1.1.0
File: logstash-filter-lynis-1.1.0.gem
[root@bogon logstash-filter-lynis]# ll
total 44
-rw-r--r--. 1 root root 63 9月 26 10:48 CHANGELOG.md
-rw-r--r--. 1 root root 407 9月 26 10:48 CONTRIBUTORS
-rw-r--r--. 1 root root 119 9月 26 10:48 DEVELOPER.md
-rw-r--r--. 1 root root 39 9月 26 10:48 Gemfile
-rw-r--r--. 1 root root 3043 9月 26 11:13 Gemfile.lock
drwxr-xr-x. 3 root root 21 9月 26 10:48 lib
-rw-r--r--. 1 root root 524 9月 26 10:48 LICENSE
-rw-r--r--. 1 root root 7680 9月 26 16:49 logstash-filter-lynis-1.1.0.gem
-rw-r--r--. 1 root root 916 9月 26 11:12 logstash-filter-lynis.gemspec
-rw-r--r--. 1 root root 33 9月 26 10:48 Rakefile
-rw-r--r--. 1 root root 3032 9月 26 10:48 README.md
drwxr-xr-x. 3 root root 41 9月 26 14:04 spec
[root@bogon logstash-filter-lynis]#
7、安装插件
- 手动安装logstash
[root@bogon opt]# tar -xvf logstash-5.6.1.tar.gz
[root@bogon opt]# ln -s logstash-5.6.1.tar.gz logstatsh
- 查看当前插件
[root@bogon logstash-5.6.1]# ./bin/logstash-plugin list
logstash-codec-cef
logstash-codec-collectd
logstash-codec-dots
logstash-codec-edn
logstash-codec-edn_lines
logstash-codec-es_bulk
logstash-codec-fluent
logstash-codec-graphite
logstash-codec-json
- 安装插件
由于gem文件路径错误，导致的错误提示
[root@bogon logstash-5.6.1]# /opt/logstash/bin/logstash-plugin install /opt/logstash_plugin/logtash-
filter-lynis/ogstash-filter-lynis-1.1.0.gem
Validating /opt/logstash_plugin/logtash-filter-lynis/ogstash-filter-lynis-1.1.0.gem
Plugin /opt/logstash_plugin/logtash-filter-lynis/ogstash-filter-lynis-1.1.0.gem does not exist
ERROR: Installation aborted, verification failed for /opt/logstash_plugin/logtash-filter-lynis/ogstash-filter-lynis-1.1.0.gem
[root@bogon logstash-5.6.1]#
安装成功
[root@bogon logstash-filter-lynis]# /opt/logstash/bin/logstash-plugin install /opt/logstash_plugin/l
ogstash-filter-lynis/logstash-filter-lynis-1.1.0.gem
Validating /opt/logstash_plugin/logstash-filter-lynis/logstash-filter-lynis-1.1.0.gem
Installing logstash-filter-lynis
执行上面命令时，会出现卡死现象，ctrl+c 终止后，安装正常。
- 验证是否安装成功
[root@bogon local_gems]# /opt/logstash/bin/logstash-plugin list
........
logstash-codec-netflow
logstash-codec-plain
logstash-codec-rubydebug
...............
logstash-filter-fingerprint
logstash-filter-geoip
logstash-filter-grok
logstash-filter-json
logstash-filter-kv
logstash-filter-lynis #新安装的lynis插件
logstash-filter-metrics
..............
logstash-input-beats
logstash-input-couchdb_changes
logstash-input-dead_letter_q
8、修改logstash配置文件调用指定插件
[root@bogon config]# vim logstash.conf
input
{
stdin{}
}
filter
{
lynis
{
source => "message"
add_tag => ["lynis_data"]
#remove_field => ["message"]
}
}
output
{
stdout { codec => rubydebug }
}
9、测试插件效果
- 运行logstash
[root@bogon config]# /opt/logstash/bin/logstash -f /opt/logstash/config/logstash.conf
2017-09-26 18:05:45,183 main ERROR Unable to locate appender "${sys:ls.log.format}_console" for logger config "root"
2017-09-26 18:05:45,184 main ERROR Unable to locate appender "${sys:ls.log.format}_rolling" for logger config "root"
2017-09-
- 数据解析情况
#键盘输入内容
hello=world
#数据解析
{
"@version" => "1",
"host" => "0.0.0.0",
"@timestamp" => 2017-09-26T10:06:02.741Z,
"hello" => "world",
"message" => "hello=world",
"tags" => [
[0] "lynis_data"
]
}
#键盘输入内容
dddd
#数据解析
{
"@version" => "1",
"host" => "0.0.0.0",
"@timestamp" => 2017-09-26T10:06:06.321Z,
"message" => "dddd",
"tags" => [
[0] "lynis_data"
]
}
#键盘输入内容
os=linux=
#数据解析
{
"@version" => "1",
"host" => "0.0.0.0",
"@timestamp" => 2017-09-26T10:06:20.110Z,
"message" => "os=linux=",
"os" => "linux=",
"tags" => [
[0] "lynis_data"
]
}
#键盘数据内容
ddd=dd==dd
#数据解析
{
"@version" => "1",
"host" => "0.0.0.0",
"@timestamp" => 2017-09-26T10:06:30.725Z,
"message" => "ddd=dd==dd",
"ddd" => "dd==dd",
"tags" => [
[0] "lynis_data"
]
}