Complete guide install elk stack

In this tutorial, I guide install ELK stack on Linux. The ELK Stack is a collection of three open-source Elasticsearch, Kibana, and Logstash. Now, let’s install ELK stack on Linux

For my example install ELK stack

• Elasticsearch, Kibana and Logstash -> 192.168.3.4
• Filebeat -> 192.168.3.5

Requirements to install elk you need JAVA. If you do not yet install java on your system. The guided install Java on server ELK as below

Installing Java

ELK requires the installation of Java 8 and higher.

$ sudo yum install java-1.8.0-openjdk

Set JAVA_HOME for Elasticsearch

# sudo cp /etc/profile /etc/profile_backup
# echo 'export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk' | sudo tee -a /etc/profile
# source /etc/profile

To check “JAVA_HOME” 

[huupv@localhost ~]$ echo $JAVA_HOME
/usr/lib/jvm/jre-1.8.0-openjdk

Change to your home directory.

# cd $HOME

Open the .bashrc file.

# vi .bashrc

Add the following line to the file

export PATH=$PATH:$JAVA_HOME/bin

Save the file and exit.

Apply the change

# source .bashrc

Install elasticsearch kibana logstash

# yum install elasticsearch kibana logstash

Another Method to install ELK use Docker as the link below

Quick start install Elasticsearch and Kibana with Docker

Elasticsearch Configure

Open the elasticsearch.yml file

$ sudo vim /etc/elasticsearch/elasticsearch.yml

The content as below

network.host: "localhost"
http.port:9200

Kibana Configure

$ sudo vim /etc/kibana/kibana.yml

The content as below

# server.port: 5601
server.port: 17000
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]

logstash Configure

Logstash: unrecognized service Centos 6. How to start it. Refer to below

sudo initctl status logstash 
sudo initctl start logstash 

Create a setting file and start Logstash. For My example, create a setting that Logstash collects sshd fail logs from [/var/log/secure]

# vi /etc/logstash/conf.d/sshd.conf

The content as below

input {
  file {
    type => "seucure_log"
    path => "/var/log/secure"
  }
}
filter {
  grok {
    add_tag => [ "sshd_fail" ]
    match => { "message" => "Failed %{WORD:sshd_auth_type} for %{USERNAME:sshd_invalid_user} from %{IP:sshd_client_ip} port %{NUMBER:sshd_port} %{GREEDYDATA:sshd_protocol}" }
  }
}

output {
  elasticsearch {
    index => "sshd_fail-%{+YYYY.MM}"
  }
}

Enable Logstash on Boot and Start Logstash:

chgrp logstash /var/log/secure 
chmod 640 /var/log/secure 
systemctl start logstash 
systemctl enable logstash

A few minutes later, Checked logs collected normally.

# curl localhost:9200/_cat/indices?v

Another server install and configure filebeat

Install Filebeat:

# yum install filebeat

Backup Filebeat configuration:

$ mkdir /home/huupv/backups/filebeat -p
$ mv /etc/filebeat/filebeat.yml /home/huupv/backups/filebeat/filebeat.yml.BAK

Create the Filebeat configuration, and specify the Logstash outputs:

$ cat > /etc/filebeat/filebeat.yml << EOF
filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/secure
  exclude_files: ['\.gz$']

output.logstash:
  hosts: ["192.168.3.4:5400"]
EOF

Testing

Thank you for reading the DevopsRoles page!