Are you interested in Elasticsearch, Logstash, Kibana, and Rivers (JDBC+MySQL)? Then this ELK guide is for you.

We are building this with Scientific Linux 7.1 and the latest versions of each available at the moment.

I sourced information on how to do this from:
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-centos-7
http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Security_Guide/sec-Using_Firewalls.html
https://github.com/jprante/elasticsearch-jdbc

Install SL 7.1
Find your local mirror: http://scientificlinux.org/downloads/sl-mirrors/

Complete a Web Server install with these options selected:
Web Server w/Java, Python, PHP

Enable EPEL:

sudo yum install epel-release

Then:

sudo yum upgrade

Download and install the public signing key:

sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

vim /etc/yum.repos.d/elk.repo

[elasticsearch-1.5]
name=Elasticsearch repository for 1.5.x packages
baseurl=http://packages.elastic.co/elasticsearch/1.5/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1
[logstash-1.4]
name=logstash repository for 1.4.x packages
baseurl=http://packages.elasticsearch.org/logstash/1.4/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

Next:

sudo yum update && sudo yum install elasticsearch logstash

Edit:

sudo vi /etc/elasticsearch/elasticsearch.yml

Find the line that specifies network.host, uncomment it, and replace its value with “localhost” so it looks like this:

network.host: localhost

If you are setting up a node environment, that system’s public IP will work or:

network.host: 0.0.0.0

Then:

sudo systemctl start elasticsearch.service
sudo systemctl enable elasticsearch.service

Download Kibana 4.0.2:

wget https://download.elastic.co/kibana/kibana/kibana-4.0.2-linux-x64.tar.gz

Expand the compressed file:

tar xvf kibana-*.tar.gz

Edit the kibana.yml file to set host:

vim ~/kibana-4*/config/kibana.yml

host: “localhost”

Make the kibana bin dir:

sudo mkdir -p /opt/kibana

Copy kibana to its running dir:

sudo cp -R ~/kibana-4*/* /opt/kibana/

Make the service file for Kibana:

sudo vim /etc/systemd/system/kibana4.service

Paste this into the new service file:

[Service]
ExecStart=/opt/kibana/bin/kibana
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=kibana4
User=root
Group=root
Environment=NODE_ENV=production

[Install]
WantedBy=multi-user.target

Now start and default enable kibana:

sudo systemctl start kibana4
sudo systemctl enable kibana4

Now we want to get Nginx install for the reverse proxy, to this, we will want the EPEL-Release installed:

sudo yum -y install epel-release

Then install nginx and httpd-tools:

sudo yum -y install nginx httpd-tools

Edit Nginx conf and remove the whole section of “server{” … “}”

sudo vim /etc/nginx/nginx.conf

Create the Kibana Nginx config:

sudo vim /etc/nginx/conf.d/kibana.conf

Paste in:

server {
    listen 80;

    server_name example.com;

    auth_basic "Restricted Access";
    auth_basic_user_file /etc/nginx/htpasswd.users;

    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;        
    }
}

Now start and enable autostart for Nginx:

sudo systemctl start nginx
sudo systemctl enable nginx

Because I don’t hate myself enough, I will remove the firewalld (thanks SystemD)

sudo systemctl disable firewalld
sudo systemctl stop firewalld

Then install the iptables-services package by entering the following command as root:

sudo yum install iptables-services

The iptables-services package contains the iptables service and the ip6tables service.
Then, to start the iptables and ip6tables services, run the following commands as root:

sudo systemctl start iptables
sudo systemctl start ip6tables
sudo systemctl enable iptables
sudo systemctl enable ip6tables

Now we can allow port 80 in:

sudo iptables -I INPUT 5 -p tcp --dport 80 -j ACCEPT
sudo iptables -I INPUT 5 -p tcp --dport 9300 -j ACCEPT

Also, save the configuration:

sudo service iptables save

You might need to install some tools to help you manage SELinux:

sudo yum install policycoreutils-python

If you want help troubleshooting any further issues or mods that SELinux will likely be a part of:

sudo yum install setroubleshoot-server

SELinux is going to block the local reconnect (reverse forward), so let’s add the new ports to http_port_t

sudo semanage port -a -t http_port_t -p tcp 5601
sudo semanage port -a -t http_port_t -p tcp 9200

Now for the river plug-in.

Install the plugin

cd /usr/share/elasticsearch/

Then execute (as root):

sudo ./bin/plugin --install jdbc --url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-river-jdbc/1.5.0.5/elasticsearch-river-jdbc-1.5.0.5-plugin.zip

Now, if you are using it to create rivers for a MySQL database, you will need the MySQL JDBC plug-in:
Download MySQL JDBC driver:

curl -o mysql-connector-java-5.1.33.zip -L 'http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.33.zip/from/http://cdn.mysql.com/'

Add MySQL JDBC driver jar to JDBC river plugin directory and set access permission for .jar file (at least chmod 644):

unzip mysql-connector-java-5.1.33.zip

Let’s copy this to the jdbc directory in $ES_HOME (/usr/share/elasticsearch):

sudo cp mysql-connector-java-5.1.33-bin.jar $ES_HOME/plugins/jdbc/

Set the correct permissions:

sudo chmod 644 $ES_HOME/plugins/jdbc/*

Now restart the node:

sudo service elasticsearch restart

Done!