Setting Up Kibana With Proxy Server in AWS

When we setup a Elastic Search Service, Kibana comes by default in AWS. However we can restrict the Search Service  by 3 different policies  Resource-based policies,  Identity based policies and IP based policies . But AWS does not grant Kibana access via any of the above three policies .

AWS suggest to use proxy server to access Elastic Search from Kibana.

We will divide this article into 2 sections :

Section 1: Creating Elastic Search Service in AWS

Section 2: Setting Up NGINX proxy in EC2

Creating Elastic Search Service in AWS

Note :You can skip this step if it is already installed.

Define Domain Name

1

Configure Cluster

This step is important when creating ES service in higher environment. The cluster works in master slave concept.
There will be a master node which will track all other data nodes in cluster . Master node will also route information and monitor the health of data nodes.
Master nodes should be either 3 or 5 in number because to increase the availability across zones in the cluster .
Data nodes can be in even number if multi zone deployment is required . In order to set the master slave,we need to enable dedicated master and for high availability , enable zone awareness.

2

EBS storage will depend on how much data we want to store in data node. EBS storage for the cluster will be EBS volume size * Data node instance count.

For this article we will go with a simple data node with 1 instance count.

3.png

 Set Up Access

If we want to keep our ES service inside VPC network, then we should select VPC access else we can mark it public. Note : It is important to keep the ES within VPC.

Here we will go with public access and restrict via IAM Users and Policies.

In Access Policies , we can select “Allow or deny access for one or more IAM users”. This policy will be updated once we have the proxy server setup ready . For now it would look like below :

{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Effect": "Allow",

     "Principal": {

        "AWS": [

          "arn:aws:iam::XXXXXXXXXXX:user/user-name",

          "arn:aws:iam:: XXXXXXXXXXX:root"

        ]

      },

      "Action": [

        "es:*"

      ],

      "Resource": "arn:aws:es:us-east-1: XXXXXXXXXXX:domain/es-dev-service/*"

    }

  ]

}

Click confirm

ES service will be ready in 10 minutes.

Setting Up NGINX proxy in EC2

Launch an EC2 instance. For this example I have used Linux instance .Keep the security group publicly accessible from network side. Note : It is important to keep EC2 within VPC subnets . Note the ssh key pair which will be used to SSH in EC2 .

  • SSH inside EC2.

Run the below commands:

sudo yum update –y

sudo yum install nginx httpd-tools –y

The above 2 commands will get all the updates in the machine and install nginx webserver .

  • Go to /etc/nginx/ and update the nginx.conf file .

Modify the server port to 8080 from 80.  4

  • Create a file in /etc/nginx/conf.d/kibana.conf

Update the file with below data:

 # Config: /etc/nginx/conf.d/kibana.conf

server {

    listen 80;

    server_name myproxy.mydomain.com;

    location / {

        proxy_pass https://search-es-dev-service-hjzpri6r37v5au24j2rjrlq2gi.us-east-1.es.amazonaws.com/

        proxy_redirect https://search-es-dev-service-hjzpri6r37v5au24j2rjrlq2gi.us-east-1.es.amazonaws.com/ /;

        proxy_http_version 1.1;

        proxy_set_header Host $host;

        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_set_header X-Real-IP $remote_addr;

        proxy_set_header Authorization "";

        proxy_hide_header Authorization;

       auth_basic "Username and Password are required";

        auth_basic_user_file /etc/nginx/.htpasswd;

    }

}

proxy_pass is the domain for your ES service which will be available in ES console.

proxy_redirect is also the same domain of ES service  .

All authentication users will be stored in /etc/nginx/.htpasswd

  • Add username and password for Kibana . This username password will be authenticated in the proxy server in ec2 nginx webserver

sudo htpasswd -c /etc/nginx/.htpasswd {username}

  • Restart the nginx server

/etc/init.d/nginx restart

  • Keep the nginx server on even if the ec2 is stopped and started

chkconfig nginx on

Congratulation now your nginx webserver is up and running . However we have few more steps before the final Kibana board to display .

Allocating an Elastic IP

Everytime we restart EC2 , Public IP will change . Hence we need to keep updating the IP in Access policy of ES service . But that is not possible to do every time there is a restart. In this case,we need to allocate an Elastic IP for the EC2 . If you already have an Elastic IP , you can ignore it .

Select the EC2 instance in Console .

Go to Actions -> Networking -> ManageIP Address.

Allocate an Elastic IP .

5.png

Click on Allocate

6.png

Note: Elastic IP will be charged certain amount if it is not associated to and EC2.

Final Step

              The final step would be to update the access policy of ES service with the elastic ip of EC2.

{

  "Version": "2012-10-17",

  "Statement": [

    {

      "Effect": "Allow",

      "Principal": {

        "AWS": [

          "arn:aws:iam::XXXXXXXXXXX:user/ user-name ",

          "arn:aws:iam::XXXXXXXXX:root"

        ]

      },

      "Action": "es:*",

      "Resource": "arn:aws:es:us-east-1:XXXXXXXXXXXXXXXXX:domain/es-dev-service/*"

   

    },

                  {

      "Sid": "",

      "Effect": "Allow",

      "Principal": {

        "AWS": "*"

      },

      "Action": "es:*",

      "Resource": "arn:aws:es:us-east-1:XXXXXXXXXXXXXXX:domain/es-dev-service/*",

      "Condition": {

        "IpAddress": {

          "aws:SourceIp": [

            "{ELASTICIP}"

          ]

        }

      }

    }

  ]

}

Access Kibana from the below url now:

http://{ELASTICIP}/_plugin/kibana

7.png

Hope this helps in setting up Kibana securely using proxy server. Please feel free to leave your comments below.

Digiprove sealCopyright secured by Digiprove © 2019 Geeks 18

2 Comments

Leave a Reply

Your email address will not be published.


*