Datadog Integration with Postgres/MongoDB/Apache/Elasticsearch

Datadog has integration with almost any popular services. I have integrated our Datadog to specifically monitor our needed services.

Here is how I did it.

Step 1: Datadog Agent

Install Datadog agent for your specific OS in your server. You can download the specific agent in the Datadog UI under Integrations -> Agent.

Step 2: Configure Agent

You need to configure the YAML file for each specific service you would like to be monitored.


1. Create a read-only Postgres role with access to pg_stat_database

In our case, we have a 3 servers clustered Postgres with PG-Pool. So you need to create the Postgres role and PG-Pool user in each 3 servers.

A.  Create PG role
Connect to your pg instance in each server and run following queries: 
$ psql -h server -U postgres -W
postgres# create user datadog with password 'datadog';
postgres# grant SELECT ON pg_stat_database to datadog;
B. Create encrypted password with MD5 for PG-Pool
Login to each server and run pg_md5:
$ sudo pg_md5 -p -m -u datadog
Supply the same password as you've provided during creating the role.

This command will create encrypted MD5 hash for the user and updates the following pg-pool config:
$ cat /etc/pgpool-II/pool_passwd
Test if you are able to connect with your created role:
$ psql -h server -U datadog postgres -c \
"select * from pg_stat_database LIMIT(1);"
&& echo -e "\e[0;32mPostgres connection - OK\e[0m" || \ 
|| echo -e "\e[0;31mCannot connect to Postgres\e[0m"
2. Configure the Datadog agent

Create or open datadog postgres.yaml config file:
$ sudo vi /etc/dd-agent/conf.d/postgres.yaml
Fill it as per your environment values


  - host: <IP>
    port: 5432
    username: datadog
    password: datadog
    ssl: true

In my case I had to define the config file in a way that datadog connects to my postgres via SSL with ssl: true as connections been defined to be always SSL encrypted to my postgres. When I didn't define the config file in this way for the first time, I received this error when Datadog tried to connect to postgres:
Error: md5 authentication for user datadog failed.
Based on my pool_hba.conf file, all connections to all databases from all sources need to be md5 encrypted:
$ sudo cat /etc/pgpool-II/pool_hba.conf
# "local" is for Unix domain socket connections only
local   all         postgres                          trust
local   all         all                               md5
# IPv4 local connections:
host    all         postgres          trust
host    all         all          md5
host    all         all             md5
3. Restart the Datadog agent and check the status:

$ sudo /etc/init.d/datadog-agent restart
$ sudo /etc/init.d/datadog-agent info


Same as Postgres, we need a user with read only role on admin database.

1. Create Mongo user
Connect to mongo server on admin database and create the datadog user:

$ mongo <server>/admin -ssl --sslAllowInvalidCertificates -u root -p <passwd>

> db.createUser({
  "pwd": "datadog",
  "roles" : [
    {role: 'read', db: 'admin' },
    {role: 'clusterMonitor', db: 'admin'},
    {role: 'read', db: 'local' }

2. Configure mongo.yaml

sudo vi /etc/dd-agent/conf.d/mongo.yaml
Fill it as per your environment values


  - server: mongodb://datadog:datadog@localhost:27017/admin

    ssl: true
    ssl_cert_reqs: 0
      - durability
      - locks
      - top
In my case, mongo is using SSL but certificate has expired. So I had to define ssl: true and ssl_cert_reqs: 0 so it ingonres the SSL cert. Fail to define that, I received `Mongodb: Connection Closed` error in datadog.

3. Restart the Datadog agent and check the status:
$ sudo /etc/init.d/datadog-agent restart
$ sudo /etc/init.d/datadog-agent info


In case of Apache, it is quiet straight forward except you need to make sure you have mod_status loaded in your apache service. Another requirement is that ExtendedStatus need to be set to on, which is set on by default in apache version 2.3.6 onward. 

1. Configure apache.yaml

sudo vi /etc/dd-agent/conf.d/apache.yaml
Fill it as per your environment values


  - apache_status_url: http://<IP>:6666/server-status

    disable_ssl_validation: true
2. Restart the Datadog agent and check the status:

$ sudo /etc/init.d/datadog-agent restart
$ sudo /etc/init.d/datadog-agent info


There is no trick here for Elasticsearch integrations.

Configure and restart the agent:
$ sudo vim /etc/dd-agent/conf.d/elastic.yaml
Fill it as per your environment values
init_config: null
-   tags:
    - role:production
    url: http://localhost:9200

$ sudo /etc/init.d/datadog-agent restart
$ sudo /etc/init.d/datadog-agent info


  1. This comment has been removed by the author.

  2. We are using total network monitor for network monitoring, because it works perfect with Windows. We also tried Paessler Network Monitor, its functions looks poor, but it's free. What are the benefits of datadog in compare with other monitoring software?

  3. It is impossible to consider certain moments without a complete understanding and study of everything that was connected with them, therefore it is worth thinking more seriously.


Post a Comment

Popular posts from this blog

Linux and AIX user non expiry and force password change at first login

Rebuild your new Yum repository after changing the packages

Running Docker Wildfly/JBoss Application Server in Debug mode via Eclipse