1. Setup¶
1.1. Requirements¶
DATAGERRY has the following system requirements:
Linux Operating System
MongoDB 4.2+
RabbitMQ
Although, DATAGERRY comes with an own webserver, we recomend Nginx as a reverse proxy for performance reasons.
There are several setup options for DATAGERRY, which are described in the sections below more in detail:
Docker Image
RPM file (for RHEL/CentOS distributions)
tar.gz archive with setup script (for Debian/Ubuntu or other distributions)
If you want to have a fast and easy start, use our Docker Image and docker-compose file.
1.2. Docker Image¶
The fastest way for getting started with DATAGERRY is using Docker. We provide a docker-compose file, which creates four containers (DATAGERRY, MongoDB, RabbitMQ, Nginx). All data were stored in MongoDB using Docker volumes on the Docker host system.
To start, copy the following docker-compose.yml in a directory of your Docker host.
version: "3.0"
services:
nginx:
image: nethinks/nginx:latest
ports:
- "80:80"
- "443:443"
depends_on:
- datagerry
environment:
NGINX_SSL_CERT: "/data/cert/cert.pem"
NGINX_SSL_KEY: "/data/cert/key.pem"
NGINX_LOCATION_DEFAULT: "/;/;http://datagerry:4000"
restart: unless-stopped
volumes:
- ./cert:/data/cert
datagerry:
image: nethinks/datagerry:latest
depends_on:
- db
- broker
environment:
DATAGERRY_Database_host: "db"
DATAGERRY_MessageQueueing_host: "broker"
restart: unless-stopped
db:
image: mongo:4.2-bionic
volumes:
- mongodb-data:/data/db
- mongodb-config:/data/configdb
broker:
image: rabbitmq:3.8
volumes:
- rabbitmq-data:/var/lib/rabbitmq
volumes:
rabbitmq-data:
mongodb-data:
mongodb-config:
Create a subdirectory called cert with an SSL certificate (called cert.pem) and key (called key.pem). Your directory structure should look like this:
./docker-compose.yml
./cert/cert.pem
./cert/key.pem
If you don’t need SSL and just want to have a quick start, use the follwing docker-compose.yml:
version: "3.0"
services:
nginx:
image: nethinks/nginx:latest
ports:
- "80:80"
depends_on:
- datagerry
environment:
NGINX_LOCATION_DEFAULT: "/;/;http://datagerry:4000"
restart: unless-stopped
datagerry:
image: nethinks/datagerry:latest
depends_on:
- db
- broker
environment:
DATAGERRY_Database_host: "db"
DATAGERRY_MessageQueueing_host: "broker"
restart: unless-stopped
db:
image: mongo:4.2-bionic
volumes:
- mongodb-data:/data/db
- mongodb-config:/data/configdb
broker:
image: rabbitmq:3.8
volumes:
- rabbitmq-data:/var/lib/rabbitmq
volumes:
rabbitmq-data:
mongodb-data:
mongodb-config:
Now, run docker-compose to start the application:
$ docker-compose up -d
To access the DATAGERRY frontend, use the following parameters:
http://<<host> or https://<host>
user: admin
password: admin
We provide Docker images for every version of DATAGERRY in Docker Hub. You can use one of the following Docker tags:
- latest
The latest tag is a symlink to the latest stable version of DATAGERRY. It is nice for a quickstart but will also do an automatic upgrade to the next major release if one is available. For production, we recommend the <release> tag below.
- <release> (e.g. 1.0.1)
Every release has its own tag. So, if you want to use a specific version (or prevent your environment from automatic upgrades, just use a specific version as a Docker tag.
- branches_<branch-name> (e.g. branches_development)
If you follow our development process and want to test a specific branch from our git repository, just use this tag. Please keep in mind, that these things aren’t released yet and a branch is a work-in-progress. So it can change at any time and break existing things. We don’t recommend to use these images in production.
To use a specific Docker tag, just replace the following line of the docker-compose.yml:
# replace this line
image: nethinks/datagerry:latest
# by the following:
image: nethinks/datagerry:<tagname>
# example:
image: nethinks/datagerry:1.0.2
1.3. RPM setup¶
For Red Hat Enterprise Linux (RHEL) or RHEL based systems like CentOS or Oracle Linux, we provide a RPM file for installing DATAGERRY.
The following RHEL/CentOS versions are supported and tested:
RHEL/CentOS 7
RHEL/CentOS 8
Before we can install DATAGERRY, we need to install the required dependencies MongoDB and Rabbit MQ.
1.3.1. Setup MongoDB¶
MongoDB 4.2+ is required as database for DATAGERRY.
Note
The setup of MongoDB is described in detail on the MongoDB website: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/ The following section is a quick install guide of MonogDB.
To setup MongoDB, place the follwing file under /etc/yum.repos.d/mongodb.repo:
[MongoDB]
name=MongoDB Repository
baseurl=http://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.2/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.2.asc
After that, install the mongodb-org package and start the server with SystemD:
$ sudo yum install -y mongodb-org
$ sudo systemctl enable mongod
$ sudo systemctl start mongod
1.3.2. Setup RabbitMQ¶
RabbitMQ 3.8+ is used as messaging bus between the processes of DATAGERRY.
Note
The setup of RabbitMQ is described in detail on the RabbitMQ website: https://www.rabbitmq.com/install-rpm.html The following section is a quick install guide of RabbitMQ
For setting up RabbitMQ, we can use the RPM repository provided by Bintray. Place the following file under /etc/yum.repos.d/rabbitmq.repo:
[bintray-rabbitmq-server]
name=bintray-rabbitmq-rpm
baseurl=https://dl.bintray.com/rabbitmq/rpm/rabbitmq-server/v3.8.x/el/$releasever/
gpgcheck=1
gpgkey=https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc
enabled=1
[bintraybintray-rabbitmq-erlang-rpm]
name=bintray-rabbitmq-erlang-rpm
baseurl=https://dl.bintray.com/rabbitmq-erlang/rpm/erlang/22/el/$releasever/
gpgcheck=0
repo_gpgcheck=0
enabled=1
Now, RabbitMQ can be installed and started:
$ sudo yum install -y rabbitmq-server
$ sudo systemctl enable rabbitmq-server
$ sudo systemctl start rabbitmq-server
1.3.3. Setup DATAGERRY¶
If all requirements were installed, we’ll can install the downloaded DATAGERRY RPM file:
$ sudo rpm -ivh DATAGERRY-<version>.x86_64.rpm
To change the parameters for connecting to MongoDB and RabbitMQ, edit the configuration file /etc/datagerry/cmdb.conf
After that, activate and start DATAGERRY with Systemd:
$ sudo systemctl enable datagerry.service
$ sudo systemctl start datagerry.service
To access the DATAGERRY frontend, use the following parameters:
http://<<host>:4000
user: admin
password: admin
Note
If you can’t access the webfrontend of DATAGERRY, check the firewall settings of your server. Port 4000 should ba accessible.
1.4. tar.gz archive setup¶
For all non rpm based Linux distributions, we provide a tar.gz archive with a setup shell script. Systemd is a requirement for that setup. This should work on most distributions, and is tested with the following distributions:
Ubuntu 18.04
Before we can install DATAGERRY, we need to install the required dependencies MongoDB and Rabbit MQ.
1.4.1. Setup MongoDB¶
MongoDB 4.2+ is required as database for DATAGERRY.
Please follow the offical MongoDB documentation <https://docs.mongodb.com/manual/administration/install-on-linux/> to setup MongoDB for your distribution.
1.4.2. Setup RabbitMQ¶
RabbitMQ 3.8+ is used as messaging bus between the processes of DATAGERRY.
Please follow the offical RabbitMQ documentation <https://www.rabbitmq.com/download.html#installation-guides> to setup RabbitMQ for your distribution.
1.4.3. Setup DATAGERRY¶
Extract the provided tar.gz archive and execute the setup script as root:
$ tar -xzvf datagerry-<version>.tar.gz
$ cd datagerry
$ sudo ./setup.sh
To change the parameters for connecting to MongoDB and RabbitMQ, edit the configuration file /etc/datagerry/cmdb.conf
After that, activate and start DATAGERRY with Systemd:
$ sudo systemctl enable datagerry.service
$ sudo systemctl start datagerry.service
To access the DATAGERRY frontend, use the following parameters:
http://<<host>:4000
user: admin
password: admin
Note
If you can’t access the webfrontend of DATAGERRY, check the firewall settings of your server. Port 4000 should ba accessible.
1.5. Setting up Nginx as reverse proxy¶
We recomend to run Nginx as reverse proxy for DATAGERRY. After installing Nginx for your platform, you can adapt the following example configuration for Nginx (nginx.conf):
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events
{
worker_connections 1024;
}
http
{
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
client_max_body_size 0;
server
{
listen 80;
server_name localhost;
# HTTPS redirect
return 301 https://$host$request_uri;
}
server
{
listen 443 ssl;
ssl_certificate /data/cert/cert.pem;
ssl_certificate_key /data/cert/key.pem;
server_name localhost;
location /
{
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
set $upstream_host http://127.0.0.1:4000;
rewrite ^/(.*) /$1 break;
proxy_pass $upstream_host;
}
}
}
This will Nginx listen on port 80 (HTTP) and 443 (HTTPS) and create a redirect from HTTP to HTTPS. If someone access https://<host>/, Nginx will contact http://127.0.0.1:4000, where DATAGERRY is listening.
1.6. Update DATAGERRY¶
To update DATAGERRY to a new version, simply install the new version and start DATAGERRY. During the first start, DATAGERRY will detect the version of the existing database and apply required updates. This may take a few seconds or minutes. After that, the application will be started.