1. Setup a development system

To set up a development environment DATAGERRY does not need any special tools. Only the generation of a productive binary is generated by additional software (more about this later). However, it is recommended to set up third-party programs for data storage and process management independently of the development environment.

1.1. Install Python

DataGerry is written in Python 3.6+. This is also the only dependency which is absolutely necessary to start the program. Python 2 is not supported. Older versions than Python 3.6 may run, but are not officially supported. The installation of Python 3 is different depending on the operating system. Please see at the official documentation for details: Download Python | Python.org

Note

We are using Python 3.6.x or higher, which is not compatible with Python 2.x.

1.2. Clone repository

Clone the git repository from our official mirror:

git clone https://github.com/NETHINKS/DATAGERRY

1.2.1. Install requirements

We recommend generating an isolated python environment like Virtualenv To install the python requirements run:

pip install -r requirements.txt

1.3. Setting up third party dependencies

1.3.1. MongoDB

MongoDB is a document-oriented NoSQL database. It is used to store content and the program uses necessary data. See the official installation guide for details: Install MongoDB Community Edition

1.3.2. RabbitMQ

RabbitMQ is an open source message broker software that implements the Advanced Message Queuing Protocol (AMQP). See the official installation guide for details: Downloading and Installing RabbitMQ

1.4. Modify configuration file

Default file is etc/cmdb.conf. The default configuration should look like this:

[Database]
host = 127.0.0.1
port = 27017
database_name = cmdb
;username = username
;password = password

[WebServer]
host = 0.0.0.0
port = 4000

[MessageQueueing]
host = 127.0.0.1
port = 5672
username = guest
password = guest
exchange = datagerry.eventbus
connection_attempts = 2
retry_delay = 6
use_tls = False

1.4.1. Database config

Configuration section for the MongoDB database.

Database config section table

Database

Description

Default value

Optional

host

IP address

127.0.0.1

port

connection port

27017

database_name

database name

cmdb

connection_timeout

timeout in ms for connection try

2000

username

authentication username

only required if the MongoDB database uses a authentication method

password

authentication password

only required if the MongoDB database uses a authentication method

1.4.2. Webserver config

Configuration section for the web(rest)-server.

Webserver config section table

Webserver

Description

Default value

Optional

host

webserver host address

0.0.0.0

if you want that the rest server is only reachable over the local machine use “127.0.0.1” or “localhost”

port

connection port

4000

1.4.3. Message queueing config

Configuration section for the message queueing server.

message queueing config section table

MessageQueueing

Description

Default value

Optional

host

RabbitMQ host adress

127.0.0.1

port

connection port

5672

username

authentication username

guest

password

authentication username

guest

exchange

used bus

datagerry.eventbus

DONT CHANGE THIS!

connection_attempts

2

retry_delay

time between connection retries

6

use_tls

using tls encryption

false

1.5. Starting the backend and frontend

For a development environment, the frontend must be started independently of the backend, since these systems are only connected during the compile process. If only one development of the backend is desired, the frontend can be ignored. This will then display only one placeholder page.

1.5.1. Starting DATAGERRY

To start DATAGERRY you can use the following command.

./datagerry -s -d

Note

For development it is recommended to start the system in debug mode.

1.5.1.1. Generate clean database

To generate an empty database, start the CMDB with the --setup parameter. This starts a startup routine. During the startup, the database structure is created and a query to the admin user is started. In addition, the asymmetric key pair is generated and stored in the database`.

Note

Datagerry is terminated after successful setup, no matter which parameter is used to start it.

Warning

If an already existing database is specified in the configuration, this database will be deleted!

1.5.1.2. Insert test data (optional)

Optionally DATAGERRY can be filled with fictitious test data.

./datagerry --test

1.5.2. Starting angular frontend

This frontend was generated with [Angular CLI](https://github.com/angular/angular-cli) version 7.3.8.

1.5.2.1. Installation

Run npm install

1.5.2.2. Development server

Run ng serve for a dev server. Navigate to http://localhost:4200/. The app will automatically reload if you change any of the source files.

1.5.2.3. Build

Run ng build to build the project. The build artifacts will be stored in the dist/ directory. Use the –prod flag for a production build.

6. Event Handling — DATAGERRY feature-NET-766_category_datatables_replacement-713e3e7c2dafc65f69fb1da649895efb611a4145 documentation

6. Event Handling

DATAGERRY uses an event based communication between its daemons. Each daemon can publish and subscribe events to a message broker. The message broker (currently, RabbitMQ is supported) will route events to the different daemons.

Each daemon will open two AMQP connections to the message broker (one for sending, one for receiving events). The connection handling will be done by cmdb.event_management.event_manager.EventManagerAmqp.

6.1. structure of an event

An event consists of a type and a set of key-value pairs as parameters.

6.1.1. example:

type:

cmdb.core.object.added

parameters:

id: id of the new created object

The set of key-value pairs depends on the event type.

6.2. event routing

Events are routed by type to the daemons. Each daemon can subscribe to a list of event type definitions. In these definitions, wildcards can be used:

# subscribe to a specific event type
cmdb.core.object.added

# subscribe to all event types starting with a specific string
cmdb.core.#

6.3. list of event types

The following event types are defined in DATAGERRY:

6.3.1. cmdb.core.object:

These events will be created by the core system if CMDB objects will be added/updated/deleted. The parameter “public_id” which includes the public ID of the specific object will be included.

  • cmdb.core.object.added

  • cmdb.core.object.updated

  • cmdb.core.object.deleted

6.3.2. cmdb.core.objecttype:

These events will be created by the core system if CMDB object types will be added/updated/deleted. The parameter “public_id” which includes the public ID of the specific object will be included.

  • cmdb.core.objecttype.added

  • cmdb.core.objecttype.updated

  • cmdb.core.objecttype.deleted