Environment installation

---

Installation

The following describes how to set up a local environment mainly using docker images. I have provided two methods of starting the environment: standalone docker images or a docker-compose method.

Docker

The docker-compose methods automate the startup of a few example components.

• Postgres

• Influxdb

• Grafana

Execute the following command to pull, startup and configure Postgres, Influxdb and Grafana.

docker-compose up -d

For Postgres all the required setting such as username, database setup and data loading is performed. For all the details take a look at:

sql/init.sql

This file can be used to setup the database if you choose not to use this method.

Joule

You will need the latest version of the Joule image for either setup approach as containers are deployed based upon passed configurations.

Pull the latest version of the Joule platform.

docker pull fractalworks/joule

The creation of the container will be described further down in the document.

Postgres

PostgreSQL is a powerful, open source object-relational database system that uses and extends the SQL language combined with many features that safely store and scale the most complicated data workloads.

This step can be skipped if Postgres is already installed or using the docker compose method.

docker pull postgres

Start a postgres instance

docker run --name telco-postgres -e POSTGRES_PASSWORD=joule -e POSTGRES_USER=joule -e POSTGRES_DB=telco -p 5432:5432 -d postgres
-v ${PWD}/conf/sql:conf/sql

The default postgres user and database are created in the entrypoint with initdb.

If you are using a local install first create the user joule with a password of joule that has the ability to create a database, tables and load data. Then run the below script

psql -h 127.0.0.1 -d telco -U joule -a -f sql/telco_database.sql

InfluxDB

InfluxDB is a time series database designed to handle high write and query loads.

docker pull influxdb

This step can be skipped if Postgres is already installed or using the docker compose method.

Apache Geode

Apache Geode is a distributed in-memory data grid supporting caching and event computation

wget https://www.mirrorservice.org/sites/ftp.apache.org/geode/1.13.2/apache-geode-1.13.2.tgz

This step can be skipped if you decide not to test the real-time enrichment process. Otherwise just simply decompress apache-geode-1.13.2.tgz into a directory and set the following environment variables in the .bashrc or similar and ensure they are available in current shell.

GEODE=/<INSTALL-LOCATION>/apache-geode-1.13.2
PATH=$GEODE/bin:$PATH

source ~/.bashrc

Start up cluster

Under the geode directory cluster startup and shutdown scripts are provided. Ensure the telco database has been created before running the startup script. This script with deploy a local distributed cluster. A locator and data member with data regions will be created.

gfsh run --file geode/scripts/geodeClusterStartup.gfsh

Processes

• locator listening on port 10334

• Data node as a cluster member

Regions

• customer

• billing

• bundles

• mobiledevices

• londonTransportStations

• ukpostcodes

• uktowns

Deployments and configuration

A number of key Fractalworks jar files are dynamically deployed to the cluster to enable data loading from a Postgres database. Further documentation will be provided that will discuss the provided Joule features.

Shutdown cluster

To shut down the cluster nicely execute the command below.

cd $GEODE
gfsh run --file scripts/geodeClusterShutdown.gfsh

Troubleshooting

1. What if I cannot connect to Geode locator using localhost? The quickest way is to use your machine IP address instead of localhost in the mobileReferenceData.yaml file

Error in log

docker logs telcoapp

Failed to add reference data store.
org.apache.geode.cache.client.NoAvailableLocatorsException: Unable to connect to any locators in the list [HostAndPort[localhost/<unresolved>:10334]]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.findServer(AutoConnectionSourceImpl.java:174)

Fix

vi conf/sources/mobileReferenceData.yaml

locatorAddress: <Your-machine-IP-address>