Comment on page
Environment installation
The following describes how to set up a local environment mainly using docker images. I have provided two methods of starting the environment: standalone docker images or a docker-compose method.
The docker-compose methods automate the startup of a few example components.
- Postgres
- Influxdb
- Grafana
Execute the following command to pull, startup and configure Postgres, Influxdb and Grafana.
docker-compose up -d
For Postgres all the required setting such as username, database setup and data loading is performed. For all the details take a look at:
sql/init.sql
This file can be used to setup the database if you choose not to use this method.
You will need the latest version of the Joule image for either setup approach as containers are deployed based upon passed configurations.
Pull the latest version of the Joule platform.
docker pull fractalworks/joule
The creation of the container will be described further down in the document.
PostgreSQL is a powerful, open source object-relational database system that uses and extends the SQL language combined with many features that safely store and scale the most complicated data workloads.
This step can be skipped if Postgres is already installed or using the docker compose method.
docker pull postgres
Start a postgres instance
docker run --name telco-postgres -e POSTGRES_PASSWORD=joule -e POSTGRES_USER=joule -e POSTGRES_DB=telco -p 5432:5432 -d postgres
-v ${PWD}/conf/sql:conf/sql
The default postgres user and database are created in the entrypoint with initdb.
If you are using a local install first create the user joule with a password of joule that has the ability to create a database, tables and load data. Then run the below script
psql -h 127.0.0.1 -d telco -U joule -a -f sql/telco_database.sql
InfluxDB is a time series database designed to handle high write and query loads.
docker pull influxdb
This step can be skipped if Postgres is already installed or using the docker compose method.
Apache Geode is a distributed in-memory data grid supporting caching and event computation
wget https://www.mirrorservice.org/sites/ftp.apache.org/geode/1.13.2/apache-geode-1.13.2.tgz
This step can be skipped if you decide not to test the real-time enrichment process. Otherwise just simply decompress apache-geode-1.13.2.tgz into a directory and set the following environment variables in the .bashrc or similar and ensure they are available in current shell.
GEODE=/<INSTALL-LOCATION>/apache-geode-1.13.2
PATH=$GEODE/bin:$PATH
source ~/.bashrc
Under the geode directory cluster startup and shutdown scripts are provided. Ensure the telco database has been created before running the startup script. This script with deploy a local distributed cluster. A locator and data member with data regions will be created.
gfsh run --file geode/scripts/geodeClusterStartup.gfsh
Processes
- locator listening on port 10334
- Data node as a cluster member
Regions
- customer
- billing
- bundles
- mobiledevices
- londonTransportStations
- ukpostcodes
- uktowns
Deployments and configuration
A number of key Fractalworks jar files are dynamically deployed to the cluster to enable data loading from a Postgres database. Further documentation will be provided that will discuss the provided Joule features.
To shut down the cluster nicely execute the command below.
cd $GEODE
gfsh run --file scripts/geodeClusterShutdown.gfsh
- 1.What if I cannot connect to Geode locator using localhost? The quickest way is to use your machine IP address instead of localhost in the mobileReferenceData.yaml file
Error in log
docker logs telcoapp
Failed to add reference data store.
org.apache.geode.cache.client.NoAvailableLocatorsException: Unable to connect to any locators in the list [HostAndPort[localhost/<unresolved>:10334]]
at org.apache.geode.cache.client.internal.AutoConnectionSourceImpl.findServer(AutoConnectionSourceImpl.java:174)
Fix
vi conf/sources/mobileReferenceData.yaml
locatorAddress: <Your-machine-IP-address>
Creating examples and a platform takes a significant amount of work. Joule is independently developed, funded and maintained by Fractalworks Ltd. If you would like to support please contact Fractalworks enquiries
Last modified 10mo ago