Geo search

Real-time view on what entities are nearby with respect to geofence radius

Overview

Often when performing real-time geospatial analytics the objective is to gain a view of what entities are nearby such as friends, businesses, local interests, cell towers etc,. This processor provides the ability to perform such a function using a configurable search area and entities that may be of concern.

A geo search is performed against a spatial index using the passed event attributes which will return 0 or more nearby entities.

Example & DSL attributes

geo search:
    name: nearbyCellTowers
    search area: 8

    spatial index:
        top left coordinates: [ 0,0 ]
        width: 512
        area: 262144
        max levels: 6
        preferred max elements: 601

    spatial entities:
        parser: com.fractalworks.streams.processors.geospatial.domain.telco.CellTowerArrowParser
        source: data/mini_cells.csv

Response

geoSearchResult:
    - CellTower{radioType=UMTS, mcc=262, net=2, area=801, cell=71555, unit=-1, lon=13.359251, lat=52.52103, range=383, samples=5, changeable=1, created=1286420769, updated=1289843835, averageSignal=-92, coordinates=Tuple{x=216.0, y=132.0}, key=null}
    - CellTower{radioType=UMTS, mcc=262, net=2, area=15, cell=6652412, unit=-1, lon=13.413759, lat=52.520469, range=0, samples=1, changeable=1, created=1288894949, updated=1288894949, averageSignal=-98, coordinates=Tuple{x=216.0, y=132.0}, key=null}
    - .....

Attributes schema

Attribute	Description	Data Type	Required
name	Descriptive name of the processor function	String Default: Random UUID
search area	Area to search	Integer Default: 8
response field	Custom field name to place result of search lookup	String Default: geoSearchResult
latitude field	Custom field name to gain latitude dimension	String Default: latitude
longitude field	Custom field name to gain longitude dimension	String Default: longitude
convertWGS84ToTileCoords	Flag to convert WGS84 coordinates to tile coordinates	Boolean Default: true
spatial index	An area is set out as a search tree. The default area is a flattened world divided out in to rectangles.	See spatial index documentation
spatial entities	Spatial entities to return on a successful search	See Spatial Entities section

Spatial Entities

These are entities that have the required location attributes that enable the search algorithm to be executed against. Since entities are specific to the use case and business domain and/or industry the ability to leverage these assets within Joule a parser API is provided. The parser translates existing entity structures, via a file, in to a support Joule data structure.

Parser example

This example parses a cell tower file using the Apache Arrow library. CellTower extends the GeoNode class to simplify the implementation.

/*
 * Copyright 2020-present FractalWorks Ltd.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 *  you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *  http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.fractalworks.streams.processors.geospatial.domain.telco;

import com.fractalworks.streams.core.exceptions.TranslationException;
import com.fractalworks.streams.sdk.referencedata.serialization.ReferenceDataArrowParser;
import org.apache.arrow.vector.FieldVector;
import org.apache.arrow.vector.VectorSchemaRoot;

/**
 * CellTower Arrow row parser.
 * <p>
 * Format
 * - radio,mcc,net,area,cell,unit,lon,lat,range,samples,changeable,created,updated,averageSignal
 * <p>
 * Created by Lyndon Adams
 */
public class CellTowerArrowParser implements ReferenceDataArrowParser<CellTower> {

    /**
     * Parse provided vectorSchemaRoot to CellTower.
     *
     * @param row
     * @return CellTower
     */
    @Override
    public CellTower translate(VectorSchemaRoot row) throws TranslationException {
        CellTower cellTower = new CellTower();
        for(FieldVector v : row.getFieldVectors()){
            var str = v.getObject(0);
            switch (v.getName().toLowerCase() ){
                case "radio" -> cellTower.setRadioType(RadioType.getEnum(str.toString().intern()));
                case "mcc" -> cellTower.setMcc( ((Long)str).intValue());
                case "net" -> cellTower.setNet(((Long)str).intValue());
                case "area" -> cellTower.setArea(((Long)str).intValue());
                case "cell" -> cellTower.setCell(((Long)str).intValue());
                case "unit" -> cellTower.setUnit( (str!=null) ? ((Long)str).intValue() : -1);
                case "lon" -> cellTower.setLon((Double)str);
                case "lat" -> cellTower.setLat((Double)str);
                case "range" -> cellTower.setRange(((Long)str).intValue());
                case "samples" -> cellTower.setSamples(((Long)str).intValue());
                case "changeable" -> cellTower.setChangeable(((Long)str).intValue());
                case "created" -> cellTower.setCreated((Long) str);
                case "updated" -> cellTower.setUpdated((Long) str);
                case "averagesignal" -> cellTower.setAverageSignal( (str!=null) ?((Long)str).intValue() : -1);
                default -> throw new TranslationException(String.format("Unknown field '%s'", v.getName()));
            }
        }
        return cellTower;
    }
}

Attributes schema

Attribute	Description	Data Type	Required
parser	Custom parser that converts to a custom type that extends GeoNode, see documentation for further details on implementing a GeoNode entity	String
source	Source of the file	String