Other recent blogs
Social Media generates a wealth of data each day. Twitter generates over 500 million tweets every 24 hours, while Facebook shares 4 new petabytes of data per day. If enterprises mine this ever-expanding gold reserve of information, they can catapult into a new stratosphere of business growth while cultivating a visionary capability to disrupt the future.
However, it’s only lamentable that most organizations still rely on manual processes to generate insights from social media. The manual approach results in an erroneous reading of data and process latencies, thereby, impacting an organization’s ability to grow and compete at pace.
Meet Software AG’s Apama
Apama is a streaming analytics platform by Software AG, leading the way forward by facilitating organizations to leverage social media data for meaningful insights and scale optimally. It works by aggregating and calibrating data to drill into market insights, recognizing trends faster, and helping enterprises engage with customers in real-time. This cumulatively results in proactive brand engagement and increased customer retention.
The feature set of Apama social media framework includes:
- Connectivity plug-in API, which allows in-correlator integration with external data sources of varying formats.
- Integration Adaptor Framework (IAF), which provides easy integration with external event sources and systems.
- Sophisticated Analytics with native support for temporal arguments. Sub-second response to detecting events and patterns of interest.
- Highly scalable, patented, and event-driven architecture, supporting tens of thousands of concurrent scenarios.
- Integrated tools for creating visually appealing user dashboards.
- Flexible Event Replay for testing new event scenarios and analyzing existing ones.
- Customizable tools for managing and monitoring your application.
- Email Monitoring Capabilities for your support centers.
- Graphical Development Tools for business access.
Use Case: Software AG’s Apama Integration with Social Media
Let us look at the use case for developing a simple weather advisory Apama application that would post tweets about weather conditions based on events defined inside its environment. To do this, we have applied complex event processing capabilities of Apama to monitor, detect, and analyze important events and patterns across vast quantities of data for usable intelligence.
Development of Apama Application in Software AG Designer
Create Apama Project
Open Software AG Designer:
Open the Apama Workbench Perspective.
You can see the project as follows with all the generated folders and files.
Apama Monitor
A monitor defines:
- One or More Listeners
Event Processing Language (EPL) provides event listeners and stream listeners
- An event listener observes the correlator event stream and analyzes each event until it finds a sequence of events that match its expression. When this happens, the event listener triggers, causing the correlator to execute the listener’s action.
- A stream listener passes stream query output to procedural code. A stream query operates on one or two streams in order to transform their contents into a single output stream. The type of stream query output need not be the same as the type of stream query input. The output for one stream query can be the input for another stream query. At the end of the chain of stream queries, a stream listener co-assigns each stream query output to a variable and executes specified code.
- One or More Actions
Action is one or more operations the correlator performs. This might be to register a listener or an operation when the correlator finds a match between an incoming event/sequence and a listener.
Once WeatherAdvisor.mon is created, make sure that you have the events, actions, and monitor defined as given below in the code:
//*****************************************************************************
// Title: WeatherAdvisor
// Description: WeatherAdvisor description
// Dependencies: None
// Author: Nageswara Reddy Chintakuntla
//
//*****************************************************************************
event Location {
string name;
}
event Temperature {
string locationName;
float tempCelcius;
}
event Rain {
string locationName;
boolean isRaining;
}
event WindSpeed {
string locationName;
integer speed;
}
monitor WeatherAdvisor{
action onload {
Location loc;
on all Location() : loc {
monitorLocation(loc);
}
}
action monitorLocation(Location locToMonitor) {
string locationName := locToMonitor.name;
log "Monitoring "+locationName at INFO;
Temperature t;
Rain r;
WindSpeed w;
//Message to compose and send out
string message := locationName+" : ";
on Temperature(locationName,*):t and Rain(locationName,*):r and WindSpeed(locationName,*):w {
if t.tempCelcius > 20.0 then {
message := message + "It's T Shirt time, ";
}
else {
message := message + "It's a bit cold out, ";
}
if r.isRaining then{
message := message + "take an umbrella, ";
}
else {
message := message + "no showers at the moment, ";
}
if w.speed > 3 then{
message := message + "watch out for crosswinds!";
}
else {
message := message + "the air is relatively still !";
}
}
}
}
Connectivity Plug-ins
Connectivity Plug-ins can be written in Java or C++ and run inside the correlator process to allow messages to be sent and received to/from external systems. They perform the same function for Apama client library, which allows Java or C++ code in an external process to send/receive messages to/from the correlator.
Add a Java plugin to interact with the monitor and send events.
The application uses Twitter4J. You can download the package from the below link, create your Twitter API key, and add the relevant library/properties to your Apama project.
http://twitter4j.org/en/index.html
From the downloaded zip file, you can copy the following two files into your project folder in the workspace:
twitter4j.properties à Will be available in twitter4j-4.0.4\bin
twitter4j-core-4.0.4.jar à Will be available in twitter4j-4.0.4\lib
Refresh the Project and add these jars from Java Build Path configuration wizard.
Make sure that you have the following code in Tweeter.java
import com.apama.epl.plugin.annotation.*;
import com.apama.jmon.annotation.*;
import twitter4j.Status;
import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
@Application(name = "WeatherAdvisorApp",
author = "",
version = "1.0",
company = "",
description = "",
classpath = "")
@EPLPlugin(description = "", name= "Tweeter")
public class Tweeter {
private static Twitter twitter = TwitterFactory.getSingleton();
public static String sendTweet(String msg){
String returnVal;
try {
Status status = twitter.updateStatus(msg);
returnVal = "Tweet Sent";
} catch (TwitterException err) {
returnVal = "Error: " + err.toString();
}
catch (Exception e) {
returnVal = "Error: " + e.toString();
}
return returnVal;
}
}
Now open WeatherAdvisor.mon file and import Tweeter class.
monitor WeatherAdvisor{
import "Tweeter" as twit;
action onload {
Location loc;
on all Location() : loc {
monitorLocation(loc);
}
}
Add statement to send weather message to Twitter:
action monitorLocation(Location locToMonitor) {
string locationName := locToMonitor.name;
log "Monitoring "+locationName at INFO;
Temperature t;
Rain r;
WindSpeed w;
//Message to compose and send out
string message := locationName+" : ";
on Temperature(locationName,*):t and Rain(locationName,*):r and WindSpeed(locationName,*):w {
if t.tempCelcius > 20.0 then {
message := message + "It's T Shirt time, ";
}
else {
message := message + "It's a bit cold out, ";
}
if r.isRaining then{
message := message + "take an umbrella, ";
}
else {
message := message + "no showers at the moment, ";
}
if w.speed > 3 then{
message := message + "watch out for crosswinds!";
}
else {
message := message + "the air is relatively still !";
}
string tweetStatus := twit.sendTweet(message);
log tweetStatus at INFO;
}
}
Define events
Events are data elements. Each event is a collection of attribute-value pairs that capture the state (or changes to state) of real-world or computer-based objects. Events consist of data and temporal attributes that represent what, when, and where of an object.
Next, create a new event file WeatherAdvisorEvents
And you can have the events as follows in WeatherAdvisorEvents.evt file
Location("NewYork")
Location("LasVegas")
WindSpeed("LasVegas",7)
Temperature("LasVegas", 25.0)
Temperature("NewYork", 15.0)
Rain("LasVegas",true)
WindSpeed("NewYork",3)
Rain("NewYork",false)
Registering Application in Twitter
Login to your twitter account with the URL https://dev.twitter.com
Click on My Apps
Create a new Twitter App for Apama weather advisor application
Creating Consumer Key and Access Token
Move to the tab ‘Keys and Access Tokens’ and generate the keys for the app.
Now, we are ready to publish the events from Apama with the above generated Consumer Key and Access Token.
Apama Correlator
Apama correlator is the engine that powers an Apama application. Correlators execute the sophisticated event pattern-matching logic that you define in your Apama application. Apama applications track inbound event streams and listen for events whose patterns match with defined conditions. The correlator's patented architecture can monitor huge volumes of events per second. When an event or an event sequence matches with an active event expression, the correlator executes the appropriate actions as defined by the application logic.
Now, the application is ready to run Apama correlator and post messages in twitter based on the events defined.
Eureka! You can now watch your Twitter posting short and interesting weather updates.
