How to setup a Real-Time data analyze using Azure Stream Analytics and PowerBI

These days’ demand is high for Real-Time data analytics and can easily implement the system using cloud technologies. Real-time analytics is the use of, or the capacity to use, data and related resources as soon as the data enters the system.

Few well-known applications of real-time analytics:

• Real-time credit card fraud detection  
• CRM real-time analytics can provide up-to-the-minute information about an enterprise's customers and present it so that better and quicker business decisions.
• Traffic detection
• Wind speed

In this post, using a real-time feed from Twitter and Azure Cloud technologies.

You can find Azure Event Hub and Azure Stream Analytics under Internet Of Things marketplace category.

Step1: Create an Azure Event Hubs

Azure Event Hubs is a highly scalable publish-subscribe service that can invest millions of events per second and stream them into multiple applications. This lets you process and analyses the massive amounts of data produced by your connected devices and applications.

Use Event Hubs to:

• Log millions of events per second in near real time.
• Connect devices using flexible authorization and throttling.
• Use time-based event buffering.
• Get a managed service with elastic scale.
• Reach a broad set of platforms using native client libraries.
• Pluggable adapters for other cloud services.

Step2: Create Azure Stream Analytics

Azure Stream Analytics is a fully managed, cost-effective real-time event processing engine that helps to unlock deep insights from data. Stream Analytics makes it easy to set up real-time analytic computations on data streaming from devices, sensors, websites, social media, applications, infrastructure systems, and more.

With a few clicks in the Azure portal, you can author a Stream Analytics job specifying the input source of the streaming data, the output sink for the results of your job, and a data transformation expressed in a SQL-like language. You can monitor and adjust the scale/speed of your job in the Azure portal to scale from a few kilobytes to a gigabyte or more of events processed per second.

Stream Analytics leverages years of Microsoft Research work in developing highly tuned streaming engines for time-sensitive processing, as well as language integrations for intuitive specifications of such.

In this case Power BI used as stored data.

Used query:

SELECT Topic,count(*) AS Count, Avg(SentimentScore) AS AvgSentiment, System.Timestamp AS Insert_Time

FROM [tweets-input] TIMESTAMP BY CreatedAt

GROUP BY TumblingWindow(second,5), Topic

Step3: Feed real-time data to Event Hub

In this case, Twitter WPF Client used as real-time data feeder. To use this, you should create a twitter app and access token key for that.

You can create apps https://apps.twitter.com

Step4: Crete a Dashboard using real-time data

Once you start your Azure Stream Analytics, you will find data source will be available in Powerbi.com

Cheers!

Uma