Azure Event Hub and Azure Data Explorer -Part II
Let’s write a simple spring application to publish events to Azure event hub and ADX.
Welcome to the 2nd one of my blog series on integrating & working with Azure resources. If you are new to Azure eventhub and ADX, I recommend to first read my previous blog[1] which gives a basic introduction to both those services.
In this blog lets build a simple spring application to stream events to the Azure event hub and ingest to ADX. Refer blog[2] as a guide on writing simple spring web applications.
- Follow step 1, 2 and 3 in my previous blog[1] to configure an Azure eventhub, ADX cluster and establish a data connection between them.
- Create a new maven project. You can refer to this pom.xml file for the project from the sample given. I have attached the complete source code for your reference[3].
3. Create a folder structure as below.
4. Add the main method as follows to make it a stand-alone application.
5. Created a model folder and create a new class LoginInfo to include the getters and setters of event payload attributes. Lets have the same set of attributes of the login event payload as in my previous blog regarding azure publishing[1] , namely; userId, eventId, eventType, userstoreDomain, tenantDomain, loginTimestamp and publishTimestamp. You can change them accordingly depending on the requirement.
6. Create a new class UserLoginController to implement the logic for publishing event batches to Azure event hub.
Here, the @RequestMapping annotation is used to map publishEvents() method to the http://localhost:8080/login GET request. The publishEvents()method handles the event publishing logic to Azure event hub. For this example, 10 dummy loginInfo objects are created, converted to json strings and those 10 json strings are appended to an array. Then a new producer client is created which initiates the connection to azure event hub when the correct connection string and event hub name is provided. Finally, the producer.send(batch) method sends the event batch to the specified event hub.
7. When the implementation is done, go to the project home and run the below command.
mvn clean install8. Finally, navigate to the war file location, and enter the following command.
java -Xms128m -Xmx256m -jar <APP_NAME>-1.0-SNAPSHOT.warThen the sample application will start as shown in the snapshot below. As you can observe in the logs, it implies that your application has started in port 8080 in Tomcat.
9. Open a new browser tab and navigate to the following url http://localhost:8080/login .
Then you will see in the terminal logs events starting to publish. Once all the events are published successfully to event hub, you will see the below message on the browser window.
10. Navigate to the event hub you created and click on the overview tab. Select the Messages graph and you will observe a spike with a height of 10 that corresponds to the batch of 10 events we just sent.
10. Go to the created ADX cluster > Data tab > Databases > Database name you created > Query. Then you will see a canvas to execute KQL queries. Type the name of the table you created and click Run. You will notice the login events recorded in a table as shown in below image.
In this way, you can publish any kind of events to the event hub and then directly ingest that data to ADX easily.
Thank you for reading and lets meet again with another interesting blog.