Building a Low-Cost, High-Throughput Data Exchange Platform for Digital Meters on Azure & Migrate your SSIS skills to Azure Data Factory (IN-PERSON @ AE)

When you are responsible for collecting loads of information from data providers and sending it out swiftly to data subscribers, how would you design and architect the central data exchange platform? How do you keep costs low but still receive, process, and send all this data in a timely fashion?

On Microsoft Azure, Function apps and Cosmos DB will give you the operational scalability you need. Azure Storage Accounts can hold enormous amounts of data safely but cheaply for a long time. And Azure Synapse can transform your basic data and store it in other formats optimized to serve their use cases very efficiently.

This session will give an overview of the design and the challenges we faced when building a data exchange platform for digital meters and how we tackled them to be ready to redistribute data, generated daily from 10 million devices.

Integration Services is the new Cobol. Well at least according to some people, but there might be some truth to that statement. Azure Data Factory (ADF) is now looked at as the replacement of SSIS in the cloud.

In this session, we will take a look at how you can leverage your SSIS skills in ADF. We will also look at some patterns that were valid for SSIS but are a bad idea in ADF, or the other way around. At the end, you should feel confident to take up new projects in ADF.

The target audience of this session are BI/data warehouse developers & data engineers who have experience with SSIS (or other ETL tools), but little to no experience with ADF.