dataMinds News Round up – May 2022
Azure
Dying to know what’s new in store for Azure Synapse Analytics? This month, Ryan Majidimehr got you covered with elaborating upon the Spark 3.2 Public Preview, the new Dataverse connector added to Synapse data flows, a revamped exploration experience in database templates, and how to clone a lake database. Other new features are in SQL, Spark, data integration, and developer experience.
Aimed at developers, DBA’s, architects & practically everyone who works with streams of data that are captured in real-time, Lionel Pénuchot reviews a common pattern of streaming data, which is real-time message ingestion, in Synapse SQL Dedicated pool.
You’ve just built an application. You’d like users to perceive it as fast and performant, even when many users are using it. What way is there to put your application to the test? Find out how Azure Load Testing can cover your need.
When you have large data sets with hundreds of millions or billions of records, ingesting that data efficiently can be a challenging feat. In many cases, these data sets may be loaded on a nightly schedule. Your nightly data ingestion pipeline must be complete within a certain time frame to avoid negatively impacting your downstream processes or analyses. So, what can you do when a job you are running each night grows too large to be processed in your time frame? How can you better manage the ingestion of these large datasets?
An incredibly powerful way to do this is to employ a partitioning pattern. A partitioning pattern essentially involves breaking a large data ingestion job into several smaller jobs. Emily Schmucker take a few steps back and look at the overall data ingestion process to see how partitioning can fit into that process.
DevOps
Writing Azure DevOps Pipelines YAML, have you thought about including some conditional expressions? In this blog post, Thomas Thornton shows how you can use If, elseif or else expressions to assist in your pipeline creation.
Merging GIT conflicts within Azure DevOps can be a serious burden to carry. Praveen Kumar Sreeram showcases an Azure DevOps Marketplace extension called Pull Request Merge Conflicts which makes life beautiful again.
Power Platform
It’s hard to believe but Power BI has now been available for over 10 years! Over the last few years, there have been a number of new features to improve the performance of queries for dashboards and reports, especially for very large datasets. But what are they actually doing and what should be aware of when using them? Sit back and relax because James Serra‘s on a roll!
Hack alert!
Ever ran into a situation where someone in your organization has authored a report in the Power BI web service, and now they want to make changes that can only be done with Power BI Desktop. So, you try to download the PBIX file from the Power BI web service, only to discover that you can’t, because if it wasn’t created as a PBIX, it can’t be downloaded as a PBIX.
James Bartlett shines his light on how you can Powershell your way out of this restriction.
It appears that there is a misplaced fear of enabling XMLA Read/Write – that somehow doing this may cause problems. Michael Kovalsky takes this opportunity to dispel these erroneous beliefs as enabling XMLA Read/Write is perhaps the most important step you can take to advance your Power BI platform.
