dataMinds News Round up – June 2021
Azure Synapse
Azure Synapse Analytics enables you to read Parquet files stored in the Azure Data Lake storage using the T-SQL language and high-performance Parquet readers. The key characteristic of these high-performance Parquet readers is that they are using the native (C++) code for reading Parquet files, unlike the existing Polybase Parquet reader technology that uses the Java code. These native readers are introduced in the serverless SQL pools in Azure Synapse Analytics workspaces. Jovan Popovic writes about how you can greatly enhance performance when reading Parquet files from dedicated SQL Pools.
Serverless Synapse SQL pools enable you to read Parquet/CSV files or Cosmos DB collections and return their content as a set of rows. In some scenarios, you would need to ensure that a reader cannot access some rows in the underlying data source. Even though, row-level security is supported in dedicated SQL pools, it is not in serverless pools. Read all about how you can implement your own custom row-level security rules using standard T-SQL code.
Azure DevOps
Azure Logic Apps can be used to automate steps like sending approval emails. Elie Bou Issa explains automating Azure DevOps with Logic Apps. By reading this article, you’ll have a good understanding of leveraging a Logic App for Azure DevOps to automate the create of work items, in addition to creating an automated approval-based workflow using Office 365.
Let’s talk about development environments… a virtual space where your data architecture team, software developers, and information curators do their development and testing work. A proper development environment is logically separated from the production environment, and is often further partitioned into different realms for initial development, data or functional validation, and user acceptance testing.
However, chances are there, you might experience some counterparties, questioning the presence of a development environment. Tim Mitchell provides you with answers on any counterarguments crossing your path.
SQL
It’s hard to set absolute rules about, “Feature X should absolutely never be used.” However, there are some features that set off alarm bells when it comes down to T-SQL code robustness. Brent Ozar shines his light on his 9 signs your T-SQL might fail a code review.
SQL Server performance tuning can seem overwhelming when you don’t know where to start. In this article Robert Sheldon explains 9 best practices for performance tuning.
Blockchain has been transformational for enterprises, digitizing trust for multi-party workflows. Blockchain decentralizes the data and the business logic which produces the data that augments, or replaces, the necessary business contracts in place between parties to ensure agreed-upon business practices are followed.
But blockchain isn’t always the right fit for all systems that require digitization of trust. What if you have a centralized system where you want to prove your data’s integrity to other parties? For example, proving to auditing or regulatory authorities that your data has not been tampered by a malicious party? Jason M. Anderson introduces you to Azure SQL Database ledger, which adds tamper-evident capabilities to your Azure SQL Databases, available in Preview starting in West Central US with all regions following shortly.
Power BI
In the slipstream of the commercial release of Daniel Otykier‘s Tabular Editor 3, Brian Julius takes some time to walk you through some of the most useful features. Good to know: Brian was fortunate enough to try the preview version of Tabular Editor 3 in December 2020 and get into the beta program and has been using TE3 every day since.
One common business requirement after publishing a Power BI report to the service is that users might want to know when there is a change or updates in their report. This is particularly helpful when the dashboard is needed frequently, but the dataset feeding the reports are updated at different intervals during the day. One very simple way of doing this is by using Power Automate to send the business users email alerts. You can use a tutorial? Kenneth A. Omorodion got you covered!
During the recent MS Build conference, Microsoft announced API support for Power BI Deployment Pipelines. Marc Lelijveld had the privilege to get his hands on this in an early stage, which gave him the opportunity to build an Azure DevOps extension, enabling you to integrate Power BI deployment pipelines within Azure DevOps. In this blog, Marc will run you through the configuration of the DevOps extension, where you can download it and get you up to speed!
Learn
Looking for a challenge? You can test your knowledge on the current technology available on the Azure Data Platform! By passing the single DP-203 exam required, you will be awarded with a Azure Data Engineer Associate certification. In this short blog post, Andy Cutler goes through the current exam, its criteria, and learning resources. Whether you are new to data engineering in Azure or if you are already experienced in developing data solutions in Azure, it’s well worth working through the skills measured list to ensure you have the required knowledge.