dataMinds News Round up – November 2020
Databricks
James Serra shares his outtakes on some of the fundamental differences between using external tables or t-sql views on files in data lake.
Leo Furlong shares his favourite features for Databricks that are a must use for all Data Engineers.
https://www.jamesserra.com/archive/2020/11/external-tables-vs-t-sql-views-on-files-in-a-data-lake
https://www.bluegranite.com/blog/microsoft-and-databricks-top-5-modern-data-platform-features-part-2
Azure Synapse Analytics
Wolfgang Strasser, lays out some of the possibilities for making sure we perform Direct Queries over Data Lake Files in Power BI, by using Azure Synapse Analytics as a virtual layer.
Fikrat Azizov dives into how you can use Azure Synapse Analytics to help you explore data inside of SQL Server.
https://workingondata.wordpress.com/2020/11/08/direct-query-your-data-lake-files-using-azure-synapse-analytics-and-power-bi/
https://www.mssqltips.com/sqlservertip/6602/azure-synapse-analytics-sql-server-data/
SSAS & SSRS
Announced during the PASS Summit keynote, we learned of newly released Azure Marketplace images, primed for workloads on SSRS and SSAS
SQL
Erik Darling wrote a quick tip on how to find out information about deadlocks in your system, and some options to start fixing them.
https://www.erikdarlingdata.com/sql-server/options-for-fixing-parallel-deadlocks/
Haroon Ashraf lays out some options for including tSQLt into our SQL Database Projects, to include Unit Testing in our database development process.
https://www.sqlshack.com/three-ways-you-can-add-tsqlt-to-your-sql-database-projects/
William Assaf shares his insights into the importance having backups for the SQL Server Master Key, to be able to rebuild from scatch in certain DR scenarios (ie. a ransomware attack)
https://www.sqltact.com/2020/11/protect-your-sql-server-from-ransomware.html
Azure Data Factory
Paul Andrew created a set of tests and checks for common pitfalls in Azure Data Factory development. Gracious as he is, he’s sharing them with the general public.
Sucharita Das shows us an incremental data loading pattern for Azure Data Factory, by leveraging Change Tracking in SQL Server.
https://www.sqlservercentral.com/articles/incremental-data-loading-through-adf-using-change-tracking
Power BI
Meagan Longoria shares her view on something we’ve all encountered while writing and testing DAX logic. Blanks are handled in a special way, and should be handled with care.
https://datasavvy.me/2020/10/29/dax-logic-and-blanks/
Marco Russo dives deep into the internal workings of the VertiPaq Engine, driving Power BI, Analysis Services and PowerPivot. He lays out the way the storage engine and formula engine work, and how they combine their efforts for optimal results.
https://www.sqlbi.com/articles/formula-engine-and-storage-engine-in-dax