Restoring Azure SQL Datawarehouse

Microsoft recently added snapshots to the Azure SQL datawarehouse recovery arsenal. What does it do? A snapshot is like a point-in-time marker, which allows you to restore a datawarehouse to a state recorded at the marker. Restore points are useful in more than just the traditional bcp scenarios, such as disaster recovery. Restore points can … Continue reading Restoring Azure SQL Datawarehouse

Capture Telemetry – Azure SQL Data Warehouse

A short article about capturing transient telemetry. Most DMVs in the Azure SQL data warehouse are transient, which means data stored in the table is not persisted usually after: A Pause / Resume / Scale event After a max rollover threshold has been reached (10000 rows) For example, the data in the sys.dm_pdw_request_steps dmv – … Continue reading Capture Telemetry – Azure SQL Data Warehouse

Upgrading The Azure Data warehouse To Gen2

The Data warehouse has recently been upgraded to the next version called Gen2. The good news is that the upgrade process is relatively seamless. Upgrading is supposed to be a one step process but I have decided to add 2 extra steps: Backup / Take a snapshot Upgrade the data warehouse Rebuild all indexes Backup / … Continue reading Upgrading The Azure Data warehouse To Gen2

Azure Data warehouse Performance Tuning 101

When it comes to performance tuning or troubleshooting, for a standard SQL server many tools and techniques exist. For example query execution plans can be analysed, profiler exists, and a host of other dmvs which a DBA cannot live without. In an Azure Data warehouse there are similar dmvs. However some basic principles can be … Continue reading Azure Data warehouse Performance Tuning 101

Azure MSI: Connect Using PowerShell Or .NET?

Managed Service Identity (MSI) was introduced last year. Since then quite a few articles have been written about it: Use a Windows VM Managed Service Identity (MSI) to access Azure Key Vault: Azure SQL authentication with a Managed Service Identity MSI gives your code an automatically managed identity for authenticating to Azure services, so that you … Continue reading Azure MSI: Connect Using PowerShell Or .NET?

Azure Parallel Data Warehouse (PDW) Errors #1: cancelled transactions

Microsoft’s Azure data warehouse is a cloud hosted PaaS offering. Which implies compute and storage resources are managed by Microsoft / Azure fabric. Scheduled and unscheduled maintenance activities occur which are supposed to be transparent, i.e. should not impact the platform. This is a false assumption. On some occasions maintenance will impact the platform. For … Continue reading Azure Parallel Data Warehouse (PDW) Errors #1: cancelled transactions

Forecasting Database Growth

Been thinking about the most accurate method for forecasting. More closely to my line of work, I’d like to forecast database growth over a specified period of time into the future? The common method I have seen used is to simple use excel and plot a trend line from previously collected data, while this can … Continue reading Forecasting Database Growth

Don’t want to learn Perl, Python, Shell … Ok, try On.Inno

There are people on this planet earth, wonderful as it is, that don’t want to learn about the glue that holds all things together. And by glue I’m referring to the workhorse scripts that never really get the attention they deserve: Perl Python Shell One cannot truly say they have experienced a production infrastructure with at least typing one command from … Continue reading Don’t want to learn Perl, Python, Shell … Ok, try On.Inno

use Parallel::ForkManager

A very powerful and easy to use Perl multitasking module. Used in every single code I now write, as it’s annoying to continuously  update code just to make it scale-able. In my scripts I usually like to “fork” things off especially say when manipulating data. With this module it’s as easy as a foreach loop: use Parallel::ForkManager; use PiroLabs::Utils::DataCruncher; … Continue reading use Parallel::ForkManager