Category: Data Warehouse
-
Amazon Redshift Views, Sort Keys, and Outer Joins
My team built a process to load from a couple of base tables, in our Amazon Redshift enterprise data warehouse, into an other table which would act as a data mart entity. The data was rolled up and it included some derived fields. The SQL query had some complicity to it. This process ran daily […]
-
Musings of a DevOps Data Pro
Continuous delivery and automation are a couple of core concepts of DevOps. As a Data Professional, I have spent countless hours pushing through the resistance of people who feel the data tier should be left out of source control, automated deployments, and who feel the need to make the data tier a separate work stream […]
-
Amazon Redshift Unexpected Commit
If you are familiar with sqlhammer.com, then you know that I heavily specialize in the Microsoft Data Platforms. My experiences with Microsoft, and specifically the SQL Server products, established an expectation that explicit transactions would always commit/rollback successfully or there would be an exception thrown to explain the problem. That expectation set the stage for […]
-
NYC Yellow Cab Data in Azure SQL Data Warehouse
Microsoft Azure provides a convenient feature when creating a new Azure SQL Data Warehouse database. You can create the database from a backup of the Microsoft sample database, AdventureWorksDW. This is super fast and easy. It also has huge tables like dbo.FactInternetSalesReason with 64,515 rows! Oh, hold on I think I typo’ed. No, actually that is […]
-
Amazon Redshift Temporary Tables
My company is in the middle of a migration to Amazon Redshift. Data warehouse design standards have been in question now that our platform is changing. One development team asked me whether they would be allowed to use temporary tables or not. They know that different database platforms handle temporary tables differently and there was […]