Category: Cloud
-
Amazon Redshift Views, Sort Keys, and Outer Joins
My team built a process to load from a couple of base tables, in our Amazon Redshift enterprise data warehouse, into an other table which would act as a data mart entity. The data was rolled up and it included some derived fields. The SQL query had some complicity to it. This process ran daily […]
-
Amazon Redshift Unexpected Commit
If you are familiar with sqlhammer.com, then you know that I heavily specialize in the Microsoft Data Platforms. My experiences with Microsoft, and specifically the SQL Server products, established an expectation that explicit transactions would always commit/rollback successfully or there would be an exception thrown to explain the problem. That expectation set the stage for […]
-
NYC Yellow Cab Data in Azure SQL Data Warehouse
Microsoft Azure provides a convenient feature when creating a new Azure SQL Data Warehouse database. You can create the database from a backup of the Microsoft sample database, AdventureWorksDW. This is super fast and easy. It also has huge tables like dbo.FactInternetSalesReason with 64,515 rows! Oh, hold on I think I typo’ed. No, actually that is […]
-
Amazon Redshift Temporary Tables
My company is in the middle of a migration to Amazon Redshift. Data warehouse design standards have been in question now that our platform is changing. One development team asked me whether they would be allowed to use temporary tables or not. They know that different database platforms handle temporary tables differently and there was […]
-
What is a data lake?
Our zatz-its are wise. Our zatz-its are dispersed. We have questions to ask the zatz-its, so we corral them in a warehouse. Now everyone can see the zatz-its and ask any question that the zatz-it can answer. But buyer beware, our zatz-its are data, try not to stare. We have more snuvs than we can […]
-
What is a cloud data warehouse?
Amazon Redshift was announced in November of 2012 and became the first cloud data warehouse, opening a whole new segment of technology. What is a cloud data warehouse exactly? On-premises data warehouse While cloud data warehouses are relatively new, at least from this decade, the data warehouse concept is not. A data warehouse is a […]
-
SQL Data Warehouse – Fast row counts
A couple of years ago I published a script to calculate fast, light-weight, row counts for SQL Server on-premises. The need to view the row counts of tables has not diminished but new technologies have come to the fore-front. Azure SQL Data Warehouse is a massively parallel processing (MPP) architecture designed for large-scale data warehouses. […]
-
T-SQL Tuesday #95 – Unstructured Cosmos
T-SQL Tuesday (#TSQL2sDAY) is a blog party founded by Adam Machanic (b | t). Each month a member of the community hosts the party and selects a topic for us to write about. This month, I am hosting. I invited everyone to write about Big Data. For my post, I chose to learn about Microsoft […]
-
Launch your database into Azure SQL Database – bacpac edition
You have decided to migrate our on-premise SQL Server database into Azure SQL Database. How do you do that? The first thought that comes to mind is a simple backup from on-premise and restore to Azure SQL Database. Unfortunately, that is not supported by Microsoft. Instead, there are three supported methods of migrating into Microsoft’s […]
-
AWS Database Migration Service
I have begun writing a few blog posts focused on launching your databases into the cloud. Specifically targeted on database migration methodologies. Choosing a migration method involves making a lot of decisions up-front. This post will assume that Managed Relational Database Service (RDS), which is a Platform as a Service offering by Amazon Web Services […]