SQL Server 2008 MCM Readiness Videos

The other day I was doing some research on the Transaction Log of SQL Server and stumbled upon the MS SQL Server MCM Readiness videos, many of which are made by Paul Randal.

After watching just one of these videos I quickly found out how useful they can be for general knowledge, even if you aren’t trying to ramp up for the MCM certification.

Below is a link to over 70 videos about all areas of SQL Server from a detailed perspective. I have found them very useful in growing my own knowledge base and recommend that others pick and chose the topics from this list that relate to them and give them a watch.

http://technet.microsoft.com/en-us/sqlserver/ff977043

sp_Blitz v18 – Lots of Bug Fixes

Brent Ozar’s sp_Blitz v18 has been released. For those of you unfamiliar with sp_blitz I highly recommend not only downloading it but also watching Brent’s video explanation and reading all of his documentation on it.

sp_Blitz is a great way to get a fairly complete overview of a server that you might be unfamiliar with especially from a configuration and performance perspective.

See his blog post and download link below.

Ten Free SQL Server Tools that you Need to Know About

Hello, I’m happy to see that the buzz word “Free” did not fail to attract readers. NEVER FEAR! You will not leave this post disappointed.

I’m not going to try and do a better job than John Sansom has already done but I felt that this information needed to be widely distributed.

On that note, I highly encourage you to check out his article here and learn about the great free tools available to us all.

Quick Links:

  1. Plan Explorer
  2. sp_Blitz
  3. sp_BlitzIndex
  4. sp_WhoIsActive
  5. Management Data Warehouse (MDW)
    1. See one possible problem if this feature is over loaded in my post here.
  6. Index Defrag Script 4.1
  7. Compression Estimator
  8. Central Management Server
  9. ClearTrace
  10. SQL Server 2012 Diagnostic Information Queries

Deadlocks – Querying Extended Events

Background

The days of restarting SQL Service with trace flags 1204 or 1222 (MSDN Detecting and Ending Deadlocks) or being forced to start up a profiler trace are no more.

Let’s face it. There are times when someone comes up to you and says, “hey we got a handful of deadlocks an hour ago can you tell me how it happened?” Now, if you are practicing pro-active DBA methods then you might already have this information but if you are left with the deer in the head-lights look then these queries might be for you.

Extended Events was new with SQL Server 2008 and by default captures xml deadlock reports. For information about Extended Events see Pinal Dave’s post here or you can read the white paper here.

The Code

The comments on the queries below are fairly self explanatory. Running the first query will return a single record with a lot of improperly formatted xml. This query is given because it is very fast. All you need to do then is paste it into notepad and search for xml_deadlock_report to find the deadlock graph xml. I provided the bottom query for use when you come across the sql handles in the xml to identify the queries that were being run.

The middle query is the best of the bunch here but can be very slow. This one will return a row per deadlock graph in properly formatted xml. Must easier to work with.

In both cases the most recent deadlock events will be at the bottom of the result set or the bottom of the xml code block copied out from query one.

-- This query is fast but the xml is not formatted properly so you
-- need to paste the result in notepad and search for xml_deadlock_report.
select CAST(target_data as xml) as TargetData
from sys.dm_xe_session_targets st
join sys.dm_xe_sessions s on s.address = st.event_session_address
where name = 'system_health'

-- This query gives you a record per deadlock graph with xml corrected
-- but this query is VERY slow. There is no server performance impact
-- but you should expect this query to run for 30 mins +
select XEventData.XEvent.value('(data/value)[1]', 'varchar(max)') as DeadlockGraph
FROM
(select CAST(target_data as xml) as TargetData
from sys.dm_xe_session_targets st
join sys.dm_xe_sessions s on s.address = st.event_session_address
where name = 'system_health') AS Data
CROSS APPLY TargetData.nodes ('//RingBufferTarget/event') AS XEventData (XEvent)
where XEventData.XEvent.value('@name', 'varchar(4000)') = 'xml_deadlock_report'

/*For viewing the sql handles*/
SELECT * FROM sys.dm_exec_sql_text(0x03001a00ea93ed5bed752b0150a100000100000000000000);

Credits: (edited – 2013-02-13)

Many thanks to Jonathan Kehayias for authoring these scripts and providing an informative article here on sqlservercentral.com.

Temporary Stored Procedures

We are all familiar with table variables, local temp tables, and global temp tables but what some people over look is that stored procedures and functions can also utilize tempdb.

I imagine that the reason for this oversight is because it can be difficult to think of a use for it. To be honest, in almost all cases a permanent stored procedure is the best way to go. But if you are a DBA that often has to look into servers that you can’t change or simply don’t want to deploy a stored procedure that might only be used once and discarded then temp stored procedures might be the thing for you.

I personally like to use these in situations (such as indicated in the example below) where I want to run Brent Ozar’s sp_Blitz on an instance that I’m unfamiliar with or if I wanted to execute a restore of all files in a directory. In that case I could loop through the results of xp_dirtree and have it call Tibor Karaszi’s RestoreDatabase_SQL2008 without having to apply the stored procedure permanently on the server.

Syntax:

Use the same # or ## prefix that you normally use on tables you can use to create these stored procedures. For detailed syntax information see MSDN CREATE PROCEDURE (Transact-SQL).

IF OBJECT_ID('tempdb..##sp_Blitz') IS NOT NULL
DROP PROC ##sp_Blitz
GO
CREATE PROCEDURE ##sp_Blitz
@CheckUserDatabaseObjects TINYINT = 1 ,
@CheckProcedureCache TINYINT = 0 ,
@OutputType VARCHAR(20) = 'TABLE' ,
@OutputProcedureCache TINYINT = 0 ,
@CheckProcedureCacheFilter VARCHAR(10) = NULL ,
@CheckServerInfo TINYINT = 0 ,
@Version INT = NULL OUTPUT
AS
SET NOCOUNT ON;
--sp_Blitz v16 - December 13, 2012 
-- (C) 2012, Brent Ozar Unlimited - http://www.BrentOzar.com/blitz
--Rest of code here
GO
EXEC ##sp_Blitz
@CheckUserDatabaseObjects = 1 ,
@CheckProcedureCache = 1 ,
@OutputType = 'TABLE' ,
@OutputProcedureCache = 0 ,
@CheckProcedureCacheFilter = NULL,
@CheckServerInfo = 0

Bug: INSERTing into a MySQL Database via Linked Server

Recently I found myself involved in a project where a tab delimited text file (updated daily) was being imported into a MS SQL Server 2008 R2 database and then the single imported table was manipulated into a handful of others in a completely different format. This was done to stage the data in a manner which was going to be exported to Excel and then uploaded to an Open Cart based eCommerce site.

Once this process was complete there was an issue discovered with the uploading extension to Open Cart and I was asked to convert this process into direct INSERTS/UPDATES into the MySQL database schema instead. Naturally, I decided to not re-invent the wheel so I set out to setup a linked server to the MySQL database and then just run the necessary logic and DML statements from MS SQL Server and pushing to MySQL.

Detailed instructions of how to setup a MySQL linked server via the SSMS GUI is located here in a post by Taylor Gerring.

After successful validating my connection with read-only queries I wrote myself a stored procedure which would basically perform a merge process based on certain business logic and as soon as I executed the proc in my testing environment I encountered this error.

Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "MSDASQL" for linked server "MYSQL" reported an error. The provider did not give any information about the error.
 Msg 7343, Level 16, State 2, Line 1
The OLE DB provider "MSDASQL" for linked server "MYSQL" could not INSERT INTO table "[MSDASQL]". Unknown provider error.

As it turns out, even though by following Taylor’s instructions I assigned a catalog to the ODBC DSN, the GUI requires and fails to set the catalog setting in the linked server. Another oddity is that this bug only affects INSERT statements because my read-only queries all worked just fine without setting the database name. Below is an example of one of my test queries which worked.

DECLARE @optionID as INT
SELECT @optionID = option_id
FROM MYSQL_OPENCART...[option_description]
WHERE name = 'Add-ons'

So now what do we do about this?

We are going to have to recreate our linked server with T-SQL instead so we can manually set the database name. Below is a statement which I was successfully with and which I first gathered from a discussion about this bug here.

EXEC master.dbo.sp_addlinkedserver @server='MYSQL', @srvproduct='MySQL',
@provider='MSDASQL', @provstr='DRIVER={MySQL ODBC 5.1
Driver};SERVER=HOST;Port=3306;USER=uid;PASSWORD=pw;OPTION=3;DATABASE=mydb;

With this annoying bug identified and worked around I’ve move forward with my project and  hope that this won’t be necessary in SQL Server 2012.

Versions Suffering from this Bug:

  • MS SQL Server 2008
  • MS SQL Server 2008 R2

Bug: Local View + Synonym + Remote Query

Recently I came across an interesting bug when upgrading a SQL Server 2000 box to 2008 R2. With SQL Server 2005, 2008, and 2008 R2, there is a situation where you are calling a local view that is also using a synonym and accessing a remote table and then the remote table’s index is rebuilt. In this situation the synonym should not fundamentally change the way the query operates but in the versions listed you will receive the below error.

The OLE DB provider "SQLNCLI" for linked server "REMOTESERVER" reported a change in schema version between compile time ("170677722198508") and run time ("170677722198546") for table ""tempdb"."dbo"."remotetable"".

This occurs because the execution plan of the view is set at compile time and the act of creating and dropping an index is in fact a schema change. As described here, with an index rebuild you are affecting a schema change and therefore the schema versions will be different and throw the error above.

This bug has been accepted by Microsoft (as referenced here) and has been fixed in SQL Server 2012 RTM. See below for a test case that reproduces this bug.

I have used the same test case with 2012 and witnessed that the bug is in fact fixed.

Work Arounds:

  • Re-write the process to exclude any one of the necessary factors which cause the bug.
  • Create a local view on the remote server and referencing that in the synonym.
  • Use sp_refreshview before calling the view.
    • This was the simplest fix in my situation because no one wanted to affect a code change, this ran in a SQL Agent job and the view was only used once a month. So I just added a new step prior to the stored procedure call to refresh the view.

Test Case:

The following script runs in SQLCMD mode and will generate the necessary test case and provides print out statements to walk you through to successful and buggy situations.

The test script was pulled from a post here by Jasper Smith.

Download: linked server view synonym tests.txt

Detach and Attach User Databases

The Need

I’ve found myself, more times than once, needing to change disk drives that my user databases are found on. This could come up because you are moving a server and need to switch SANs, migrate from local storage to SAN storage, or need to move to a new LUN because it was over expanded and your SAN techs can’t shrink it without a drop/create.

This exercise can be done with the GUI or with typed T-SQL but in my cases there are usually a couple of dozen databases with file in a number of different sub-directories. Today I’m going to go over each method but conclude on a T-SQL script that I wrote to do almost all of the work for us.

Method #1: T-SQL

To detach a database and re-attach you will be using two different types of T-SQL. For the detach method you will be using the stored procedure sp_detach_db while for the attachment you will be using a CREATE DATABASE command with the FOR ATTACH option.

The syntax for the detachment on SQL Server 7.0 through 2012:

USE master;
GO
EXEC sp_detach_db @dbname = N'AdventureWorks2012';
GO

Syntax for the attachment on SQL Server 2005 through 2012:

USE master;
GO
CREATE DATABASE MyAdventureWorks 
    ON (FILENAME = 'C:\MySQLServer\AdventureWorks2012_Data.mdf'),
    (FILENAME = 'C:\MySQLServer\AdventureWorks2012_Log.ldf')
    FOR ATTACH;
GO

Syntax for the attachment on SQL Server 7.0 and 2000:

use master
go
sp_attach_db 'mydb','E:\Sqldata\mydbdata.mdf','E:\Sqldata\mydblog.ldf'
go

Method #2: GUI via SSMS

  1. Detach
    1. Right click the database that you wish to detach and select Tasks->detach.
    2. Check “Drop Connections” so that your detach won’t be prone to timing out due to a persisted connection.
    3. Click OK.
  2. Attach
    1. Right click Databases and select Attach…
    2. When the Attach Databases window opens, click Add…
    3. Locate and select the mdf file for your database and click OK.
    4. When your mdf file is selected the “database details” group box will be populated with a list of all of the files that are associated to the mdf. In my screenshot below the log file was discovered because it was in the same directory. If your ldf or ndf files are not in that same location there will be text in the “Message” field that will indicate that you need to tell it where the file is. You would, in this case, click the ellipsis that I have highlighted and navigate to the file in question.
    5. Click OK and your databases will be attached.

Method #3: T-SQL Script Generation

Below is the script I wrote to write all of my detach and attach statements for me. It first prints out the detach statements but it queries for the database file locations and builds your attachment statements as well.

Important: Once you run the detach scripts you will no longer be able to query the file information so don’t make the mistake of running these in the same query window as the print statements.

DECLARE @name VARCHAR(50)
DECLARE @path VARCHAR(MAX)
DECLARE @execSQL VARCHAR(MAX)
DECLARE @filename VARCHAR(MAX)
DECLARE @database_ID INT

--Place the directory paths here
DECLARE @newdatapath VARCHAR(MAX) = 'Q:\InstanceName\'
DECLARE @newlogpath VARCHAR(MAX) = 'Z:\InstanceName\'

--populate file locations
SELECT name, physical_name AS CurrentLocation, database_id
INTO #temptable
FROM sys.master_files
WHERE [database_id] > 4

--detach
DECLARE detach_cursor CURSOR FAST_FORWARD FOR
SELECT name, physical_name AS CurrentLocation, database_id
FROM sys.master_files
WHERE file_id = 1
AND name <> 'master'
AND name <> 'tempdev'
AND name <> 'msdbdata'
AND name <> 'modeldev'

OPEN detach_cursor
FETCH NEXT FROM detach_cursor INTO @name, @path, @database_ID

PRINT  'USE master;' + CHAR(13) + CHAR(10)

WHILE @@FETCH_STATUS = 0
BEGIN
	SET @execSQL = 'EXEC sp_detach_db @dbname = N''' + DB_NAME(@database_ID) + ''';'
	PRINT @execSQL

	FETCH NEXT FROM detach_cursor INTO @name, @path, @database_ID
END
CLOSE detach_cursor
DEALLOCATE detach_cursor

--attach
PRINT CHAR(13) + CHAR(10) + CHAR(13) + CHAR(10)

DECLARE attach_db_cursor CURSOR FAST_FORWARD
FOR
        SELECT DISTINCT [database_id]
        FROM    #temptable

OPEN attach_db_cursor
FETCH NEXT FROM attach_db_cursor INTO @database_ID

PRINT 'USE master;' + CHAR(13) + CHAR(10)

WHILE ( SELECT  fetch_status
        FROM    sys.dm_exec_cursors(@@SPID)
        WHERE   name = 'attach_db_cursor'
      ) = 0 
      BEGIN

			SET @execSQL = 'CREATE DATABASE ' + DB_NAME(@database_ID) + ' ON' + CHAR(13) + CHAR(10)

            DECLARE attach_file_cursor CURSOR FAST_FORWARD
            FOR
                    SELECT  [name]
                          , [CurrentLocation]
                          , [database_id]
                    FROM    #temptable
                    WHERE [database_id] = @database_ID

            OPEN attach_file_cursor
            FETCH NEXT FROM attach_file_cursor INTO @name, @path, @database_ID

            WHILE ( SELECT  fetch_status
                    FROM    sys.dm_exec_cursors(@@SPID)
                    WHERE   name = 'attach_file_cursor'
                  ) = 0 
                  BEGIN
                        SET @filename = @path
                        WHILE CHARINDEX('\', @filename, 0) <> 0 
                              BEGIN
                                    SET @filename = RIGHT(@filename,LEN(@filename) - CHARINDEX('\',@filename, 0))
                              END

						IF RIGHT(@filename,3) = 'ldf'
						BEGIN
							SET @execSQL = @execSQL + '(FILENAME = ''' + @newlogpath + @filename + '''),' + CHAR(13) + CHAR(10)
						END
						ELSE
						BEGIN
							SET @execSQL = @execSQL + '(FILENAME = ''' + @newdatapath + @filename + '''),' + CHAR(13) + CHAR(10)
						END

                        FETCH NEXT FROM attach_file_cursor INTO @name, @path,
                              @database_ID
                  END

                  SET @execSQL = LEFT(@execSQL,LEN(@execSQL)-3) + CHAR(13) + CHAR(10)
                  SET @execSQL = @execSQL + ' FOR ATTACH;'

                  CLOSE attach_file_cursor
				  DEALLOCATE attach_file_cursor

				  PRINT @execSQL

             FETCH NEXT FROM attach_db_cursor INTO @database_ID
      END

CLOSE attach_db_cursor
DEALLOCATE attach_db_cursor

DROP TABLE #temptable

I currently have this script setup to accept a data and log path in the variables @newdatapath and @newlogpath at the top of the script. This is for if you know the directories that you would like to move the databases to and if they are all the same for each database. I find this situation common but I have come across situations where I wanted to move drives but not change the folder structure. In that case I would build the folder structure and then replace the below lines of the script.

--Before
IF RIGHT(@filename,3) = 'ldf'
BEGIN
	SET @execSQL = @execSQL + '(FILENAME = ''' + @newlogpath + @filename + '''),' + CHAR(13) + CHAR(10)
END
ELSE
BEGIN
	SET @execSQL = @execSQL + '(FILENAME = ''' + @newdatapath + @filename + '''),' + CHAR(13) + CHAR(10)
END

--With change
IF RIGHT(@filename,3) = 'ldf'
BEGIN
	SET @execSQL = @execSQL + '(FILENAME = ''' + REPLACE(REPLACE(@path,'E:\','Q:\'),'F:\','Z:\') + '''),' + CHAR(13) + CHAR(10)
END
ELSE
BEGIN
	SET @execSQL = @execSQL + '(FILENAME = ''' + REPLACE(REPLACE(@path,'E:\','Q:\'),'F:\','Z:\') + '''),' + CHAR(13) + CHAR(10)
END

Success

Using method #3 I was able to relocate the database files on 2 servers with 5 instances each and 1-2 dozen databases per instance in less than 10 minutes.

Now I just need to move the system databases.

How to: Transfer Logins to a New Server

Often enough we encounter a situation where we need security to be migrated. Microsoft hasn’t made this task as easy as moving other things such as SQL Agent jobs since restoring the master database to another server is tricky at best but they do provide us some tools.

A couple of common reasons that you’d want to move logins is if you are standing up a test server and want the servers to match identically or you are standing up a disaster recovery server and might need to re-sync logins as changes are made to the primary.

Preferred Method: T-SQL

I personally prefer using T-SQL over SSIS whenever the situation leaves to the two methods on an even playing field so I use the Microsoft provided stored procedures even though I will go over the SSIS method below.

sp_help_revlogin is a stored procedure which will satisfy two main feats while printing out all of the necessary create statements for all of your instance logins. The first is that it returns as part of the create statements all of the SIDs for your SQL accounts. Domain accounts don’t require this step but when creating a new SQL account and then restoring a legacy database to that instance you often are left with orphaned users because the database level user exists and so does the login but their SIDs don’t match. The second feat is that it returns the SQL account passwords in hex format so that you aren’t exposing the passwords to anyone. But DBAs who have no need to know those passwords can still create the new user.

As found in KB918992 and KB246133 there are two stored procedures that Microsoft provides but are not installed implicitly. The first is sp_hexadecimal which is a function used to pull SQL account passwords in hex so that they aren’t directly exposed to the person executing the stored procedure sp_help_revlogin. sp_help_revlogin is the primary procedure that you execute and it takes a single optional parameter @login_name (sysname). @login_name is NULL by default and omitting it will return all logins for the instance. If you pass in the login name then you will receive the statement for just that user.

NOTE: The KB articles where you get the stored procedure only list versions 7.0 through 2005 as applicable but I have tested this on 2008 and 2008 R2 and it works fine.

 Example Syntax:

EXEC dbo.sp_help_revlogin
--OR
EXEC dbo.sp_help_revlogin @login_name = 'BlogTester' -- sysname

 Example Output:

/* sp_help_revlogin script 
** Generated Aug 20 2012  8:24AM on V-DEV-DB-011\vb18 */

-- Login: BlogTester
CREATE LOGIN [BlogTester] WITH PASSWORD = 0x01000D1F43BB2A1F306EC90F5352291E8DD273549DB4AF950845 HASHED, SID = 0xD831296B0274D448A5748A52B8C796EA, DEFAULT_DATABASE = [master], CHECK_POLICY = OFF, CHECK_EXPIRATION = OFF

Permissions:

Only members of the fixed server role ‘sysadmin’ can access the system table master..sysxlogins which is required to execute sp_help_revlogin therefore you will need to be a sysadmin in order to run this procedure.

SSIS Method: Transfer Logins Task

BIDS provides us with the Transfer Logins Task. This task will transfer logins from one SQL Server source to one SQL Server destination. It can transfer all logins, a single login, or specific logins selected as a collection. It also can be configured to overwrite, skip, or throw an error when it reaches a login that already exists.

See the technet post for more details.

Permissions:

This method also requires you to be a sysadmin of the source server, in addition, it requires sysadmin to the destination server.

Beware of this method

The SSIS Method does not pull over existing passwords like the T-SQL method does. Instead it will assign random passwords to each of the SQL accounts that it creates. You will then after to go through and manually (or via script) update every password for the accounts that were created.

Important Planning Note

The create statements with both methods will include the default database for the login. If this is not set to master or a database that already exists on the destination then there will be errors and at least the create statement in question will fail. I find it easiest to handle security after I’ve done most other tasks involved in my migration or instance sync.

Huge distribution database

The Problem:

Today I noticed the data drive of one of my Production servers growing. It wasn’t growing overly fast and it had been quite some time since the last SAN drive expansion so I gave the thumbs up to my SAN techs to add 5GBs of space. I started with 5GBs just out of a hunch even though I knew in a couple weeks I’d be giving the green light to add more space again.

It turned out that adding 5GBs at first was a good idea because what I assumed with normal data growth turned out to be a ballooning distribution database. When I noticed my disk space alert again in the same day is when I got wise because this server should not grow 5 GBs in a day.

After a quick look it was easy to notice the distribution database was the issue with it being 48GBs which is far above normal for the publications that it handles.

The Cause:

I went ahead and pulled up my friend, the replication monitor, and noticed a red X over the publisher with the ballooned distribution database. Upon further investigation I noticed 5 publications were red and the offending subscription in each was for the same subscriber (our source database for our data warehouse which is used to perform warehouse tuning operations separate from the transactional data).

I pulled up the subscriber and noticed that it has no subscriptions. What apparently happened was, one of my DBAs had done a restore without using the KEEP_REPLICATION flag. This restored the database who was a subscriber but broke the transfer. Since the source still had the subscription registered in it all the transactions for the last 72 hours were being held and not purged because the default retention period for a distribution database is 72 hours.

The Fix:

So what I ended up doing was first dropping all of the subscriptions that were broken and then recreating them. This fixed the root cause but my distribution database was still large and had 0% free space to shrink.

What I had to do next was hit up the Google. I knew that the job ‘Distribution clean up: distribution’ was running and would eventually clean up my database but when I have a problem I like to stick to it until fixed and this job was running slowly. That is when I found clsilva’s post Replication: Distribution Cleanup and Paul Ibison’s post How to Massively Optimise Transactional Replication.

The command that clsilva talks about, EXEC dbo.sp_MSdistribution_cleanup @min_distretention = 0, @max_distretention = 72, is actually exactly what the job, Distribution clean up: distribution is running that is slow.

This process is slow because it is trying to purge a row count, which in my case, is very large. I ended up stopping this job and running the commands myself in steps as recommended by clsilva. I started with a 2 hour block and found that I was deleting records at a 2 row per second rate. This was sufficient evidence that to me that the process was running successfully but was simply very slow. When you have a working system stored procedure I never dive down deeper and try to tweak things myself. So what I ended doing was executed a script that ran this command decrementing the max and min values by 10 hours a piece in separate batches. The only purpose of this was so that I could have a 10 hour incremented print out in SSMS that would reassure me and give me a pseudo-progress bar.

The End

Upon completion of my script I shrunk my distribution database down to its normal size and went on with my day.