Chris May

About a year ago, we tried using a tool to get our SQL schema into a souce control system.  It was supposed to work like this: every night the process would run and checkin each object as a file into TFS. 

Well, aside from not working, it also added Extended Properties to all our objects in SQL Server. 

Not really a big deal, except when we would run our database comparison tools between production and development these would cause it so that almost everything would appear out of sync.

I ended up writing my own service to check in all our SQL objects into TFS, but we were left with all these Extended Properties all over the place. 

So I finally got around to cleaning them up using the script below:

select 'EXEC sp_dropextendedproperty 
@name = ''' + sys.extended_properties.name + ''' 
,@level0type = ''schema'' 
,@level0name = ' + object_schema_name(extended_properties.major_id) + '
,@level1type = ''' + 
    CASE WHEN  
        xtype = 'U' THEN 'table'
        WHEN 
        xtype = 'FN' THEN 'function'
        WHEN 
        xtype = 'V' THEN 'view'
        WHEN 
        xtype = 'P' THEN 'procedure' end
    + '''
,@level1name = ''' + object_name(extended_properties.major_id) + ''''
from sys.extended_properties
INNER JOIN sysobjects ON sys.extended_properties.major_id  = sysobjects.id
where extended_properties.class_desc = 'OBJECT_OR_COLUMN'
and extended_properties.minor_id = 0
and extended_properties.name like 'VSS%'

This script will produce a record set that you can copy and paste into another query window to run to clean out all the extended properties. 

Note: all the generated extended properties that this tool created on our DB started with "VSS" so that is why I have that extra item in my where clause.  You'll probably want to remove that.

Last night I had another instance where doing a:

SELECT * FROM
TBL WHERE ID NOT IN (SELECT ID FROM TBL2)

would fail every time to return any rows.

After longer than it should have been, I realized that it was this stupid issue that I blogged about already (SQL "NOT IN" Will Fail If The List Contains A Null).

Here is a great resource for debugging deadlocks in sqlserver:

Part 1:
http://blogs.msdn.com/bartd/archive/2006/09/09/Deadlock-Troubleshooting_2C00_-Part-1.aspx

Part 2:
http://blogs.msdn.com/bartd/archive/2006/09/13/751343.aspx

UPDATE 2:

Either this never worked in the first place as I wanted, or MS has changed how they are doing things in SSRS 2008.

I now don't see any of the errors in the tables I had been looking in, and in fact, the errors don't show up in any of their logs either.  This is crap.

If you are like me, you don't like the fact that the first time you realize that a data driven subscriptions in SQL Server Reporting Services has been failing is when someone comes up to you and says "Hey, I did XYZ and I never saw the server kick off an email." 

Really?  Let me check.... oh, looks like it has been broken for god knows how long.  What the heck?

Well, I wrote a script to notify you when there are errors on the report server.

First, you need to setup Database Mail.  Expand Management, and pick Database Mail.  For my script I used the name "Email Profile" for the "Email Profile" get it?

Then create a trigger on the ExecutionLogStorage table like this:

USE [ReportServer]
GO
/****** Object:  Trigger [dbo].[ExecutionLogStorage_Insert]    Script Date: 05/13/2009 10:20:44 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER TRIGGER [dbo].[ExecutionLogStorage_Insert]
ON [dbo].[ExecutionLogStorage]
AFTER INSERT
AS 
BEGIN
    DECLARE @sStatus AS NVARCHAR(32)
    SET @sStatus = (SELECT Status FROM inserted)
    
    IF(@sStatus <> 'rsSuccess')
    BEGIN
        DECLARE @sReportName AS NVARCHAR(425)
        DECLARE @sPath AS NVARCHAR(425)
        DECLARE @ReportId AS UNIQUEIDENTIFIER 
        DECLARE @sUserName AS NVARCHAR(260)
        DECLARE @sParameters AS NVARCHAR(4000)
        DECLARE @dTimeEnd AS DATETIME 
        DECLARE @iTimeProcessing as int
        
        
        SELECT 
          @sReportName = Name,
          @sPath =PATH,
          @ReportId = ReportId,
          @sUserName = username,
          @sParameters = (select CAST(ISNULL(ELS.Parameters,'') AS NVARCHAR(4000)) from ExecutionLogStorage ELS Where LogEntryId = Inserted.LogEntryId),
          @dTimeEnd = TimeEnd,
          @iTimeProcessing = ISNULL(TimeProcessing,0)
        FROM inserted INNER JOIN [Catalog] 
        ON inserted.ReportId = CATALOG.[ItemID] 
        
        DECLARE @sBody VARCHAR(8000)
        SET @sBody = 'Error with SQL Report ' 
                    + CAST(@sReportName AS VARCHAR(100))
                    + ' at path ' 
                    + CAST (@sPath AS VARCHAR(250))
                    + ' with ReportId ' 
                    + CAST(@ReportId AS VARCHAR(100)) 
                    + ' and username ' 
                    + CAST(@sUserName AS VARCHAR(100))
                    + ' Parameters ' 
                    + CAST(@sParameters  AS VARCHAR(5000))
                    + ' End time ' 
                    + CAST(@dTimeEnd AS VARCHAR(50))
                    + ' Status ' 
                    + CAST(@sStatus AS VARCHAR(50))
                    + ' Time Processing '
                    + CAST(@iTimeProcessing AS VARCHAR(50));
        
        DECLARE @sSubject VARCHAR(500)
        SET @sSubject = 'Error With Report ' + @sReportName
        
        EXEC msdb.dbo.sp_send_dbmail 
         @profile_name = 'Email Profile', 
         @recipients = 'email@address.com', 
         @body =  @sBody  ,
         @subject = @sSubject;

    END
    
END

Presto!

UPDATE: Added logic to get the status and parameters.

I'm working on something where I wanted to know if some parameters to a SQL Server Stored Procedure were nullable (or, in otherwords, if they had a default value).

The API I was working with provided a way to find info about the SP, but the isNullable value was never accurate.

I tried working with with the MS DAABs which have a way to fetch SP parameter info, but this method also produced faulty information.

Looking into the code of the DAAB, it was internally calling:

Dim command As SqlCommand

SqlCommandBuilder.DeriveParameters(command)

Somewhere I came across some code for the DeriveParameters method (I think from the Mono project maybe?) which showed it calling this system SP:

sp_procedure_params_rowset

Which returns output fields,:

PARAMETER_HASDEFAULT

IS_NULLABLE

But, again, neither of these appear correctly.

However, I finally came across this thread where a MSFT poster indicates that this data is simply not available:

http://social.msdn.microsoft.com/forums/en-US/transactsql/thread/900756fd-3980-48e3-ae59-a15d7fc15b4c/

Instead, you need to parse the SP definition to see what the default parameter value is, which is a pain, but at least I know why these other methods kept not working.

One of the challenges of using code generation is to avoid overwriting your custom changes when you re-generate the code later.

This is a trick I am using to avoid overwriting my stored procedures if I update them manually.  The trick is to include a comment indicating that the SP has been code generated, and then to remove that comment if you update the SP.  So lets say that you generate:

CREATE PROCEDURE OrderDetails_Get
@OrderId as int
AS
-- CODE GENERATED
SELECT * FROM
OrderDetails
WHERE
OrderId = @OrderId

and you made some customizations to the SP so it now looks like this:

CREATE PROCEDURE OrderDetails_Get
@OrderId as int
AS
SELECT * FROM
OrderDetails
INNER JOIN
Orders ON
Orders.OrderId = OrderDetails.OrderId
WHERE
OrderId = @OrderId

The following code will help you when generating your SPs to avoid overwriting this change:

IF EXISTS(
  SELECT [definition] AS objectText 
  FROM sys.sql_modules 
  WHERE [object_id] = OBJECT_ID(N'OrderDetails_Get', 'P') 
  AND definition LIKE '%CODE GENERATED%')
BEGIN
   -- overwrite is ok
END

Today we started getting this error when trying to do full text search with SQL Server 2008.

Msg 30046, Level 16, State 1, Procedure CodeMaster_GetCostCodesByCriteria_FullText, Line 47

EXEC sp_fulltext_service 'restart_all_fdhosts'

If you want to check and update the identity on tables in SQL Server, you can use the following code:

select ident_current('tablename') as ident
dbcc checkident('tablename',RESEED,100)
select ident_current('tablename') as newident

We utilize data driven subscriptions in SQL Server Reporting Services (SSRS) to automate several reports and their distribution to a group of people.  For example maybe when a work order is created in the database a report with info about that order is emailed to everyone who will have to be involved in fulfiling the order. 

We realized that some of these reports were not going out.  By looking at the log files (located at ) it became clear that we were hitting a permissions issue:

ReportingServicesService!library!4!09/25/2008-08:15:04:: 
e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: 
The permissions granted to user 'MYDOMAIN\someuser' are insufficient for performing this operation., ;
Info: Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: 
The permissions granted to user 'MYDOMAIN\someuser' are insufficient for performing this operation.

Ok that seemed to make sense.  The user "someuser" had left our company and so I'm sure his account was disabled.  After looking around at the report, it's definition, the subscriptions, the data access, nothing was tied to this old employee. 

But.... the subscription itself still is.

The downside of this is that there is no way to change who "owns" the subscription.

However, you can make the changes manually in the database with the following code:

DECLARE @OldUserID uniqueidentifier
DECLARE @NewUserID uniqueidentifier 

SELECT @OldUserID = UserID FROM dbo.Users 
WHERE UserName = 'MYDOMAIN\someuser'

SELECT @NewUserID = UserID FROM dbo.Users 
WHERE UserName = 'MYDOMAIN\newuserhere'

UPDATE dbo.Subscriptions 
SET OwnerID = @NewUserID 
WHERE OwnerID = @OldUserID

UPDATE: I am going to try to work on something that will monitor the subscriptions and notify me if one of them fail.  Check back later.

I have been poking around with NHibernate for a while now, but I am actually writing a small app with it at the moment.

During my time getting it up and running, I came across a few well written tutorials that I want to catalog here in case I want to return to them at some point for more in depth reading.

Great NHibernate Faq:
http://www.tobinharris.com/2007/2/3/nhibernate-faq

Fluent Interface for NHibernate:
http://blogs.hibernatingrhinos.com/...nhibernate.aspx

Alan Northam's Tutorials:
http://devlicio.us/blogs/alan_northam/...part-i.aspx
http://devlicio.us/blogs/alan_northam/...part-ii.aspx
http://devlicio.us/blogs/alan_northam/...part-iii.aspx
http://devlicio.us/blogs/alan_northam/...part-iv.aspx

Simple-talk.com has a pretty in depth article about deployment and mangement of databases.

Rolling out changes to the application is only 1 part of a deployment.  Updating the database can be more difficult depending on your schema.

We could have more process around the DB development and deployment process.  I will be giving this a read shortly.

Has anyone been getting these errors when serving up SQL Reporting Services Reports?

A connection that was expected to be kept alive was closed by the server

All of a sudden we have been getting a lot of these, and I am not sure at the moment what is causing them.

We have 2 webservers that are hosting the reportviewer control, which loads up reports from a single reporting server.

This guy suggested that this problem was related to some 10 minute timeout, but I haven't been able to replicate his 10 minute problem.

Some other people are talking about overrideing the webservice proxy classes to set KeepAlive = false, but we have no webservice proxy to override as we are using the reportviewer.

Update:

This guy is talking about changing the SSRS session timeout.

Could "rc:Toolbar=false" be the problem?

Update 2:
Looks like this was all caused by the introduction of an IDP by our network guys.

Recently I wanted to find the time between 2 datetimes, but I wanted to exclude weekends.  To further complicate matters, I wanted to know hours, not just days.

The "standard" way to do stuff like this is to build out a calendar table that has every date for the next 50 years, along with info like "IsWeekend" or "IsHoliday".

But I found some snippets around that I hoped to use to my advantage, and what I came up with works very well:

DECLARE @start DATETIME
DECLARE @end DATETIME

SET @start = '3/7/08 12:50 pm'
SET @end = '3/11/08 1:50 am'

SELECT (DateDiff(hh, @start, @end) - DateDiff(ww, @start, @end)*2*24)/24.0
       

This is some useful stuff from 4guysfromrolla.com. 

BEGIN TRY
   BEGIN TRANSACTION    -- Start the transaction

   -- Your code here

   -- If we reach here, success!
   COMMIT
END TRY
BEGIN CATCH
  -- Whoops, there was an error
  IF @@TRANCOUNT > 0
     ROLLBACK

  -- Raise an error with the details of the exception
  DECLARE @ErrMsg nvarchar(4000), @ErrSeverity int
  SELECT @ErrMsg = ERROR_MESSAGE(),
         @ErrSeverity = ERROR_SEVERITY()

  RAISERROR(@ErrMsg, @ErrSeverity, 1)
END CATCH

We recently ran into a problem with SSRS where our email subscriptions were processing, but no emails were going out.

SSRS doesn't really give you any info in the web interface to give you any clue that something isn't work right, let alone help you figure out what the problem is.

I was able to solve the problem by looking at the various log files that SSRS creates:  ReportServerWebApp__X, ReportServerService__X, and ReportServer__X.

In these config files I was able to find some errors coming back from our mail server. 

This lead me to discover that the problem was with how the email addresses were formatted. 

The issue was that we recently changed which mail server we used for sending out these emails, and we have moved from an Exchange 2003 server to an Exchange 2007 server.

Exchange 2003 would accept domain accounts as destinations for emails, so you could send an email to "cmay" and it would be delivered.  It seems that Exchange 2007 requires that you provide the full email address: cmay@company.com. 

After changing some of the SPs that served up the data, all worked once again.

Every now and then I go to my blog to lookup something I wrote before only to find that I never posted it.

I would have bet someone $20 that I had posted something on how to strip off the time from a DateTime in SQL.  Why would you do this?

Well, lets say that have some autogenerated DateTime field (CreatedOn?) that you later want to join with another field, but the other field only knows the DATE when something happened, not the exact Date and Time.

So, you can use the following code to just get the DATE and zero out the time component:

SELECT CAST( CONVERT( CHAR(8), GetDate(), 112) AS DATETIME)

If you are using the NOT IN clause with a SQL Select statement with SQL Server it will not work if there is a null value in the list.  Say for example you want to do:

SELECT * FROM 
Employees 
Where 
Employee.Id NOT IN 
(  
   Select EmployeeId from 
   Salaries 
   where Salary < 100000
)

If the inner select returns a null value then the outer select statement will return no rows.

To try it, just run these 2 SQL statments against any table in your DB.

-- will return rows
SELECT * FROM 
sometable
WHERE ID NOT IN (1,2,3)

-- will return nothing
SELECT * FROM 
sometable
WHERE ID NOT IN (1,2,3, null)

When working with a SSRS report, the report framework will already "Denull" everything.  But in some cases, for example when you are working with numbers, you might want to display a value in lue of the "" that is displayed for nulls.

Now you would THINK, that they would just use the same ISNULL syntax that everyone knows from sql programming.

HAHAHHAHA... of course they didn't do that.

Instead you have to use a combo of IIF and IsNothing like so:

=IIF(IsNothing(Fields!Quantity.Value),0,Fields!Quantity.Value)

It isn't THAT big of a pain, but I wish MS would sometimes just realize how much easier it would be for them to just create an IsNull function in their own codebase than make their users write this out.

Oh well.  I guess when it comes to problems with SSRS, this is waaaaaaayyyyy down my list.

Speaking of "my problems with SSRS"... when are you guys going to support TBLR text??  Drives me nuts.  Everyone does "vertical text" in TBLR format, for pretty much every application, but now in SSRS you are forced to use TBRL.  When you give these reports to engineers that is the very first thing they say "You need to turn this text around the other way".... yes I know, I got my degree in Civil Engineering with a focus on structure design, so when you come from a background of seeing all vertical text (like on any plans) as ALWAYS, by requirement, aligned in a TBLR manner, and then you are forced to cock your head the other way to read it, it is really a pain.

But I guess I would still like to see MS fix their *terrible* PDF rendering first.  I just LOVE watching my reports take 4x as long as in CR, and end up 100x bigger in file size than they are after being saved in acrobat (15 MB for a 1 page PDF, open in Acrobat, save the file again, down to 150KB, with no visible change in appearance or quality).

ALTER TABLE tablename
ALTER COLUMUN
   columnname varchar(50) NOT NULL
GO


ALTER TABLE tablename
ADD CONSTRAINT constraintname UNIQUE( columnname ) 

GO

You just screwed up the database, and now you need to make some surgical repairs to the database.

Or maybe you have data in development and production tables and you want to move some data changes from one to the other.

Basically the problem is that you have 2 tables with the same schema where 1 column has some different values in one table when compared with the other.

You can't do an update on an inner join statement, but you can create an updatable view containing the fields you need and then run an update statement on it.

A view is updatable if it meets the following conditions:

A view is updatable if:

• select_statement has no aggregate functions in the select list and does not contain the TOP, GROUP BY, UNION, or DISTINCT clauses. Aggregate functions can be used in a in a subquery in the FROM clause as long as the values returned by the functions are not modified. For more information, see Aggregate Functions.
• select_statement has no derived columns in the select list. Derived columns are result set columns formed by anything other than a simple column expression, such as using functions or addition or subtraction operators.
• The FROM clause in the select_statement references at least one table. select_statement must have more than nontabular expressions, which are expressions that are not derived from a table. For example, this view is not updatable:

CREATE VIEW NoTable AS 
 SELECT GETDATE() AS CurrentDate, 
 @@LANGUAGE AS CurrentLanguage, 
 CURRENT_USER AS CurrentUser 

SQL Servers' GETDATE() function will return the current DateTime.  But what if you just want the current date.

CAST(FLOOR(CAST(GETDATE() AS float)) AS datetime)

or

CAST(CONVERT(VARCHAR(10), GETDATE(), 111) AS DATETIME)

See this page for even more options.

Or, if you want to get just the current time out of GETDATE or any datetime for that matter, check out this page (it is one of the FAQs).

When you are trying to get the last Identity value in SQL Server, the old way was to use the @@IDENTITY variable.

This, however, can easily give you the wrong answer back.  For example, if you have a trigger inserting a history record that also contains an identity column.

Here is an article that discusses why you should probably stick to SCOPE_IDENTITY().

When you try to assign the web reportviewer control to a height of 100%, it displays as a very short (height wise) format.  For me, adding the style attributes to the form did the trick.

The solution to this problem was found on this page: http://msdn2.microsoft.com/en-us/library/ms252090.aspx.

Considerations for XHTML and ReportViewer Web Server Control

I know Access is old news, but I have recently been asked to take a look at an old Access app.

This application replaces the existing menu at the top with a custom one, which really limits what you can do.

The solution to getting the menu back is to right click in the menu area, select customize, highlight "Menu Bar" and select properties, and change "Allow hide/show..." checkbox.  After you do this, you can then add it back into your form.

I frequently get asked how you can update a join statement in SQL.  Of course, the UPDATE syntax doesn't allow for you do actually perform an update directly against a JOIN statement, but there are some things you can do to work around this problem.

Probably the best way is to do the update like the example below:

UPDATE Job_Opportunities
SET AwardedId = 1
WHERE JobOpportunityId = (
    SELECT JO.JobOpportunityId 
    FROM Job_Opportunities JO
    INNER JOIN
    Job_results JobResults
    ON BO.JobOpportunityID = BR.Job_id 
    AND JR.is_us = 1 
    AND JR.rank = 1 
    AND JO.JobOpportunityId = Job_Opportunities.JobOpportunityId
) 

When dealing with money, or numbers, sql server provides a few different datatypes: Decimal, Money, Float among others.

The main difference is that Decimal and Money datatypes are "exact", as opposed to Float which is a "Floating Point Representation".

When dealing with Floats, you can get some odd mathematical results:
http://www.chrismay.org/2005/09/29/BinaryRepresentationOfDecimalValues.aspx

e.g.
0.4 - 0.1 = 0.30000000000000000000004
1.4-1.1 = 0.29999999999999999999998

So when a number is going to represent money, you should use the Decimal or Money datatypes.  Money is (19,4) which means 19 total digits, 4 of which are right of the decimal point.  Decimal can be setup however you want, like Decimal(10,2).

SmallMoney can also be used, which can represent money values under 1,000,000 w/ 4 decimal places.

One of the challenges in doing merge replication in SQL Server is dealing with IDENTITY columns.

One of the tricks you can use is "automatic identity range management". 

Basically you tell DB 1 to start at seed 1 and DB 2 to seed at 20,000, and when DB 1 reaches 16,000 or DB 2 reaches 36,000 it will reset those seeds to 40,000 and 60,000.

http://msdn2.microsoft.com/en-us/library/ms152543.aspx

http://msdn2.microsoft.com/en-us/library/ms146907.aspx

It appears to work well.

The problem is that if you restore a sql server database on a different computer than the backup was created, you can have the problem that the usernames and permissions are somewhat lost.

Each username has an associated SID.  So even though you might have a user named 'Bill' and the restore database might have a user named 'Bill', they will have different SIDs and so the permissions will not be applied to the correct user.

To fix this problem you can follow the following steps:

1) run this TSQL:

sp_change_users_login 'report'

2) This will produce a table showing you Users that having issues.

3) run the following TSQL:

sp_change_users_login 'update_one', 'UserNameFromStep2Table','NameOfUserToUse'

In many cases the 2nd and 3rd parameters will be the same string.

More about this SP can be found here: http://doc.ddart.net/mssql/sql70/sp_ca-cz_4.htm

Many times you can have a database that has access to it restricted to SPs.  If the database is application specific, then there is a good chance that 1 user account will need to access all these SPs to run the application.

My friend Phil found and updated this script to run in SQL Server 2005 to generate the necessary code to grant execute rights to all SPs in a database for a given user.

SELECT 'GRANT EXECUTE ON ' + sysobjects.name + ' TO AccountName' + 

CHAR(10) + CHAR(13) + CHAR(10) + CHAR(13) 

FROM sysobjects 

WHERE type = 'P' and category = 0 AND name not like 'sp_%'

There are many times when I just want to run a query and send the results to someone in an excel spreadsheet.

You can easliy copy the and past the results, but you don't get column headers, and you lose formatting in many cases.

Dinakar Nethi has a good article about how to directly import the results of a query into excel.

I just used his method and it worked great.

Ok so, on a recent project, for some reason, I set up a few fields, and the corrosponding SP parameters as "decimal" datatypes. What a mistake.

After lots of rounding errors, which were partly ignored because the revised "specs" called for these figures to be integers (later changed to decimals), I found out that a Decimal datatype has no decimal places, unless you specifically declare it.

The reason for this is that a "decimal" is really just another name for a "numeric". So I guess I can understand that part... but how dumb is it that a decimal, by default, has no decimal places.

I went back and changed everything to float.

Use decimal(x,x) instead of float.  See here for the reason why.