DTC Error and unresolved SQL transaction

I have a procedure that reads data from linked server, a SQL2005 box, and
writes a row in a SQL2000 database. This procedure and configuration have
been working successfully for several years. This Sunday at 4am, this
procedure failed to complete, leaving an unresolved transaction. The symptom
is an insert on this table will timeout and fail because the unresolved
transaction has a lock on the table, and it shows as a blocking transaction.
Otherwise the database is functional and responsive. I have tried to KILL the
unresponsive process, but it wont clear, and just reads "KILLED/ROLLED BACK"
under the Activity Monitor command column.
I had this same problem last weekend, and re-starting the SQL Server Service
resolved the transaction. However, this is not a viable option during
production hours.
I tried using "KILL 51 WITH STATUSONLY", and it returned:
SPID 51: transaction rollback in progress. Estimated rollback completion:
100%. Estimated time remaining: 0 seconds.
I tried using KILL "51457D54-4FD7-408A-B5CA-AFF33D601D00"
It it cam back with:
Server: Msg 6114, Level 16, State 1, Line 1
Distributed transaction with UOW {51457D54-4FD7-408A-B5CA-AFF33D601D00} is
being used by another user. KILL command failed.
My two questions are:
1) Is there anyway to clear this blocking transaction short of re-starting
the SQL Server?
2) Is there anyway to figure out the root cause of the problem? I believe it
is some sort of MSDTC issue, that seems to only happen early on Sunday
mornings.
The following DTC error happened at exactly the same timestamp as the SQL
procedure was executed.
___________________________________________
Application Event Log Error
___________________________________________
Date7/1/2007 4:09:31 AM
LogWindows NT (Application)
SourceMSDTC
Category(3)
Event3221229829
ComputerSERVER002
Message
The description for Event ID '-1073737467' in Source 'MSDTC' cannot be
found. The local computer may not have the necessary registry information or
message DLL files to display the message, or you may not have permission to
access them. The following information is part of the
event:'.\iomgrclt.cpp:204, Pid: 1300, CmdLine: C:\WINNT\System32\msdtc.exe'
___________________________________________
Thank you in advance,
Ken
Hi Ken
"KenL" wrote:

> I have a procedure that reads data from linked server, a SQL2005 box, and
> writes a row in a SQL2000 database. This procedure and configuration have
> been working successfully for several years. This Sunday at 4am, this
> procedure failed to complete, leaving an unresolved transaction. The symptom
> is an insert on this table will timeout and fail because the unresolved
> transaction has a lock on the table, and it shows as a blocking transaction.
> Otherwise the database is functional and responsive. I have tried to KILL the
> unresponsive process, but it wont clear, and just reads "KILLED/ROLLED BACK"
> under the Activity Monitor command column.
> I had this same problem last weekend, and re-starting the SQL Server Service
> resolved the transaction. However, this is not a viable option during
> production hours.
> I tried using "KILL 51 WITH STATUSONLY", and it returned:
> SPID 51: transaction rollback in progress. Estimated rollback completion:
> 100%. Estimated time remaining: 0 seconds.
> I tried using KILL "51457D54-4FD7-408A-B5CA-AFF33D601D00"
> It it cam back with:
> Server: Msg 6114, Level 16, State 1, Line 1
> Distributed transaction with UOW {51457D54-4FD7-408A-B5CA-AFF33D601D00} is
> being used by another user. KILL command failed.
> My two questions are:
> 1) Is there anyway to clear this blocking transaction short of re-starting
> the SQL Server?
> 2) Is there anyway to figure out the root cause of the problem? I believe it
> is some sort of MSDTC issue, that seems to only happen early on Sunday
> mornings.
> The following DTC error happened at exactly the same timestamp as the SQL
> procedure was executed.
> ___________________________________________
> Application Event Log Error
> ___________________________________________
> Date7/1/2007 4:09:31 AM
> LogWindows NT (Application)
> SourceMSDTC
> Category(3)
> Event3221229829
> ComputerSERVER002
> Message
> The description for Event ID '-1073737467' in Source 'MSDTC' cannot be
> found. The local computer may not have the necessary registry information or
> message DLL files to display the message, or you may not have permission to
> access them. The following information is part of the
> event:'.\iomgrclt.cpp:204, Pid: 1300, CmdLine: C:\WINNT\System32\msdtc.exe'
> ___________________________________________
> Thank you in advance,
> Ken
I am not a MSDTC expert!!! Which process did you kill? I would expect a
process on the remote and local (originator) machines, and if there was an
order to be killed then local would be the first. If you stopped the DTC
services (NET STOP MSDTC) it should also rollback, but all distributed
transactions would be affected.
Is this the only time distributed transaction are used? If not then it would
narrow the issue down to either something with the process or something that
happens at that time. If the process is scheduled and works at other times
then it would rule the process out. If it is something that happens at a
specific time, check things like firewalls or antivirus updates/scans etc.
http://support.microsoft.com/default.aspx/kb/306843
Also look for blocking occuring during the process and how you handle errors
such as deadlocks in the code.
You could use DTCTester http://support.microsoft.com/kb/293799 or DTCPing to
check that DTC works ok.
John
|||Thank you for the response John.
<Is this the only time distributed transaction are used?
No, there are many procedures on this server that link to databases on
another server. The stored procedure that is failing runs hundreds of times
in a day. It had been reliable for years, up until last Sunday and this
Sunday when I have seen the two failures
<Which process did you kill?
I killed the spid on the SQL server initiating the link
I will review the kb's you referenced
Thanks,
Ken
"John Bell" wrote:

> Hi Ken
> "KenL" wrote:
>
> I am not a MSDTC expert!!! Which process did you kill? I would expect a
> process on the remote and local (originator) machines, and if there was an
> order to be killed then local would be the first. If you stopped the DTC
> services (NET STOP MSDTC) it should also rollback, but all distributed
> transactions would be affected.
> Is this the only time distributed transaction are used? If not then it would
> narrow the issue down to either something with the process or something that
> happens at that time. If the process is scheduled and works at other times
> then it would rule the process out. If it is something that happens at a
> specific time, check things like firewalls or antivirus updates/scans etc.
> http://support.microsoft.com/default.aspx/kb/306843
> Also look for blocking occuring during the process and how you handle errors
> such as deadlocks in the code.
> You could use DTCTester http://support.microsoft.com/kb/293799 or DTCPing to
> check that DTC works ok.
> John