Process Terminated Unexpectedly

tlbroadbenttlbroadbent Posts: 25 Bronze 3
edited January 17, 2006 11:29PM in SQL Backup Previous Versions
The SQL Backup XP fails with the following messages. Executing from the GUI results in the same error.

SQL Backup 3.2.0.10, (c) Red Gate Software Ltd 2004 - 2005
Serial number: 010-015-027314-030A
Backing up DatabaseName (transaction log) to \\BackupServer\SQLBACKUP\ServerName\DatabaseName_tlog_20060116180634.sqb ...
Thread 0 error:
Process terminated unexpectedly. Error code: -2139684860
Msg 3013, Level 16, State 1, Server ServerName, Line 1
BACKUP LOG is terminating abnormally.

Here is the backup command that generated the above error. As you can see, we backup to a network drive. Backing up to a local drive also gets the same error.

-SQL "BACKUP Log DatabaseName
TO DISK = [\\BackupServer\SQLBACKUP\ServerName\DatabaseName_tlog_20060116180634.sqb]
WITH NAME = [ServerName.DatabaseName tlog Backup 2006-01-16 18:06:34],
DESCRIPTION = [ServerName.DatabaseName tlog Backup on 2006-01-16 18:06:34],
PASSWORD = [password], COMPRESSION = 1"

I thought we had overcome this problem by installing version 3.2.0.10. However, the error now seems to occur more frequently on this particular server.

Usually, the backups will work for a few days if we restart the SQL Service. However, we restarted the service on this server today and T-Log backups started failing about 6 hours later. Restarting the service regularly is not a good option in our environment.

We have tried starting SQL Server with the –g option to increase the virtual address space. This did not fix the problem. This setting seemed to cause other problems on the server last week and we have returned to the default value.

The server is a Dell PE6850 with four hyper-threaded Xeon processors and four gigabytes of memory. The OS is Windows Server 2003. SQL Server version is 2000 Standard Edition with SP4 installed. As noted above, Red Gate SQLBackup version is 3.2.0.10. There is plenty of disk space available – 260GB on the backup drive.

SQL Backup on another server was failing earlier today but now runs successfully. These two servers are the most heavily used SQL servers. SQL Backup is installed on ten other servers. Three of them have experienced the same error. Seven have been running SQLBackup for weeks without failure. The servers that have not had failures don’t handle as much data or as many users as the servers that experience the failures.

We’d like some help resolving the problem before management tells us to use native backup. The compression and encryption features are too valuable for us to lightly discard them. By the way, we have tried backups without encryption and compression. SQL Backup still fails.

Thanks in advance for your assistance.

Terry Broadbent
Terry L Broadbent
Database Administrator
L-3 Communications, Inc.
· Share on Twitter

Comments

  • peteypetey Posts: 2,358 New member
    What is the error that's logged in SQL Server's vdi.log file?

    Does the error happen immediately after you start the process, or some time after the process has been running?
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    We perform multiple backups within the backup job. The first time the error occurs, it may happen on the first or any subsequent backup. After that it usually fails on the first backup attempted.

    Here are the VDI contents for the last error that occurrred.
    2006/01/16 18:46:30 pid(3208) tid(5496)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at ServerBufferAreaManager::Allocate: Map buffers Status Code: 8, x8
    Explanation: Not enough storage is available to process this command.
    2006/01/16 18:46:30 pid(3208) tid(5496)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    2006/01/16 18:46:30 pid(2860) tid(4624)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at CVD::GetCommand: AbortDetected
    2006/01/16 18:46:30 pid(3208) tid(3728)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    2006/01/16 18:46:30 pid(3208) tid(3728)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at SVDS::CloseDevice: Abort detected
    2006/01/16 18:46:30 pid(3208) tid(3728)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    2006/01/16 18:46:30 pid(2860) tid(4624)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    2006/01/16 18:46:30 pid(3208) tid(5496)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    2006/01/16 18:46:30 pid(2860) tid(4368)
    Error on Global\SQLBACKUP_{8123B2FB-D86A-46DE-801D-BF12C8A5DAE1}
    Error at TriggerAbort: invoked
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    It looks like there isn't enough contiguous memory to serve SQL Backup's needs. Can you pls download this utility (http://www.yohz.com/downloads/memstat.zip), and run it? Provide the process id of the SQL Server process to get a dump of the memory allocations.

    In the meantime, I'll try to dig out some docs from Microsoft relating to this.

    Thanks.
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    Here is the MEMSTAT output. I hope you will explain it as there is no documentation with the tool.
    memstat - Yohz Software (c) 2005 - 2006

Type             Maximum     Minimum     Average     Blk count   Total
---------------- ----------- ----------- ----------- ----------- -----------
Commit           4096        879820800   1714066     1048        1796341760
Reserve          8192        1032192     359424      152         54632448
Free             4096        169521152   1976292     150         296443904
Private          4096        879820800   2632996     683         1798336512
Mapped           4096        1060864     148650      48          7135232
Image            4096        7569408     97020       469         45502464
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    The headings for Minimum and Maximum are reversed.

    I was looking at the Free values. Maximum free indicates the largest contiguous free block of memory in the process space of SQL Server. It looks like there's enough contiguous memory for SQL Backup to run.

    Can SQL Backup back up your databases on this server at this moment?
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    Yes. SQLBackup is working now because I restarted the SQL Server service. Had to get the nightly backups done. I'll run memstat next time backups begin to fail and post the output.

    Thanks for your help.
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    >> I'll run memstat next time backups begin to fail and post the output.

    Yes please. The previous stats would be a good baseline to compare against.
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    SQLBackup has run without any problem, today. Memstat shows that the maximum free space is now 75,522,048. That is less than half of 169,521,152 that was free last night. How much free space is needed for SQLBackup? Is the server approaching the threshold?

    Thanks,
    Terry
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    The amount varies, but what's more important is the contiguous amount of memory. If you have a lot of free memory, but is heavily fragmented, it will affect SQL Server's ability to work with SQL Backup.

    What is the block count value for the Free memory? A large number would indicate heavily fragmentation.
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    The Free Blk count is 210.
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    Looks ok.

    Just for your information, SQL Backup 4 dumps the SQL Server memory stats to the log file whenever it encounters errors with the SQL Server backup interface, for troubleshooting purposes.
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
  • tlbroadbenttlbroadbent Posts: 25 Bronze 3
    When do you anticipate release of version 4?
    Terry L Broadbent
    Database Administrator
    L-3 Communications, Inc.
    · Share on Twitter
  • peteypetey Posts: 2,358 New member
    It's already out (http://www.red-gate.com/products/SQL_Backup/index.htm). You may need to manually refresh the page if you don't see it the first time.
    Peter Yeoh
    SQL Backup Consultant Developer
    Associate, Yohz Software
    Beyond compression - SQL Backup goodies under the hood, updated for version 8
    · Share on Twitter
Sign In or Register to comment.