Help with diagnosing source of error
AaronBertrand
Posts: 62 New member
I realize there is some kind of hardware issue at play here. But I am trying to determine whether the I/O issue is:
(a) on the source server
(b) on the destination server
(c) on the server where red gate backup is running
(d) some combination of the above
Any help with diagnosing this would be greatly appreciated. Backups that go directly to a drive on the local machine (the source server where the databases are active) works fine, as does an after-the fact copy from source to destination (also initiated by a script on the third server, where RG is running).
SQL Backup v5.4.0.55
Backing up test_db (full database) to:
\\192.168.5.208\share\test_db_0.sqb
\\192.168.5.208\share\test_db_1.sqb
\\192.168.5.208\share\test_db_2.sqb
\\192.168.5.208\share\test_db_3.sqb
\\192.168.5.208\share\test_db_4.sqb
Thread 0 error:
Process terminated unexpectedly. Error code: -2139684860 Process terminated unexpectedly. Error code: -2139684860
Thread 1 error:
Error 620: Error writing to backup file(s).
Process terminated unexpectedly. Error code: -2139684860 Warning 210: Thread 1 warning:
Warning 210: Error writing to backup file:
\\192.168.5.208\share\test_db_1.sqb
Warning: System error 1450 (Insufficient system resources exist to complete the requested service)
(...three more of these...)
SQL error 3013: SQL error 3013: BACKUP DATABASE is terminating abnormally.
SQL error 3271: SQL error 3271: A nonrecoverable I/O error occurred on file "SQLBACKUP_E6B29B3F-F428-4475-B167-53AC0789D8DD:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).
SQL error 3202: SQL error 3202: Write on "SQLBACKUP_E6B29B3F-F428-4475-B167-53AC0789D8DD01" failed: 1117(The request could not be performed because of an I/O device error.)
SQL Backup exit code: 790
SQL error code: 3202
name value
exitcode 790
sqlerrorcode 3202
(a) on the source server
(b) on the destination server
(c) on the server where red gate backup is running
(d) some combination of the above
Any help with diagnosing this would be greatly appreciated. Backups that go directly to a drive on the local machine (the source server where the databases are active) works fine, as does an after-the fact copy from source to destination (also initiated by a script on the third server, where RG is running).
SQL Backup v5.4.0.55
Backing up test_db (full database) to:
\\192.168.5.208\share\test_db_0.sqb
\\192.168.5.208\share\test_db_1.sqb
\\192.168.5.208\share\test_db_2.sqb
\\192.168.5.208\share\test_db_3.sqb
\\192.168.5.208\share\test_db_4.sqb
Thread 0 error:
Process terminated unexpectedly. Error code: -2139684860 Process terminated unexpectedly. Error code: -2139684860
Thread 1 error:
Error 620: Error writing to backup file(s).
Process terminated unexpectedly. Error code: -2139684860 Warning 210: Thread 1 warning:
Warning 210: Error writing to backup file:
\\192.168.5.208\share\test_db_1.sqb
Warning: System error 1450 (Insufficient system resources exist to complete the requested service)
(...three more of these...)
SQL error 3013: SQL error 3013: BACKUP DATABASE is terminating abnormally.
SQL error 3271: SQL error 3271: A nonrecoverable I/O error occurred on file "SQLBACKUP_E6B29B3F-F428-4475-B167-53AC0789D8DD:" 995(The I/O operation has been aborted because of either a thread exit or an application request.).
SQL error 3202: SQL error 3202: Write on "SQLBACKUP_E6B29B3F-F428-4475-B167-53AC0789D8DD01" failed: 1117(The request could not be performed because of an I/O device error.)
SQL Backup exit code: 790
SQL error code: 3202
name value
exitcode 790
sqlerrorcode 3202
Comments
You at least are not getting the 1010 error where SQLBackup is using the MemToLeave memory and needs this to be contiguous. SQLBackup is pretty resource hungry and if you use the GUI to run the jobs rather than schedule them you can exhaust the resources.
I have had this problem on occasion and just had to make sure I was not running too many SQLBackups jobs at the same time and I think that the system drive was low on space. I believe that it is this last point that is causing your problem.
Chris
Plenty of disk space. And like I said, the backups work fine if the 3rd server tells the source, back up to your own local Y: drive (SAN). It fails when the 3rd server tells the source, back up to that other server's Y: drive (same SAN underneath).
As an experiment, I ran the same commands from the source server and experienced the same issue. So the 3rd server is not part of the equation, it would seem.
The backup hangs on a particular file with errors showing up in the log:
BackupMedium::ReportIoError: write failure on backup device 'SQLBACKUP_5FE0E104-6CA8-4FE8-99AC-DF5901C107C605'. Operating system error 995(The I/O operation has been aborted because of either a thread exit or an application request.).
The backup location is a SAN backup storage device and I am running from a SQLAgent job as well.
Oddly I am not seeing this error on other DB's on the same server to the same backup device.
Does the error message shed any light on the source of the problem?
--Dan