Backup Copy to S3 Loses Access to Bucket

jcricegpjcricegp Posts: 18 Bronze 3
in SQL Backup
I am currently copying SQL Backups to S3 Storage using build 9.1.4.722. I have linked the Bucket using the interface and successfully tested the link. 

I have been experiencing an issue where the copy to S3 will get an 'Access Denied' error when attempting to copy the file. When the interface shows that the file copy error, I check the link in 'Service Options' and it shows that the Bucket is still linked. If I re-enter my Access, Secret Keys, and Bucket Name, the copy begins and is successful. This has been a problem for several of my Amazon EC2 instances. I have to check each instance to see if the error is occurring. Is there an alarm for this problem? 

I have upgraded this instance and several to 9.2.1.128 (Latest Build), in case the 9.1 build is the problem. 

I am including the log file of the error. 

Thanks, John R.

2018-04-10 05:20:12,326 [1] INFO  Program - SQL Backup upload client 9.1.4.722
2018-04-10 05:20:12,582 [1] INFO  Program - Copyright ©  Red Gate Software Ltd
2018-04-10 05:20:12,660 [1] INFO  Program - UseSignatureVersion4 was 'True'
2018-04-10 05:20:12,669 [1] INFO  Program - UseSignatureVersion4 is now 'True'
2018-04-10 05:20:12,707 [1] INFO  AccountDetails - AWS Keys are 'NOT' set'
2018-04-10 05:20:12,816 [1] INFO  S3UploadClientFactory - Setting Region to Default
2018-04-10 05:20:12,832 [1] INFO  S3UploadClientFactory - Defaulting EndPoint to 'US East (Virginia)'
2018-04-10 05:20:12,895 [1] INFO  ReadWriteRegistry - Instance name does not exist: (local)
2018-04-10 05:20:13,426 [1] ERROR S3UploadClientFactory - Failed to get Bucket '<S3 Bucket Name>' location 'Access Denied'
Amazon.S3.AmazonS3Exception: Access Denied ---> Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (403) Forbidden. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at System.Net.HttpWebRequest.GetResponse()
   at Amazon.Runtime.Internal.HttpRequest.GetResponse()
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpRequest.GetResponse()
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeSync(IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleException(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext executionContext)
   at Amazon.Runtime.AmazonServiceClient.Invoke[TRequest,TResponse](TRequest request, IMarshaller`2 marshaller, ResponseUnmarshaller unmarshaller)
   at Amazon.S3.AmazonS3Client.GetBucketLocation(GetBucketLocationRequest request)
   at RedGate.SQLBackup.CloudBackup.App.S3.S3UploadClientFactory.GetS3RegionForBucket(String accessKeyId, String secretAccessKeyId, String bucket, RegionEndpoint defaultEndpoint)
2018-04-10 05:20:13,504 [1] ERROR Program - ExitCode  AuthorizationError
RedGate.SQLBackup.CloudBackup.App.Utils.ExitCodeException: Amazon Authorization Error: Access Denied
   at RedGate.SQLBackup.CloudBackup.App.S3.S3UploadClientFactory.GetS3RegionForBucket(String accessKeyId, String secretAccessKeyId, String bucket, RegionEndpoint defaultEndpoint)
   at RedGate.SQLBackup.CloudBackup.App.S3.S3UploadClientFactory.GetRegionForBucket(String region, String accessKeyId, String secretAccessKeyId, String bucket)
   at RedGate.SQLBackup.CloudBackup.App.S3.S3UploadClientFactory.CreateUploadClient(String instanceName, String s3Key, String s3SecretKey, String region, String bucket)
   at RedGate.SQLBackup.CloudBackup.App.Commands.Upload.FileUploader.UploadFile(IUploadClientFactory s3ClientFactory, IProgressEventSinkFactory progressSinkFactory, IFileFactory fileFactory, IAccountDetails accountDetails, ICommandLineArguments args)
   at RedGate.SQLBackup.CloudBackup.App.Program.<>c__DisplayClass6_0.<UploadFile>b__0()
   at RedGate.SQLBackup.CloudBackup.App.Utils.RetryLogic.Retry(Action action, Func`2 exceptionFilter, Int32 maxTries, BackOffAlgorithm backOffAlgorithm)
   at RedGate.SQLBackup.CloudBackup.App.Program.UploadFile(CommandLineArguments commandLineArguments)
   at RedGate.SQLBackup.CloudBackup.App.Program.Main(String[] args)


Tagged:
· Share on Twitter

Answers

  • Eddie DEddie D Posts: 1,807 Rose Gold 5
    Hi

    I suspect that we may already have an open support ticket with you on this problem.

    We are are unable to replicate your problem.

    We use components are from the Amazon S3 SDK to connect to the S3 server.

    The AWS keys are stored in the CredentialManager on the relevant db server (under whichever user the SQB server components uses), under key "Redgate.SqlBackup.S3.{InstanceName}.{BucketName}". That may not be helpful if you are running the server components as Local System, but if you are using a loggable-in account then you could check them. 

    You can view the Credentials Manager via Control Panel ->User Accounts ->Credentials Manager

    Credentials in the Manager don't expire, so is it possible some other process is clearing them out occasionally?

    Many Thanks
    Eddie
    Eddie Davis
    Senior Product Support Engineer
    Redgate Software Ltd
    Email: support@red-gate.com
    · Share on Twitter
Sign In or Register to comment.