7.3.0.383 Issue: Refreshing Connection for 20m & counting

PDinCA

Just upgraded the SQL Cluster's 7.2.x to 7.3 and am STILL waiting for the only connection defined to the UI to refresh, while actually RDP'd to the primary node.

The server components were installed first, then the UI started, which gave me the updates-ready notification, so downloaded, installed and started the updated UI. Said YES to the Cache Compress that appears compulsory.

I can see jobs, in progress, but the green-spinner just keeps on spinning...

UI is utterly unusable for any Activity Monitoring.

Failing the cluster over/rebooting is not an option.

Actions:

1. Open the UI. 2. Click on the Server. 3. Wait forever.

I closed the UI and restarted it. Same issue.

D.O.A. can we say?

Windows Server 2008 R2 Enterprise SP1 Build 7601.
SQL Server 2012 Enterprise CU5.

===== TWO UPDATES=====

1. UI also fails for the same server from my laptop - Windows-8 Pro.

2. Trust you received the {sa} DUMP generated by the utter failure of the UI to refresh the Activity tab for the connection?

James B

Hi Stephen,

We've had one or two reports of this in the latest version and I'm not sure the underlying cause has been identified so far. In short, it seems like the local data store is encountering trouble after the cache compress.

You may find it springs into life after some time - how long have you left it?

If it's still not working after being left alone for ages, the quickest way to get things working again is to let it create a fresh file. It'll import history from SQL Server, but you'll lose the more detailed history specific to SQL Backup (such as compression rates etc.)

To do this, stop the SQL Backup service, and then locate the data.sdf file. On a cluster, this is usually installed to a shared folder between the nodes somewhere, and on a single server would be in c:\programdata\red gate\sql backup\data\<instance name>. Rename / move the file elsewhere, and then start the service again. A new file should get created, and things should come back to life.

PDinCA

I left it over 20 minutes before posting a bug report. When I returned to the server, a {sa} crash had been trapped and sent to Red Gate.

I don't care about the detailed stats as I'll have a fresh set after the cache is rebuilt and the daily Full backups have run. Thanks for the workaround.

MINOR POTENTIAL PROBLEM: one of the SQL Backup instances is on a cluster and the cluster manage is aware of the service - do NOT want to accidentally fail the beast over Any advice?

Can we say NO to the Cache Compress? I have FIVE more machines to upgrade and don't want to have to fudge around on all ten machines, so far...

James B

We're currently investigating this; as soon as I hear more I'll post back. I'm not sure there's a way to skip the compress option (we actually are also adding a couple of extra fields to the data store too, so even trying to trick it by keeping a copy and putting it back afterwards will most likely not work...)

RBA

PDinCA wrote:

Can we say NO to the Cache Compress? I have FIVE more machines to upgrade and don't want to have to fudge around on all ten machines, so far...

Hi,

You can delete the server.dat and {number}.dat files from:
C:\Users\USER.NAME\AppData\Local\Red Gate\SQL Backup\Server Data (on the workstation, not server) to bypass the cache compact step.

The path mentioned in a previous post is on the machine hosting SQL Server and performance will dramatically improve if you purge it, however I would clear the local cache files at the same time for good measure.

I recall you've had problems in the past with the UI - if you note the size of your .sdf and .dat files before purging them, I'll make sure our test files are at least this large if they're not already.

Thanks,

PDinCA

One laptop I'm having issues with is a brand new Dell M6700 running Windows 8 Pro.

The 1.dat is only 12,669 KB.

I don't have a server.dat file.

I have a localDataCache.dat file of 31KB, though.

Do I delete the latter?

On the Server, the 1.dat is 25,548 KB, 2.dat is 30 KB and localDataCache.dat is 30 KB.

Please advise.

James B

The server.dat is located under the "roaming" part of your user-profile rather than local.

I'd start by just removing the numbered .dat file and the localDataCache.dat. If you delete the servers.dat you'll need to re-add your servers into the console.

PDinCA

Thanks, James. Servers.dat is only 3KB, to complete the file size portfolio.

James B

Thanks- that'll be handy for Robin's test cases.

PDinCA

I deleted all the recommended files.

Activity History is at 10 minutes and counting...

I can see the Jobs, no problem.

Next steps, please, as this version is definitely broken...

James B

Is this in relation to the server where you already removed the data.sdf file? If so, I'm not sure what else to try; it may just be slow because it's rebuilding the cache file but if it's sticking once again there's something more underlying going on and we'll need advice from the dev-team on it (you're not the only person seeing this problem, a couple of other users have reported similar)

PDinCA

Hi James,

I was waiting for advice on the data.sdf file on the SQL Cluster due to the configuration of the SQL Backup Service with respect to the cluster manager.

Apologies for misleading you all, I didn't delete THAT file, just the client files.

PDinCA

UPDATE:

I didn't delete the data.sdf files on the stand-alone Dev and QA Servers and the Activity History just came up fine, immediately after adding the servers to the new laptop.

Only the CLUSTER Activity History is now a problem.

James B

Ah, okay- so, if that file is quite large it'll contain a lot of history which will take some time to repopulate.

If it turns out that one does need removing then I guess the issue is indeed that stopping the service may fail the cluster over as the backup is seen as a cluster resource... on the installation notes here it does seem that we recommend a policy to restart it on the *current* node though, rather than failing over, so you may be OK. The other thing is whether you can temporarily remove / ignore the SQL Backup service from the clustering side of things. I'd test this out myself, but unfortunately I don't have a cluster here to try it on

PDinCA

Cluster's data.sdf is an enormous 148KB ( )

PDinCA

After TWO HOURS the cluster's Activity History is finally back!

It also incurred a SQL "Long Running Query" Alert from SQL Monitor - the query ran for 8718.0400001 seconds!!!

PDinCA

It also turns out that the server has Activity dating back to 9 Oct 2012. The sqbdata process is responsible for the long running query, hence the 6 full months of activity may be the underlying issue, not the client or server dat or sdf files...

Maintenance plan for History Cleanup is now configured, executed, and we're down to 30-days.

James B

I'd already left for the night when you posted that last message- unlikely it's going to be busy with lots of data then...
Did you manage to remove the file in the end?

PDinCA

No need to remove the file after I whacked the extraneous backup history. All clients now refresh the activity history just fine.

I deployed the server bits to three more production servers that have only 30-day history retention and had no issues at all. Their backup schedules are very similar to the 6-month-retention machine, BTW.

The fact that SQL Monitor kicked out a Long Running Query Alert for sqbdata as the underlying process tells me that the issue lies in THAT process and that the volume of data gave it a two hour twenty five minute headache!

mglenn

Has Red-Gate looked into performance tuning sqbdata? Activity History still takes forever to populate--GT 5 minutes for us on a server that contains 1 year of history.

Customers should not be forced to delete important backup history as a workaround for a poorly performing SQL Backup process.