Link Static Data - Takes 5 Minutes to Compare Databases
JackAce
Posts: 75 New member
We currently have 17 tables with about 115,000 rows statically linked via SQL Source Control. This is just a small fraction of the total list of tables that we would like to link. When clicking on the "Commit Changes" tab for the SSMS, it takes about 5 minutes before the change list shows up.
This is currently tolerable under optimal conditions. Under sub-optimal conditions (e.g. when you connect remotely via VPN), I have timeout issues.
Looking forward, I am not sure that we will be able to link all of the lookup tables that we would like to ultimately manage. We have dozens of tables and I'm afraid that the time it takes to view the change list would take over an hour, which would not be acceptable.
Are there ways to minimize the time that it takes to determine the change list when you are versioning the data in many large tables? We use these linked tables to generate the update scripts (via SQL Data Compare) from the Dev to QA to Staging to Production environments.
Some possible things that I was thinking of:
1) Link the tables and generate the scripts and then copy the scripts to a separate folder and then unlink the table.
Obviously, this is a clunky solution because you have to repeat the process every time you insert/update/delete a row in the table.
2) Don't link the tables at all. Just use the SQL Data Compare command line executable and hand pick the tables that you want to deploy via the /Include switch. I am afraid of doing this because it would be very easy to have test data accidentally make it to production, since developers aren't manually committing changes to source control.
3) In SSMS, you could create two Registered Database connections to the same database (using different logins, for example). One connection could just link the schema to one folder in source control (e.g. http://mysvnserver/myDbRepository/trunk/SchemaScripts). The other connection could link the Data to source control (http://mysvnserver/myDbRepository/trunk/DataScripts) where you filter out all objects except the tables involved (with no indexes, no foreign key constraints, no defaults) and just link the data. This seems like a viable solution, but a little clunky and awkward to say the least. This would also mean that you have the schema of the tables checked into multiple places.
Is there a more elegant solution for versioning monstrous tables?
This is currently tolerable under optimal conditions. Under sub-optimal conditions (e.g. when you connect remotely via VPN), I have timeout issues.
Looking forward, I am not sure that we will be able to link all of the lookup tables that we would like to ultimately manage. We have dozens of tables and I'm afraid that the time it takes to view the change list would take over an hour, which would not be acceptable.
Are there ways to minimize the time that it takes to determine the change list when you are versioning the data in many large tables? We use these linked tables to generate the update scripts (via SQL Data Compare) from the Dev to QA to Staging to Production environments.
Some possible things that I was thinking of:
1) Link the tables and generate the scripts and then copy the scripts to a separate folder and then unlink the table.
Obviously, this is a clunky solution because you have to repeat the process every time you insert/update/delete a row in the table.
2) Don't link the tables at all. Just use the SQL Data Compare command line executable and hand pick the tables that you want to deploy via the /Include switch. I am afraid of doing this because it would be very easy to have test data accidentally make it to production, since developers aren't manually committing changes to source control.
3) In SSMS, you could create two Registered Database connections to the same database (using different logins, for example). One connection could just link the schema to one folder in source control (e.g. http://mysvnserver/myDbRepository/trunk/SchemaScripts). The other connection could link the Data to source control (http://mysvnserver/myDbRepository/trunk/DataScripts) where you filter out all objects except the tables involved (with no indexes, no foreign key constraints, no defaults) and just link the data. This seems like a viable solution, but a little clunky and awkward to say the least. This would also mean that you have the schema of the tables checked into multiple places.
Is there a more elegant solution for versioning monstrous tables?
Comments
We have one database with 304 static data linked tables. The Calculating changes spinner has been going for two hours now... The get latest might take all day for our other programmers.
So have you been doing anything besides just waiting the X hours for the change list to show up? I was hoping that others have figured out a workaround.
I'm even getting the "timeout expired" with another database that has very few linked tables. It's a small database and it's brand new, so there are just a few dozen tables and stored procedures. I'm still trying to figure out why.
I stopped the last attempt to commit of 304 tables around noon our time today. I started it back at 12:45PM CST/CDT. It gets to the calculating changes spinner in about two minutes and is spinning... I plan on letting this and nothing else run until.
I noticed that the Calculating changes is not spinning at this time and neither is the yy on the databases.
I believe ssms is stalled, locked up or otherwise waiting for something from rrssc.
I plan on stopping it soon if I don't have success in tracing or troubleshooting the issue.
I am going to attempt to add static linked tables in groups and check in. This may work around the check in issue, but probably not the get latest issue.
This is a wall, not a speed bump.
A problem caused this program to stop interacting with Windows.
Problem signature:
Problem Event Name: AppHangB1
Application Name: Ssms.exe
Application Version: 2009.100.1600.1
Application Timestamp: 4bb679e7
Hang Signature: 5d4e
Hang Type: 6400
OS Version: 6.1.7600.2.0.0.256.1
Locale ID: 1033
Additional Hang Signature 1: 5d4e89e34b78d1eaec09604964415018
Additional Hang Signature 2: 7339
Additional Hang Signature 3: 7339e7c669c193a7ad276d15bf9f2609
Additional Hang Signature 4: 5d4e
Additional Hang Signature 5: 5d4e89e34b78d1eaec09604964415018
Additional Hang Signature 6: 7339
Additional Hang Signature 7: 7339e7c669c193a7ad276d15bf9f2609
Read our privacy statement online:
http://go.microsoft.com/fwlink/?linkid= ... cid=0x0409
If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt
We need to log a support call for you regarding your issue.
Can you please send an e-mail to support@red-gate.com with your contact details and include a reference to this forum post?
We will then be able to create a support call for you that has the correct contact information. A member of the Product Support Team will then be able contact you and investigate your problem.
Many Thanks
Eddie
Senior Product Support Engineer
Redgate Software Ltd
Email: support@red-gate.com