Generating data for large databases

sbendayan

Greetings,

I am trying to generate a large data set and it is taking many hours in SQL Data Generator to generate a 20 million row table. Any suggestions to speed this up?

sbendayan

More specifically, it appears that I have 1 table that is taking much longer to generate than all the others, and I can't find the reason why. I see that on another table we are inserting thousands of rows every few seconds but on this table it's more like dozens of rows.

Both tables are inserting 20 million rows, have several FKs, and they have similar rowsizes (626 bytes for the faster table and 778 for the slower). Any ideas?

Eddie D

Hi, thank you for your forum post.

If you increase the minimum logging level to verbose, as per the instructions in this help document:
https://documentation.red-gate.com/display/SDG3/Logging+and+log+files

Are there any entries that explain the performance problem you are reporting?

Any entries in the SQL logs to indicate problem inserting data into the database?

Does the table in question have any in triggers that fire each time a row is added? Or dependencies to other data sources outside of the database?

Are there any growth issues as you insert the 20 million rows into this table?

Many Thanks
Eddie