Options
Memory Issues with large syncs
MarkLFT
Posts: 15
I am using the SDK in a Windows Service, to, based on a timer, sync two databases. The service runs fine but seems to suffer a memory leak. I have tracked this down to occasionally there is a large amount of data to sync. Infact the service memory in Windows grows from 50K to 1.2M. These rises are when the SDK is obtaining the difference.
Usually this memory is released after the sync, but as I said if it is a big sync, the memory is not full released.
I nothice in the sample code it mentions using GetFileStream() to avoid memory issues, but I am not sure how to use the resulting stream, as there is no execute method that takes a stream. Could someone give me an example of how to use the Stream option.
Many thanks
Usually this memory is released after the sync, but as I said if it is a big sync, the memory is not full released.
I nothice in the sample code it mentions using GetFileStream() to avoid memory issues, but I am not sure how to use the resulting stream, as there is no execute method that takes a stream. Could someone give me an example of how to use the Stream option.
Many thanks
Comments
I don't think there is a way to make this synchronization any more efficient, and if the memory was not returned after use, that would be a memory leak. So there doesn't appear to be a memory leak because the memory is being freed.
Also, you have to be careful to look at the correct metric to determine the memory usage because the .net Framework will not always release memory back to the OS when it is done with it. Most times it keep the memory but marks it as free for use.
The GetFileStream method increases the efficiency of writing the synchronization script to disk only, and won't help you with the synchronization. For instance, if you wantes to write the script to disk without using GetFileStream, you would first have to turn the execution block to a string which will be completely in memory, and then writing that string to disk. Using a FileStream, you can read small bits (512 bytes) and write each buffer to disk instead.
If you have any more questions, please let us know.
If I run the service, but disable the SDK methods that perform the sync, the service always uses exactly the same amount of memory, and runs for extended periods of time with no problems.
However, if I leave the SDK methods in place, the service starts using about 3% of available system memory, within 24 hours that is up to 50%, with two to three days the system crashes as out of memory.
I would say that is the definition of a memory leak irrespective of which metric I use.
Thanks