SQL HyperBac - High memory usage

pjackman

We installed the recently released HyperBac Components v5.4.0.27 on Thursday, Aug 4th around 7:30 PM Pacific. Since then we are observing the memory footprint of HyperBacSrv.exe increasing roughly 12 K per minute. Over the last six hours it increased from 470,836 K to 474,976 K. Is anyone else experiencing this issue?

We posted our observations to RedGate support two hours after the install at 9:40 PM Pacific and several times since then and have not received a response.

Patrick Jackman
Vancouver, BC

pjackman

Roughly 25 hours since the last check and HyperBacSrv's memory footprint has increased from 474,976 K to 494,892 K.

Patrick Jackman

javen

This is normal and not a memory leak, the HyperBac service allocates virtual memory as available, with emphasis on virtual and available. The memory is virtual so it does not consume valuable physical memory, moreover memory will be released to any process which requests or requires this. The virtual buffers are allocated automatically to improve performance and reliability for all operations. This number and growth is quite normal, it should not go much higher than this however.

pjackman

Can you give an idea of where the memory growth should be reasonably expected to level off? Since my last report HyperBacSrv.exe claim has grown from 494,892 K to 513,134 K. This relentless growth of 700 K per hour is troubling to watch.

Patrick Jackman

pjackman

HyperBacSrv.exe is now laying claim to 527,644 K.

pjackman

We're a small office with 2 SQL Server instances on a Win 2003 box running inside VMWare. One instance if for the company's Backup Exec. The other instance holds one production database that is 6 GB at present. HyperBac is compressing backups on this second instance. We are using HyperBac primarily to conserve drive space.

In response to my posts regarding High Memory Usage, Jeffery Aven at RedGate suggested that we change two settings in our hyperbac.conf file as follows:

MaxCacheBufferSize=0x800
MaxCompBufferSize=0x1000

This restricts the compression buffers in the service however Jeffery warns that doing so can be detrimental to performance in some cases.

We do a full backup once per day. Differential backups occur every 2 hours and log backups every 15 minutes from from 9 am to 9 pm. Here are some comparisons before and after the .conf update.

Our 5:00 pm differential backup durations were:
8/03 00:19 (19 seconds)
8/04 00:19
8/05 00:16
8/06 00:12
8/07 00:12
8/08 00:21
8/09 00:24
8/10 00:19 (after the .conf update)
8/11 00:20
8/12 00:17
8/13 00:14

The 1st log backup after our nightly processes had the following durations:
8/03 01:06 (1 minutes, 6 seconds)
8/04 01:03
8/05 01:04
8/06 01:03
8/07 01:07
8/08 01:05
8/09 01:03
8/10 01:06 (after the .conf update)
8/11 01:06
8/13 01.06
The resulting log file sizes ranged from 1,048,034 KB to 1,064,157 KB during this period.

The primary difference that we are seeing with memory usage since the .conf change is the starting value. Prior to the change the first full backup would ramp up the memory value to the high 400,000 Ks and begin accumulating at 12 K per minute from there. Since the change it appears that the first full backup resulted in a memory value in the high 100,000 K's and accumulation began from this lower starting value. Seveny four hours later it was at 247,296 K, up 700 K per hour. So we could be back up to the 500,000 K's by the end of August.

However, as Jeffery Aven has pointed out me several times, the Memory attributed to HyperBacSrv and being displayed in Task Manager is virtual memory and it will be released if requested by any other program. So there shouldn't be a problem with any amount of memory claimed by HyperBacSvr, even 1,000,000 K.

If the memory claimed by HyperBacSrv is required by another program we should see HyperBacSrv's displayed allocation in Task Manager knocked back and accumulating at 700 K per hour from that lower point. Eventually the overall system will achieve a dynamic balance. We will always see HyperBacSrv's Memory in Task Manager increasing by 700 K per hour.

I asked Jeffery: "do you think that we've really accomplished anything of technical merit with our .conf change? Would HyperBacSrv's performance be more efficient and reliable under the original settings?"

After reviewing some of our system data he responded: "You are correct in that this should not be an issue with the allocation. Sometimes the value in task manager has a sticker shock to it, but it is really only a constraint if you do not have a large enough page file. The virtual memory is allocated in anticipation of a HyperBac operation or in the case of the initial rise you saw, a reaction to an operation. You should be OK with the new settings. The main bellwether should be system performance overall (backup, restore performance relative to native, query throughput and application responsiveness). If these drop off then changes should be made."

I'll report again to the forum when we appear to reach that dynamic balance.

Patrick Jackman
Vancouver, BC