Confused about effects of pagefile usage on real-time performance

Hello,

I'm using a couple of PCs to stream audio for virtual musical instruments. Basically, there's a master PC that receives commands from a piano keyboard then sends those commands to the slave PCs to play back the audio associated with each key press. That process needs to happen quickly in order to "feel" correct when playing.

Here's my dilemma: every time I load up the audio data on the slave, it gets within 5-10 GB of the total amount of physical RAM (64 GB) and then it starts dumping data to the page file, thereby reducing the RAM usage. That creates disruptions in the audio stream because it can't pull the data from the page file as quickly as it can from RAM (at least I presume that's what's happening). The pagefile usage goes to about 10 GB (as reported by perfmon.exe) and the memory usage goes to about 47 GB (as reported by task manager). So the total memory requirement is about 57 GB, leaving me about 7 GB to spare.

But why is it using the page file instead of using physical memory? My understanding is that the OS tries to free up memory that it thinks I won't need. But how does it decide what RAM is less necessary? In my case it always seems to guess wrong because I can drop the page file down to 1 GB and the system performs better.

But even with a tiny page file I still get 5-7 GB of RAM that is "modified" and, apparently, not available for use.

It seems to me that the best setup for a real-time application like this one is to have no page file because there are no other apps loaded or executed - the memory gets filled at boot up and then never changes. So why would you ever need a pagefile? But even with a tiny page file Windows won't let me access all my RAM and it puts a bunch of it into some kind of quarantine that prevents me from making use of it.

Is there a solution to this problem? Or do I just have to accept that my 64 GB of RAM is really only about 55 GB, and I've lost the other 9 GB to Windows because it thinks it's smarter than I am?

Thanks in advance,

rgames

OK - my page file is on my L drive (why does that matter?) and has been set to between 1 and 15 GB. It's an SSD. As I said above, when set to a larger value I get many breaks in the audio stream. The frequency with which those breaks occur decreases with decreasing page file size.

I have tried both with compressed RAM and without and the behavior is the same.

System specs:
i7 6700k on Gigabyte/Asus motherboard
64 GB RAM
Various SSDs and HDDs
One machine uses a $20 AMD video card and the other has a GTX 970.

How does this help answer my questions above? I'm not sure what you do with this extra information.

Thanks,

rgames

A weird thought just surfaced from my MS-DOS 5 days. I frequently created a RAM disk in memory to copy data from floppies before recopying to a blank disk. MS-DOS 6.xx was much better as it would copy to the HDD first. Anyway, the thought was running the Pro version of Windows on a motherboard that supports 128GB RAM and create that RAM disk about half the RAM and designate it for the paging file/Virtual Memory. Or in other word, speed. It would be a bit expensive but the end result may be worth it.

EdTittel said:

Best place to learn about paging behavior in Windows is in Mark Russinovich et al's outstanding book Windows Internals, 6th edition. Chapter 10 in Part 2 is entitled Memory Management and it explains how modern Windows manages memory and, by extension, how it works with the paging file. Turns out that at least some paging file space is a good idea to maintain on Windows, even on systems with max memory, simply to leave buffering space for pages waiting to be written to memory while pages marked for deletion are being cleaned out. Fascinating stuff!
HTH,
--Ed--

Hi Ed - thanks for the info. I'll check it out.

Quick question - it seems to me that leaving the page file intact does, indeed, make sense for most use cases (spreadsheets, web browsing, e-mail, etc). But do you think the same thinking applies to real-time applications like I described above? Does that book specifically discuss how the page fille affects real-time applications?

My understanding is that the pagefile is kind-of a hedge against what the user might do in the future - load other apps, load other data, etc. so Windows tries to guess what's not really necessary and free up memory for what might be necessary in the future. Right?

But in the case I'm describing, there are no other apps or data loaded. Once everything is loaded at boot-up, nothing changes. The computer just responds to requests to dump data that's already in memory into a buffer and transfer it to the master computer.

So, given that the real-time system performance would improve without the pagefile (as evidenced by my testing), doesn't it make sense for Windows to just forget about the pagefile and "modified" memory?

More to the point, how do you force it to do so? I can disable the pagefile but I still lose all that memory to "modified" status, which, as far as I can tell, is Windows waiting for the pagefile size to increase so that it can dump it there.

Thanks again,

rgames

So, given that the real-time system performance would improve without the pagefile (as evidenced by my testing), doesn't it make sense for Windows to just forget about the pagefile and "modified" memory?

Always have to keep in mind that other/third-party program installs look for certain things to be available, downside of Microsoft having to "cooperate" with those programmers. Apple not so much, Linux usually even more open than Windows.