system will not use over 800 megs of ram

How much RAM have you configured for each machine?

What virtualization software do you use?

oops, previous one have been an information request

I am using VMware server 2.0 for Linux. (Fit all virtual machine memory into reserved host RAM)

Are you saying you have 4GB of RAM installed and it is showing 4GB (via your system properties, etc.), but you only "have access" to 3.2 by what you can tell? Thus, you want access to ALL of it (the other 800mb)?

If so, I believe you are using the 32bit version (i386/i686) installation of Ubuntu. You will need to use the 64bit version of Ubuntu to be able to utilize all 4GB of your RAM.

If you are just referring to your virtual machine(s) and their settings:

If not, ensure that your VM(s) are set to use the proper amount of RAM. If it is VMWare, make sure this is set in the settings and RESTART the Virtual Machine. If it is VirtualBox, the same applies.

Do not do it while the machine is running because it will not take affect until the machine is completely powered off.

VMWare (how to): http://www.vmware.com/support/gsx3/doc/performance_mem_reserved_gsx.html

VirtualBox: It's a little bit different, but it tries to guess (like VMWare) upon setup. It's usually in the range of 256MB or 512MB based on your machine. Please double check this in the machine's settings.

no! this is an answer to the 32 bit system accessing more than 3.2 gb. I am not saying this.

I am saying I have the 32 bit system. It sees all the memmory it is supposed too. i.e. per 32 bits. Using the system monitor and the system slowing down with hard drive thrashing somehow there is a limit set for the system to only use 749 megs. The system sees 3.9gb.

And just a while ago it was using 23% swapfile and 9% memory. Now what is that?

Thanks,
Steve Janke

> To: <email address hidden>
> From: <email address hidden>
> Subject: Re: [Question #65132]: system will not use over 800 megs of ram
> Date: Tue, 24 Mar 2009 02:49:15 +0000
>
> Your question #65132 on Ubuntu changed:
> https://answers.edge.launchpad.net/ubuntu/+question/65132
>
> Status: Open => Answered
>
> Craig Huffstetler proposed the following answer:
> Are you saying you have 4GB of RAM installed and it is showing 4GB (via
> your system properties, etc.), but you only "have access" to 3.2 by what
> you can tell? Thus, you want access to ALL of it (the other 800mb)?
>
> If so, I believe you are using the 32bit version (i386/i686)
> installation of Ubuntu. You will need to use the 64bit version of Ubuntu
> to be able to utilize all 4GB of your RAM.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.edge.launchpad.net/ubuntu/+question/65132/+confirm?answer_id=3
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.edge.launchpad.net/ubuntu/+question/65132
>
> You received this question notification because you are a direct
> subscriber of the question.

So... where and if there are settings that sets a ceiling of memory to use is my question as well.

Thanks,
Steve Janke

> To: <email address hidden>
> From: <email address hidden>
> Subject: Re: [Question #65132]: system will not use over 800 megs of ram
> Date: Tue, 24 Mar 2009 02:54:23 +0000
>
> Your question #65132 on Ubuntu changed:
> https://answers.edge.launchpad.net/ubuntu/+question/65132
>
> Craig Huffstetler proposed the following answer:
> If you are just referring to your virtual machine(s) and their settings:
>
> If not, ensure that your VM(s) are set to use the proper amount of RAM.
> If it is VMWare, make sure this is set in the settings and RESTART the
> Virtual Machine. If it is VirtualBox, the same applies.
>
> Do not do it while the machine is running because it will not take
> affect until the machine is completely powered off.
>
> VMWare (how to):
> http://www.vmware.com/support/gsx3/doc/performance_mem_reserved_gsx.html
>
> VirtualBox: It's a little bit different, but it tries to guess (like
> VMWare) upon setup. It's usually in the range of 256MB or 512MB based on
> your machine. Please double check this in the machine's settings.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.edge.launchpad.net/ubuntu/+question/65132/+confirm?answer_id=4
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.edge.launchpad.net/ubuntu/+question/65132
>
> You received this question notification because you are a direct
> subscriber of the question.

I'm not familiar with VMware server, I have used only VMware workstation. But I guess they might work similarly.

If you create a virtual machine, you define how much memory will be allocated for the machine. (And can can change it later). So how much have you allocated for each machine running? Are you sure its more than what they use?

It's the memsize parameter in the vmx file.

One case where I have seen heavy disk thrashing is related to the *.vmem files maintained by VMware. Do you have such files? Check what filefrag reports for them. I've seen cases with several hundred thousand extents. Try to read such a file sequentially (e.g. using md5sum command). Does the disk thrashing happen in this case, too?

If that's your problem there are various options to deal with *.vmem files. Making them unnamed helps a lot (I've done that), obviously they can also be turned off altogether. I don't remember the relevant options now. Please post again if your problem is related to *.vmem and you cannot find the relevant option in VMWare's knowledge base.

Please also post the output of the "free" command. (run in Terminal / command line)

The case is I just happen to be using VMware. Vmware uses extensive amount of memory. I mentioned this so people would have an idea of the applications I am using. What is happening is if I am using VMware, whatching a movie opening up large files and/or doc files the system memory will not show it being used over 800 meg. it goes so slow that it crashes.

Thanks,
Steve Janke

> To: <email address hidden>
> From: <email address hidden>
> Subject: RE: [Question #65132]: system will not use over 800 megs of ram
> Date: Tue, 24 Mar 2009 11:09:17 +0000
>
> Your question #65132 on Ubuntu changed:
> https://answers.launchpad.net/ubuntu/+question/65132
>
> Status: Open => Answered
>
> Uwe Geuder proposed the following answer:
> I'm not familiar with VMware server, I have used only VMware
> workstation. But I guess they might work similarly.
>
> If you create a virtual machine, you define how much memory will be
> allocated for the machine. (And can can change it later). So how much
> have you allocated for each machine running? Are you sure its more than
> what they use?
>
> It's the memsize parameter in the vmx file.
>
> One case where I have seen heavy disk thrashing is related to the *.vmem
> files maintained by VMware. Do you have such files? Check what filefrag
> reports for them. I've seen cases with several hundred thousand extents.
> Try to read such a file sequentially (e.g. using md5sum command). Does
> the disk thrashing happen in this case, too?
>
> If that's your problem there are various options to deal with *.vmem
> files. Making them unnamed helps a lot (I've done that), obviously they
> can also be turned off altogether. I don't remember the relevant options
> now. Please post again if your problem is related to *.vmem and you
> cannot find the relevant option in VMWare's knowledge base.
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/ubuntu/+question/65132/+confirm?answer_id=7
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/ubuntu/+question/65132
>
> You received this question notification because you are a direct
> subscriber of the question.

Here is what free -m shows:

             total used free shared buffers cached

Mem: 3973 2989 984 0 10 2518

-/+ buffers/cache: 459 3514

Swap: 0 0 0

Now I see there is 2518 cached. What is this? Why is this?

Thanks,
Steve Janke

> To: <email address hidden>
> From: <email address hidden>
> Subject: RE: [Question #65132]: system will not use over 800 megs of ram
> Date: Tue, 24 Mar 2009 11:15:43 +0000
>
> Your question #65132 on Ubuntu changed:
> https://answers.launchpad.net/ubuntu/+question/65132
>
> Uwe Geuder requested for more information:
> Please also post the output of the "free" command. (run in Terminal /
> command line)
>
> --
> To answer this request for more information, you can either reply to
> this email or enter your reply at the following page:
> https://answers.launchpad.net/ubuntu/+question/65132
>
> You received this question notification because you are a direct
> subscriber of the question.

2518 Megs are files, which the kernel has loaded into RAM.

I could guess a lot of it are *.vmem file(s), which represent the RAM of your virtual machine(s).

So your virtual machines RAM is indeed in the pyhsical machine's RAM, but it's backed by a file.

But if those files are badly fragmented it can get extremely slow. Please search whether you have *.vmem files and investigate them using filefrag as I've written before.

Instead of using md5sum as I wrote before you could use the following command to investigate in what shape your vmem files are

time cat name.vmem >/dev/null

(that should also read the whole file, but not cause any CPU load)

Does it take a long time and your disk thrashes during that operation?

Example from my system:

$ cp Windows\ XP\ Professional.vmem-fragmented Windows\ XP\ Professional.vmem-copied
$ time cat Windows\ XP\ Professional.vmem-fragmented >/dev/null

real 4m7.352s
user 0m0.032s
sys 0m0.800s
$ time cat Windows\ XP\ Professional.vmem-copied >/dev/null

real 0m15.215s
user 0m0.028s
sys 0m0.616s
$ sudo filefrag Windows\ XP\ Professional.vmem-*
[sudo] password for user:
Windows XP Professional.vmem-copied: 20 extents found, perfection would be 9 extents
Windows XP Professional.vmem-fragmented: 105739 extents found, perfection would be 9 extents

Because you have a lot RAM you might need to read some other big files before the measurements, to make sure that the file to be measured is no longer in the cache. And of the course the VM using this file must not be running during the measurement.

BTW, have you tried the System Monitor Applet? (In your upper panel: Add to panel, System Monitor.

It can be configured in many ways (Right click, Preferences). I don't remember what the defaults were, you might have to turn on memory manually. It shows memory in use a cache in a different color

This just as an addition, please still investigate the issues mentioned in my previous posting.

Hi Uwe,

Based on your input I was able to find the solution with goole search. The link explains it:
http://communities.vmware.com/thread/96239

And here as well:
http://www.ubuntu-unleashed.com/2008/04/free-up-cache-memory-in-linux.html

Good to hear you got it solved.

I've used the solution as described in the vmware community post you linked. I once got this undocumented parameter from VM support after pressing them really hard that VMware performance is unacceptable in some use cases. When I wondered why I need an undocumented parameter to make it work, why they don't do that by default, what this parameter really does, and what is the downside of using it etc. etc, they were unable to answer. Typical tech support, who don't understand how their own product works... Anyway the parameter improves performance drastically in the critical use cases and I have not found any problems.

There is another parameter (probably undocumented, too, but Google finds it), which turns the vmem files off altogether. I have never used it, because the unnamed files work well enough for me.

The unleashed link is interesting. I wasn't aware of that functionality. On the other side I cannot see how that would help the vmem file problem. If accessing the vmem file is slow (although a lot of it is in the cache), I don't see how dropping the cache could make things faster. But well, I'm far from completely understanding Linux memory handling and caching, so it might be worth experimenting when I have some extra time...

When thinking about your original 800 Meg problem description: The big unanswered Ubuntu question is of course, why the full System Monitor gives such misleading information not mentioning the cache at all, although the small panel applet can show it better. Maybe we should make a feature request...

Here is more tuning that can be done for this:
http://www.fewt.com/2008/06/performance-tuning-vmware-server-on.html