Linux memory usage demystified.. !!!!

Have you ever worried about Memory usage in the Linux server?

Ok, let’s take a look into the Linux memory.

Memory Architecture

To execute a process, the Linux kernel allocates a portion of the memory area to the requesting process. The process uses the memory area as workspace and performs the
required work. Assume a desktop or laptop assigned to you in your office cubicle. Take a look at your desktop (read as your Windows/Linux Dekstop) most of us will use this place in our system to scatter text files, documents, pdf’s, ppt’s and other things to perform your work, look how muddled up it looks. We are very poor in managing our desktop area, we are speaking about a maximum area of 1366×768 or more. Think of our Linux kernel managing GB’s of memory how cluttered it will be, But the Linux kernel is perfectly efficient in managing its memory area. The difference is that the kernel has to allocate space in a more dynamic manner. The number of running processes sometimes comes to tens of thousands and amount of memory is usually limited. Therefore, Linux kernel must handle the memory efficiently or you will end up with an unresponsive system.

32 Bit Vs 64 bit

On 32-bit architectures such as the i386, the Linux kernel can directly address only the first gigabyte of physical memory (896 MB when considering the reserved range). Memory above the so-called ZONE_NORMAL must be mapped into the lower 1 GB. This mapping is completely transparent to applications, but allocating a memory page in ZONE_HIGHMEM causes a small performance degradation.

On the other hand, with 64-bit architectures such as x86_64, ZONE_NORMAL extends all the way to 64 GB or to 128 GB in the case of IA-64 systems. As you can see, the overhead of mapping memory pages from ZONE_HIGHMEM into ZONE_NORMAL can be eliminated by using a 64-bit architecture

On 32-bit architectures, the maximum address space that single process can access is 4GB.This is a restriction derived from 32-bit virtual addressing. In a standard implementation, the virtual address space is divided into a 3 GB user space and a 1 GB kernel space. There are some variants like 4 G/4 G addressing layout implementing.

On the other hand, on 64-bit architecture such as x86_64 and IA64, no such restriction exists.Each single process can benefit from the vast and huge address space.

Memory Usage

On a Linux system, many programs run at the same time. These programs support multiple users, and some processes are more used than others. Some of these programs use a portion of memory while the rest are “sleeping.” When an application accesses cache, the performance increases because an in-memory access retrieves data, thereby eliminating the need to access slower disks. The OS uses an algorithm to control which programs will use physical memory and which are paged out. This is transparent to user programs.

Paging and Swapping

In Linux, genrally *NIXes, there are differences between paging and swapping. Paging moves individual pages to swap space on the disk and swapping is a bigger operation that moves the entire address space of a process to swap space in one operation.

If your server is always paging to disk (a high page-out rate), buddy..! its time to increase your physical RAM. However, for systems with a low page-out rate, it might not affect performance. If a process behaves poorly. Paging can be a serious performance problem when the amount of free memory pages falls below the minimum amount specified, because the paging mechanism is not able to handle the requests for physical memory pages and the swap mechanism is called to free more pages. This significantly increases I/O to disk and will quickly degrade a server’s performance.

Memory Available

This indicates how much physical memory is available for use. If, after you start your application, this value has decreased significantly, you might have a memory leak. Check the application that is causing it and make the necessary adjustments.

System Cache

This is the common memory space used by the file system cache.

Page Faults

There are two types of page faults: soft page faults, when the page is found in memory, and hard page faults, when the page is not found in memory and must be fetched from disk. Accessing the disk will slow your application considerably.

Private Memory

This represents the memory used by each process running on the server.

Oh.. ok.. at least some of you seems its too boring by now, So let’s think it real time.

To explain the memory usage I’ll try to illustrate with examples from my virtual test server running  PostgreSQL DB Instance on Centos 6.3.

The above output is a typical output for the free command used for memory monitoring. Just looking at this numbers, we can conclude that the system is having approx 2GB of ram and nearly 95% of it is used. If we look at the Swap: line in the output, we see that the swap space appears to be unused.

In between the Mem: and Swap: lines, we see a line labeled -/+ buffers/cache. This is probably the trickiest part in interpreting free command output. This line shows how much of the physical memory is used by the buffer/cache. In other words, this shows how much memory is being used (a better word would be borrowed) for disk caching. Disk caching makes the system run much faster.

So, while at first glance, this system appears to be running short of memory, it’s actually just making good use of memory that’s currently not needed for anything else. The key number to look at in the output above is, therefore, 1703. This is the amount of memory that would be made available to your applications if they need it.

Now we will see the Swap and Paging activity

NOTE: If your system is busy and you want to watch how memory is changing, you can run free with a -s (seconds) argument that causes the command to give you totals every X seconds. You need to start worrying about memory only and only if you see the swap usage is high.