Académique Documents
Professionnel Documents
Culture Documents
Article Name: SYSTEM195 Last Updated: 11/2/2004 10:03:39 AM Products: Topic: While not unthinkable only a few years ago, tumbling memory prices and increasing hard drive capacities have presented PC users the ability to acquire gigabytes of memory at relativity low cost. As a result of this hardware environment, users are attempting to access large areas of contiguous memory in IDL, but failing for various reasons. This document outlines some of the limitations affecting memory allocation and how they are related to IDL. While focused on the Windows operating system, most of the issues described here relate to any modern operating system. This is a particularly useful Tech Tip for users wondering why on 32-bit Windows they get system "Unable to allocate memory" errors after calls to initialize single IDL arrays requiring 1GB to 2GB of storage space. Discussion: Limiting Factors When a process requests memory from the operating system, various factors determine if the request will be successful or not. The major factors that can cause a request to fail include:
ENVI; IDL
OS Platforms: WIN
Available Memory The amount of physical storage space on the system allocated for memory. OS Limits The operating system has limits to the amount of memory that it can support. Memory Fragmentation Affects the size of contiguous memory blocks available to the process.
Available Memory
Issue This is a fairly straightforward issue: if the amount of memory requested exceeds the amount available, the request will fail. To determine the amount of free memory, the operating system takes into account the following factors:
Physical Memory The amount of physical memory (RAM) available to the system. Virtual Memory The amount of virtual memory available on the system. Modern operating systems can use areas on hard drives to present a set of memory to the system that exceeds the amount of physical memory. Memory systems on modern operating systems, Virtual Memory
Managers (VMM), segregate memory into blocks or pages of a specific size. Pages that aren't being accessed by the current process are copied to the hard drive page file (sometimes referred as a "swap file") and retrieved to physical memory when requested. This action allows the memory pool used by a process to exceed actual physical RAM.
Free Memory While the system might have a large amount of memory available, if that memory is in use, the memory request will fail. The operating system, other processes, and other memory requested by the application affect the pool of available memory. If a memory request fails, closing other running applications could release enough memory to allow it to succeed.
Solution Exit any executing applications. Purchase more memory for the system. Increase the virtual memory manager's page file size.
Certain Windows Server-class systems (including NT/2000/2003 Server and XP Professional) 3 Gigabytes To the best of our knowledge, Windows XP also has these same limits. As you will read below, IDL, as a window-based application, does not have access to this full 2 - 3 GB in one contiguous block. Solution The only solution for this type of limitation is for the operating system to change. For Windows, a user could move to Windows NT Server, Enterprise Edition (or other systems mentioned above) or a 64 bit version of Windows. Of course, a user could also try another platform such as Linux (32bit) or Solaris (64-bit).
Memory Fragmentation
Issue Even when a sufficient amount of free memory is available and the memory request being made is below the limits set by the operation system, memory requests can fail. Often this results from memory fragmentation. When memory is requested from the system, a contiguous free space of the requested size must exist in the process's address space. If such a free space does not exist, the request will fail, even if the total free memory is greater than the request. The larger the memory allocation request, the greater the chance that this failure will occur. This scenario is exacerbated when memory fragmentation occurs. Memory fragmentation takes place when a block of memory is allocated that divides a region of the process's address space, reducing the maximum size of a free block of memory. The following diagram demonstrates this issue.
An indication that memory fragmentation is taking place is when a request for a large block of memory fails, but smaller requests that total the larger request succeed. Memory fragmentation is often the result of software coding scenarios like the following: A large block of memory is allocated. A smaller block of memory is allocated. This allocation places that memory above the large block of memory in the address space. The large block of memory is released to the system. This can occur often when working with large dynamic memory blocks, so developers must be
aware of the issue when designing and implementing software. Solution The key to resolving this type of issue is to be aware that it exists during application development, allocating memory in a correct manner: Persistent allocations first, transient allocations, particularly the large ones, after, and freed at the earliest possible moment.