Allocating memory dynamically helps with many of our problems with statically sized buffers. There's no need to place hard restrictions to buffer sizes and there's no memory loss because of the unused space in buffers.
Having no limits is tempting, but there are some tradeoffs. Dynamic allocations are always slower because they're allocated at run time. Luckily there are ways to keep that slowdown nearly non-existent for some allocation types.
Memory allocations could be divided into two groups. Either you want to save the data permanently or you only need it for a short time within the function or possibly as the return value. I'll call the latter "temporary allocations". Usually programs use a lot more temporary allocations than permanent.
malloc() is the only ANSI-C compatible way to allocate memory dynamically. C99 also has variable length arrays that could be called dynamic allocations but their usage is quite limited. Usually the same malloc() implementation is used by all programs running in your system, so it's optimized to common usage patterns.
Commonly used malloc() implementations allocate a few extra bytes for it's own internal bookkeeping. The memory is allocated from heap which is a contiguous memory area that is grown when it gets full. The important thing to remember is that heap doesn't shrink. So if you allocate a few megabytes of memory and free() it, the process still uses those few megabytes of memory, it's only internally marked as being free for future malloc()s. malloc() doesn't have to work this way and there are a couple of common exceptions, see anonymous memory maps bellow.
Using malloc() doesn't directly help with buffer overflows. You still have to be careful to allocate enough memory for the written data. There's two common ways to do this:
First calculate how much memory you need, then allocate it and copy the data there. This requires you to be very careful with calculating the size correctly.
With strings you could do for example:
str = malloc(strlen(a)+strlen(b)+2);
strcat(strcat(strcpy(str, a), " "), b);. Note that strcat() isn't
exactly fast operation as it needs to traverse the destination string from
beginning to end before writing the new string.
With larger datasets you could also execute the code twice. First time by only calculating the length without any writing. For example:
calc_length = 1;
do {
length = 0;
/* ... subdata = ..; sublen = ..; */
if (!calc_length)
memcpy(data + length, subdata, sublen);
length += sublen;
/* more additions like above */
if (calc_length)
data = malloc(length);
calc_length = !calc_length;
} while (!calc_length);
Start with a small amount of memory and realloc() more whenever the buffer gets full. realloc() may not be very fast operation, so you should try to minimize how often it's called. For temporarily allocated memory you should try to pick good initial size for malloc(), so that the buffer wouldn't need to be grown in most cases. Pretty good way to handle growing is to allocate twice the old size, that way realloc() isn't constantly called even for larger datasets. Of course that could cause unneededly large memory usage.
Allocate and grow method has one huge benefit to calculate and allocate method. You can create a few simple to use functions to write to the buffer and handle the growing automatically. Here's a simple example:
char *malloc_strcat(char *buf, const char *str)
{
size_t buflen = strlen(buf);
size_t slen = strlen(str);
char *mem;
/* realloc() new buffer every time. slow, but works.
faster way would be to use struct { char *buf; size_t alloc_size;
size_t used_size; } and allocate extra memory every time. */
mem = realloc(buf, buflen + slen + 1);
memcpy(mem, buf, buflen);
memcpy(mem + buflen, str, slen + 1); /* +1 for \0 */
free(buf);
return mem;
}
The most annoying problem with malloc() is that you specifically have to deallocate the memory. This often causes memory leaks with programs when it's forgotten. It also produces sometimes very ugly code when using many functions that dynamically allocate their return value and require you to free it.
Especially error handling gets ugly when you have to return from function while still having several allocations that has to be freed. Typically people either free() them in multiple places or use gotos. More about this later in Coding Style chapter.
To ease the pain of freeing memory with temporary allocations, alloca() was created. It allocates memory from caller function's stack frame so it's automatically freed when the function returns. Because allocating means only increasing the stack pointer, it's a very fast operation.
alloca() isn't in any standard, but it exists in most of today's UNIX platforms. If you want your code to be portable, you probably shouldn't be using it. You also need to be careful enough not to let the allocated memory get stored permanently and get accessed outside the function's stack frame. Especially function must not return a value allocated with alloca().
Quoting BUGS section from Linux's alloca(3) manual page:
The alloca function is machine and compiler dependent. Its use is discouraged.
On many systems alloca cannot be used inside the list of arguments of a function call, because the stack space reserved by alloca would appear on the stack in the middle of the space for the function arguments.
Quoting BUGS section from OpenBSD's alloca(3) manual page:
The alloca() function is slightly unsafe because it cannot ensure that the pointer returned points to a valid and usable block of memory. The allocation made may exceed the bounds of the stack, or even go further into other objects in memory, and alloca() cannot determine such an error. Avoid alloca() with large unbounded allocations.
mmap() function can be used to map files and devices into memory. It makes it very easy to access files and provides efficient caching at operating system level. The file is lazily read into memory, so if you never access a part of file it may not be never read into memory. OS can also drop unmodified mmaped pages when it's running low on memory so it's definitely better than malloc()ing huge amounts of memory and reading the file to it.
You shouldn't rely too much on perfect mmap()ing behviour however. If you need to fully read large files sequentially, mmap()ing it entirely could take memory away from other processes. Also processes may have virtual memory size limits and the mmap() function could fail with ENOMEM. It's possible to mmap() only parts of the file, so you can still use it instead of read().
Some systems also support anonymous memory maps. Using them gets rid of the overhead of malloc() headers and makes it possible to free the memory back to operating system for other uses. Some malloc() implementations use anonymous mmaps internally, for example glibc 2.3 uses it for larger allocations. They can also be useful with preventing some buffer overflows from overwriting any important data, more about this in Buffer Overflows chapter.
Anonymous mmaps can be created using MAP_ANONYMOUS or MAP_ANON flag with mmap(). With some older systems that don't support those flags you can also mmap() /dev/zero. Linux also supports mremap() for growing mmaped areas.