RE: Avoiding OOM on overcommit...?

From: David Schwartz (davids@webmaster.com)
Date: Sat Mar 25 2000 - 20:19:32 EST

  • Next message: David Elliott: "Re: Cylinder limits jumper for drives over 32GB"

    > Sandy Harris writes:
    >
    > > Earlier in this thread, though, someone suggested replacing malloc()
    > > with calloc() to ensure that space actually gets allocated. That should
    > > not be necessary.
    >
    > Nor should it work, because the implementation of calloc relies on the
    > kernel zeroing new pages, and therefore won't touch new pages.

            Which is a good thing, because if it actually did that, it would cause your
    process to get killed in cases where it otherwise wouldn't.

            Imagine if a process allocates 128Mb with 'malloc'. After the process has
    touched 64Mb of it through normal processes, we get to a case where we are
    nearly out of RAM+swap.

            Case 1: We don't touch the memory until we really need to.

            The system has not used 64Mb of extra swap to hold the garbage data we
    threw in the pages and never used (which, of course, were paged out). As a
    result, the system never runs low on swap and we run to completion.

            Case 2: We touch all the memory when we 'malloc' it. The memory we don't
    'really' use gets swapped out as we run low on RAM+swap.

            The system has paged out the 64Mb of data we touched but didn't really
    need. As a result, the system is out of RAM+swap (64Mb earlier than in case
    1). Now we touch one of the pages we earlier wrote _real_ data to, the
    system needs to page it into physical RAM. Unfortunately, since it's out of
    RAM+swap, it can't page out, so it can't page in. So the kernel kills the
    process that attempts to access the memory it previously wrote to. We die
    when we touch the memory.

            The people who keep concentrating on malloc and memory touching don't
    understand the problem.

            DS

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/



    This archive was generated by hypermail 2b29 : Sat Mar 25 2000 - 20:26:37 EST