Re: Endless overcommit memory thread.

From: David Whysong (dwhysong@physics.ucsb.edu)
Date: Fri Mar 24 2000 - 21:03:15 EST

  • Next message: Austin Schutz: "Re: Update: Subtle data corruption of TCP streams"

    On Fri, 24 Mar 2000, Riley Williams wrote:

    >The basic problem as I understand it is not the memory allocated
    >by applications, but the memory allocated by the kernel AFTER the
    >various applications have grabbed theirs. In particular, this is
    >memory used for the buffer cache, the networking stack, the
    >system stack, and the like.

    No. Application dynamic memory is the problem, in addition to the kernel
    memory allocation.

    >As I understand the arguments presented so far, they can be
    >summarised like this:
    >
    > 1. The anti-overcommit brigade:
    >
    > No matter how much memory is used by applications when the
    > kernel is running, if the kernel requires more memory for
    > some essential task (such as the system stack) than is
    > available, we let the system crash by denying that memory
    > to the kernel.

    The system crash issue is completely separate from memory overcommit. The
    fact that you can crash Linux by running OOM is a kernel bug. The best
    anti-overcommit argument I've seen is that programs shouldn't be killed
    just because somebody else called malloc() too much.

    I don't agree with this, because I think the problem isn't really due to
    memory overcommit -- it's a problem with handling OOM situations, which
    can occur with or without overcommit. The problem can be solved using
    quotas and guaranteeing some memory to each user. I REALLY don't like
    quotas myself, but I see why others want them.

    The anti-overcommit people claim this is better because malloc() can
    return NULL. As Alan pointed out, the problem with this argument is that
    there are other implicit ways that programs can use memory, and you can't
    easily notify the users program that they can't, for example, allocate
    another variable on the stack.

    > 2. The pro-overcommit brigade:
    >
    > No matter whether a program really needs the 300M of memory
    > it's just asked for, we will tell the program we can give
    > it that memory, and then have the program crash when it
    > validly assumes we've told the truth.

    Partly right. But the real reason for overcommit is practical: you can do
    more with overcommitted memory, and there are no additional failure modes.

    If we define "failure" as a situation where a user program must be killed
    because we are out of memory, then non-overcommit systems should always
    fail more often than overcommitted ones. So I don't see any downside, AT
    ALL, to memory overcommit.

    [...]

    Dave

    David Whysong dwhysong@physics.ucsb.edu
    Astrophysics graduate student University of California, Santa Barbara
    My public PGP keys are on my web page - http://www.physics.ucsb.edu/~dwhysong
    DSS PGP Key 0x903F5BD6 : FE78 91FE 4508 106F 7C88 1706 B792 6995 903F 5BD6
    D-H PGP key 0x5DAB0F91 : BC33 0F36 FCCD E72C 441F 663A 72ED 7FB7 5DAB 0F91

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/



    This archive was generated by hypermail 2b29 : Fri Mar 24 2000 - 22:10:05 EST