[Solved] How can I reduce the virtual memory required by gccgo compiled executable?


I was able to locate where gccgo is asking for so much memory. It’s in the libgo/go/runtime/malloc.go file in the mallocinit function:

// If we fail to allocate, try again with a smaller arena.
// This is necessary on Android L where we share a process
// with ART, which reserves virtual memory aggressively.
// In the worst case, fall back to a 0-sized initial arena,
// in the hope that subsequent reservations will succeed.
arenaSizes := [...]uintptr{
  512 << 20,
  256 << 20,
  128 << 20,
  0,
}

for _, arenaSize := range &arenaSizes {
  // SysReserve treats the address we ask for, end, as a hint,
  // not as an absolute requirement. If we ask for the end
  // of the data segment but the operating system requires
  // a little more space before we can start allocating, it will
  // give out a slightly higher pointer. Except QEMU, which
  // is buggy, as usual: it won't adjust the pointer upward.
  // So adjust it upward a little bit ourselves: 1/4 MB to get
  // away from the running binary image and then round up
  // to a MB boundary.
  p = round(getEnd()+(1<<18), 1<<20)
  pSize = bitmapSize + spansSize + arenaSize + _PageSize
  if p <= procBrk && procBrk < p+pSize {
    // Move the start above the brk,
    // leaving some room for future brk
    // expansion.
    p = round(procBrk+(1<<20), 1<<20)
  }
  p = uintptr(sysReserve(unsafe.Pointer(p), pSize, &reserved))
  if p != 0 {
    break
  }
}
if p == 0 {
  throw("runtime: cannot reserve arena virtual address space")
}

The interesting part is that it falls back to smaller arena sizes if larger ones fail. So limiting the virtual memory available to a go executable will actually limit how much it will successfully allocate.

I was able to use ulimit -v 327680 to limit the virtual memory to smaller numbers:

VmPeak:   300772 kB
VmSize:   300772 kB
VmLck:         0 kB
VmPin:         0 kB
VmHWM:      5712 kB
VmRSS:      5712 kB
VmData:   296276 kB
VmStk:       132 kB
VmExe:      2936 kB
VmLib:         0 kB
VmPTE:        56 kB
VmPMD:         0 kB
VmSwap:        0 kB

These are still big numbers, but the best that a gccgo executable can achieve. So the answer to the question is, yes you can reduce the VmData of a gccgo compiled executable, but you really shouldn’t worry about it. (On a 64 bit machine gccgo tries to allocate 512 GB.)

solved How can I reduce the virtual memory required by gccgo compiled executable?