GCC compilation performance under RAM starvation

[Krystian Kazmierczak (Nokia)]

Hello,

I'd like to continue this topic. The context of that is that I am testing distcc to perform distributed compilation across many hosts which are in fact pods (containers) in k8s env. What I observe is that when all compilation which run on certain container cause memory pressure (memory utilization hits memory limit defined by cgroupv2 for container, swap is disabled) then cpu load for cc1 processes significantly increase (container looks like hanged) and all compilations may last indefinitely. OOM killer is triggered when we hit the memory limit (that is expected). So some memory has been released and other compilations can be continued until we reach memory limit again, however in that point OOM killer is not invoked again. I observe that memory utilization almost equal to memory limit and this is permanent state. It behaves like gcc is aware of lacking memory and trying garbage collect continuously in order to free some memory to continue. Is it possible that gcc somehow prevents exceeding memory limit? My expectation in such scenario is that OOM killer is invoked each time when we hit memory limit. Any insights into whether this is GCC-internal behaviour or if it's more likely tied to Linux's / kernel memory management would be appreciated.

In my test I run 10 compilations of dummy_1.c in parallel (one compilation per cpu) where each compilation utilize ~1GB of RAM in peak. Command which I invoke:
# gcc -std=c99 -Wall -Wextra -pedantic -O3 -funroll-loops -finline-functions -fexpensive-optimizations -S -c "$src_file" -o "/tmp/test/build/${src_file%.c}.o"

BR,
Krystian

From: Krystian Kazmierczak (Nokia)
Sent: Wednesday, April 30, 2025 2:49 PM
To: 'gcc-help@gcc.gnu.org' <gcc-help@gcc.gnu.org>
Subject: GCC compilation performance under RAM starvation

Hello,

I'm observing that gcc compilation slows down significantly when the system experiences RAM starvation. I understand that some slowdown is expected under memory pressure, but I'm trying to determine whether this level of degradation is expected GCC behavior or is primarily due to Linux kernel memory management.
What I observe:

  *   During large builds (make -jN), when total RAM is nearly or fully exhausted, compilation slows drastically. Same source file compile ~20s when we have free memory comparing to 40s-60s when we reach the threshold of memory (if not killed by OOM meanwhile).
  *   Sometimes the Linux kernel's OOM killer is triggered, mostly by other processes not gcc itself, then gcc is selected to be killed e.g.:

[Fri Apr 18 16:14:49 2025] python invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=815
[Fri Apr 18 16:14:49 2025] Memory cgroup out of memory: Killed process 108353 (cc1plus) total-vm:5550480kB, anon-rss:5421224kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:10876kB oom_score_adj:815


  *   The system has swap disabled.
Questions:

  *   Is this performance degradation considered normal for gcc under memory pressure?
  *   Is there any GCC-specific tuning or flags that can help in constrained memory environments? (I'm aware of ggc-min-expand and ggc-min-heapsize but these in most cases leads to lower memory consumption at the cost of higher user time).
  *   I want to understand whether this observation is only an outcome of kernel's memory management or gcc itself is aware of reaching limit of RAM and then somehow "slows down".
System Info:

  *   GCC version: gcc (GCC) 13.1.0
  *   OS: AlmaLinux release 9.3
Any insights into whether this is GCC-internal behaviour or if it's more likely tied to Linux's memory management would be appreciated.

BR,
Krystian

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dummy_1.c
URL: <https://gcc.gnu.org/pipermail/gcc-help/attachments/20250626/25cae2b6/attachment.c>