Extending and Benchmarking the "Big Memory" implementation on Blue Gene/P

Kazutomo Yoshii, Harish Naik, Chenjie Yu, and Pete Beckman

Despite the fact that Linux is a popular operating system for
high-performance computing, it does not ensure maximum performance for
compute-intensive workloads.  In our previous work we presented "Big
Memory"-an alternative, transparent memory space that successfully removes
the memory performance bottleneck on Blue Gene/P Linux.  The initial Big
Memory worked only as a per node resource.  In this work we extend it to a
per core resource and describe the details of the implementation.  We
evaluate our new implementation by running various benchmarks and the
Nek5000 application.  Compared with IBM's Compute Node Kernel that is
noise-free and lightweight, the Nek5000 application runs with only 1.2%
performance loss on Linux with Big Memory at 32K cores.  Our benchmark
results show no significant performance degradation from OS noise caused by
Linux at a scale of up to 32K cores, although irregular OS events are still
present.