MELT Performance Tests

Here are a few tests which could be used to check the MELT performance and give an idea of the time that a plugin might add.

Tests June 15, 2011

The tests were run on the following computer:

Computer

Processors

Quad-core Intel(R) Xeon(R) CPU W3520 @ 2.67GHz

OS

Ubuntu 11.04 (x86_64)

kernel

2.6.38-10-generic

Ram

7079 MB

GCC version

gcc version 4.6.1 20110611 (prerelease) (Debian 4.6.0-13)

MELT Plugin version

MELT plugin 0.7 for GCC 4.6

Sadly, I had to run the tests on a virtual machine (wmware)... That can slightly slow down the tests, however the equivalence between test with and without the MELT plugin stay.

Readline compilation

The first test was to see the duration of compilation of the library readline 6.2 with and without the use of a small MELT module (about 180 lines). The MELT plugin adds a pass at gimple level inspecting every gimple (so there is a loop for each basic_block, for each gimple). In this test we expect the MELT module to be already compiled into a .so, so we don't count the time to compile the MELT module.

Readline compilation

Without module

With module

User Time

8.26

14.06

System Time

0.69

1.58

Compilation of a big file with the melt plugin

I tried to compile an important single file (gcc/tree.c from trunk revision 174908, it is about 11217 lines of codes) with a melt module to see the time spend by the plugin. The plugin is a small MELT module (about 180 lines). It adds a pass at gimple level inspecting every gimple (so there is a loop for each basic_block, for each gimple).

I compiled it using the following options:

administrateur@administrateur-virtual-machine:~/programmation/compil_gcc/gcc_trunk_build/gcc$ gcc -c -ftime-report -fplugin=/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.6.1/plugin/melt.so -fplugin-arg-melt-mode=test_fopen -fplugin-arg-melt-init=@@:test_fopen -fplugin-arg-melt-module-path=. -fplugin-arg-melt-source-path=/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/ -fkeep-inline-functions -DIN_GCC   -W -Wall -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Wold-style-definition -Wc++-compat -fno-common  -DHAVE_CONFIG_H -I. -I. -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/. -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/melt/generated -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/../include -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/../libcpp/include  -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/../libdecnumber -I/home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/../libdecnumber/bid -I../libdecnumber  -DCLOOG_INT_GMP -DCLOOG_ORG  /home/administrateur/programmation/compil_gcc/gcc_melt_source/gcc/tree.c -o tree.o

Result is the following:

Execution times (seconds)

garbage collection    :   9.62 (70%) usr   0.14 (14%) sys   9.80 (66%) wall       0 kB ( 0%) ggc
callgraph construction:   0.06 ( 0%) usr   0.02 ( 2%) sys   0.06 ( 0%) wall    3938 kB ( 1%) ggc
callgraph optimization:   0.26 ( 2%) usr   0.10 (10%) sys   0.35 ( 2%) wall  237936 kB (38%) ggc
varpool construction  :   0.01 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall      41 kB ( 0%) ggc
cfg construction      :   0.00 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall     470 kB ( 0%) ggc
cfg cleanup           :   0.01 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall      10 kB ( 0%) ggc
trivially dead code   :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.02 ( 0%) wall       0 kB ( 0%) ggc
df scan insns         :   0.15 ( 1%) usr   0.00 ( 0%) sys   0.16 ( 1%) wall     199 kB ( 0%) ggc
df live regs          :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.02 ( 0%) wall       0 kB ( 0%) ggc
df reg dead/unused notes:   0.08 ( 1%) usr   0.02 ( 2%) sys   0.10 ( 1%) wall    2340 kB ( 0%) ggc
register information  :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.02 ( 0%) wall       0 kB ( 0%) ggc
alias analysis        :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.02 ( 0%) wall     648 kB ( 0%) ggc
rebuild jump labels   :   0.03 ( 0%) usr   0.00 ( 0%) sys   0.04 ( 0%) wall       0 kB ( 0%) ggc
preprocessing         :   0.21 ( 2%) usr   0.08 ( 8%) sys   0.26 ( 2%) wall   11173 kB ( 2%) ggc
lexical analysis      :   0.04 ( 0%) usr   0.08 ( 8%) sys   0.19 ( 1%) wall       0 kB ( 0%) ggc
parser                :   0.30 ( 2%) usr   0.09 ( 9%) sys   0.35 ( 2%) wall   30008 kB ( 5%) ggc
inline heuristics     :   0.03 ( 0%) usr   0.00 ( 0%) sys   0.03 ( 0%) wall       0 kB ( 0%) ggc
integration           :   0.02 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall      26 kB ( 0%) ggc
tree gimplify         :   0.07 ( 1%) usr   0.05 ( 5%) sys   0.12 ( 1%) wall   17770 kB ( 3%) ggc
tree eh               :   0.01 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall       0 kB ( 0%) ggc
tree CFG construction :   0.00 ( 0%) usr   0.02 ( 2%) sys   0.02 ( 0%) wall    6919 kB ( 1%) ggc
tree CFG cleanup      :   0.03 ( 0%) usr   0.01 ( 1%) sys   0.05 ( 0%) wall      31 kB ( 0%) ggc
tree find ref. vars   :   0.01 ( 0%) usr   0.02 ( 2%) sys   0.03 ( 0%) wall    1000 kB ( 0%) ggc
tree PHI insertion    :   0.02 ( 0%) usr   0.02 ( 2%) sys   0.03 ( 0%) wall     668 kB ( 0%) ggc
tree SSA rewrite      :   0.00 ( 0%) usr   0.01 ( 1%) sys   0.01 ( 0%) wall    3527 kB ( 1%) ggc
tree SSA other        :   0.00 ( 0%) usr   0.03 ( 3%) sys   0.01 ( 0%) wall     415 kB ( 0%) ggc
tree operand scan     :   0.05 ( 0%) usr   0.07 ( 7%) sys   0.12 ( 1%) wall    5386 kB ( 1%) ggc
dominance computation :   0.04 ( 0%) usr   0.00 ( 0%) sys   0.05 ( 0%) wall       0 kB ( 0%) ggc
out of ssa            :   0.07 ( 1%) usr   0.00 ( 0%) sys   0.07 ( 0%) wall       0 kB ( 0%) ggc
expand vars           :   0.03 ( 0%) usr   0.01 ( 1%) sys   0.03 ( 0%) wall    3022 kB ( 0%) ggc
expand                :   0.32 ( 2%) usr   0.05 ( 5%) sys   0.39 ( 3%) wall   30444 kB ( 5%) ggc
post expand cleanups  :   0.00 ( 0%) usr   0.01 ( 1%) sys   0.01 ( 0%) wall    1801 kB ( 0%) ggc
jump                  :   0.01 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall       0 kB ( 0%) ggc
integrated RA         :   0.60 ( 4%) usr   0.03 ( 3%) sys   0.59 ( 4%) wall    8510 kB ( 1%) ggc
reload                :   0.35 ( 3%) usr   0.02 ( 2%) sys   0.38 ( 3%) wall    7851 kB ( 1%) ggc
thread pro- & epilogue:   0.04 ( 0%) usr   0.00 ( 0%) sys   0.04 ( 0%) wall    3789 kB ( 1%) ggc
final                 :   0.34 ( 2%) usr   0.02 ( 2%) sys   0.37 ( 2%) wall     844 kB ( 0%) ggc
uninit var analysis   :   0.01 ( 0%) usr   0.00 ( 0%) sys   0.01 ( 0%) wall       0 kB ( 0%) ggc
plugin execution      :   0.66 ( 5%) usr   0.04 ( 4%) sys   0.74 ( 5%) wall  235648 kB (38%) ggc
rest of compilation   :   0.22 ( 2%) usr   0.05 ( 5%) sys   0.29 ( 2%) wall    7211 kB ( 1%) ggc
unaccounted todo      :   0.01 ( 0%) usr   0.03 ( 3%) sys   0.04 ( 0%) wall       0 kB ( 0%) ggc

TOTAL                 :  13.80             1.02            14.89             622726 kB

MELT simple module compilation

This test measures the time to compile a melt module into a .so library. The module tested here is a 180 lines plugin inserting a pass (I will make it available from this page soon).

User Time

0.87

System Time

0.17

MELT do_nothing module compilation

This test measures the time to compile an almost empty melt module into a .so library. This is to check the minimal time taken to compile a simple module.

The file is the following:

(defun really_do_nothing ()
)
(really_do_nothing)

User Time

0.35

System Time

0.03

to do for MELT

Thanks to Pierre Vittet for the nice work.

Probably, the main issue could be that MELT calls ggc_collect too often. I perhaps might fix it with an additional PLUGIN_GGC_START handler which would call MELT minor GC. Done with MELT plugin 0.8

Basile Starynkevitch

None: MELTPerformanceTest (last edited 2011-07-07 06:07:58 by BasileStarynkevitch)