Re: [tree-ssa] DCE with control dependence again (with numbers, for a change)

[law at redhat dot com]

In message <200401101831.59589.s.bosscher@student.tudelft.nl>, Steven Bosscher 
writes:
 >On Friday 09 January 2004 18:26, Steven Bosscher wrote:
 >> Hi Jeff, all,
 >> I cannot give you benchmark performance numbers right now, I'm still
 >> trying to get SPEC numbers with Andreas Jaeger.
 >
 >And the SPEC CINT2000 numbers are...
 >
 >Base Compiler: GCC Tree-SSA as of 2003-01-09
 >Peak Compiler: base+patch.DCE_control_dep
 >cflags base: -O2
 >cflags peak: -O2
 >Iterations: 3
 >
 >Size of binaries:
 >Total: Base: 5064657 bytes
 >Total: Peak: 5064369 bytes
 >gcc -192 bytes, crafty -64 bytes, gap -64 bytes;
 >eon +32 bytes.
 >rest is unchanged.
 >
 >Not significant, I think.
Odd (particularly the gcc number).  Any thought on why we're not seeing
as much improvement in the codesize with your spec2000 test for gcc
compared to the earlier tests using the components of cc1?

The data above is more along the lines of what I expected based on my
earlier work control dependence based DCE.



 >Compile times for benchmarks:
 >164.gzip base: 3 s
 >164.gzip peak: 3 s
 >175.vpr base: 8 s
 >175.vpr peak: 8 s
 >176.gcc base: 92 s
 >176.gcc peak: 91 s
 >181.mcf base: 1 s
 >181.mcf peak: 1 s
 >186.crafty base: 14 s
 >186.crafty peak: 14 s
 >197.parser base: 6 s
 >197.parser peak: 6 s
 >252.eon base: 174 s
 >252.eon peak: 173 s
 >253.perlbmk base: 46 s
 >253.perlbmk peak: 45 s
 >254.gap base: 30 s
 >254.gap peak: 30 s
 >255.vortex base: 28 s
 >255.vortex peak: 28 s
 >256.bzip2 base: 2 s
 >256.bzip2 peak: 2 s
 >300.twolf base: 17 s
 >300.twolf peak: 16 s
 >======================================
 >Total time for base compilation: 421 s
 >Total time for peak compilation: 417 s
Presumably this was a single compile of the testsuite.  It would be good
to have multiple runs -- we're looking at a 1% improvement across a single
run, but we don't have any idea if that is significant or not since we
don't know how much compilation time is varying from one run to the next.

 >So with the patch, GCC is never slower, and sometimes faster.
Possibly.  Again, I'd prefer to see a couple more compile timings to
make that determination.

[ ... ]
 >In summary: vrp is better,
I'm not sure you can really make that statement.  If we look at the vpr results
we have:

   175.vpr           1400  196             714*     1400  195             717*
   175.vpr           1400  196             714      1400  196             716 
   175.vpr           1400  194             720      1400  195             719 

You'll see the base ranges for 714-720 and the peak ranges from 716-719.
The base range entirely subsumes the peak range.  With only 3 runs I don't
think you can make the claim that the change is statistically significant
given the variance in the results.

The "improvement" could easily be run-to-run noise.  In fact, the absolute
first thing you should do is actually verify that the base & beak binaries
for vpr actually have different code!   I've seen this kind of variation
using the same binary.  





 > crafty and parser drop a point but not
 >significantly judging from the three runs. twolf up one point, but
 >also not significantly. 
Agreed.

 >perlmbk droms two points, I'll have a look at that when I get the
 >opportunity. 
Again, the first thing is to actually verify that the binaries are
in fact different.  I don't know offhand if the change is statistically
significant or not -- though it's the one that is most likely to be
significant IMHO.

 >Overall, unchanged.
Definitely agreed.


 >Other than that, the compiler is slightly faster,
Possibly.  The data from testing 8361 and gcc itself tends to make me
think this statement might be true.  Having a couple more spec runs
would help solidify that.


 > and produces a bit smaller binaries.
True.  Though I'd like a little more analysis on why we saw so much better
improvements for your tests of 8361 and gcc vs such small improvements on
spec.

 >I think I can claim from these results and from the results that:
 >- The patch will not cause performance problems, in fact it will
 >  make the compiler a bit faster overall.
Agree with the former.  Unsure on the latter.

 >- The effect on the performance of the resulting code is not very
 >  significant overall.
Agreed.

 >- Binary sizes are typically smaller, especially for C++ code.
Yes, though there's some questions remaining about how much we really
gain by this code.

Jeff