Optimization Bug in g77

["Steven E. Williamson" <sew at uiuc dot edu>]

I have run into a strange problem with g77 when run with the -O or -O1
option but NOT with -O2, -O3, or no optimization.  Below is piece
of Fortran code (removed from a larger program) which illustrates the
problem.  The basic function of the code is to go through a list
of data (the RAW array) and accumulate an average and uncertainty
in the average, excluding "outlier points".  This is done iteratively
by calculating an average and standard deviation using only points
that differ by less than some number (NSIG) of standard deviations
from the average calculated by the previous iteration (in the first
iteration all points are accumulated).  The iteration is repeted until
the average does not change between two iterations or until some
maximum number of iterations is reached.  In the program below, some
fake data is first generated.  For technical reasons, the number of
"rejected points" and the number of iterations (normally integers) are
accumulated in double precision reals.

The problem with this code is that when compiled with either the -O or -O1
options, the maximum number of iterations is performed.  When compiled
with -O2, -O3, or no optimization options, only 2 iterations are done.
Some how the check for the averages of two successive iterations is
being disabled when -O or -O1 is selected.

Two other symptoms that I have noticed: if the -ffloat-store option is
used, then the program always works correctly (does only 2 iterations).
And, if the write statement immediately preceding the check for
equal averages is un-commented, the program also works correctly.

-----------Program optimization-test.F CUT HERE--------------------------
C	This program produces two different results depending on
C	the g77 optimization.
	IMPLICIT NONE
C	Maximum number of iterations of the filtering process
	INTEGER NITRMX
	PARAMETER (NITRMX = 50)
C	Number of SIGMA outside of which data is rejected.
	INTEGER NSIG
	PARAMETER (NSIG = 40)
C	Number of data points
	INTEGER NDAT
	PARAMETER (NDAT = 200)
C
	INTEGER I
	REAL RAW(200)
	DOUBLE PRECISION AV,SD,AVOLD,SDOLD,FIL(2),SUM,SUM2
C
	FIL(2) = 1.D0
	SDOLD = 1.D37
	AVOLD = 0.D0
C	Generate some fake data (Gaussian tail with flat-top)
	DO 100 I = 1,NDAT
	    RAW(I) = EXP(-((I-100.)/10.)**2)
100	CONTINUE
	DO 110 I = NDAT/2-20,NDAT/2+20
	    RAW(I) = 1
110	CONTINUE
C
C	Scan data to calculate mean and uncertainty.  Reject
C	any data that is NSIG SIGMA from the mean (unless this
C	is the first iteration).
10	SUM = 0.D0
	SUM2 = 0.D0
	FIL(1) = 0.D0
	DO 70 I = 1,NDAT
	    IF (FIL(2).LE.1.D0.OR.ABS(RAW(I)-AVOLD).LE.NSIG*SDOLD) THEN
	        SUM = SUM+RAW(I)
	        SUM2 = SUM2+RAW(I)**2
	    ELSE
	        FIL(1) = FIL(1)+1.D0
	    ENDIF
70	CONTINUE
C	Calculate a new mean and SIGMA.
	AV = SUM/(NDAT-FIL(1))
C	This is the standard deviation of the mean (which is not the
C	same as the standard deviation of the measurement distribution,
C	i.e. the sample standard deviation.).
	SD = SQRT(SUM2/(NDAT-FIL(1))-AV**2)/SQRT(NDAT-1-FIL(1))
c	write (6,*) av
	IF (FIL(2).LE.1.D0.OR.AV.NE.AVOLD) THEN
	    IF (FIL(2).LT.NITRMX) THEN
C	        If this is the first iteration, or if the process
C	        hasn't converged, go back and re-scan.
	        FIL(2) = FIL(2)+1.D0
	        AVOLD = AV
	        SDOLD = SD
	        GO TO 10
	    ENDIF
	ENDIF
	write (6,*) fil(2),av,fil(1)
	if (fil(2).eq.nitrmx) then
	    write (6,*) 'Wrong answer!'
	else
	    write (6,*) 'Right answer.'
	endif
	STOP 'All done.'
	END
------------------------------------------------------------------------

Here is the Makefile that I use to genererate the executable of the
above program.  It assumes that the above Fortran is in a file
called optimization-test.F.  A number of different choices for
the FFLAGS (most commented out) are divided into a group that doesn't
work and a group that works.

-----------------------Makefile CUT HERE--------------------------------
# This is a make file for optimization-test.  The following commands are
# defined:
# make			Create program on source directory (don't install
#                       in destination)
# make all			"
# make program			"
# make clean		Remove object files from the source directory
# make install		Move the program from the source directory to the
#                       destination
# make update		Combines program creation with installation in
#                       destination directory

# Final result of this make file
PROGRAM=optimization-test

OBJS=optimization-test.o

# Source file names (syntax is the same as SRCS=$(patsubst %.o,%.F,$(OBJS)))
SRCS=$(OBJS:%.o=%.F)

# Place to store the final result (no last /)
DEST=$(HOME)/bin/$(OSNAME)

# Name of this file
MAKEFILE=Makefile

# Library directories 
LIBDIR=

# Libraries to link to (other operating system specific libraries are defined
# by OSLIBS).
LIBS=

# Fortran command
FC=g77

# Fortran (g77) flags for Linux
# -fcase-lower		Map to lower case (case insensitive)
#			(note that most libraries are compiled
#			with lower case symbols).
# -O			Optimize the code -- same as O1 (could be O2 for even
#			beter optimization, or O3 for the best optimization)
# -Wall			Warn when variables are unused (except
#			for declaration) and when uninitialized
# 			variables are used. 
# The following options produce a program that does NOT work
# FFLAGS=-W -Wall -fcase-lower -O
# FFLAGS=-W -Wall -fcase-lower -O1
#
# The following options produce a program that works:
# FFLAGS=-W -Wall -fcase-lower -O2
# FFLAGS=-W -Wall -fcase-lower -O3
# FFLAGS=-W -Wall -fcase-lower -ffloat-store -O1
# FFLAGS=-W -Wall -fcase-lower -ffloat-store -O
# FFLAGS=-W -Wall -fcase-lower -ffloat-store
# FFLAGS=-W -Wall -fcase-lower -ffloat-store -O2
# FFLAGS=-W -Wall -fcase-lower -ffloat-store -O3
FFLAGS=-W -Wall -fcase-lower

# Loader options for Linux
# -L <directory>        Look in <directory> for libraries
LDFLAGS=

# Libraries required on this operating system
OSLIBS=

# "make all" and "make program" are the same as "make"
all:		$(PROGRAM)
program:        $(PROGRAM)
$(PROGRAM):     $(OBJS)
		$(FC) $(LDFLAGS) $(OBJS) $(LIBS) $(OSLIBS) -o $(PROGRAM)

# "make install" transfers the program to the destination directory
install:	$(PROGRAM)
		mv $(PROGRAM) $(DEST)

# "make update" combines the creation of the program with its installation.
update:		$(DEST)/$(PROGRAM)
$(DEST)/$(PROGRAM): $(SRCS)
		@make -f $(MAKEFILE) DEST=$(DEST) install

# "make clean" removes all old object files.  This uses a "phony target"
# (ie target without dependencies)
.PHONY:		clean
clean:
		rm -f $(OBJS)

Here is the result of doing g77 -v on my system:


------------------------------------------------------------------------
>g77 -v
g77 version 2.95.4 20011002 (Debian prerelease) (from FSF-g77 version
0.5.25 20010319 (prerelease))
Driving: g77 -v -c -xf77-version /dev/null -xnone
Reading specs from /usr/lib/gcc-lib/i386-linux/2.95.4/specs
gcc version 2.95.4 20011002 (Debian prerelease)
 /usr/lib/gcc-lib/i386-linux/2.95.4/cpp0 -lang-c -v -D__GNUC__=2
-D__GNUC_MINOR__=95 -D__ELF__ -D__unix__ -D__i386__ -D__linux__ -D__unix
-D__linux -Asystem(posix) -D_LANGUAGE_FORTRAN -traditional
-Acpu(i386) -Amachine(i386) -Di386 -D__i386 -D__i386__ /dev/null /dev/null
GNU CPP version 2.95.4 20011002 (Debian prerelease) (i386 Linux/ELF)
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/gcc-lib/i386-linux/2.95.4/include
 /usr/include
End of search list.
The following default directories have been omitted from the search path:
 /usr/lib/gcc-lib/i386-linux/2.95.4/../../../../include/g++-3
 /usr/lib/gcc-lib/i386-linux/2.95.4/../../../../i386-linux/include
End of omitted list.
 /usr/lib/gcc-lib/i386-linux/2.95.4/f771 -fnull-version -quiet -dumpbase
g77-version.f -version -fversion -o /tmp/ccCss7xb.s /dev/null
GNU F77 version 2.95.4 20011002 (Debian prerelease) (i386-linux) compiled
by GNU C version 2.95.4 20011002 (Debian prerelease).
GNU Fortran Front End version 0.5.25 20010319 (prerelease)
 as -V -Qy -o /tmp/ccOS8gud.o /tmp/ccCss7xb.s
GNU assembler version 2.12.90.0.1 (i386-linux) using BFD version
2.12.90.0.1 20020307 Debian/GNU Linux
 ld -m elf_i386 -dynamic-linker /lib/ld-linux.so.2 -o /tmp/cc8iVYBd
/tmp/ccOS8gud.o /usr/lib/crt1.o /usr/lib/crti.o
/usr/lib/gcc-lib/i386-linux/2.95.4/crtbegin.o
-L/usr/lib/gcc-lib/i386-linux/2.95.4 -lg2c -lm -lgcc -lc -lgcc
/usr/lib/gcc-lib/i386-linux/2.95.4/crtend.o /usr/lib/crtn.o
 /tmp/cc8iVYBd
__G77_LIBF77_VERSION__: 0.5.25 20010319 (prerelease)
@(#)LIBF77 VERSION 19990503
__G77_LIBI77_VERSION__: 0.5.25 20010319 (prerelease)
@(#) LIBI77 VERSION pjw,dmg-mods 19990503
__G77_LIBU77_VERSION__: 0.5.25 20010319 (prerelease)
@(#) LIBU77 VERSION 19980709
-----------------------------------------------------------------------

Obvously I have a work-around to make my code do what I want it to do,
but this makes me worry that there are other programs, compiled with
-O, that are not really doing what they are supposed to do.

Thank you for any help (patches, suggestions).  Let me know if I can
supply more information.

				Steve Williamson
				Physics Dept.
				University of Illinois