[PATCH] math: Refactor how to use libm-test-ulps

[Adhemerval Zanella Netto]

On 27/01/25 18:47, Joseph Myers wrote:
> On Mon, 27 Jan 2025, Adhemerval Zanella wrote:
> 
>> Regressions are only reported if the implementation shows larger errors
>> than 9 ulps (13 for IBM long double) unless it is overridden by
>> libm-test-ulps and the maximum error is not printed at the end of tests.
>> The regen-ulps rule is also removed since it does not make sense to
>> update the libm-test-ulps automatically.
> 
> I think the tests *should* report the maximum observed error for each 
> function, even when it's not causing the tests to fail, to make it easy to 
> see which functions have the largest errors.  (For example, say we wished 
> to reduce the maximum error accepted from 9 ulps to something smaller.  
> It's desirable to have a quick way to check, by running the tests for a 
> target, which functions would need improving.)

Oops, I meant that 'maximum error is *now* printed at the end of tests'.
I forgot to handle the complex value, I will add them on v2.

The output should be:

$ math/test-double-cos
testing double (without inline functions)

Test suite completed:
  Maximum error found of `1' ulp
  4 max error test cases,
  440 input tests,
  - with 1768 tests for exception flags,
  - with 432 tests for errno executed.
  All tests passed successfully.

$ math/test-double-ccos
testing double (without inline functions)

Test suite completed:
  Maximum error found of `1' ulp for real part and `1' ulp for imaginary part
  4 max error test cases,
  576 input tests,
  - with 2280 tests for exception flags,
  - with 108 tests for errno executed.
  All tests passed successfully.

> 
>> +math testsuite only flag results larger than 9ulp (or 16 for IBM
> 
> flag -> flags

Ack.

> 
>> +@code{long double} format) as errors; although most of the implementation
>> +shows errors smalls than the limit.
> 
> "implementation shows errors smalls" -> "implementations show errors 
> smaller"

Ack.

> 
>> +A more comprehensive analysis of @theglibc math function precision could
>> +be found in 'Accuracy of Mathematical Functions in Single, Double, Double
>> +Extended, and Quadruple Precision'; Brian Gladman, Vincenzo Innocente,
>> +John Mather, and Paul Zimmermann at
>> +@url{https://members.loria.fr/PZimmermann/papers/accuracy.pdf}
> 
> Note that this reference does not cover complex functions, or jn/yn, or 
> any architecture variations (in particular IBM long double is out of 
> scope).
> 
> (The known cases of very large errors are cpow, jn and yn for all formats, 
> and j0/j1/y0/y1 except for binary32.  The previous table in the manual 
> didn't deal with those cases of very large errors since they didn't go in 
> the testsuite.)

Indeed, although I am not sure if the current table based on the limited
coverage shows any real indication of the expected precision for the math
functions (even with the current caveats in the manual description). It 
sounds more a reason for development of glibc itself, than an indication of
user expectations.

Paul Zimmermann has added multiple inputs that shows worse ULPs cases that 
the previous advertised, so the information that we currently present is 
really limited.

I will add a comment with your remarks.

> 
>> +A "libm-test-ulps" is required only to test for extra contraints in
> 
> contraints -> constraints

Ack.

> 
>> +The keywords are float, double, ldouble, and float128.
> 
> I think it would be better for the keywords to be based on the format, not 
> the type.  (In particular, if in future we have correctly rounded 
> binary128 implementations, of rsqrt or cbrt or hypot, say, it would be 
> desirable to have a single libm-test-ulps file describing those, rather 
> than needing separate files for when those implementations are used for 
> long double and when they are used for _Float128.  Though since those 
> cases get the implementations via different sysdeps directories, our 
> sysdeps system isn't really designed for this; the file would need to go 
> somewhere more generic such as sysdeps/ieee754/.)

Are you suggesting changing to something line binary32, binary64, binary80,
binary128, and ibmlonglong? If so it makes sense; although I think it would
be a different and more extensive patch.

I have extended the keyword description mostly to adequate to current usage.

At least with current scheme, it would indeed require at least two files
with same contents (one for sysdeps/ieee754/ldbl-128 and another for
sysdeps/ieee754/float128), which is not ideal but also way less complex
than a per-architecture file.

> 
> (Really the rounding mode shouldn't be treated as part of the function 
> name either, but that might be a more complicated change.)

Agree, I might work on this in the future.

> 
>> +Also, muultiple "libm-test-ulps" can be added, "gen-libm-test.py" will
> 
> muultiple -> multiple
> 

Ack.