Bug 87096 - "Optimised" snprintf is not POSIX conformant
Summary: "Optimised" snprintf is not POSIX conformant
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: tree-optimization (show other bugs)
Version: 6.4.0
: P3 normal
Target Milestone: ---
Assignee: Martin Sebor
URL: http://pubs.opengroup.org/onlinepubs/...
Keywords: patch, wrong-code
: 87111 (view as bug list)
Depends on:
Blocks:
 
Reported: 2018-08-24 18:39 UTC by A. Wilcox (awilfox)
Modified: 2018-12-14 22:39 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail: 4.7.4, 5.3.0, 6.4.0, 7.3.0, 8.1.0, 9.0
Last reconfirmed: 2018-08-24 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description A. Wilcox (awilfox) 2018-08-24 18:39:39 UTC
POSIX Issue 7, 2018 edition (see URL)

The snprintf() function shall fail if:

[EOVERFLOW]
    The value of n is greater than {INT_MAX}.


This behaviour may or may not be ISO C compatible: C99 WG14 and C11 specifically state that intending the printf family of functions to return larger than INT_MAX is undefined behaviour: J.2.  I am unclear on the status of this behaviour with regards to C89.  It is definitely required by POSIX.

The following line of C, which is the minimum reproduction case:

snprintf(buf, (size_t)INT_MAX + 1, "");

returns 0 when compiled with GCC unless -fno-builtin is passed.
Comment 1 Martin Sebor 2018-08-24 19:23:57 UTC
Confirmed.  This goes back to the original folding of snprintf to memcpy.

$ cat f.c && gcc -O2 -S -Wall -fdump-tree-optimized=/dev/stdout f.c
int f (char *d)
{
  return __builtin_snprintf (d, __INT_MAX__ + 1LLU, "");
}
f.c: In function ‘f’:
f.c:3:53: warning: zero-length gnu_printf format string [-Wformat-zero-length]
   return __builtin_snprintf (d, __INT_MAX__ + 1LLU, "");
                                                     ^~

;; Function f (f, funcdef_no=0, decl_uid=1756, cgraph_uid=0, symbol_order=0)

f (char * d)
{
  <bb 2>:
  __builtin_memcpy (d_2(D), "", 1);
  return 0;
}


GCC 8 warns on this case but still folds it into memcpy and return 0.

f.c:3:10: warning: specified bound 2147483648 exceeds ‘INT_MAX’ [-Wformat-truncation=]
   return __builtin_snprintf (d, __INT_MAX__ + 1LLU, "");
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Comment 2 Andrew Pinski 2018-08-27 05:24:04 UTC
*** Bug 87111 has been marked as a duplicate of this bug. ***
Comment 3 Richard Biener 2018-08-27 06:56:05 UTC
I don't think we "preserve" exceptional behavior consistently.  That is, we happily change code with exceptional behavior to code without if the main computation result is the same.
Comment 4 Rich Felker 2018-08-27 13:47:25 UTC
I'm aware, but that makes it an invalid transformation. To be valid, the transformation must only be performed in cases where it's provably not exceptional.

The same principle applies to all the pathological results where p=malloc(SIZE_MAX);if(!p)...;free(p) is getting optimized out (breaking code that relies on "if malloc succeeds, pointer arithmetic in the allocated object is valid"). This kind of error is a high-level optimization strategy mistake in gcc that needs to be addressed and fixed.
Comment 5 Martin Sebor 2018-12-12 17:50:33 UTC
I find the POSIX requirement to fail when n is greater than INT_MAX to be in conflict with C.  I've submitted a defect to the Austin Group:
  http://austingroupbugs.net/view.php?id=1219
At the same time, the POSIX requirement makes sense to me since it makes the function safer.  So I've also brought this up in WG14 to see if C would be willing to adopt the POSIX requirement.
Comment 6 Rich Felker 2018-12-12 17:58:53 UTC
I don't see how the POSIX requirement makes the function safer. On the contrary, it makes it less safe by introducing failure cases (that an application might fail to check for, assuming it knows it has a good implementation, with no spurious failures) for calls that should no fail. For example:

char foo[3];
snprintf(buf, size, "%d", 42);
strcpy(foo, buf);

Assuming snprintf succeeds, the strcpy is safe. If snprintf spuriously fails, buf contains whatever it previously held, possibly uninitialized data, and the strcpy produces dangerous undefined behavior/overflows.

This is a stupid constructed example, but there are lots of cases where an application might not check the result of snprintf because it's happy with truncation and because the format string lacks anything that could fail (like wchar_t conversions which can fail from EILSEQ), but where it would not be happy/safe with uninitialized or stale data.
Comment 7 jsm-csl@polyomino.org.uk 2018-12-12 18:03:30 UTC
On Wed, 12 Dec 2018, msebor at gcc dot gnu.org wrote:

> I find the POSIX requirement to fail when n is greater than INT_MAX to be in
> conflict with C.  I've submitted a defect to the Austin Group:
>   http://austingroupbugs.net/view.php?id=1219

Should note this is a resubmission of: 
http://austingroupbugs.net/view.php?id=761
Comment 8 Martin Sebor 2018-12-12 19:46:28 UTC
The POSIX requirement prevents buffer overflow when the size of the destination is incorrectly computed.  I realize it's common practice to ignore snprintf  return value, but defensively written code should check it.  The return value can safely be ignored only when the function can neither fail nor truncate (in the latter case sprintf is just as safe).  Otherwise, the return value should be tested and either the failure or the more likely truncation should be handled somehow.  The -Wstringop-truncation warning is based on this premise.

GCC can mitigate some of the buffer overflow cases when it can determine the size of the destination on its own, but that's only possible in a subset of cases.

That said, I'm not sure how to proceed here.  I see three ways forward:

1) disable the folding in this case and call the library function
2) suspend this until C/POSIX have resolved the conflict
3) fold the call to -1 and set errno to EOVERFLOW

I have a trivial patch to do (1) but my testing shows that while Solaris 11 implements the POSIX requirement AIX and Glibc don't, so it won't solve the conformance/portability problem.  (2) is the easy way out for now, until C and POSIX have either converged or decided not to.  (3) is out of scope for GCC 9.

From your comments, Rich, it's not clear to me what you are arguing for.  It sounds like you don't agree with the POSIX requirement but also disapprove of the GCC optimization.
Comment 9 Martin Sebor 2018-12-12 19:50:48 UTC
(In reply to joseph@codesourcery.com from comment #7)

Thanks, that has some useful background.
Comment 10 Rich Felker 2018-12-12 19:58:05 UTC
Indeed, I think the POSIX requirement is both in conflict with the requirements of the C language and a bad requirement in itself.

As for what GCC should do, since there room for debate about which behavior is correct, and library implementations might even continue to disagree/diverge from one or both of the standards for some time, I do not think GCC should be overruling the choices made by the library implementation here. The folding for this case should either happen only for constant size arguments <=INT_MAX, or should produce a runtime check and force a function call for the case where it's >INT_MAX.
Comment 11 Martin Sebor 2018-12-12 23:30:50 UTC
GCC 9 patch: https://gcc.gnu.org/ml/gcc-patches/2018-12/msg00886.html
Comment 12 Martin Sebor 2018-12-14 22:38:40 UTC
Author: msebor
Date: Fri Dec 14 22:38:08 2018
New Revision: 267157

URL: https://gcc.gnu.org/viewcvs?rev=267157&root=gcc&view=rev
Log:
PR tree-optimization/87096 - Optimised snprintf is not POSIX conformant

gcc/ChangeLog:

	PR rtl-optimization/87096
	* gimple-ssa-sprintf.c (sprintf_dom_walker::handle_gimple_call): Avoid
	folding calls whose bound may exceed INT_MAX.  Diagnose bound ranges
	that exceed the limit.

gcc/testsuite/ChangeLog:

	PR tree-optimization/87096
	* gcc.dg/tree-ssa/builtin-snprintf-4.c: New test.


Added:
    trunk/gcc/testsuite/gcc.dg/tree-ssa/builtin-snprintf-4.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/gimple-ssa-sprintf.c
    trunk/gcc/testsuite/ChangeLog
Comment 13 Martin Sebor 2018-12-14 22:39:09 UTC
Fixed for GCC 9 in r267157.