cpp: Add UTF-16/UTF-32 encoding auto-detection in C preprocessor

[Joseph Myers]

On Thu, 27 Nov 2025, katahiromz wrote:

> Hello, I'm katahiromz. Thank you for your great software.
> I want to add UTF-16/UTF-32 support to your C preprocessor.
> 
> This patch (attached) might add automatic character encoding detection
> to `libcpp/files.cc` by examining the first 4 bytes of input files.
> I hope this patch helps.

It's not conventional for Unix-like utilities accepting text files to do 
this sort of thing; text files are expected to be in the locale's 
encoding, or at least in some ASCII-compatible encoding; UTF-16 and UTF-32 
are binary files, and in practice the world utilities such as GCC operate 
in has settled on UTF-8 as the standard Unicode encoding for input and 
output.  Furthermore, the existing logic is documented in cppopts.texi, 
which this patch doesn't change.

-- 
Joseph S. Myers
josmyers@redhat.com