Bug 115190 - -fmodule-mapper does not accept CRLF files
Summary: -fmodule-mapper does not accept CRLF files
Status: NEW
Alias: None
Product: gcc
Classification: Unclassified
Component: c++ (show other bugs)
Version: 14.1.0
: P3 normal
Target Milestone: ---
Assignee: Nathaniel Shead
URL:
Keywords:
Depends on:
Blocks: c++-modules
  Show dependency treegraph
 
Reported: 2024-05-22 14:30 UTC by huangqinjin
Modified: 2025-02-02 05:25 UTC (History)
7 users (show)

See Also:
Host:
Target: *-*mingw
Build:
Known to work:
Known to fail:
Last reconfirmed: 2025-02-02 00:00:00


Attachments
potential fix (884 bytes, patch)
2025-02-02 05:25 UTC, Nathaniel Shead
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description huangqinjin 2024-05-22 14:30:38 UTC
In MSYS2/UCRT64 shell,install gcc (version 14.1.0),
 
$ printf '$root .\r\n' > a.modmap
$ g++ -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -E -o /dev/null -x c++ /dev/null
nul: error: failed reading mapper 'a.modmap'


CMake related issue: https://gitlab.kitware.com/cmake/cmake/-/issues/25974
Comment 1 Ben Boeckel 2024-05-22 16:44:36 UTC
My analysis points to the change needing to happen in 1module_resolver::read_tuple_file` in `c++tools/resolver.cc`.
Comment 2 Peter Damianov 2024-05-22 17:11:41 UTC
I could only reproduce this on MSYS2's gcc packages. On w64devkit, I couldn't reproduce it. Nor with a gcc 15 I built myself.

I suspect perhaps it has something to do with whether gcc uses mmap or not? But I'm not certain of this.
Comment 3 huangqinjin 2024-05-23 10:35:23 UTC
(In reply to Peter Damianov from comment #2)
> I could only reproduce this on MSYS2's gcc packages. On w64devkit, I
> couldn't reproduce it. Nor with a gcc 15 I built myself.
> 
Oh yes, I made another test, but the error message is different.

$ printf 'test test.gcm\r\n' > a.modmap
$ echo 'export module test;' > test.cpp
$ g++ -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -c test.cpp
test.cpp:1:8: error: unknown Compiled Module Interface: no such module
    1 | export module test;
      |        ^~~~~~
test.cpp:1:8: warning: not writing module 'test' due to errors
Comment 4 huangqinjin 2024-05-23 10:36:05 UTC
The repro in description shows that MSYS2 gcc read the module mapper early even it is not used, seems to me it doesn't match the document https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Module-Mapper.html

> The mapper is connected to or loaded lazily,
> when the first module mapping is required.
Comment 5 huangqinjin 2024-05-23 10:40:18 UTC
(In reply to huangqinjin from comment #3)
> (In reply to Peter Damianov from comment #2)
> > I could only reproduce this on MSYS2's gcc packages. On w64devkit, I
> > couldn't reproduce it. Nor with a gcc 15 I built myself.
> > 
> Oh yes, I made another test, but the error message is different.
> 
> $ printf 'test test.gcm\r\n' > a.modmap
> $ echo 'export module test;' > test.cpp
> $ g++ -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -c test.cpp
> test.cpp:1:8: error: unknown Compiled Module Interface: no such module
>     1 | export module test;
>       |        ^~~~~~
> test.cpp:1:8: warning: not writing module 'test' due to errors

The line ending of last line is also required. Personally feel strange.

$ printf 'test test.gcm' > a.modmap
$ echo 'export module test;' > test.cpp
$ g++ -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -c test.cpp
test.cpp:1:8: error: unknown Compiled Module Interface: no such module
    1 | export module test;
      |        ^~~~~~
test.cpp:1:8: warning: not writing module 'test' due to errors
Comment 6 Ben Boeckel 2024-05-23 11:29:20 UTC
> The line ending of last line is also required. Personally feel strange.

This is explicitly handled (as a "no, not supported" case): https://github.com/gcc-mirror/gcc/blob/4efa7ec85a85c6024d0907a0e735ad5df7fca952/c%2B%2Btools/resolver.cc#L143
Comment 7 Christoph Reiter 2024-05-23 19:11:03 UTC
If the problem is indeed in read_tuple_file(), then I see one potential cause there:

The file is opened in text mode and it checks if the read count matches the size of the actual data read. But in text mode where newline normalization happens the read part can be smaller if there are CRLF in there that got converted to LF:

https://github.com/gcc-mirror/gcc/blob/0b3b6a8df77b0ae15078402ea5fb933d6fccd585/c%2B%2Btools/resolver.cc#L129-L130
Comment 8 Peter Damianov 2024-05-24 13:21:43 UTC
I have been totally unable to reproduce this outside of MSYS2, with any compilers I built myself. The "MSYS2 MSYS" shell also doesn't have this problem.

I can't explain why. I tried investigating, but didn't have success.

Peter@DESKTOP-1SHN9FA UCRT64 ~
$  g++ -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -E -o /dev/null -x c++ /dev/null
nul: error: failed reading mapper 'a.modmap'

Peter@DESKTOP-1SHN9FA UCRT64 ~
$ /c/Users/Peter/3D\ Objects/mydevkit/bin/g++ -B /c/Users/Peter/3D\ Objects/mydevkit/bin/  -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -E -o /dev/null -x c++ /dev/null

Peter@DESKTOP-1SHN9FA UCRT64 ~
$ g++ -B /c/Users/Peter/3D\ Objects/mydevkit/bin/  -std=c++20 -fmodules-ts -fmodule-mapper=a.modmap -E -o /dev/null -x c++ /dev/null
nul: error: failed reading mapper 'a.modmap'
Comment 9 Nathaniel Shead 2025-02-02 05:25:11 UTC
Created attachment 60351 [details]
potential fix

Confirmed.  There's actually two separate errors here depending on whether `MAPPED_READING` is defined.

When using mmap, any '\r' are treated as normal characters, and so e.g. a modmap with 'test test.gcm\r\n' would (attempt) to output to a module file 'test.gcm\r'; this would work fine on Linux (or WSL), but Windows forbids control characters in filenames.

When not using mmap, since the file descriptor is opened in text mode (in module_client::open_module_client), as noted in comment #7 the size of the read may not match the size of the file.

In general I'm not convinced that mmap is super appropriate for this case where we're reading a file in once, start-to-finish.  So maybe the way to go is to remove the mmap case to reduce complexity and fix the assumption on file size, like the attached.

But maybe it'd even be worth handling '\r' in module mapper files on Linux (e.g. if copied from Windows?)