haifa-sched: Avoid overflows in extend_h_i_d [PR112411]
On Thu, Dec 07, 2023 at 09:36:23AM +0100, Jakub Jelinek wrote:
> Without the dg-skip-if I got on 64-bit host with
> -O3 --param min-nondebug-insn-uid=0x40000000:
> cc1: out of memory allocating
571230784744 bytes after a total of
2772992 bytes
I've looked at this and the problem is in haifa-sched.cc:
9047 h_i_d.safe_grow_cleared (3 * get_max_uid () / 2, true);
get_max_uid () is 0x4000024d with the --param min-nondebug-insn-uid=0x40000000
and so 3 * get_max_uid () / 2 actually overflows to -
536870028 but as vec.h
then treats the value as unsigned, it attempts to allocate
0xe0000374U * 152UL bytes, i.e. those 532GB. If the above is fixed to do
3U * get_max_uid () / 2 instead, it will get slightly better and will only
need 0x60000373U * 152UL bytes, i.e. 228GB.
Perhaps more could be helped by making the vector indirect (contain pointers
to haifa_insn_data_def rather than the structures themselves) and pool allocate
those, but the more important question is how sparse are uids in normal
compilations without those large --param min-nondebug-insn-uid= parameters.
Because if they aren't enough, such a change would increase compile time memory
just to help the unusual case.
2023-12-08 Jakub Jelinek <jakub@redhat.com>
PR middle-end/112411
* haifa-sched.cc (extend_h_i_d): Use 3U instead of 3 in
3 * get_max_uid () / 2 calculation.