Re: Inlining, nested functions, and VLAs

[kenner at vlsi1 dot ultra dot nyu dot edu (Richard Kenner)]

    What I had in mind was something different -- modify the inliner
    to notice this specific case and add the V_C_E there.

    We *know* the conversion to be safe at that point; barring bugs
    elsewhere, the types logically ought to match.  It's just this
    phase ordering problem (for lack of a better description) that's
    causing the mismatch.  

Well, detecting this case isn't at all trivial because it's the *second*
inline that has the error.  So we have to know that the call that's being
currently inlined was copied before as part of a previous inlining.  It
seems to me that if we're going to do this flagging, we might as well
flag the outer routine as not being inlinable in the first place.

Did you have any ideas in mind as to how to detect it?

    And adding the V_C_E is definitely safer than rearranging the
    inliner for phase ordering at this point.

I'm not sure what you mean by "rearranging": I think we should just disable
inlining for a function with nested functions whose parameters have
variable size.