Some RTL pointers are used in very few places.
Look at REG_ORIGINAL for example. It should be easy to replace REG_ORIGINAL with a flag, saving a pointer per created register. Since REGs are not shared, this can be a significant memory saving.
Similar cleanups might be possible for other RTXes.