[2/2] i386 ROP mitigation

[Bernd Schmidt]

This adds a new -mmitigate-rop option to the i386 port. The idea is to 
mitigate against certain forms of attack called "return oriented 
programming" that some of our security folks are concerned about. The 
basic idea is that the stack gets smashed and then, just by chaining 
function returns and some preceding instructions, you can have a 
Turing-complete program to perform an attack. The function returns can 
be either normal, intended ones that are part of the program, or parts 
of the instruction encoding of other sequences.

This patch is a small step towards preventing this kind of attack. I 
have a few more steps queued (not quite ready for stage 1), but 
additional work will be necessary to give reasonable protection. Here, 
I'm only concerned with modr/m bytes, and avoiding certain specific 
opcodes that encode a "return" instruction. Two strategies are 
available: rename entire chains of registers, or insert extra reg-reg 
copies if there is a free scratch register.

The modrm byte computation is not a full one, it is only intended to be 
able to tell whether a value is risky or not.

This was bootstrapped and tested on x86_64-linux. I thought I'd also 
done a full test with -mmitigate-rop forced always on, but a typo 
thwarted that. An earlier set of test results looked reasonable but I 
did not have a baseline to compare against, so I'll be retesting this.


Bernd
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rop1.diff
Type: text/x-patch
Size: 15526 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/gcc-patches/attachments/20151113/b64a3a50/attachment.bin>