modifying the ARM generation behavior?

[Richard Earnshaw]

> Richard:
> > Hm, which compiler release are you using?
> 
> 2.95.2, as stated above.  moving it out of the function body and to the
> function end doesn't change my underlying problem.

Indeed though it does help to explain some of the oddities.

> Richard:
> > Hm, what you are describing is a poition-independent data model (in ARM's
> > ATPCS parlance, RWPI -- read-write position independent), but taken to the
> > extreme that even constants are pushed into the global data tables.
> 
> i'll have to read up about this... and the % efficiency drop you mention.
> i'd like to know if that drop is seen with *all* ARM compilers, or just 
> the ARM compiler.  i note that the ARM compiler does not generate code like
> gcc does ... as is true on all architectures i've dealt with, gcc is a good
> compiler in general, but if you want completely optimized code, you have
> to use the platform specific compiler (intel c, sun devwkshp c, etc).  

I'll go into it further below, but I remain to be convinced you are 
enabling the optimizer.  The ARM compiler very definitely puts constants 
that can't be synthesized into the code segment.  Where possible these 
will be placed between functions, but occasionally, when a function is 
large, then they will get placed at convenient points in the function 
body.  Where possible this will be after a natural branch instruction; but 
very occasionally even this isn't possible and the compiler will have to 
insert a jump around the data table.

> Nick:
> > What happens if there is too much data to fit into the area pointed to
> > by r11 ?  (or whichever register is used).  Since this may only be
> 
> Richard:
> > time.  Further, any moderately large program is going to exceed the 4k
> > offset range of your base register, meaning that you will either need to
> > create one base value per module (= more code at the start of each module
> > to set the base register up) or you will have to compile on the assumption
> > that a single ldr can't load a constant, something like
> 
> you both jumped on this one rather pointedly ;)  it deserves it.  i'm not
> sure how silly the idea is.  but ... to answer ...
> 
> not necessarily.  i won't put it past some people to have a buttload of 
> files in their projects, so the smart decision would be to have the register
> be an indirect index itself.  think of something like the idea that you have
> 1K of 4-byte addresses to "data tables" ... these addresses are in turn
> addresses to function- or file-specific "data tables" ... and then there you
> are.  you have one setup of the register at program start, and then each
> function would have two loads at the top to get the right table into the
> register value ... this doesn't seem much of a penalty to me.  makes more
> overhead in the data segment... but that could be massaged a bit to reduce
> inefficiency.

Again, this is similar to the RWPI (or even the ROPI) shared library model 
which allows for multiple tables; though that model has a more efficient 
way of handling multiple tables that normally avoids more than one 
additional indirection per function.

> 
> Richard:
> > [re: alternate address load model]
> >
> >         add     Rtmp, Rbase, #OFFSET_HIGH(offset)
> >         ldr     Rx, [Rtmp, #OFFSET_LOW(offset)]
> 
> this is sort of how Sparc (and other) systems work. they do have a different
> instruction pattern for it, but it does the two-stage load... and it's very
> easy to catch in the code by a parser like mine.  i know exactly what its
> doing... in the sparc.  because it doesn't use register offset addressing.
> makes me wish very fervently that the ARM had an instruction like the "bl"
> or "b" -- something like "ldrhi r3, <high-imm-16bit>", and the "ldrlo"
> follow-on.  

Yes, but the SPARC can access the full 32-bit address space with that 
model.  On the ARM that still only buys you 20-bits of offset (probably 
enough for most cases from a single pointer, but still 12 bits short of 
the full range).

> Richard:
> > normal use.  You can't use r11 since it is already used, so you would have
> > to use r9 (or for some compilations r8), that would use up 15-20% of the
> > remaining call-saved registers -- that's likely to have a significant
> > effect on the efficiency of the rest of your code, since the compiler will
> > now have to spill more often.
> 
> maybe, i don't think so.  (ignoring which particular rN is being used.)  in
> the test code i've generated (prior to using -ffixed-r8), i've looked through
> the assembly output quite a bit.  i have yet to see (this is working through
> adpcm codecs, mpeg codecs, jpeg codecs, and some custom test apps) anything
> use *more* than r0-r5.  i have never seen an r6, r7, r8 reference *anywhere*.

I'm not convinced you are turning the optimizer on (or you have *very* 
small functions).

> here's a fairly simple function i was testing through the system, and noted
> the behavior on first:
> 
> test.c:
> ---------
> #include <stdio.h>
> 
> extern int s1( int );
> extern int s2( int );
> extern int s3( int );
> 
> extern int g1;
> 
> int debug( void )
> {
>    int g;
> 
>    printf("debugging s1/2/3...\n");
>    for (g=0; g<10; g++)
>       printf("s1(%d) = %d, s2(%d)=%d, s3(%d)=%d\n", g, s1(g), g, s2(g), g, s3(g) );
>    printf("end debug...\n");
> 
>    g1 *= s3(g);
>    printf("g1 is now %d\n", g1);
> 
>    g = s1(10) + s2(20);
>    return g;
> }
> 

I cannot get the compiler to generate the following output that you have:

> here's a snippet of the asm output from gcc ...
> 
>         ldr     r3, .L7
>         ldr     r0, [r3, #0]
>         bl      s1
>         mov     r4, r0
>         ldr     r3, .L7

> sure looks like a lot of "ldr r3, .L7" to me ;)  however, i note that i'm using
> a very long string of flags to gcc, as well as an older version.  

This doesn't make sense for your source code.  The variable passed to s1 
is "g", a local variable.  The only places this can exist are in a 
register, or on the stack.  In no case can it then be referenced by 
looking it up through a constant data pointer -- there's no way the 
compiler could know where on the stack it would be at compile time.  Are 
you sure this example was compiled from your posted code?

>when i went back
> and undid many of the flags, and put the optimization level at O2 (i use O1 at
> present, O2 has some side effects i haven't figured out how to deal with yet)
> i *do* get different output that looks different:
> 
> .LM5:
>         ldr     r0, [sp, #12]
>         bl      s1
>         mov     r4, r0
>         ldr     r0, [sp, #12]
>         bl      s2
>         mov     r5, r0
>         ldr     r0, [sp, #12]
>         bl      s3
>         mov     r3, r0
>         str     r5, [sp, #0]
>         ldr     r2, [sp, #12]

I can get this sort of output if I use -O0 -fomit-frame-pointer, but in no 
other way.

Compiling with ANY level of optimization on gives

.L6:
        mov     r0, r6
        bl      s1
        mov     r5, r0
        mov     r0, r6
        bl      s2
        mov     r4, r0
        mov     r0, r6
        bl      s3

If you've got a long list of flags that are being passed to the compiler, 
please check them carefully to ensure that a flag later on the command 
line isn't turning the optimizer off again.

> [ begin research description : ignore rest of email if uninterested ]
> 
> for a research project we're looking at a different model of CPU design
> that would have *no* caches.  think of a remote sensor device with a 
> very small (~4-8K) on-chip memory footprint and that's it - plus some way
> to interface a sensor array.  (sensor = uninteresting black box here.)
> so essentially all the space that would be cache is now the RAM we have
> available.  we want to dynamically page in/out code and data from this
> space, for our little sensor uP is connected to a backend powerful server
> via some link (serial, IR, ethernet, wireless, whatever).  so the 
> uP will run a little set of "stubs" that will obtain code snippets from
> the server, run them, and intercept ld/st and b/bl situations ... to
> remap the instruction to (a) the proper address if resident; or (b) to
> fetch the proper chunk needed and then remap - this may involve shipping
> code/data back to the server.  the concept is that assuming our memory is
> sufficiently large for our "hot" code (adpcm coding, whatever) that 
> eventually we reach steady-state and no longer talk to the server except
> to send "cooked" sensor data back.

This doesn't really sound any different from a demand-paged virtual memory 
system, except, perhaps that you are trying to change the code directly, 
rather than having additional hardware in the CPU to manage that for you.

> but here, when the server is parsing the code chunk to send to the 
> client, it replaces "bl myfunc" with "bl bl_intercept" ... the intercept
> will do the negotiation with the server.  for me to use finer grain chunks 
> and break up the function (say, on any "b offs" or "bl func") then i need
> to move the code chunks around in memory to be non-contiguous.  the
> problem here is that now i need to have more knowledge of the code 
> and be on the lookout for "ld r3, <some ofs addr in I-space>" and 
> replace that with something like "bl ld_intercept" where i go and
> do all the address work elsewhere for the load.  in all probability,
> i'd wind up exceeding the limited offset range of the ldr instruction
> to just remap it...
> 
> you may think "so what?" - i'm taking a huge performance hit with
> the bl-intercept routine, so what difference does the ld-intercept
> make?  the reason is that as we page code in to us, we self-modify
> those "bl b_intercept" to actually become "bl <new-real-address>" ...
> and when we page code out, we replace any call sites to the now-
> removed code with "bl b_intercept" so we can reload the code as 
> needed.  so in essence, we take the hit once, and then never again,
> when we reach that "hot code" steady state...

Ok, so presumably your server has to remember what the original address 
was when fixing up the bl (so that when executed it can repair the 
damage).  Why can't you extend this to the load/store and replace them 
with something like
	ldr rd, [r0, -r0]

This will always cause a load/store to address zero, and it would be easy 
to make either your memory system or MMU fault such an access.  Then you 
could catch that with a segmentation fault handler and put in the correct 
address before resuming execution.

R.