Memory model release/acquire mode interactions of relaxed atomic operations

[Andrew Haley]

On 05/05/17 10:09, Toebs Douglass wrote:
> On 05/05/17 10:48, Andrew Haley wrote:
>> On 04/05/17 21:18, Toebs Douglass wrote:
> 
>>> The problem I have in mind is store buffers and that store barriers do
>>> not cause stores to complete. 
>>
>> StoreLoad barriers do indeed flush the store buffer.  That's their
>> job.  (Or at least they must act as though they do, according to the
>> synchronization rules.  I don't know of any processor on which
>> StoreLoad doesn't flush the store buffer, but I suppose it's possible
>> that somebody could invent another way to do it.)
> 
> I think StoreLoad is a particular ARM barrier, right?

No, it's standard barrier terminology.

> So this is I think a point where we think different things.
> 
> I would qualify my view to say this : I can't say in what may or may not
> happen on any particular architecture as I simply don't know details
> about most implementations, but, although I may be wrong, I do think
> that in general memory barriers do not cause stores to complete.  I mean
> to say this in the sense of writing cross-platform code which assumes
> the minimum guarantees provided across platforms.

You're using the word "complete" without explaining what you really
mean by it.  I think you mean that a store becomes visible to other
processors, so that's how I'll treat it.

>>> The processor performing the store will think it has issued the
>>> store and see the world accordingly *prior even to the store
>>> reaching the first level cache*, and there is no guarantee about how
>>> long this state of affairs persists.
>>>
>>> So if we perform a store and then a store barrier,
>>
>> What exactly do you mean by "a store barrier"?
> 
> A barrier issued to the processor such that all stores prior to that
> barrier must complete before any store after the barrier completes.

Right, that's a StoreStore, i.e. it only orders stores and other stores.

>>> we have nothing - there is no guarantee any other core has seen this
>>> store or ever will.  We only have a guarantee that IF a store
>>> *after* the store barrier is going to complete, all stores prior to
>>> the barrier will be forced to actually complete, so that they
>>> complete first (and thus honour the store barrier).
>>
>> We don't know how long it will take for loads an stores to propagate.
>> But if Processor A does a store with some kind of synchronization
>> operation and then Processor B then performs an action which is
>> synchronized after Processor A's store, then I can absolutely
>> guarantee that Processor B has seen Processor A's store.
> 
> Right - but all here hinges upon what that synchronization action is.
> By definition here, it does the job and therefore everything is fine.

It doesn't matter what the synchronization action is, as long as it is
ordered.

>>> In other words, all stores which do not use LL/SC or LOCK (or equivalent
>>> thereof) can in effect never occur.
>>
>> There's nothing special about LL/SC, compare-and-exchange, or
>> whatever.  It has its part in the synchronization order, just like
>> anything else.
> 
> Where I think store barriers do not cause stores to complete, LL/SC
> / LOCK is important, because they force a store to complete (the
> atomic operation) and this in turn forces previously issued store
> barriers to be honoured (i.e. that the stores before those barriers
> must now complete, before the LL/SC / LOCK store completes).

That's because LL/SC or LOCK store often have implicit StoreLoad
barriers associated with them.  That's true on x86, for example.  But
it's a dangerous way to think, because they don't always: on ARMv8,
for example, it's optional.  But this is the crux of the matter: it's
the StoreLoad barrier that causes all of the previous stores to become
visible, not the LL/SC or LOCK store.

> So for me this is how I guarantee other cores will be able to see
> stores (after those other cores issue a load barrier, of course).
> Where I think a store barrier does not cause stores to complete, I
> need a way to cause stores to complete.

That's a StoreLoad.  I don't know what your processor calls it.  It's
usually a full barrier.

Andrew.