OpenStack Object Storage (swift)

Preventing stale reads

Asked by David Kranz

I am trying to understand what prevents the following from happening:

1. Overwrite an existing object to swift. Two replicas are written but the the third's storage node is unreachable so the write returns but only two copies of the new file exist.
2. Read the file, but the unreachable node is now reachable and the proxy chooses it to get the data before replication can
update the third copy of the file. The read returns the old data.

I looked at the proxy code for PUT/GET but could not figure this out. Is there an easy answer?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
John Dickinson
Solved:
Last query:
Last reply:
Revision history for this message
Best John Dickinson (notmyname) said : 		#1

Nothing prevents that from happening. That is part of the eventual consistency model of swift. Understanding this is very important to writing good clients for swift (or any eventually consistent system).

However, if you ABSOLUTELY MUST HAVE the very newest data every time, you can add an "X-Newest: true" header to your request, and the proxy will query all three replicas and return the one with the newest timestamp. Note, however, that this will cause the result to be slower that if the x-newest header is not used.

Revision history for this message
David Kranz (david-kranz) said : 		#2

Thanks John Dickinson, that solved my question.

Revision history for this message
David Kranz (david-kranz) said : 		#3

I remember there being some kind of 'etag' that can be provided with a read to make sure the data is OK. Would it be possible for the proxy to know what the tag is for an object and check it when doing the read from the object server? It could then try other replicas if the tag was wrong before bailing with an error. That would seem to give the attributes of your X-Newest suggestion but usually not have the overhead of checking multiple copies.

Revision history for this message
John Dickinson (notmyname) said : 		#4

If the etag doesn't match the hash of the data, the object is quarantined and replication replaces it. A hash mismatch means that the data has been corrupted on disk (filesystem corruption or bit rot).

On Dec 14, 2011, at 1:20 PM, David Kranz wrote:

> Question #181905 on OpenStack Object Storage (swift) changed:
> https://answers.launchpad.net/swift/+question/181905
>
> David Kranz posted a new comment:
> I remember there being some kind of 'etag' that can be provided with a
> read to make sure the data is OK. Would it be possible for the proxy to
> know what the tag is for an object and check it when doing the read from
> the object server? It could then try other replicas if the tag was wrong
> before bailing with an error. That would seem to give the attributes of
> your X-Newest suggestion but usually not have the overhead of checking
> multiple copies.
>
> --
> You received this question notification because you are a member of
> Swift Core, which is an answer contact for OpenStack Object Storage
> (swift).