Can Swift function well if some zones have a slow or less reliable network connection?

Asked by David Kranz

The description of zones implies that they could be separated geographically, which could be desirable for more protection against loss of an entire data center. In such a deployment, some connections between the proxies and storage nodes, or between storage nodes, could be somewhat slow or unreliable. How well does Swift handle such a situation? Is it reasonable to use this kind of deployment to protect against loss of an entire data center or is some higher-level replication needed? What are the performance requirements of the storage network?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
gholt
Solved:
Last query:
Last reply:
Revision history for this message
Best gholt (gholt) said :
#1

This is uncharted territory. There have been discussions around doing such a thing, but the software wasn't designed with differing performance characteristics across zones -- of course, it could certainly have its design improved to do so, at the cost of complexity.

Of the many things to consider is performance: When an object GET comes in, it is routed randomly to one of the three replicas in the system. With slower zones, one or even all of the replicas might be on the slow zones. Swift would need to be enhanced to understand fast vs. slow and try to distribute writes to be on both (with maybe the slow ones happening in the background) and distribute reads to the fast and fall back to the slow.

For a different solution, the upcoming container sync feature may fill the gap. Container sync simply sends updates from one container to another, presumably on different clusters. One might run two clusters each running with 2 replicas and then sync the two. There would be 4 total replicas, which means higher cost. The performance requirements would be met by using the cluster closest to a given client and falling back to the other cluster.

The main benefit of the container sync approach is that the Swift core is altered only slightly and the vast majority of the new complexity is contained within the sync feature itself. This keeps any new bugs outside of the standard Swift usage.

Revision history for this message
David Kranz (david-kranz) said :
#2

Thanks for the clear answer. I looked at the blueprint for container sync. Is there a more detailed description of this functionality somewhere?

Revision history for this message
gholt (gholt) said :
#3

Well, I haven't had time to get everything in order yet. But there is this:

https://dl-web.dropbox.com/get/Public/consync/Swift%20Container%20Sync.pdf?w=1835cb5d

And of course the code itself:

https://code.launchpad.net/~gholt/swift/consync/+merge/63317

But it will need documentation before it goes out.