How to recovery ring builder files?

Asked by Alex Yang on 2012-09-19

I delete all the ring builder file.
Can I recovery them by ring.gz ?

Question information

English Edit question
OpenStack Object Storage (swift) Edit question
No assignee Edit question
Solved by:
Samuel Merritt
Last query:
Last reply:
Revision history for this message
Best Samuel Merritt (torgomatic) said :

Using existing Swift tools, there is no way to recover a builder file from a ring.gz file.

However, if you're willing to put your Python developer hat on, you can get back a builder file that's pretty close to the one you lost. Here's what you'll need to do:

Disclaimer: I think this will work, but I haven't tried it.

First, you'll want to load up the ring and a new ringbuilder object in a Python REPL:

>>> from swift.common.ring import RingData, RingBuilder
>>> ring = RingData.load('/path/to/account.ring.gz')

Now we start copying the data we have in the ring into the builder.

>>> import math
>>> partitions = len(ring._replica2part2dev_id[0])
>>> replicas = len(ring._replica2part2dev_id)

>>> builder = RingBuilder(int(Math.log(partitions, 2)), replicas, 1)
>>> builder.devs = ring.devs
>>> builder._replica2part2dev = ring.replica2part2dev_id
>>> builder._last_part_moves_epoch = 0
>>> builder._last_part_moves = array('B', (0 for _ in xrange(
>>> builder._set_parts_wanted()
>>> for d in builder._iter_devs():
            d['parts'] = 0
>>> for p2d in builder._replica2part2dev:
            for dev_id in p2d:
                builder.devs[dev_id]['parts'] += 1

That'll do it for the recoverable fields. For min_part_hours, you'll either have to remember what the value you used was, or just make up a new one.

>>> builder.change_min_part_hours(24) # or whatever you want it to be

Try some validation: if this doesn't raise an exception, you may feel some hope. Not too much, though.

>>> builder.validate()

Save the builder:

>>> import pickle
>>> pickle.dump(builder.to_dict(), open('account.builder', 'wb'), protocol=2)

You should now have a file called 'account.builder' in the current working directory.

Then, run `swift-ring-builder account.builder write_ring` and compare the new account.ring.gz to the account.ring.gz that you started from. They probably won't be byte-for-byte identical, but if you load them up in a REPL and their "_replica2part2dev_id" and "devs" attributes are the same (or nearly so), then you're in good shape.

Now, repeat that for container.ring.gz and object.ring.gz, and you might just get usable builder files.

Good luck.

Revision history for this message
Alex Yang (alexyang) said :

Thanks Samuel Merritt, that solved my question.

Revision history for this message
Alex Yang (alexyang) said :

Thanks a lot.