A "swift-ring-builder *.builder rebalance" issue

Asked by zen

Hi everyone.

I'm trying to build a little Swift Infrastructure from this tutorial : http://docs.openstack.org/openstack-object-storage/admin/os-objectstorage-admin-book.pdf

Actually i have 3 VMs:
- Proxy Node
- Authentication Node
- Storage Node with : 4 logical partitions : /dev/sda5, /dev/sda6, /dev/sda7, /dev/sda8

In the 7th step of how configuring a Proxy Node, i added the ressources this way :

swift-ring-builder account.builder create 18 3 1
swift-ring-builder account.builder add z1-192.168.1.18:6001/sda5 100
swift-ring-builder account.builder add z1-192.168.1.18:6002/sda6 100
swift-ring-builder account.builder add z1-192.168.1.18:6003/sda7 100
swift-ring-builder account.builder add z1-192.168.1.18:6004/sda8 100
swift-ring-builder container.builder create 18 3 1
swift-ring-builder container.builder add z1-192.168.1.18:6011/sda5 100
swift-ring-builder container.builder add z1-192.168.1.18:6012/sda6 100
swift-ring-builder container.builder add z1-192.168.1.18:6013/sda7 100
swift-ring-builder container.builder add z1-192.168.1.18:6014/sda8 100
swift-ring-builder object.builder create 18 3 1
swift-ring-builder object.builder add z1-192.168.1.18:6021/sda5 100
swift-ring-builder object.builder add z1-192.168.1.18:6022/sda6 100
swift-ring-builder object.builder add z1-192.168.1.18:6023/sda7 100
swift-ring-builder object.builder add z1-192.168.1.18:6024/sda8 100

Here is the result :

# swift-ring-builder object.builder
object.builder, build version 4
262144 partitions, 3 replicas, 1 zones, 4 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 192.168.1.18 6021 sda5 100.00 0 -100.00
             1 1 192.168.1.18 6022 sda6 100.00 0 -100.00
             2 1 192.168.1.18 6023 sda7 100.00 0 -100.00
             3 1 192.168.1.18 6024 sda8 100.00 0 -100.00

# swift-ring-builder container.builder
container.builder, build version 4
262144 partitions, 3 replicas, 1 zones, 4 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 192.168.1.18 6011 sda5 100.00 0 -100.00
             1 1 192.168.1.18 6012 sda6 100.00 0 -100.00
             2 1 192.168.1.18 6013 sda7 100.00 0 -100.00
             3 1 192.168.1.18 6014 sda8 100.00 0 -100.00

# swift-ring-builder account.builder
account.builder, build version 4
262144 partitions, 3 replicas, 1 zones, 4 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 192.168.1.18 6001 sda5 100.00 0 -100.00
             1 1 192.168.1.18 6002 sda6 100.00 0 -100.00
             2 1 192.168.1.18 6003 sda7 100.00 0 -100.00
             3 1 192.168.1.18 6004 sda8 100.00 0 -100.00

But when i try to rebalance the ring, i got those errors :

# swift-ring-builder account.builder rebalance
Traceback (most recent call last):
  File "/usr/bin/swift-ring-builder", line 571, in <module>
    Commands.__dict__.get(command, Commands.unknown)()
  File "/usr/bin/swift-ring-builder", line 443, in rebalance
    parts, balance = builder.rebalance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 200, in rebalance
    self._initial_balance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 351, in _initial_balance
    while available_devs[index]['zone'] in other_zones:
IndexError: list index out of range

# swift-ring-builder container.builder rebalance
Traceback (most recent call last):
  File "/usr/bin/swift-ring-builder", line 571, in <module>
    Commands.__dict__.get(command, Commands.unknown)()
  File "/usr/bin/swift-ring-builder", line 443, in rebalance
    parts, balance = builder.rebalance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 200, in rebalance
    self._initial_balance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 351, in _initial_balance
    while available_devs[index]['zone'] in other_zones:
IndexError: list index out of range

# swift-ring-builder object.builder rebalance
Traceback (most recent call last):
  File "/usr/bin/swift-ring-builder", line 571, in <module>
    Commands.__dict__.get(command, Commands.unknown)()
  File "/usr/bin/swift-ring-builder", line 443, in rebalance
    parts, balance = builder.rebalance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 200, in rebalance
    self._initial_balance()
  File "/usr/lib/pymodules/python2.6/swift/common/ring/builder.py", line 351, in _initial_balance
    while available_devs[index]['zone'] in other_zones:
IndexError: list index out of range

Does anyone know why i can't rebalance the ring please?

Thanks.

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
zen
Solved:
Last query:
Last reply:
Revision history for this message
Colin Nicholson (colin-nicholson) said :
#1

Hi,

You are only creating 1 zone - try using at least 3 and see if that works.

Colin

Revision history for this message
zen (zenn) said :
#2

Hi Colin,

Thanks for the answer. Is there a way to have 3 zones without necessarily creating 3 nodes/VMs ?

Thanks.

Revision history for this message
Colin Nicholson (colin-nicholson) said :
#3

Hi,

You can make each of your partitions a different zone -

swift-ring-builder account.builder create 18 3 1
swift-ring-builder account.builder add z1-192.168.1.18:6001/sda5 100
swift-ring-builder account.builder add z2-192.168.1.18:6002/sda6 100
swift-ring-builder account.builder add z3-192.168.1.18:6003/sda7 100
swift-ring-builder account.builder add z4-192.168.1.18:6004/sda8 100

See this page for more info on setting up on a single machine

http://swift.openstack.org/development_saio.html#setting-up-scripts-for-running-swift

Colin

Revision history for this message
zen (zenn) said :
#4

Thanks Colin, that worked just fine.

Do you it's recommended to use 5 zones or it's not compulsary?

Regards.

Revision history for this message
Colin Nicholson (colin-nicholson) said :
#5

Only for a production system - 3 or 4 is fine for a test setup.

Colin

On 20 Feb 2011, at 00:45, zen <email address hidden> wrote:

> Question #146048 on OpenStack Object Storage (swift) changed:
> https://answers.launchpad.net/swift/+question/146048
>
> Status: Answered => Open
>
> zen is still having a problem:
> Thanks Colin, that worked just fine.
>
> Do you it's recommended to use 5 zones or it's not compulsary?
>
> Regards.
>
> --
> You received this question notification because you are an answer
> contact for OpenStack Object Storage (swift).

Revision history for this message
zen (zenn) said :
#6

Thanks Colin.

Revision history for this message
Andy Chang (s90752) said :
#7

Hi, I have three servers, one for proxy node, other twos for storage nodes.
One storage node as one zone.
If I just have 2 zones, can I have three replicas of each object?

thanks,

Andy Chang

Revision history for this message
Andy Chang (s90752) said :
#8

Another case:
If I just have three zones, and I have three replicas of each object, then one zone is dead, what will happen in this situation?

Thanks,

Andy Chang

Revision history for this message
Tudor Holton (tudor) said :
#9

Note the age of this bug. The 3 zone requirement no longer exists in newer versions of Swift. In fact, since zones are supposed to notify swift of potential points of failure, you should *not* use separate zones if the node shares a switch or power supply, etc.

Revision history for this message
rudal (rudy-rudal) said :
#10

Hi there, I am currently using 12.04 LTS ubuntu.

I tried to follow your instruction but I couldnt seem able to do what you have with 1,2,3 and so zones.

This is what I have:

root@proxy-node:/etc/swift# swift-ring-builder account.builder
account.builder, build version 2
262144 partitions, 3 replicas, 2 zones, 2 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 10.188.188.101 6002 sdb1 100.00 0 -100.00
             1 2 10.188.188.102 6002 sdb1 100.00 0 -100.00
root@proxy-node:/etc/swift# swift-ring-builder container.builder
container.builder, build version 2
262144 partitions, 3 replicas, 2 zones, 2 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 10.188.188.101 6001 sdb1 100.00 0 -100.00
             1 2 10.188.188.102 6001 sdb1 100.00 0 -100.00
root@proxy-node:/etc/swift# swift-ring-builder object.builder
object.builder, build version 2
262144 partitions, 3 replicas, 2 zones, 2 devices, 100.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id zone ip address port name weight partitions balance meta
             0 1 10.188.188.101 6000 sdb1 100.00 0 -100.00
             1 2 10.188.188.102 6000 sdb1 100.00 0 -100.00
root@proxy-node:/etc/swift#

Then I tried to rebalance them:

root@proxy-node:/etc/swift# swift-ring-builder object.builder rebalance
Traceback (most recent call last):
  File "/usr/bin/swift-ring-builder", line 690, in <module>
    Commands.__dict__.get(command, Commands.unknown)()
  File "/usr/bin/swift-ring-builder", line 528, in rebalance
    parts, balance = builder.rebalance()
  File "/usr/lib/python2.7/dist-packages/swift/common/ring/builder.py", line 254, in rebalance
    self._initial_balance()
  File "/usr/lib/python2.7/dist-packages/swift/common/ring/builder.py", line 412, in _initial_balance
    while available_devs[index]['zone'] in other_zones:
IndexError: list index out of range

Anything that I did wrong / missing?

Thank you

Revision history for this message
clayg (clay-gerrard) said :
#11

did you install from packages - do you know what version of swift you have installed?

I think this error is only present in older versions of swift when using less zones than replicas - newer swift's don't have that restriction. Does ubuntu cloud archive have a newer version available to install - or can you try to re-image on trusty?

Also this question is really old, even if you want to reference it - it'd be better to start your own thread. And I'm not even sure openstack still uses launchpad answers - everything is moved to ask.openstack.org I think? Probably better to either start a new question there, or the operators mailing list, or just jump into #openstack-swift on freenode.