OpenStack Object Storage (swift)

The partitions do not follow the weight value?

Asked by yixuan zhang

Hi,
       I found the partitions don't change with the weight. Could you please check the below test:
 There is only 1 zone. Initial create ring:
swift-ring-builder object.builder create 16 3 0

1. The ring info after add 9.111.67.11 with 6disks.

command:
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sdc 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sdc 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sdc 20
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sdg 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sdg 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sdg 20
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sdd 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sdd 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sdd 20
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sdb 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sdb 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sdb 20
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sde 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sde 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sde 20
swift-ring-builder account.builder add r1z1-9.111.67.11:6002/sdf 20
swift-ring-builder container.builder add r1z1-9.111.67.11:6001/sdf 20
swift-ring-builder object.builder add r1z1-9.111.67.11:6000/sdf 20
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance

Check result:

swift-ring-builder object.builder

65536 partitions, 3.000000 replicas, 1 regions, 1 zones, 6 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 0
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
             0 1 1 9.111.67.11 6000 9.111.67.11 6000 sdc 20.00 32768 0.00
             1 1 1 9.111.67.11 6000 9.111.67.11 6000 sdg 20.00 32768 0.00
             2 1 1 9.111.67.11 6000 9.111.67.11 6000 sdd 20.00 32768 0.00
             3 1 1 9.111.67.11 6000 9.111.67.11 6000 sdb 20.00 32768 0.00
             4 1 1 9.111.67.11 6000 9.111.67.11 6000 sde 20.00 32768 0.00
             5 1 1 9.111.67.11 6000 9.111.67.11 6000 sdf 20.00 32768 0.00

2. The ring info after add 9.111.67.12 with 7 disks.

swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sde 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sde 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sde 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdb 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdb 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdb 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdh 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdh 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdh 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdc 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdc 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdc 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdg 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdg 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdg 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdd 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdd 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdd 20
swift-ring-builder account.builder add r1z1-9.111.67.12:6002/sdf 20
swift-ring-builder container.builder add r1z1-9.111.67.12:6001/sdf 20
swift-ring-builder object.builder add r1z1-9.111.67.12:6000/sdf 20
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance

check ring:

swift-ring-builder object.builder
65536 partitions, 3.000000 replicas, 1 regions, 1 zones, 13 devices, 45.39 balance
The minimum number of hours before a partition can be reassigned is 0
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
             0 1 1 9.111.67.11 6000 9.111.67.11 6000 sdc 20.00 21879 44.67
             1 1 1 9.111.67.11 6000 9.111.67.11 6000 sdg 20.00 21850 44.48
             2 1 1 9.111.67.11 6000 9.111.67.11 6000 sdd 20.00 21575 42.66
             3 1 1 9.111.67.11 6000 9.111.67.11 6000 sdb 20.00 21900 44.81
             4 1 1 9.111.67.11 6000 9.111.67.11 6000 sde 20.00 21880 44.67
             5 1 1 9.111.67.11 6000 9.111.67.11 6000 sdf 20.00 21988 45.39
             6 1 1 9.111.67.12 6000 9.111.67.12 6000 sde 20.00 9362 -38.10
             7 1 1 9.111.67.12 6000 9.111.67.12 6000 sdb 20.00 9362 -38.10
             8 1 1 9.111.67.12 6000 9.111.67.12 6000 sdh 20.00 9363 -38.09
             9 1 1 9.111.67.12 6000 9.111.67.12 6000 sdc 20.00 9363 -38.09
            10 1 1 9.111.67.12 6000 9.111.67.12 6000 sdg 20.00 9362 -38.10
            11 1 1 9.111.67.12 6000 9.111.67.12 6000 sdd 20.00 9362 -38.10
            12 1 1 9.111.67.12 6000 9.111.67.12 6000 sdf 20.00 9362 -38.10

3. The ring info after add 9.111.67.89 with 2 disks.

swift-ring-builder account.builder add r1z1-9.111.67.89:6002/sdc 20
swift-ring-builder container.builder add r1z1-9.111.67.89:6001/sdc 20
swift-ring-builder object.builder add r1z1-9.111.67.89:6000/sdc 20
swift-ring-builder account.builder add r1z1-9.111.67.89:6002/sde 20
swift-ring-builder container.builder add r1z1-9.111.67.89:6001/sde 20
swift-ring-builder object.builder add r1z1-9.111.67.89:6000/sde 20
swift-ring-builder account.builder add r1z1-9.111.67.89:6002/sdd 20
swift-ring-builder container.builder add r1z1-9.111.67.89:6001/sdd 20
swift-ring-builder object.builder add r1z1-9.111.67.89:6000/sdd 20
swift-ring-builder account.builder add r1z1-9.111.67.89:6002/sdb 20
swift-ring-builder container.builder add r1z1-9.111.67.89:6001/sdb 20
swift-ring-builder object.builder add r1z1-9.111.67.89:6000/sdb 20
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance

65536 partitions, 3.000000 replicas, 1 regions, 1 zones, 17 devices, 41.67 balance
The minimum number of hours before a partition can be reassigned is 0
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
             0 1 1 9.111.67.11 6000 9.111.67.11 6000 sdg 20.00 10922 -5.56
             1 1 1 9.111.67.11 6000 9.111.67.11 6000 sdc 20.00 10923 -5.55
             2 1 1 9.111.67.11 6000 9.111.67.11 6000 sdd 20.00 10923 -5.55
             3 1 1 9.111.67.11 6000 9.111.67.11 6000 sde 20.00 10922 -5.56
             4 1 1 9.111.67.11 6000 9.111.67.11 6000 sdf 20.00 10923 -5.55
             5 1 1 9.111.67.11 6000 9.111.67.11 6000 sdb 20.00 10923 -5.55
             6 1 1 9.111.67.12 6000 9.111.67.12 6000 sdc 20.00 9362 -19.05
             7 1 1 9.111.67.12 6000 9.111.67.12 6000 sde 20.00 9362 -19.05
             8 1 1 9.111.67.12 6000 9.111.67.12 6000 sdg 20.00 9363 -19.04
             9 1 1 9.111.67.12 6000 9.111.67.12 6000 sdf 20.00 9362 -19.05
            10 1 1 9.111.67.12 6000 9.111.67.12 6000 sdb 20.00 9362 -19.05
            11 1 1 9.111.67.12 6000 9.111.67.12 6000 sdh 20.00 9363 -19.04
            12 1 1 9.111.67.12 6000 9.111.67.12 6000 sdd 20.00 9362 -19.05
            13 1 1 9.111.67.89 6000 9.111.67.89 6000 sdd 20.00 16384 41.67
            14 1 1 9.111.67.89 6000 9.111.67.89 6000 sdb 20.00 16384 41.67
            15 1 1 9.111.67.89 6000 9.111.67.89 6000 sde 20.00 16384 41.67
            16 1 1 9.111.67.89 6000 9.111.67.89 6000 sdc 20.00 16384 41.67

Question1 :
It was found the partitions do not follow the weight value.
If all the disk's wight are the same, why they don't have the same partitions?

4. Change the 4 disks' weight from 20 to 40, on 9.111.67.89,do rebalance, it was found the partition don't change.

swift-ring-builder account.builder set_weight r1z1-9.111.67.89:6002/sdc 40
swift-ring-builder container.builder set_weight r1z1-9.111.67.89:6001/sdc 40
swift-ring-builder object.builder set_weight r1z1-9.111.67.89:6000/sdc 40
swift-ring-builder account.builder set_weight r1z1-9.111.67.89:6002/sde 40
swift-ring-builder container.builder set_weight r1z1-9.111.67.89:6001/sde 40
swift-ring-builder object.builder set_weight r1z1-9.111.67.89:6000/sde 40
swift-ring-builder account.builder set_weight r1z1-9.111.67.89:6002/sdd 40
swift-ring-builder container.builder set_weight r1z1-9.111.67.89:6001/sdd 40
swift-ring-builder object.builder set_weight r1z1-9.111.67.89:6000/sdd 40
swift-ring-builder account.builder set_weight r1z1-9.111.67.89:6002/sdb 40
swift-ring-builder container.builder set_weight r1z1-9.111.67.89:6001/sdb 40
swift-ring-builder object.builder set_weight r1z1-9.111.67.89:6000/sdb 40
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance

swift-ring-builder object.builder

65536 partitions, 3.000000 replicas, 1 regions, 1 zones, 17 devices, 16.67 balance
The minimum number of hours before a partition can be reassigned is 0
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
             0 1 1 9.111.67.11 6000 9.111.67.11 6000 sdc 20.00 10923 16.67
             1 1 1 9.111.67.11 6000 9.111.67.11 6000 sdg 20.00 10923 16.67
             2 1 1 9.111.67.11 6000 9.111.67.11 6000 sdd 20.00 10922 16.66
             3 1 1 9.111.67.11 6000 9.111.67.11 6000 sdb 20.00 10923 16.67
             4 1 1 9.111.67.11 6000 9.111.67.11 6000 sde 20.00 10923 16.67
             5 1 1 9.111.67.11 6000 9.111.67.11 6000 sdf 20.00 10922 16.66
             6 1 1 9.111.67.12 6000 9.111.67.12 6000 sde 20.00 9362 -0.00
             7 1 1 9.111.67.12 6000 9.111.67.12 6000 sdb 20.00 9362 -0.00
             8 1 1 9.111.67.12 6000 9.111.67.12 6000 sdh 20.00 9362 -0.00
             9 1 1 9.111.67.12 6000 9.111.67.12 6000 sdc 20.00 9362 -0.00
            10 1 1 9.111.67.12 6000 9.111.67.12 6000 sdg 20.00 9362 -0.00
            11 1 1 9.111.67.12 6000 9.111.67.12 6000 sdd 20.00 9363 0.01
            12 1 1 9.111.67.12 6000 9.111.67.12 6000 sdf 20.00 9363 0.01
            13 1 1 9.111.67.89 6000 9.111.67.89 6000 sdc 40.00 16384 -12.50
            14 1 1 9.111.67.89 6000 9.111.67.89 6000 sde 40.00 16384 -12.50
            15 1 1 9.111.67.89 6000 9.111.67.89 6000 sdd 40.00 16384 -12.50
            16 1 1 9.111.67.89 6000 9.111.67.89 6000 sdb 40.00 16384 -12.50

Question 2:
Why the partitions do not change with the weight change? only balance change.
Does the partitions do not only related with the weight, but something else, like IP address?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
Hugo Kou
Solved:
Last query:
Last reply:
Revision history for this message
Best Hugo Kou (tonytkdk) said : 		#1

it's balanced as possible as it can tho.
As you can see that each node holds 65536 partitions that the total partitions of a replica.
IP address is one of the condition for partition assigning.
If the total weight is same on each node, then you will get the result as your expectation.

Revision history for this message
yixuan zhang (yixuan-z) said : 		#2

Thanks Hugo Kou, that solved my question.