where is the code about objects mapping?
When I upload a object "ppt.pdf" to the container "container_test1" of swift SAIO, I find one of the path of the object is
"/mnt/sdb1/
Question information
- Language:
- English Edit question
- Status:
- Solved
- Assignee:
- No assignee Edit question
- Solved by:
- Kun Huang
- Solved:
- Last query:
- Last reply:
Revision history for this message
|
#1 |
The strategies and background is subtle, but you could find the codes in: L416~L427
https:/
It's now whole, but an enough hint for you
If you're interested in the theory about this, try to read this blog: http://
or swiftstack‘s book http://
Revision history for this message
|
#2 |
Thanks, Kun Huang.
When I input "account:user, container, object", I want to get the address of one of the replicas in swift.
Such as when I input "test:tester, container_test1, ppt.pdf", I hope to get "/mnt/sdb1/
If there is a method without modifying the source code of swift, could you tell me how can I do that? It is perfect to obtain the address without modifying the source code.
If it is essential to modify the source code to get the address, could you give me some advices?
Revision history for this message
|
#3 |
It's easy, and you don't need change source codes. Just try "swift-get-codes", following its help message and you could find physical address of an object, a container or an account.
Revision history for this message
|
#4 |
Thanks, Kun Huang.
As an Administrator, I want to have an interface to obtain the physical address of an object.
Perhaps I need many objects addresses on time.
I want to achieve such a function that objects not being accessed for a long long time move from swift to other places. So I need to obtain the physical address of an object easy and fast. And it is better not disturbing the running swift. It is perfect to obtain the address without modifying the source code.
I think that your above advices can help me. Could you explain it with more details?
Revision history for this message
|
#5 |
I think you don't need worried about changes in source codes, because reasonable functions may have been alreadt implemented in community.
In your case, swift-get-nodes could help you:
swift-get-nodes [-a] <ring.gz> <account> [<container>] [<object>] could output real/physical address of an account/
Revision history for this message
|
#6 |
Thanks, , Kun Huang. You are so kind.
But when I have a try, I meet this
swift@qwn-
Traceback (most recent call last):
File "/usr/local/
execfile(
File "/home/
from swift.common.utils import hash_path
ImportError: cannot import name hash_path
I installed swift SAIO, and its version is swift 1.6.0.1
I find this is a bug. And Swift 1.10.0 has resolved it.
Can I upgrade my swift SAIO instead of re install it?
Revision history for this message
|
#7 |
If this is the first time you try swift, please use the newest codes. That error seems a very old bug.
Revision history for this message
|
#8 |
Thanks, , Kun Huang. You are so kind. Your advices almost resolve my problems.
But there is one thing left. I want to get the object's full path, including the last "******.data".
"swift-get-nodes" method tells me the path, not including the object's *.data file name.
Such as when I used "swift-get-nodes", I got "/mnt/sdb1/
Thank you very much.
swift@cloud1:~$ swift -A http://
cert.key
proxy-server.conf
swift@cloud1:~$ swift-get-nodes -a /etc/swift/
Account test
Container test1
Object cert.key
Partition 34233
Hash 216e50dc72887cf
Server:Port Device 192.168.1.3:6010 sdb1
Server:Port Device 192.168.1.5:6010 sdb1
Server:Port Device 192.168.1.4:6010 sdb1
curl -I -XHEAD "http://
curl -I -XHEAD "http://
curl -I -XHEAD "http://
Use your own device location of servers:
such as "export DEVICE=/srv/node"
ssh 192.168.1.3 "ls -lah ${DEVICE:
ssh 192.168.1.5 "ls -lah ${DEVICE:
ssh 192.168.1.4 "ls -lah ${DEVICE:
swift@cloud1:~$
Revision history for this message
|
#9 |
See the last part of your output, and you can find that is ssh command:
ssh <address> "<command>"
That is used to run <command> in target address. And what is output of ls? That is what you need now.
So try paste© and run that ssh command: )
Revision history for this message
|
#10 |
I meet another strange problem.
I use ssh to connect 192.168.1.3. And I can find the path "/srv/node/
It is really confused me.
swift@cloud1:~$ ssh 192.168.1.3 "ls -lah ${DEVICE:
swift@192.168.1.3's password:
ls: 初始化月份字符串出错
ls: 无法访问/srv/
Revision history for this message
|
#11 |
notice this:
Use your own device location of servers:
such as "export DEVICE=/srv/node"
If you installed your cluster by SAIO document, I think you should export your own environment variable DEVICE by your own config files.
Revision history for this message
|
#12 |
Thanks, Kun Huang.
I used both swift installed in 4 DELL servers and Swift SAIO for testing.
I can upload and download.
but when I used "swift-get-nodes", the Partition returned is not a folder in the storage node.
my DEVICE is set "/srv/node". There are many folders in “/srv/node/
There are some output of swift in 4 DELL servers, not of SAIO.
103748 126837 149607 173774 199160 224454 245960 35322 59406 81540
103759 126936 150199 173928 199274 224706 246115 35640 59883 81645
104006 127348 150388 174134 199316 224827 246328 35849 60625 82238
104023 127379 150421 174314 199607 22492 246504 35853 61046 8236
104036 12739 150443 174445 199656 224949 247236 3593 61104 82599
104155 127440 150555 174603 199993 22504 247474 36058 61285 829
104330 127876 150775 174775 200217 225222 248137 36533 61321 82982
104503 1279 151115 174919 200408 225251 249085 36593 6133 8299
10460 127971 152044 174941 200664 225523 249204 36968 61354 83013
104800 12821 152185 174971 200776 225548 249277 3722 61411 83035
104976 128482 152222 175017 201129 225639 249300 37808 61812 83806
104999 128577 152835 175140 201141 22576 24942 38130 62058 83874
105016 128778 153060 175340 201340 225873 249644 38514 62190 83956
105082 128972 153126 175360 201619 225888 249910 38592 62233 84005
105114 129266 153247 175482 201692 226246 249933 38613 62235 84290
105560 129368 153340 175781 201952 226464 250062 38616 62547 84448
105587 129420 153406 175810 202096 226583 2502 38885 62577 84471
105712 129613 153427 176219 202921 226675 250392 3896 62667 8464
105918 129644 153446 176406 203144 226927 250484 38968 62739 84936
swift@cloud1:
boto-2.6.0.tar.gz
swift@cloud1:
swift@cloud1:~$ swift-get-nodes -a /etc/swift/
Account test
Container cont1
Object boto-2.6.0.tar.gz
Partition 61178
Hash 3bbe955693bec81
Server:Port Device 192.168.1.4:6010 sdb1
Server:Port Device 192.168.1.3:6010 sdb1
Server:Port Device 192.168.1.5:6010 sdb1
curl -I -XHEAD "http://
curl -I -XHEAD "http://
curl -I -XHEAD "http://
Use your own device location of servers:
such as "export DEVICE=/srv/node"
ssh 192.168.1.4 "ls -lah ${DEVICE:
ssh 192.168.1.3 "ls -lah ${DEVICE:
ssh 192.168.1.5 "ls -lah ${DEVICE:
swift@cloud1:~$ ssh 192.168.1.3 "ls -lah ${DEVICE:
swift@192.168.1.3's password:
ls: 初始化月份字符串出错
ls: 无法访问/srv/
swift@cloud1:~$ ls
openstack
swift@cloud1:~$ swift -A http://
boto-2.6.0.tar.gz [headers 0.039s, total 0.087s, 7.358 MB/s]
swift@cloud1:~$ ls
boto-2.6.0.tar.gz openstack
Revision history for this message
|
#13 |
In SAIO, I upload 3 objects. And there are 3 folders in 4 nodes. But there is no such a folder
“Partition 72
Hash 12365c8a2807d62
My SAIO is just installed the newest one.
swift@qwn-
swift@qwn-
. .. 336 577
swift@qwn-
swift@qwn-
. .. 336 497 577
swift@qwn-
swift@qwn-
. .. 336 497 577
swift@qwn-
swift@qwn-
. .. 497
swift@qwn-
PPT2.pdf
etc/swift/
home/swift/PPT.pdf
swift@qwn-
Account test
Container container_test1
Object PPT2.pdf
Partition 72
Hash 12365c8a2807d62
Server:Port Device 127.0.0.1:6030 sdb3
Server:Port Device 127.0.0.1:6010 sdb1
Server:Port Device 127.0.0.1:6020 sdb2
Server:Port Device 127.0.0.1:6040 sdb4 [Handoff]
curl -I -XHEAD "http://
curl -I -XHEAD "http://
curl -I -XHEAD "http://
curl -I -XHEAD "http://
Use your own device location of servers:
such as "export DEVICE=/srv/node"
ssh 127.0.0.1 "ls -lah ${DEVICE:
ssh 127.0.0.1 "ls -lah ${DEVICE:
ssh 127.0.0.1 "ls -lah ${DEVICE:
ssh 127.0.0.1 "ls -lah ${DEVICE:
Revision history for this message
|
#14 |
Your account "test" actually is "AUTH_test". Try this instead : )
Revision history for this message
|
#15 |
Thanks, Kun Huang.
Your advices are so powerful.
In SAIO, it returned the right folder. and ssh line return the object name I wanted.
swift@qwn-
swift@qwn-
swift@127.0.0.1's password:
总用量 82M
drwxr-xr-x 2 swift swift 34 11月 12 21:28 .
drwxr-xr-x 3 swift swift 45 11月 12 21:28 ..
-rw------- 1 swift swift 82M 11月 12 21:28 1384262912.
But in the distributed swift in 4 DELL servers, the folder is not right. It is changed to another folder, but it is not exist neither.
The authentication of account and user is tempauth in SAIO, and it is swauth in distributed swift in 4 DELL servers.
Thank you for your warm-hearted help. It really encourage me. And make the testing interesting and fast.
I must try the "swift-get-nodes" line in 4 DELL servers. That will be the system deployed.
Revision history for this message
|
#16 |
Thanks Kun Huang, that solved my question.