Where will be the objects stored in the filesystem ?

Asked by sujay on 2012-10-30

I followed http://docs.openstack.org/developer/swift/development_saio.html URL to install openstack swift in Ubuntu 64 bit server with loopback as storage. I have the following doubts?

1. When I do swift upload file where (which container) will the file get stored in ? please absolute give the path in the filesystem

2. Will Replication occur ? If yes How does it store the objects ? How can i find multiple copies

3. How can i view the log of object storage ?

4. Can i create my own authentication mechanism without using the curl ?

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
Samuel Merritt
Solved:
2012-11-12
Last query:
2012-11-12
Last reply:
2012-11-05

This question was reopened

Samuel Merritt (torgomatic) said : #1

1. The command is "swift upload <container> <file> [<file> ...]", so the container is whatever you specify.

2. http://docs.openstack.org/developer/swift/index.html and http://docs.openstack.org/developer/swift/overview_replication.html

3. it's in /var/log/swift

4. http://docs.openstack.org/developer/swift/development_auth.html

sujay (sujay-m17) said : #2

1. In the URL it is specified as /srv/[1-4]/node for devices in the configuration files.

But in my opinion was when i do
swift upload container file
i thought the file would go to the container and i can see the container directory on my hard disk. But i tried searching for the file on my hard disk and i couldn't find one.

sujay (sujay-m17) said : #4

Thanks Samuel Merritt, that solved my question.

sujay (sujay-m17) said : #5

Thanks Samuel. Could you please check the URL and let me know where exactly i can find the file on the hard disk.

There are 4 containers given in the URL 1,2,3,4
But when i do
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list
㜄툍툐ं蔉蔋툏툐᠁蔉鐉鐉
鐋㜅蔈嘅᠄᠂㜃鐌嘇᠄툍嘅댎
댋᠂᠃ं댍嘅鐉☃嘆蔊蔋嘇鐉蔋
툍᠃㜆蔊댋ँऀ댌댎᠄㜃댌鐊댍᠄
툐蔉툎᠁鐉㜆ँ᠁툎鐋᠁᠂鐋댎툐
᠄᠂᠃鐌鐌鐋툎嘇蔈ँ툏댌툎

I get this output. So i think there is a problem in the container names. Please help me out why i am getting this instead of 1 2 3 4. I have followed the URL in the question

Samuel Merritt (torgomatic) said : #6

If you want to know where something is on the disk, run "swift-get-nodes /etc/swift/object.ring <account> <container> <object>".

I can't tell you where things are on your disks since I don't have your rings.

As for your container names, did you create them with names "1", "2", "3", and "4"? What commands did you use to upload the objects?

sujay (sujay-m17) said : #7

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list
㜄툍툐蔉蔋툏툐᠁蔉鐉鐉
鐋㜅蔈嘅᠄᠂㜃鐌嘇᠄툍嘅댎
댋᠂᠃ं댍嘅鐉☃嘆蔊蔋嘇鐉蔋
툍᠃㜆蔊댋댌댎᠄㜃댌鐊댍᠄
툐蔉툎᠁鐉㜆᠁툎鐋᠁᠂鐋댎툐
᠄᠂᠃鐌鐌鐋툎嘇蔈툏댌툎

instead of 1 2 3 4

root@ubuntu:~# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload ᠄᠂᠃鐌鐌鐋툎嘇蔈툏댌툎 Testfile

Testfile

root@ubuntu:~# swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list ᠄᠂᠃鐌鐌鐋툎嘇蔈툏댌툎
Testfile
댌᠂댎鐊㜅蔈嘅᠃ं댌᠃鐉㜄
댌댌Ω嘅툐Ωँं嘅㜆

using the given solution
root@ubuntu:/# /usr/local/bin/swift-get-nodes /etc/swift/object.ring.gz test ᠄᠂᠃鐌鐌鐋툎嘇蔈툏댌툎

Account test
Container ᠄᠂᠃鐌鐌鐋툎嘇蔈툏댌툎
Object None

Partition 166850
Hash a2f0b58577e532298c10a13675c77699

Server:Port Device 127.0.0.1:6020 sdb2
Server:Port Device 127.0.0.1:6040 sdb4
Server:Port Device 127.0.0.1:6010 sdb1
Server:Port Device 127.0.0.1:6030 sdb3 [Handoff]

curl -I -XHEAD "http://127.0.0.1:6020/sdb2/166850/test/%EF%84%92%E1%A0%84%E1%A0%82%E1%A0%83%E9%90%8C%E9%90%8C%E9%90%8B%ED%88%8E%E5%98%87%EF%84%91%E8%94%88%ED%88%8F%EB%8C%8C%ED%88%8E"
curl -I -XHEAD "http://127.0.0.1:6040/sdb4/166850/test/%EF%84%92%E1%A0%84%E1%A0%82%E1%A0%83%E9%90%8C%E9%90%8C%E9%90%8B%ED%88%8E%E5%98%87%EF%84%91%E8%94%88%ED%88%8F%EB%8C%8C%ED%88%8E"
curl -I -XHEAD "http://127.0.0.1:6010/sdb1/166850/test/%EF%84%92%E1%A0%84%E1%A0%82%E1%A0%83%E9%90%8C%E9%90%8C%E9%90%8B%ED%88%8E%E5%98%87%EF%84%91%E8%94%88%ED%88%8F%EB%8C%8C%ED%88%8E"
curl -I -XHEAD "http://127.0.0.1:6030/sdb3/166850/test/%EF%84%92%E1%A0%84%E1%A0%82%E1%A0%83%E9%90%8C%E9%90%8C%E9%90%8B%ED%88%8E%E5%98%87%EF%84%91%E8%94%88%ED%88%8F%EB%8C%8C%ED%88%8E" # [Handoff]

ssh 127.0.0.1 "ls -lah /srv/node/sdb2/containers/166850/699/a2f0b58577e532298c10a13675c77699/"
ssh 127.0.0.1 "ls -lah /srv/node/sdb4/containers/166850/699/a2f0b58577e532298c10a13675c77699/"
ssh 127.0.0.1 "ls -lah /srv/node/sdb1/containers/166850/699/a2f0b58577e532298c10a13675c77699/"
ssh 127.0.0.1 "ls -lah /srv/node/sdb3/containers/166850/699/a2f0b58577e532298c10a13675c77699/" # [Handoff]
root@ubuntu:/# ls -lah /srv/node/sdb2/containers/166850/699/a2f0b58577e532298c10a13675c77699/
ls: cannot access /srv/node/sdb2/containers/166850/699/a2f0b58577e532298c10a13675c77699/: No such file or directory
root@ubuntu:/# ls -lah /srv/2/node/sdb2/containers/166850/699/a2f0b58577e532298c10a13675c77699/
ls: cannot access /srv/2/node/sdb2/containers/166850/699/a2f0b58577e532298c10a13675c77699/: No such file or directory

There is no directory 166850 inside the below file hierarchy

This is my folder structure

/-srv--1---node----sdb1
        --2---node----sdb2
        --3---node----sdb3
        --4---node----sdb4

How to create containers? I thought that when you do swift upload containerY file, if containerY is not present then it'll create containerY.

Best Samuel Merritt (torgomatic) said : #8

If you're looking up where a container is on the filesystem, use /etc/swift/container.ring.gz. Objects, use object.ring.gz, and similarly for accounts.

sujay (sujay-m17) said : #9

Thanks Samuel Merritt, that solved my question.