how to change sqlite database to postgres for account server and container server?

Asked by Atul

We want to use Postgres instead of sqlite for storing listing of objects and containers (on container server and account server respectively). We were looking through swift documentation but did not find relevant steps to configure this.

Can anybody help on how this can be done or point us to any relevant documentation.

Thanks in advance,
Atul

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
Mike Barton
Solved:
Last query:
Last reply:
Revision history for this message
Best Mike Barton (redbo) said :
#1

That's not a configurable option, and would require changing a decent amount of code to do.

Revision history for this message
Atul (atul-kulkarni) said :
#2

Thanks Mike.
I was looking through code and thought so. I have noted few files which I would probably need to change or are relevant to this like swift/common/db.py.
As I'm very new to this code (and Python too :)) I would appreciate if you can help me figure out at high level which all files/modules would need changes to make this work.

I'm determined to get this done, and any help to achieve this goal faster is highly appreciated.

Thanks a lot for your support.
-Atul

Revision history for this message
John Dickinson (notmyname) said :
#3

One of the main advantages to using sqlite3 is that each db is represented by a file in the filesystem. Many of swift's processes rely on this fact (eg replication to ensure 3 copies are made, the account and container servers on where to write and read the data, all of the auditors to do integrity checks, etc).

I think moving Mike's comment of this change requiring "changing a decent amount of code to do" may be read as "just about every major subsystem in swift would need to be changed". That's not to say it couldn't be done or wouldn't work, just that the end result would be something different than swift.

As an alternative, one could keep metadata (ie listings) client-side in Postgres. This may be useful in cases where listing order needs to be changed (swift only sorts listings by name) or when the cardinality of the listings gets large (swift returns 10K "pages"--iterating over several million or more objects gets cumbersome) or when server-side eventual consistency needs to be spoofed as fully consistent to the client.

--John

On Jan 2, 2012, at 10:50 PM, Atul wrote:

> Question #183504 on OpenStack Object Storage (swift) changed:
> https://answers.launchpad.net/swift/+question/183504
>
> Status: Answered => Open
>
> Atul is still having a problem:
> Thanks Mike.
> I was looking through code and thought so. I have noted few files which I would probably need to change or are relevant to this like swift/common/db.py.
> As I'm very new to this code (and Python too :)) I would appreciate if you can help me figure out at high level which all files/modules would need changes to make this work.
>
> I'm determined to get this done, and any help to achieve this goal
> faster is highly appreciated.
>
> Thanks a lot for your support.
> -Atul
>
> --
> You received this question notification because you are a member of
> Swift Core, which is an answer contact for OpenStack Object Storage
> (swift).

Revision history for this message
Atul (atul-kulkarni) said :
#4

Ok. You have frightened me enough! :)
Let me check the alternative suggested.

Revision history for this message
Atul (atul-kulkarni) said :
#5

Thanks Mike Barton, that solved my question.