DB files incompatible with non-debian files

Asked by Josh Littlefield on 2017-04-19

We've found that a database files written on Debian are not readable on CentOS, and vice versa.
The issue stems from a very old commit from close to with tokyocabinet came to Debian:

  https://anonscm.debian.org/cgit/collab-maint/tokyocabinet.git/commit/?h=debian-sid/1.0.6-3&id=50842aff3da3e9526b262f40426921b5cc24f77f

From what I can tell, this commit did little more than ensure that the files no longer conform to the spec (http://fallabs.com/tokyocabinet/spex-en.html#fileformat), which specifies little-endian, when run on a platform that is already little-endian.
The Tokyocabinet code seems to default to I/O in host order, and this change makes all Debian platforms write files in anti-host order. A change aimed at portable files (as the comment implies) would have made byte-swapping conditional on the architecture.

This is all water under the bridge because it's been this way since 2007. But should there be a package variant which offers portable files conforming to the spec (little-endian)?

Question information

Language:
English Edit question
Status:
Open
For:
Ubuntu tokyocabinet Edit question
Assignee:
No assignee Edit question
Last query:
2017-04-20
Last reply:

Also noticed that the configure script has always supported auto-detecting endianess, and doing byte-swapping when needed to maintain file consistency. So this blanket use of --enable-swab, which doesn't even take the architecture into account, is just completely wrong.

Can you help with this problem?

Provide an answer of your own, or ask Josh Littlefield for more information if necessary.

To post a message you must log in.