xtrabackup_56 fails on log prepare with " Segmentation fault" error

Asked by Natasha

I create a full backup of mysql 5.6 DB on Ubuntu 12.04 with command:

innobackupex --defaults-file=/etc/mysql/my.cnf --socket=/var/run/mysqld/mysqld.sock --user=[user_name] --password=[password] /usr/local/backups

Backup is created successfully: 14:23:21 innobackupex: completed OK!

When I try to prepare the full backup with command:

xtrabackup_56 --defaults-file="/etc/mysql/my.cnf" --defaults-group="mysqld" --prepare --target-dir=/usr/local/backups/2013-04-13_09-33-04/ --use-memory=2G --tmpdir=/tmp

I get an error: Segmentation fault (core dumped)

Here is the xtrabackup_56 output:

xtrabackup_56 version 2.0.6 for MySQL server 5.6.10 Linux (x86_64) (revision id: undefined)

xtrabackup: cd to /usr/local/backups/2013-04-13_09-33-04
xtrabackup: This target seems to be not prepared yet.
xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(60558998078)
xtrabackup: Temporary instance for recovery is set as followings.
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 2097152
xtrabackup: Temporary instance for recovery is set as followings.
xtrabackup: innodb_data_home_dir = ./
xtrabackup: innodb_data_file_path = ibdata1:10M:autoextend
xtrabackup: innodb_log_group_home_dir = ./
xtrabackup: innodb_log_files_in_group = 1
xtrabackup: innodb_log_file_size = 2097152
xtrabackup: Starting InnoDB instance for recovery.
xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
InnoDB: Last MySQL binlog file position 0 52921, file name mysql-bin.000001
Segmentation fault (core dumped)

And here is a core dump:

<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /usr/bin/xtrabackup_56...Reading symbols from /usr/lib/debug/usr/bin/xtrabackup_56...done.
done.
[New LWP 32379]
[New LWP 32382]
[New LWP 32388]
[New LWP 32390]
[New LWP 32386]
[New LWP 32387]
[New LWP 32385]
[New LWP 32389]
[New LWP 32383]
[New LWP 32381]
[New LWP 32384]

warning: Can't read pathname for load map: Input/output error.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `xtrabackup_56 --defaults-file=/etc/mysql/my.cnf --defaults-group=mysqld --prepa'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000000000692a4f in explain_filename (thd=0x0, from=<optimized out>, to=0x7fffa6cee7ca "p200906", to_length=<optimized out>, explain_mode=EXPLAIN_PARTITIONS_AS_COMMENT)
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/sql/sql_table.cc:360
360 /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/sql/sql_table.cc: No such file or directory.

What do I do wrong on prepare stage?

Thank you.

Question information

Revision history for this message
Alexey Kopytov (akopytov) said :
#1

This is most likely a bug in xtrabackup_56. Can you provide the output of the "bt full" command in gdb?

Revision history for this message
Natasha (natasha-v) said :
#2

Thanks, Alexey.

Sure I can.

Here is an outpout of "bt full":
(gdb) bt full
#0 0x0000000000692a4f in explain_filename (thd=0x0, from=<optimized out>, to=0x7fffa6cee7ca "p200906", to_length=<optimized out>, explain_mode=EXPLAIN_PARTITIONS_AS_COMMENT)
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/sql/sql_table.cc:360
        res = <optimized out>
        to_p = 0x7fffa6cee6cd ""
        db_name = <optimized out>
        subpart_name = <optimized out>
        subpart_name_len = 0
        tmp_p = <optimized out>
        db_name_len = <optimized out>
        table_name_len = 7
        name_type = <optimized out>
        end_p = <optimized out>
        table_name = 0x7fffa6cee7c0 "hbvault#P#p200906"
        part_name = <optimized out>
        part_name_len = 7
#1 0x0000000000571d0d in innobase_convert_identifier (buf=0x7fffa6ceef4b "\"h200912\"", buflen=644, id=<optimized out>, idlen=17, thd=0x0, file_id=<optimized out>)
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/storage/innobase/handler/ha_innodb.cc:2450
        nz = "hbvault#P#p200906\000sts\000ons\000\021\a\000\000\000\000\060\350ні\377\177\000\000\070\361Z\000\000\000\000\000\250\000\000\000\000\000\000\000@\357ні\377\177", '\000' <repeats 27 times>"\271, -\223\226\032u\001\000\000\000\000\000\000\000\000\070H\022\a\000\000\000\000@\357ні\377\177\000\000\032\000\000\000\000\000\000\000\317\361ні\377\177\000\000@H\022\a\000\000\000\000\300\350ні\377\177\000\000\223.W\000\000\000\000\000\230*\233\000\000\000\000\000\001\000\000\000\000\000\000\000d\002\000\000\000\000\000\000\221|M\000\000\000\000\000"
        nz2 = "\"hbvault\" /* \000lues\"\000ts\"\000ns\"\000\324\177\000\000\000\000\000\000\001\000\000\000\232ч²\000\000\000\000\000@\347ні\377\177\000\000\224\222A\000\000\000\000\000\350\224\321\006\000\000\000\000\r\000\000\000\000\000\000\000\220\347ні\377\177\000\000\356\032U\000\000\000\000\000\230\215\321\006", '\000' <repeats 12 times>, "\b\000\000\000\000\000\000\000\b", '\000' <repeats 23 times>, "ahbvault\000lab\000\000\000\000\200J\002I\324\177\000\000\001\000\000\000\000\000\000\000\000K\002I\324\177\000\000\240\352ні\377\177\000\000`\350ні\377\177\000\000\211\352N\000\000\000\000\000\250", '\000' <repeats 15 times>"\220, U\235\000\000\000\000\000\370J\002I\324\177\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000"
        s = 0x7fffa6cee6c0 "\"hbvault\" /* "
        q = <optimized out>
#2 0x000000000057ab67 in innobase_format_name (buf=0x7fffa6ceef40 "\"ahbvault\".\"h200912\"", buflen=655, name=0x7124838 "ahbvault/hbvault#P#p200906", is_index_name=0)
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/storage/innobase/handler/ha_innodb.cc:2574
        bufend = 0x7fffa6cee6cd ""
#3 0x000000000048c429 in dict_check_tablespaces_and_store_max_id (in_crash_recovery=1)
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/storage/innobase/dict/dict0load.cc:1029
        len = 26
        space_id = 26
        flags = <optimized out>
        table_name = "\"ahbvault\".\"h200912\"\000ers\"\000_values\"\000ts\"\000ns\"\000\000\000\000\000\000\270%\021\a", '\000' <repeats 68 times>"\260, ", '\000' <repeats 15 times>, "\t\000\000\000\062\000\000\000[\000\000\000n\000\000\000w\000\000\000|\000\000\000\000\000\b> \000\000\000 7-\322\324\177\000\000\200\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\270%\021\a\000\000\000\000\270\361\001\001\000\000\000\000\060\207\021\a\000\000\000\000\200\360ні\377\177\000\000\316\363U\000\000\000\000\000\200\360ні\377\177\000\000\200\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\363vM\000\000\000\000\000\200\000\000\000\000\000\000\000x?\022\a\000\000\000\000"...
        is_temp = <optimized out>
        discarded = <optimized out>
        field = 0x7fd453741540 ""
        pcur = {btr_cur = {index = 0x6d24608, page_cur = {rec = 0x7fd44c61cd08 "ahbvault/hbvault#P#p200906", block = 0x7fd449024a80}, purge_node = 0x7fd44c01c000, left_block = 0x7fffa6cee9d0, thr = 0x1751a96932db900,
            flag = 2798578368, tree_height = 5083220, up_match = 0, up_bytes = 140735991966364, low_match = 0, low_bytes = 300, n_fields = 140735991966400, n_bytes = 5080912, fold = 16384, path_arr = 0x7fd4d3341a74},
          latch_mode = 1, old_stored = 122766467, old_rec = 0x711883f "ahbvault/h200912usersim_valuesistsions", old_n_fields = 1, rel_pos = 1, block_when_stored = 0x7fd449024a80, modify_clock = 0,
          pos_state = 1997660512, search_mode = 1, trx_if_known = 0x0, old_rec_buf = 0x7118838 "\020", buf_size = 48}
        mtr = {memo = {heap = 0x0, used = 16,
            data = "\001\000\000\000\000\000\000\000\200J\002I\324\177\000\000\067", '\000' <repeats 15 times>, "\003", '\000' <repeats 15 times>, "\003", '\000' <repeats 15 times>, "\001", '\000' <repeats 15 times>"\200, \353ні\377\177\000\000\356\032U\000\000\000\000\000\b\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000\b", '\000' <repeats 15 times>, " \306U\323\324\177\000\000\220\353ні\001\000\000\000@\v\000I\324\177\000\000`\215\235\006\000\000\000\000ч⌠A\000\000\000\000\000`\215\235\006\000\000\000\000\200\n\000I\324\177\000\000\000\v\000I\324\177\000\000\270\361\001\001\000\000\000\000\260\354ні\377\177---Type <return> to continue, or q <return> to quit---return
\000\000d\344N", '\000' <repeats 13 times>"\200, A-\322\324\177\000\000\n\000\000\000\000\000\000\000п╨\017\001\000\000\000\000\001\000\000\000\000\000\000\000\270"..., base = {count = 140735991968160,
              start = 0x48edb5, end = 0x9c2378}, list = {prev = 0x67, next = 0x7fffa6ceece0}}, log = {heap = 0x0, used = 0,
            data = "\020\000\000\000\000\000\000\000\002", '\000' <repeats 71 times>, "\020\000\000\000\000\000\000\000@%\021\a\000\000\000\000\000\002\000\001\000\000\000\000\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>, "\002", '\000' <repeats 15 times>..., base = {count = 0, start = 0x2, end = 0x0}, list = {prev = 0x2, next = 0x0}}, inside_ibuf = 0, modifications = 0, made_dirty = 0, n_log_recs = 0, n_freed_pages = 0,
          log_mode = 21, start_lsn = 2, end_lsn = 0}
        sys_index = <optimized out>
        rec = 0x7fd44c61cd08 "ahbvault/hbvault#P#p200906"
        max_space_id = 2126
#4 0x000000000054f29b in innobase_start_or_create_for_mysql ()
    at /mnt/workspace/percona-xtrabackup-2.0-debs/label_exp/ubuntu-precise-64bit/target/percona-xtrabackup-2.0.6-521.precise/mysql-5.6/storage/innobase/srv/srv0start.cc:2353
        sum_of_data_file_sizes = <optimized out>
        err = <optimized out>
        i = <optimized out>
        logfile0 = 0x0
        unit = <optimized out>
        min_flushed_lsn = 60558993856
        srv_n_log_files_found = 1
        dirnamelen = 2
        size = <optimized out>
        io_limit = 256
        mtr = {memo = {heap = 0x7fffa6cf3510, used = 140735991988257,
            data = "\200@оі\377\177\000\000\021>оі\377\177\000\000/tmp", '\000' <repeats 12 times>"\200, ?оі\377\177\000\000\000\000\000\000\000\000\000\000\210G\000\000\000\000\000\000\220?оі\377\177\000\000\003\000\000\000\000\000\000\000OAG", '\000' <repeats 13 times>"\210, G\000\000\000\000\000\000\240Aоі\377\177\000\000\200Q\354\002\000\000\000\000\000\t\000\000\000\000\000\000\272<\226\002\000\000\000\000\001\000\000\000\000\000\000\000\244\201\000\000g\000\000\000l", '\000' <repeats 15 times>"\210, G\000\000\000\000\000\000\000\020\000\000\000\000\000\000(\000\000\000\000\000\000\000\205^iQ\000\000\000\000\006\237\253\031\000\000\000\000t\032\064\323\324\177\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\v\000\000\000\000\000\000\000\360\065T\323\324\177\000\000\250\063\262"..., base = {count = 5, start = 0x476952,
              end = 0x7fffa6cf88aa}, list = {prev = 0x4, next = 0x7fffa6cf4080}}, log = {heap = 0x7fffa6cf88ae, used = 140735991988320,
            data = "\352\224G\000\000\000\000\000`\200\021\001\000\000\000\000\252\210оі\377\177\000\000\257\210оі\377\177\000\000`\200\021\001\000\000\000\000\260Bоі\377\177\000\000[\207G", '\000' <repeats 14 times>, "R\354\002\000\000\000\000/tmp", '\000' <repeats 12 times>, "\001\000\000\000\324\177\000\000\310\302U\323\324\177\000\000\000\000\000\000\000\000\000\000 \306U\323\324\177\000\000\220\066T\323\001\000\000\000\060Bоі\377\177\000\000\310\302U\323\324\177\000\000\017\213A\000\000\000\000\000\377\377\377\377", '\000' <repeats 20 times>, "п⌡\351\002\000\000\000\000H5\377", '\000' <repeats 14 times>, "@0\321\324\177\000\000\000\000 \000\000\000\000\000\330K\361\322\324\177\000\000\260\231U\323\324\177\000\000\005", '\000' <repeats 15 times>"\350, G\361\322\324\177\000\000\000@0\321\324\177\000\000\377\377\377\377\000\000\000\000п⌡\351\002\000\000\000\000 Bоі\377\177\000\000 7"..., base = {count = 0, start = 0x0, end = 0x1}, list = {prev = 0x8, next = 0x54c5fa}}, inside_ibuf = 0, modifications = 0, made_dirty = 0, n_log_recs = 49042048, n_freed_pages = 140735991988912,
          log_mode = 49041962, start_lsn = 23, end_lsn = 140735991993248}
        logfilename = "./ib_logfile0", '\000' <repeats 7131 times>"\356, \213\371\321\324\177\000\000\000:оі\377\177\000\000\360\031оі\377\177\000\000\350Aоі\377\177\000\000\356\213\371\321\324\177\000\000 :оі\377\177\000\000\020\032оі\377\177\000\000\bBоі\377\177\000\000\020 \233\000\000\000\000\000\220 \233\000\000\000\000\000\254K\366\321\324\177\000\000\000\000\000\000\000\000\000\000\067\000\000\000\000\000\000\000Y \233\000\000\000\000\000\n", '\000' <repeats 63 times>"\371, \377\377\377\377\377\377\377\000\000\000\000\000\000\000\000 ", '\000' <repeats 15 times>"\366, \377\377\377\377\377\377\377\000\000\000\000\000\000\000\000 \000 ", '\000' <repeats 13 times>, "\n", '\000' <repeats 18 times>"\200, ", '\000' <repeats 12 times>, "\n", '\000' <repeats 11 times>"\377, \377\377\377\316\030оі\377\177\000\000\000\000\000\000\001", '\000' <repeats 11 times>"\377"...
        create_new_db = 0
        max_flushed_lsn = 60558993856
        sum_of_new_sizes = 0
        tablespace_size_in_header = <optimized out>
        ib_bh = <optimized out>
#5 0x000000000046b028 in innodb_init () at xtrabackup.cc:3336
        err = <optimized out>
#6 xtrabackup_prepare_func () at xtrabackup.cc:7327
        err = <optimized out>
#7 0x000000000045330c in main (argc=0, argv=0x2e99ed0) at xtrabackup.cc:7955
 ho_error = <optimized out>

That is it.

Thanks.

Revision history for this message
Alexey Kopytov (akopytov) said :
#3

Thanks, created https://bugs.launchpad.net/percona-xtrabackup/+bug/1169169 for further investigation.

Can you help with this problem?

Provide an answer of your own, or ask Natasha for more information if necessary.

To post a message you must log in.