GET 24/7 LIVE HELP NOW

Announcement

Announcement Module
Collapse
No announcement yet.

tar4ibd too small read block size for free space on ibdata

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • tar4ibd too small read block size for free space on ibdata

    Description:

    tar4ibd make too many small block read for free .ibd files. I am xtrabacking on slave database,but the query sql request on slave will much slower while dump ibdata2. Is there some way to reslove it?

    for ibdata1,which have data on it,16k block request to os. this is correct.
    # tar4ibd -d -c ibdata1 |gzip > /bak/a.tar.gz
    tar4ibd: using O_DIRECT for the input file.
    Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
    sda 6.50 0.00 1672.00 0.00 26640.00 0.00 31.87 0.11 0.06 0.06 10.75

    12:55:37 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    12:55:38 PM 18 11.88 0.00 2.97 13.86 0.00 0.99 0.00 70.30 1678.00

    for ibdata2,maybe no data on it. you will see 24000 seq read iops to os,which block size is 1kb at all.cpu iowait much higher.
    Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
    sda 0.00 3.00 24281.00 2.50 24281.00 22.00 2.00 0.82 0.04 0.03 67.80

    12:55:10 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
    12:55:11 PM 18 5.05 0.00 40.40 41.41 5.05 8.08 0.00 0.00 24763.00


    REPLAY:
    dd if=/dev/zero of=a.z count=10000 bs=1024k
    strace tar4ibd -d -c a.z|gzip > /bak/a.tar.gz
    and you will see
    read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 1024) = 1024
    write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 512) = 512
    write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 512) = 512
    read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 1024) = 1024
    write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 512) = 512
    write(1, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 512) = 512
    read(3, " \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0 \0\0\0\0\0 \0\0 "..., 1024) = 1024

  • #2
    modified page_size from 1*1024 to 8*1024 on function tar_append_regfile(TAR *t, char *realname),
    which is on tar4ibd_libtar-1.2.11.patch line 396
    or
    on libtar-1.2.11/lib/append.c line 539。
    539c539
    < int page_size = 1 * 1024; /* check 1K, 2K, 4K, 8K and 16K */
    ---
    > int page_size = 8 * 1024; /* check 1K, 2K, 4K, 8K and 16K */

    after that,the block size will change from 1k to 8k,it will help to my issue,but i don't konw if it will hit bugs?

    Comment


    • #3
      Right, tar4ibd reads file inefficiently.

      For compressed tablespaces, tar4ibd has to start with the smallest possible block size (i.e. 1KB) to verify page integrity. However, it can be optimized by:

      - reading in large chunks and then processing it in smaller ones;
      - using default page size (16KB) for uncompressed files

      In other words, do what xtrabackup binary does when reading the input files.

      I have reported https://bugs.launchpad.net/percona-x...up/+bug/899931 so this issue is on our radar. However, we are going to remove tar4ibd (and use xtrabackup binary for streaming) in the next major version. So it's unlikely we will invest any efforts into fixing this in 1.6.

      Comment

      Working...
      X