Announcement

Announcement Module
Collapse
No announcement yet.

Node Shutdown after start

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Node Shutdown after start

    Hello!


    After fail start node i see log-file:

    Code:
    2014-06-02 18:59:06 7162 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
    2014-06-02 18:59:06 7162 [Note] WSREP: wsrep_load(): Galera 3.5(r178) by Codership Oy <info@codership.com> loaded successfully.
    2014-06-02 18:59:06 7162 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm.
    2014-06-02 18:59:06 7162 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
    2014-06-02 18:59:06 7162 [Note] WSREP: Passing config to GCS: base_host = 10.10.48.8; base_port = 4567; cert.log_conflicts = no; debug = no; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 1; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.listen_addr = tcp://10.10.48.8; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.version = 0; pc.wait_prim = true; pc.wait_prim_t
    2014-06-02 18:59:06 7162 [Note] WSREP: Service thread queue flushed.
    2014-06-02 18:59:06 7162 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
    2014-06-02 18:59:06 7162 [Note] WSREP: wsrep_sst_grab()
    2014-06-02 18:59:06 7162 [Note] WSREP: Start replication
    2014-06-02 18:59:06 7162 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
    2014-06-02 18:59:06 7162 [Note] WSREP: protonet asio version 0
    2014-06-02 18:59:06 7162 [Note] WSREP: Using CRC-32C (optimized) for message checksums.
    2014-06-02 18:59:06 7162 [Note] WSREP: backend: asio
    2014-06-02 18:59:06 7162 [Note] WSREP: GMCast version 0
    2014-06-02 18:59:06 7162 [Note] WSREP: (d3a72715-ea6e-11e3-9862-2a026dbf4089, 'tcp://10.10.48.8:4567') listening at tcp://10.10.48.8:4567
    2014-06-02 18:59:06 7162 [Note] WSREP: (d3a72715-ea6e-11e3-9862-2a026dbf4089, 'tcp://10.10.48.8:4567') multicast: , ttl: 1
    2014-06-02 18:59:06 7162 [Note] WSREP: EVS version 0
    2014-06-02 18:59:06 7162 [Note] WSREP: PC version 0
    2014-06-02 18:59:06 7162 [Note] WSREP: gcomm: connecting to group 'rxmydb', peer '10.10.48.8:,10.10.48.10:'
    2014-06-02 18:59:07 7162 [Note] WSREP: declaring ed87b5f4-ea6c-11e3-b5f2-6782c2dc0b6f stable
    2014-06-02 18:59:07 7162 [Note] WSREP: Node ed87b5f4-ea6c-11e3-b5f2-6782c2dc0b6f state prim
    2014-06-02 18:59:07 7162 [Note] WSREP: view(view_id(PRIM,d3a72715-ea6e-11e3-9862-2a026dbf4089,6) memb {
            d3a72715-ea6e-11e3-9862-2a026dbf4089,0
            ed87b5f4-ea6c-11e3-b5f2-6782c2dc0b6f,0
    } joined {
    } left {
    } partitioned {
    })
    2014-06-02 18:59:07 7162 [Note] WSREP: gcomm: connected
    2014-06-02 18:59:07 7162 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
    2014-06-02 18:59:07 7162 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
    2014-06-02 18:59:07 7162 [Note] WSREP: Opened channel 'rxmydb'
    2014-06-02 18:59:07 7162 [Note] WSREP: Waiting for SST to complete.
    2014-06-02 18:59:07 7162 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
    2014-06-02 18:59:07 7162 [Note] WSREP: STATE_EXCHANGE: sent state UUID: d3f3d7f1-ea6e-11e3-82ae-f38d73c4898d
    2014-06-02 18:59:07 7162 [Note] WSREP: STATE EXCHANGE: sent state msg: d3f3d7f1-ea6e-11e3-82ae-f38d73c4898d
    2014-06-02 18:59:07 7162 [Note] WSREP: STATE EXCHANGE: got state msg: d3f3d7f1-ea6e-11e3-82ae-f38d73c4898d from 0 (mysql)
    2014-06-02 18:59:07 7162 [Note] WSREP: STATE EXCHANGE: got state msg: d3f3d7f1-ea6e-11e3-82ae-f38d73c4898d from 1 (db)
    2014-06-02 18:59:07 7162 [Note] WSREP: Quorum results:
            version    = 3,
            component  = PRIMARY,
            conf_id    = 5,
            members    = 1/2 (joined/total),
            act_id     = 5587,
            last_appl. = -1,
            protocols  = 0/5/2 (gcs/repl/appl),
            group UUID = 07f3e316-ea5c-11e3-a67b-7f95f5026990
    2014-06-02 18:59:07 7162 [Note] WSREP: Flow-control interval: [23, 23]
    2014-06-02 18:59:07 7162 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 5587)
    2014-06-02 18:59:07 7162 [Note] WSREP: State transfer required:
            Group state: 07f3e316-ea5c-11e3-a67b-7f95f5026990:5587
            Local state: 00000000-0000-0000-0000-000000000000:-1
    2014-06-02 18:59:07 7162 [Note] WSREP: New cluster view: global state: 07f3e316-ea5c-11e3-a67b-7f95f5026990:5587, view# 6: Primary, number of nodes: 2, my index: 0, protocol version 2
    2014-06-02 18:59:07 7162 [Warning] WSREP: Gap in state sequence. Need state transfer.
    2014-06-02 18:59:09 7162 [Note] WSREP: Running: 'wsrep_sst_xtrabackup --role 'joiner' --address '10.10.48.8' --auth 'replicate:password' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '7162'  '' '
    WSREP_SST: [INFO] Streaming with xbstream (20140602 18:59:09.569)
    WSREP_SST: [INFO] Using socat as streamer (20140602 18:59:09.572)
    WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20140602 18:59:09.592)
    2014-06-02 18:59:09 7162 [Note] WSREP: Prepared SST request: xtrabackup|10.10.48.8:4444/xtrabackup_sst
    2014-06-02 18:59:09 7162 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
    2014-06-02 18:59:09 7162 [Note] WSREP: REPL Protocols: 5 (3, 1)
    2014-06-02 18:59:09 7162 [Note] WSREP: Service thread queue flushed.
    2014-06-02 18:59:09 7162 [Note] WSREP: Assign initial position for certification: 5587, protocol version: 3
    2014-06-02 18:59:09 7162 [Note] WSREP: Service thread queue flushed.
    2014-06-02 18:59:09 7162 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (07f3e316-ea5c-11e3-a67b-7f95f5026990): 1 (Operation not permitted)
             at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
    2014-06-02 18:59:09 7162 [Note] WSREP: Member 0.0 (mysql) requested state transfer from '*any*'. Selected 1.0 (db)(SYNCED) as donor.
    2014-06-02 18:59:09 7162 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 5587)
    2014-06-02 18:59:09 7162 [Note] WSREP: Requesting state transfer: success, donor: 1
    WSREP_SST: [ERROR] xtrabackup process ended without creating '/var/lib/mysql//xtrabackup_galera_info' (20140602 18:59:10.064)
    2014-06-02 18:59:10 7162 [Warning] WSREP: 1.0 (db): State transfer to 0.0 (mysql) failed: -22 (Invalid argument)
    2014-06-02 18:59:10 7162 [ERROR] WSREP: gcs/src/gcs_group.c:gcs_group_handle_join_msg():723: Will never receive state. Need to abort.
    2014-06-02 18:59:10 7162 [Note] WSREP: gcomm: terminating thread
    2014-06-02 18:59:10 7162 [Note] WSREP: gcomm: joining thread
    2014-06-02 18:59:10 7162 [Note] WSREP: gcomm: closing backend
    WSREP_SST: [INFO] Contents of datadir (20140602 18:59:10.067)
    WSREP_SST: [INFO] -rw-rw---- 1 mysql mysql     8820 Jun  2 17:52 /var/lib/mysql//mysql/columns_priv.frm
    -rw-rw---- 1 mysql mysql        0 Jun  2 17:52 /var/lib/mysql//mysql/columns_priv.MYD
    -rw-rw---- 1 mysql mysql     4096 Jun  2 17:52 /var/lib/mysql//mysql/columns_priv.MYI
    ***
    more tables.... (cut)
    ***
    (20140602 18:59:10.161)
    WSREP_SST: [ERROR] Cleanup after exit with status:32 (20140602 18:59:10.173)
    WSREP_SST: [INFO] Removing the sst_in_progress file (20140602 18:59:10.176)
    2014-06-02 18:59:10 7162 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup --role 'joiner' --address '10.10.48.8' --auth 'replicate:pass' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '7162'  '' : 32 (Broken pipe)
    2014-06-02 18:59:10 7162 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
    2014-06-02 18:59:10 7162 [ERROR] WSREP: SST failed: 32 (Broken pipe)
    2014-06-02 18:59:10 7162 [ERROR] Aborting
    
    2014-06-02 18:59:11 7162 [Note] WSREP: view(view_id(NON_PRIM,d3a72715-ea6e-11e3-9862-2a026dbf4089,6) memb {
            d3a72715-ea6e-11e3-9862-2a026dbf4089,0
    } joined {
    } left {
    } partitioned {
            ed87b5f4-ea6c-11e3-b5f2-6782c2dc0b6f,0
    })
    2014-06-02 18:59:11 7162 [Note] WSREP: view((empty))
    2014-06-02 18:59:11 7162 [Note] WSREP: gcomm: closed
    2014-06-02 18:59:11 7162 [Note] WSREP: /usr/sbin/mysqld: Terminated.
    140602 18:59:11 mysqld_safe mysqld from pid file /tmp/mysql.pid ended
    How start node 'mysql' without errors?


    Node db running at CentOS 6 (selinux disabled, iptables disabled) and percona DB (Percona XtraDB Cluster (GPL), Release rel35.0, Revision 756, WSREP version 25.10, wsrep_25.10.r3985)
    Node mysql running at Ubuntu 12.04 and (Percona XtraDB 5.6.15-rel63)

  • #2
    Server up only on settings wsrep_sst_method is rsync, but no xtrabackup. Any ideas?

    Comment


    • #3
      Node (db) down after many instert data to node (mysql)
      Code:
      140603 10:45:20 [Note] WSREP: Signalling provider to continue.
      140603 10:45:20 [Note] WSREP: SST received: 07f3e316-ea5c-11e3-a67b-7f95f5026990:5613
      140603 10:45:20 [Note] WSREP: 1.0 (mysql): State transfer from 0.0 (mysql) complete.
      140603 10:45:20 [Note] WSREP: Shifting JOINER -> JOINED (TO: 5613)
      140603 10:45:20 [Note] WSREP: Member 1.0 (mysql) synced with group.
      140603 10:45:20 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 5613)
      140603 10:45:20 [Note] WSREP: Synchronized with group, ready for connections
      140603 10:45:20 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
      140603 10:45:20 [Note] /usr/sbin/mysqld: ready for connections.
      Version: '5.5.37-35.0-55-log'  socket: '/tmp/mysql.sock'  port: 3306  Percona XtraDB Cluster (GPL), Release rel35.0, Revision 756, WSREP version 25.10, wsrep_25.10.r3985
      
      
      
      
      
      140603 12:35:40 [ERROR] Error in Log_event::read_log_event(): 'Sanity check failed', data_len: 50, event_type: 30
      140603 12:35:40 [ERROR] WSREP: applier could not read binlog event, seqno: 5614, len: 0
      140603 12:35:40 [Warning] WSREP: Failed to apply app buffer: seqno: 5614, status: 1
               at galera/src/trx_handle.cpp:apply():340
      Retrying 2th time
      140603 12:35:40 [ERROR] Error in Log_event::read_log_event(): 'Sanity check failed', data_len: 50, event_type: 30
      140603 12:35:40 [ERROR] WSREP: applier could not read binlog event, seqno: 5614, len: 0
      140603 12:35:40 [Warning] WSREP: Failed to apply app buffer: seqno: 5614, status: 1
               at galera/src/trx_handle.cpp:apply():340
      Retrying 3th time
      140603 12:35:40 [ERROR] Error in Log_event::read_log_event(): 'Sanity check failed', data_len: 50, event_type: 30
      140603 12:35:40 [ERROR] WSREP: applier could not read binlog event, seqno: 5614, len: 0
      140603 12:35:40 [Warning] WSREP: Failed to apply app buffer: seqno: 5614, status: 1
               at galera/src/trx_handle.cpp:apply():340
      Retrying 4th time
      140603 12:35:40 [ERROR] Error in Log_event::read_log_event(): 'Sanity check failed', data_len: 50, event_type: 30
      140603 12:35:40 [ERROR] WSREP: applier could not read binlog event, seqno: 5614, len: 0
      140603 12:35:40 [ERROR] WSREP: Failed to apply trx: source: f4882da2-eaed-11e3-9489-b66dd25d467d version: 3 local: 0 state: APPLYING flags: 1 conn_id: 4434 trx_id: 10039 seqnos (l: 5, g: 5614, s: 5613, d: 5613, ts: 63733643875911)
      140603 12:35:40 [ERROR] WSREP: Failed to apply trx 5614 4 times
      140603 12:35:40 [ERROR] WSREP: Node consistency compromized, aborting...
      140603 12:35:40 [Note] WSREP: Closing send monitor...
      140603 12:35:40 [Note] WSREP: Closed send monitor.
      140603 12:35:40 [Note] WSREP: gcomm: terminating thread
      140603 12:35:40 [Note] WSREP: gcomm: joining thread
      140603 12:35:40 [Note] WSREP: gcomm: closing backend
      140603 12:35:41 [Note] WSREP: view(view_id(NON_PRIM,f4882da2-eaed-11e3-9489-b66dd25d467d,26) memb {
              f6af48e1-eaf2-11e3-9723-db01c76922f6,0
      } joined {
      } left {
      } partitioned {
              f4882da2-eaed-11e3-9489-b66dd25d467d,0
      })
      140603 12:35:41 [Note] WSREP: view((empty))
      140603 12:35:41 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
      140603 12:35:41 [Note] WSREP: gcomm: closed
      140603 12:35:41 [Note] WSREP: Flow-control interval: [16, 16]
      140603 12:35:41 [Note] WSREP: Received NON-PRIMARY.
      140603 12:35:41 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 5614)
      140603 12:35:41 [Note] WSREP: Received self-leave message.
      140603 12:35:41 [Note] WSREP: Flow-control interval: [0, 0]
      140603 12:35:41 [Note] WSREP: Received SELF-LEAVE. Closing connection.
      140603 12:35:41 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 5614)
      140603 12:35:41 [Note] WSREP: RECV thread exiting 0: Success
      140603 12:35:41 [Note] WSREP: recv_thread() joined.
      140603 12:35:41 [Note] WSREP: Closing replication queue.
      140603 12:35:41 [Note] WSREP: Closing slave action queue.
      140603 12:35:41 [Note] WSREP: /usr/sbin/mysqld: Terminated.
      140603 12:35:41 mysqld_safe Number of processes running now: 0
      140603 12:35:41 mysqld_safe WSREP: not restarting wsrep node automatically
      140603 12:35:41 mysqld_safe mysqld from pid file /tmp/mysql.pid ended

      Comment


      • #4
        How run Cluster with xtrabackup?

        Comment


        • #5
          Can you paste your my.cnf file? Did you also try xtrabackup-v2?
          http://www.percona.com/doc/percona-x...rep_sst_method

          Comment


          • #6
            Option xtrabackup-v2 work good, but not successfully join node3 (node1 ex db and node2 ex mysql working):

            Code:
            140610 11:49:21 [Note] WSREP: gcomm: connected
            140610 11:49:21 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
            140610 11:49:21 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
            140610 11:49:21 [Note] WSREP: Opened channel 'mydb'
            140610 11:49:21 [Note] WSREP: Waiting for SST to complete.
            140610 11:49:21 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
            140610 11:49:21 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 1da6f842-f07c-11e3-b4e6-170c73cd6cd1
            140610 11:49:21 [Note] WSREP: STATE EXCHANGE: sent state msg: 1da6f842-f07c-11e3-b4e6-170c73cd6cd1
            140610 11:49:21 [Note] WSREP: STATE EXCHANGE: got state msg: 1da6f842-f07c-11e3-b4e6-170c73cd6cd1 from 0 (mydb03)
            140610 11:49:21 [Note] WSREP: STATE EXCHANGE: got state msg: 1da6f842-f07c-11e3-b4e6-170c73cd6cd1 from 1 (db)
            140610 11:49:21 [Note] WSREP: STATE EXCHANGE: got state msg: 1da6f842-f07c-11e3-b4e6-170c73cd6cd1 from 2 (mysql)
            140610 11:49:21 [Note] WSREP: Quorum results:
                    version    = 3,
                    component  = PRIMARY,
                    conf_id    = 57,
                    members    = 2/3 (joined/total),
                    act_id     = 850,
                    last_appl. = -1,
                    protocols  = 0/5/2 (gcs/repl/appl),
                    group UUID = 4c570ccd-ebfe-11e3-916a-3223af6b893e
            140610 11:49:21 [Note] WSREP: Flow-control interval: [28, 28]
            140610 11:49:21 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 850)
            140610 11:49:21 [Note] WSREP: State transfer required:
                    Group state: 4c570ccd-ebfe-11e3-916a-3223af6b893e:850
                    Local state: 00000000-0000-0000-0000-000000000000:-1
            140610 11:49:21 [Note] WSREP: New cluster view: global state: 4c570ccd-ebfe-11e3-916a-3223af6b893e:850, view# 58: Primary, number of nodes: 3, my index: 0, protocol version 2
            140610 11:49:21 [Warning] WSREP: Gap in state sequence. Need state transfer.
            140610 11:49:23 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.10.48.21' --auth 'replicator:Shoh4ool' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '26671''
            WSREP_SST: [INFO] Streaming with xbstream (20140610 11:49:23.589)
            WSREP_SST: [INFO] Using socat as streamer (20140610 11:49:23.590)
            WSREP_SST: [INFO] Evaluating timeout 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | pv -f  -i 10 -N joiner  2>>/tmp/sstl.progress | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20140610 11:49:23.728)
            140610 11:49:23 [Note] WSREP: Prepared SST request: xtrabackup-v2|10.10.48.21:4444/xtrabackup_sst
            140610 11:49:23 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
            140610 11:49:23 [Note] WSREP: REPL Protocols: 5 (3, 1)
            140610 11:49:23 [Note] WSREP: Service thread queue flushed.
            140610 11:49:23 [Note] WSREP: Assign initial position for certification: 850, protocol version: 3
            140610 11:49:23 [Note] WSREP: Service thread queue flushed.
            140610 11:49:23 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (4c570ccd-ebfe-11e3-916a-3223af6b893e): 1 (Operation not permitted)
                     at galera/src/replicator_str.cpp:prepare_for_IST():447. IST will be unavailable.
            140610 11:49:23 [Note] WSREP: Member 0.0 (mydb03) requested state transfer from '*any*'. Selected 1.0 (db)(SYNCED) as donor.
            140610 11:49:23 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 850)
            140610 11:49:23 [Note] WSREP: Requesting state transfer: success, donor: 1
            2014/06/10 11:49:24 socat[26946] E write(1, 0xd963e0, 133): Broken pipe
            WSREP_SST: [ERROR] Error while getting data from donor node:  exit codes: 1 127 0 (20140610 11:49:24.305)
            WSREP_SST: [ERROR] Cleanup after exit with status:32 (20140610 11:49:24.306)
            WSREP_SST: [INFO] Removing the sst_in_progress file (20140610 11:49:24.307)
            140610 11:49:24 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.10.48.21' --auth 'xxx:yyyyy' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --parent '26671': 32 (Broken pipe)
            140610 11:49:24 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
            140610 11:49:24 [ERROR] WSREP: SST failed: 32 (Broken pipe)
            140610 11:49:24 [ERROR] Aborting
            
            140610 11:49:26 [Note] WSREP: Closing send monitor...
            140610 11:49:26 [Note] WSREP: Closed send monitor.
            140610 11:49:26 [Note] WSREP: gcomm: terminating thread
            140610 11:49:26 [Note] WSREP: gcomm: joining thread
            140610 11:49:26 [Note] WSREP: gcomm: closing backend
            140610 11:49:26 [Note] WSREP: view(view_id(NON_PRIM,1d5a4878-f07c-11e3-a03d-caab901a1248,58) memb {
                    1d5a4878-f07c-11e3-a03d-caab901a1248,0
            } joined {
            } left {
            } partitioned {
                    4aff8818-f079-11e3-b2be-2b3e4a2741ed,0
                    f8408c3e-f078-11e3-ab86-da1737463817,0
            })
            140610 11:49:26 [Note] WSREP: view((empty))
            140610 11:49:26 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
            140610 11:49:26 [Note] WSREP: gcomm: closed
            140610 11:49:26 [Note] WSREP: Flow-control interval: [16, 16]
            140610 11:49:26 [Note] WSREP: Received NON-PRIMARY.
            140610 11:49:26 [Note] WSREP: Shifting JOINER -> OPEN (TO: 850)
            140610 11:49:26 [Note] WSREP: Received self-leave message.
            140610 11:49:26 [Note] WSREP: Flow-control interval: [0, 0]
            140610 11:49:26 [Note] WSREP: Received SELF-LEAVE. Closing connection.
            140610 11:49:26 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 850)
            140610 11:49:26 [Note] WSREP: RECV thread exiting 0: Success
            140610 11:49:26 [Note] WSREP: recv_thread() joined.
            140610 11:49:26 [Note] WSREP: Closing replication queue.
            140610 11:49:26 [Note] WSREP: Closing slave action queue.
            140610 11:49:26 [Note] WSREP: Service disconnected.
            140610 11:49:26 [Note] WSREP: rollbacker thread exiting
            140610 11:49:27 [Note] WSREP: Some threads may fail to exit.
            140610 11:49:27 [Note] /usr/sbin/mysqld: Shutdown complete


            My.cnf:
            Code:
            [sst]
            streamfmt=xbstream
            #incremental=1
            #rebuild=1
            progress=/tmp/sstl.progress
            #nsferfmt=socat
            
            [mysql]
            port                           = 3306
            # socket                         = /tmp/mysql.sock
            
            [mysqld]
            bind-address                   = 0.0.0.0
            user                           = mysql
            default-storage-engine         = InnoDB
            # socket                         = /tmp/mysql.sock
            # pid-file                       = /tmp/mysql.pid
            character-set-server=utf8
            collation-server=utf8_general_ci
            init-connect                   = 'SET NAMES utf8'
            
            wsrep_cluster_address=gcomm://ALL_NODES_IPS
            
            key-buffer-size                = 128Mb
            myisam-recover                 = FORCE,BACKUP
            
            # SAFETY #
            max-allowed-packet             = 16M
            max-connect-errors             = 1000000
            skip-name-resolve
            sql-mode                       = STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_AUTO_VALUE_ON_ZERO,NO_ENGINE_SUBSTITUTION,NO_ZERO_DATE,NO_ZERO_IN_DATE,ONLY_FULL_GROUP_BY
            sysdate-is-now                 = 1
            innodb                         = FORCE
            innodb-strict-mode             = 1
            #innodb_force_recovery         = 1
            log-slave-updates
            # DATA STORAGE #
            datadir                        = /var/lib/mysql/
            
            # BINARY LOGGING #
            log-bin                        = /var/lib/mysql/mysql-bin
            expire-logs-days               = 60
            sync-binlog                    = 1
            server-id                      = 2
            max_binlog_size                = 1G
            binlog_format                  = row
            wsrep_provider                 = /usr/lib64/libgalera_smm.so
            wsrep_slave_threads            = 8
            wsrep_cluster_name             = mydb
            wsrep_provider_options         = "gmcast.listen_addr=tcp://mydb03;"
            
            wsrep_sst_method               = xtrabackup-v2
            
            wsrep_sst_auth                 = xxxx:yyyy
            wsrep_node_name                = mydb03
            
            # CACHES AND LIMITS #
            tmp-table-size                 = 32M
            max-heap-table-size            = 32M
            query-cache-type               = 0
            query-cache-size               = 0
            max-connections                = 5000000
            thread-cache-size              = 1000
            open-files-limit               = 65535
            table-definition-cache         = 4096
            table-open-cache               = 128
            
            # INNODB #
            innodb-flush-method            = O_DIRECT
            innodb-log-files-in-group      = 2
            innodb-log-file-size           = 256M
            innodb-flush-log-at-trx-commit = 1
            innodb-file-per-table          = 1
            innodb-buffer-pool-size        = 5000M
            innodb_locks_unsafe_for_binlog = 1
            innodb_autoinc_lock_mode       = 2
            innodb_locks_unsafe_for_binlog = 1
            # LOGGING #
            log-error                      = /var/log/mysql-error.log
            log-queries-not-using-indexes  = 0
            slow-query-log                 = 0
            slow-query-log-file            = /var/log/mysql-slow.log
            general_log_file               = /var/log/mysql.log
            general_log                    = 0
            
            
            [mysqld_safe]
            wsrep_cluster_address=gcomm://ALL_NODES_IPS

            Comment


            • #7
              any ideas?

              Comment


              • #8
                Please help to fix this problem with perconaDB

                Comment


                • #9
                  PLEASE HELP!

                  Comment


                  • #10
                    140610 11:49:24 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
                    140610 11:49:24 [ERROR] WSREP: SST failed: 32 (Broken pipe)

                    Seems that SST failed on donor (db). When a SST is executed on a server the log of that xtrabackup process goes to files with the name innobackup.backup.log and/innobackup.prepare.log. They are on the data dir. If you share them (with error log too) we can check it to see what could be the problem. Maybe the answer is there

                    Comment


                    • #11
                      i found file /tmp/sstl.progress with contents:

                      Code:
                      /usr//bin/wsrep_sst_xtrabackup-v2: line 106: pv: command not found
                      PS: Prompts to me that percona DB is not yet stable enough and ready to run on the stable projects. Because more time to start cluster and so more errors + no support.
                      Last edited by panaceya; 06-27-2014, 02:59 AM.

                      Comment


                      • #12
                        Percona XtraDB Cluster is stable, being used in really large projects and you have all the support you need in this link:

                        http://www.percona.com/products/mysql-support

                        This is a community forum

                        I still would need innobackup.backup.log and/innobackup.prepare.log files, but the error you pasted indicates that you need to install pv tool (you can use yum or apt). pv is used to throttle the backup, as you can read in our documentation:

                        http://www.percona.com/doc/percona-x...kx_stream.html

                        So, install the tool on all the nodes and try again please.

                        Comment

                        Working...
                        X