Emergency

Can't join node to cluster after restart

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Can't join node to cluster after restart

    Hi everyone!. I have three nodes which installed Percona-Xtra-DB 5.7.19-29.22-1.xenial on Ubuntu 16.04, the cluster.cnf configure file.


    ```
    [mysqld]
    datadir = /var/lib/mysql
    user = mysql
    default-time-zone = '+7:00'
    max_connections = 500
    bind-address = 10.5.68.19
    slow-query-log = /var/log/mysql/mysql-slow.log
    long_query_time = 2

    # UTF8
    default-storage-engine = innodb
    innodb_file_per_table = on
    collation-server = utf8_general_ci
    init-connect = 'SET NAMES utf8'
    character-set-server = utf8

    # Path to Galera library
    wsrep_provider = /usr/lib/galera3/libgalera_smm.so
    # Cluster connection URL contains IPs of node#1, node#2 and node#3
    wsrep_cluster_address = gcomm://10.5.68.31,10.5.68.12,10.5.68.33,

    # In order for Galera to work correctly binlog format should be ROW
    binlog_format = ROW
    # MyISAM storage engine has only experimental support
    default_storage_engine = InnoDB
    # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
    innodb_autoinc_lock_mode = 2
    # Node #2 address
    wsrep_node_address = 10.5.68.31
    # Cluster name
    wsrep_cluster_name = MySQL_CLUSTER_DEV_TEAM
    # SST method
    wsrep_sst_method = xtrabackup-v2
    #Authentication for SST method
    wsrep_sst_auth = "sstuser:aQq0azaaaaaaa
    ```

    when I reconfigure with value of `max_allowed_packet` , I restart node -1 then restart node-2 and node3, But node2 , node-3 can;t join cluster with loop message

    ``
    2017-12-06T09:58:42.560757Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:43.063215Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:43.562064Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:44.064023Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:44.562928Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:45.064980Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:45.564467Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:46.066043Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:46.565168Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:47.067126Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:47.566116Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:48.068213Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:48.567062Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:49.069196Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:49.568081Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:50.070429Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:50.570048Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:51.071358Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:51.570904Z 0 [Warning] WSREP: Member 0.0 (percona-xtradb-3) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    2017-12-06T09:58:52.072327Z 0 [Warning] WSREP: Member 2.0 (percona-xtradb-2) requested state transfer from '*any*', but it is impossible to select State Transfer donor: Resource temporarily unavailable
    ```

    I removed /var/lib/mysql/* on node-3, But It can't solve this problem.

    Please help me. Thanks

  • #2
    * what are states of other nodes of the cluster. From the message, it sounds like it is unable to find a valid DONOR. This could be because existing nodes are already in DONOR/DESYCNED state or some other state that prevent it from donating data.

    Comment

    Working...
    X