Announcement

Announcement Module
Collapse
No announcement yet.

Xtradb cluster only replicating ddl statements

Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Xtradb cluster only replicating ddl statements

    I'm having a hard time with percona xtradb cluster. The cluster has 3 nodes, and is up and running (all tip top according to wsrep status indicators). It did the SST perfectly fine from one node to the other 2. BUT every time i do an update in a table, the write is not propagated to the other nodes, even though ddl statements such as table creation are! any idea of what’s going on? i turned all sorts of logging on and i keep seeing:
    “Skipping empty log_xid” messages in error log, but not affecting the tables im testing so not sure this is related..
    Im talking about innodb tables by the way, that where inherited from mysql 5.5 (not percona mysql). Attaching the status of wsrep_ of a single node, the other 2 are very similar. At the very least there should be some logging indicating where the problem is, the debug level logs of galera are silent about these queries!

    wsrep_local_state_uuid | 16819721-b6c1-11e2-0800-8d78b46e9a6d |
    | wsrep_protocol_version | 4 |
    | wsrep_last_committed | 38795 |
    | wsrep_replicated | 715 |
    | wsrep_replicated_bytes | 9288264 |
    | wsrep_received | 87 |
    | wsrep_received_bytes | 1760 |
    | wsrep_local_commits | 710 |
    | wsrep_local_cert_failures | 0 |
    | wsrep_local_bf_aborts | 0 |
    | wsrep_local_replays | 0 |
    | wsrep_local_send_queue | 0 |
    | wsrep_local_send_queue_avg | 0.000000 |
    | wsrep_local_recv_queue | 0 |
    | wsrep_local_recv_queue_avg | 0.000000 |
    | wsrep_flow_control_paused | 0.000000 |
    | wsrep_flow_control_sent | 0 |
    | wsrep_flow_control_recv | 0 |
    | wsrep_cert_deps_distance | 3.000000 |
    | wsrep_apply_oooe | 0.000000 |
    | wsrep_apply_oool | 0.000000 |
    | wsrep_apply_window | 1.000000 |
    | wsrep_commit_oooe | 0.000000 |
    | wsrep_commit_oool | 0.000000 |
    | wsrep_commit_window | 1.000000 |
    | wsrep_local_state | 4 |
    | wsrep_local_state_comment | Synced |
    | wsrep_cert_index_size | 685 |
    | wsrep_causal_reads | 0 |
    | wsrep_incoming_addresses | 10.0.2.17:3306,10.0.2.51:3306,10.0.2.11:3306 |
    | wsrep_cluster_conf_id | 3 |
    | wsrep_cluster_size | 3 |
    | wsrep_cluster_state_uuid | 16819721-b6c1-11e2-0800-8d78b46e9a6d |
    | wsrep_cluster_status | Primary |
    | wsrep_connected | ON |
    | wsrep_local_index | 0 |
    | wsrep_provider_name | Galera |
    | wsrep_provider_vendor | Codership Oy <info@codership.com> |
    | wsrep_provider_version | 2.5(r150) |
    | wsrep_ready | ON |
    Last edited by rveloso; 05-16-2013, 04:07 AM.

  • #2
    Can you please attach the my.cnf and the error log? That would be helpful.

    Comment


    • #3
      dump of mysql variables (i have too many config files)
      https://gist.github.com/anonymous/d6bae549ea9d4bae9ea6

      error log:
      https://gist.github.com/anonymous/8050335826568394cbf3

      Comment


      • #4
        Any update on this? Would be really nice to understand what else i can do to solve this. Either im doing something very basically wrong or there's a serious bug in the code. In any case, just the fact is not logging errors is bad...

        Comment


        • #5
          Are your tables all Innodb?

          Comment


          • #6
            yes, and im using a test table that's innodb, the table creation statement gets into the logs, but any change i do to the table afterwards gets dropped in the floor and i dont even see anything in the logs (including logs from other nodes). in debug mode, shouldnt wsrep log all statements? or is it just ddl?

            30520 17:28:23 [Note] WSREP: TO BEGIN: 57423, 2130520 17:28:24 [Note] WSREP: avoiding binlog rotate due to TO isolation: 1
            130520 17:28:24 [Note] WSREP: TO END: 57423, 2 : ALTER TABLE `test` CHANGE `id` `id` INT(11) UNSIGNED NOT NULL
            130520 17:28:24 [Note] WSREP: TO END: 57423

            Comment


            • #7
              Seems ddl statements have special treatment, how are non-ddl statements replicated and what would be the reason for that not working?
              http://www.percona.com/doc/percona-x...imitation.html
              "However, DDL statements are replicated in statement level, and changes to mysql.* tables will get replicated that way."

              Comment


              • #8
                Rveloso,

                We have talked about this problem and we have realized that it was caused by replicate_wild_do_table. Those filtering variables affects also cluster's replication. More info:

                https://groups.google.com/forum/?fro...am/1uw7cDIA510

                Comment


                • #9
                  Miguel: good catch! Replication filters (counter-intuitively IMO) do apply to Galera replication

                  Comment

                  Working...
                  X