Mysql Galera WSREP_SST: [ERROR] xtrabackup_checkpoints missing

Adding new node to galera fails with following error :

WSREP_SST: [INFO] Waiting for SST streaming to complete! (20160607 12:21:20.988)
160607 12:21:23 [Note] WSREP: (3d7453d3, 'tcp://0.0.0.0:4567') turning message relay requesting off
WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20160607 12:21:30.966)
WSREP_SST: [ERROR] Cleanup after exit with status:2 (20160607 12:21:30.968)
160607 12:21:30 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.103.34.27' --auth 'sstuser:dev2' --datadir '/var/untd/mysql/data/' --defaults-file '/etc/my.cnf' --parent '4858': 2 (No such file or directory)
160607 12:21:30 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
160607 12:21:30 [ERROR] WSREP: SST failed: 2 (No such file or directory)
160607 12:21:30 [ERROR] Aborting

160607 12:21:30 [Warning] WSREP: 1 (hadoop-dn1qa.cs1cmates.com): State transfer to 0 (node2) failed: -22 (Invalid argument)
160607 12:21:30 [ERROR] WSREP: gcs/src/gcs_group.cpp:long int gcs_group_handle_join_msg(gcs_group_t*, const gcs_recv_msg_t*)():717: Will never receive state. Need to abort.
160607 12:21:30 [Note] WSREP: gcomm: terminating thread
160607 12:21:30 [Note] WSREP: gcomm: joining thread
160607 12:21:30 [Note] WSREP: gcomm: joining thread
160607 12:21:30 [Note] WSREP: gcomm: closing backend
160607 12:21:31 [Note] WSREP: gcomm: closed
160607 12:21:31 [Note] WSREP: /usr/sbin/mysqld: Terminated.
160607 12:21:32 mysqld_safe mysqld from pid file /var/untd/mysql/log/mysqld.pid ended

Fix :

  • Verify wsrep_cluster_address is set to donor node address in following format :
    wsrep_cluster_address=’gcomm://<donor-ip>’
  • wsrep_sst_method is configured properly.
  • wsrep_sst_auth is configured properly in following format and sstuser exists on donor node with proper privileges. This parameter should be under [mysqld] section
    wsrep_sst_auth=<sstuser>:<password>
  • sstuser on primary/donor is configured as ‘sstuser’@’localhost’ instead of ‘sstuser’@’%’
  • wsrep_provider points correctly to libgalera_smm.so file, for example it may be pointing to /usr/lib/libgalera_smm.so instead of /usr/lib64/libgalera_smm.so

4 thoughts to “Mysql Galera WSREP_SST: [ERROR] xtrabackup_checkpoints missing”

  1. Way cool! Some extremely valid points! I appreciate you writing this
    post plus the rest of the website is also really good.

  2. HI Goutham,

    I have one doubt, i have 3 node cluster in our environment and we are facing the same error.

    on all 3 nodes we have same provider (/usr/lib64/libgalera_smm.so). as per your article if we changed wsrep_provider = /usr/lib64/libgalera_smm.so to /usr/lib/libgalera_smm.so , is there any problem to reaming two nodes ? or we have to change the parameter on all 3 nodes ?

    1. Hi Sandeep,

      Are you facing this issue while adding a new node to the cluster?
      > is there any problem to remaining two nodes ? or we have to change the parameter on all 3 nodes?
      No, there should not be any problem with remaining nodes and we need to change this variable(if needed) only on the node which is failing.

Leave a Reply

Your email address will not be published. Required fields are marked *