Mysql Galera WSREP_SST: [ERROR] xtrabackup_checkpoints missing

Adding new node to galera fails with following error :

WSREP_SST: [INFO] Waiting for SST streaming to complete! (20160607 12:21:20.988)
160607 12:21:23 [Note] WSREP: (3d7453d3, 'tcp://') turning message relay requesting off
WSREP_SST: [ERROR] xtrabackup_checkpoints missing, failed innobackupex/SST on donor (20160607 12:21:30.966)
WSREP_SST: [ERROR] Cleanup after exit with status:2 (20160607 12:21:30.968)
160607 12:21:30 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '' --auth 'sstuser:dev2' --datadir '/var/untd/mysql/data/' --defaults-file '/etc/my.cnf' --parent '4858': 2 (No such file or directory)
160607 12:21:30 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
160607 12:21:30 [ERROR] WSREP: SST failed: 2 (No such file or directory)
160607 12:21:30 [ERROR] Aborting

160607 12:21:30 [Warning] WSREP: 1 ( State transfer to 0 (node2) failed: -22 (Invalid argument)
160607 12:21:30 [ERROR] WSREP: gcs/src/gcs_group.cpp:long int gcs_group_handle_join_msg(gcs_group_t*, const gcs_recv_msg_t*)():717: Will never receive state. Need to abort.
160607 12:21:30 [Note] WSREP: gcomm: terminating thread
160607 12:21:30 [Note] WSREP: gcomm: joining thread
160607 12:21:30 [Note] WSREP: gcomm: joining thread
160607 12:21:30 [Note] WSREP: gcomm: closing backend
160607 12:21:31 [Note] WSREP: gcomm: closed
160607 12:21:31 [Note] WSREP: /usr/sbin/mysqld: Terminated.
160607 12:21:32 mysqld_safe mysqld from pid file /var/untd/mysql/log/ ended

Fix :

  • Verify wsrep_cluster_address is set to donor node address in following format :
  • wsrep_sst_method is configured properly.
  • wsrep_sst_auth is configured properly in following format and sstuser exists on donor node with proper privileges. This parameter should be under [mysqld] section
  • sstuser on primary/donor is configured as ‘sstuser’@’localhost’ instead of ‘sstuser’@’%’
  • wsrep_provider points correctly to file, for example it may be pointing to /usr/lib/ instead of /usr/lib64/

5 thoughts to “Mysql Galera WSREP_SST: [ERROR] xtrabackup_checkpoints missing”

  1. Way cool! Some extremely valid points! I appreciate you writing this
    post plus the rest of the website is also really good.

  2. HI Goutham,

    I have one doubt, i have 3 node cluster in our environment and we are facing the same error.

    on all 3 nodes we have same provider (/usr/lib64/ as per your article if we changed wsrep_provider = /usr/lib64/ to /usr/lib/ , is there any problem to reaming two nodes ? or we have to change the parameter on all 3 nodes ?

    1. Hi Sandeep,

      Are you facing this issue while adding a new node to the cluster?
      > is there any problem to remaining two nodes ? or we have to change the parameter on all 3 nodes?
      No, there should not be any problem with remaining nodes and we need to change this variable(if needed) only on the node which is failing.

  3. Hit the same issue: the solution was to read the donor’s log (/var/lib/mysql/innobackup.backup.log) and not the joiner’s log!

    For instance, my donor says:
    InnoDB: Operating system error number 24 in a file operation.
    InnoDB: Error number 24 means ‘Too many open files’.
    Now I know what to look for.

Leave a Reply

Your email address will not be published.