A simple way for adding future partitions or purging old partitions is through mysql procedures. Later we can use any scheduler or a cronjob to call this procedure daily.
Here I will provide a sample procedure for maintainance of a daily partitioned table based on a datetime column. Please do look at my previous post on mysql range partitioning in the following post. I have taken the example from this post and modified it based on our requirement.
First we will create a sample table along with a partition for our testing purposes.
mysql> create table test_partitions(id int,name varchar(50),tx_date datetime);
mysql> alter table test_partitions PARTITION BY RANGE (to_seconds(tx_date)) (
partition test_partitions_16092015 values less than ( to_seconds('2015-09-17 00:00:00') ));
Now lets create a table for holding list of tables and number of partitions to preserve/drop.
mysql> create table test.part_tables(dbname varchar(50),tabname varchar(50),numpart int,nextpart int);
Lets go ahead and insert a sample row there :
mysql> insert into test.part_tables values('test','test_partitions',30,7);
So, we will be preserving only last 30 days partitions and make sure we have next 7 days partitions for ‘test_partitions’ table.
Mysql Procedure 1 (part_maint) : This procedure takes in table list from our meta-table i.e part_tables and runs partition maintainance for each table :
DELIMITER $
DROP PROCEDURE IF EXISTS `test`.`part_maint` $
CREATE PROCEDURE `test`.`part_maint` ()
BEGIN
DECLARE var_dbname varchar(50);
DECLARE var_tabname varchar(50);
DECLARE var_numpart int;
DECLARE var_nextpart int;
DECLARE done INT DEFAULT FALSE;
DECLARE _output TEXT DEFAULT '';
DECLARE cursor_p CURSOR FOR select dbname,tabname,numpart,nextpart from test.part_tables;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
open cursor_p;
read_loop: LOOP
FETCH cursor_p INTO var_dbname, var_tabname, var_numpart, var_nextpart;
IF done THEN
LEAVE read_loop;
END IF;
SET _output = concat('Partitioning - ', var_dbname,'.', var_tabname);
SELECT _output;
CALL test.UpdatePartitions(var_dbname,var_tabname,var_numpart,var_nextpart);
SET _output = concat('---------------------------------------------');
SELECT _output;
END LOOP;
CLOSE cursor_p;
END $
DELIMITER ;
Above procedure is calling ‘UpdatePartitions’ procedure for each table.
Mysql Procedure 2 (UpdatePartitions) :
DELIMITER $$
DROP PROCEDURE IF EXISTS `test`.`UpdatePartitions` $$
CREATE PROCEDURE `test`.`UpdatePartitions` (dbname varchar(50), tblname varchar(50), numpart int, nextpart int)
BEGIN
-- numpart is the number of previous partitions to keep
-- nextpart is the number of next partitions to create
DECLARE maxpart decimal(38);
DECLARE partition_count int;
DECLARE minpart decimal(38);
DECLARE newpart decimal(38);
DECLARE _output TEXT DEFAULT '';
SELECT COUNT(*)
INTO partition_count
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_NAME=tblname AND TABLE_SCHEMA=dbname;
SET _output = concat('Number of partitions present=',partition_count);
SELECT _output;
WHILE (partition_count > numpart+nextpart)
DO
SELECT MIN(PARTITION_DESCRIPTION)
INTO minpart
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_NAME=tblname AND TABLE_SCHEMA=dbname;
SET _output = concat('Dropping partition - ',tblname,'_',date_format(date_sub(from_seconds(minpart),INTERVAL 1 DAY),'%d%m%Y'));
SELECT _output;
SET @sql := CONCAT('ALTER TABLE ',tblname,' DROP PARTITION ',CONCAT(tblname,'_',date_format(date_sub(from_seconds(minpart),INTERVAL 1 DAY),'%d%m%Y')),';');
PREPARE stmt FROM @sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SELECT COUNT(*)
INTO partition_count
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_NAME=tblname AND TABLE_SCHEMA=dbname;
END WHILE;
SELECT MAX(PARTITION_DESCRIPTION)
INTO maxpart
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_NAME=tblname AND TABLE_SCHEMA=dbname;
-- create enough partitions based on nextpart
WHILE (maxpart < to_seconds(CURDATE() + INTERVAL nextpart+1 DAY))
DO
SET newpart := maxpart;
SET _output = concat('Adding partition - ',tblname,'_',date_format(from_seconds(newpart),'%d%m%Y'));
SELECT _output;
SET @sql := CONCAT('ALTER TABLE ',tblname,' ADD PARTITION (PARTITION ',CONCAT(tblname,'_',date_format(from_seconds(newpart),'%d%m%Y')), ' values less than(' ,to_seconds(date_add(from_seconds(newpart),INTERVAL 1 DAY)), ') );');
PREPARE stmt FROM @sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SELECT MAX(PARTITION_DESCRIPTION)
INTO maxpart
FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_NAME=tblname AND TABLE_SCHEMA=dbname;
END WHILE;
END $$
DELIMITER ;
Lets add a cronjob for running the ‘part_maint’ procedure daily.
1) Write a simple shell script to call the procedure :
cat /tmp/part_maint.sh #/bin/bash date echo '----------------------------------------' mysql -uroot -S/var/lib/mysql1/data/mysql.sock test -N -B -e "CALL part_maint" printf "\n"
2) Add crontab entry to run daily at 3PM :
00 16 * * * /tmp/part_maint.sh >> /tmp/part_maint.log