Mike Conigliaro

Tokyo Tyrant Update Log (ulog) Management

When setting up dual-master replication for Tokyo Tyrant, it won’t be long before you discover that old update logs hang around forever and eventually cause your disk to fill up. I found this sample script that attempts to remedy the problem, but I also found it pretty scary that it blindly deletes old logs without taking the replication delay into account. This means that (however unlikely it may be) it’s possible to delete logs prematurely. The script below uses the replication delay to figure out which logs are safe to delete, and it also allows you retain a certain number of old logs just in case. Stick this in a cron job, and you’re good to go:

#!/bin/bash
#
# This script queries a slave to see how far behind it is in its replication
# (i.e. the 'delay'). This number is used to figure out which update logs are
# safe to delete.
#
# Author: Michael Conigliaro <mike [at] conigliaro [dot] org>
#

SLAVE=ttslave.example.com
ALWAYS_KEEP=1440
ULOG_DIR=/mnt/tokyo/data/ttserver.ulog

# get all slave stats (exit on failure)
OUTPUT=`tcrmgr inform -st $SLAVE`
if [ $? -ne 0 ]; then
    exit 1
fi

# use the delay value to figure out how many minutes worth of ulogs to keep
DELAY=`printf "%s" "$OUTPUT" | grep delay | cut -f 2`
OLDER_THAN=$((${DELAY/\.*/}/60+$ALWAYS_KEEP))

# delete old ulogs
printf "delay=%ssec (sec), older_than=%s (mins)\n" $DELAY $OLDER_THAN
find $ULOG_DIR -maxdepth 1 -name "*.ulog" -type f -mmin +${OLDER_THAN} -exec rm -v {} \;
blog comments powered by Disqus