This is Alex Zhang     About Me

The Internal of Redis Transaction

If you have the relational database background, you should be familiar with the transaction, execution of a transaction is atomic, which means either all the commands or none of them are performed.

Although Redis is well-known as a NO-SQL typed database, it really supports the transaction facility long before. Now I’d like to dissect the essence of Redis transaction in this post, the content is based on a relatively new redis version (actually on my own branch which has the newest commit 5e8cac3), so the description might diverge from your expectation slightly.

The behavior

It’s not surprising that there are only four transaction-relevant commands. They are:

The MULTI command is a trigger, which marks the current client into the transaction state when this command was deliverd by your Redis instance. After MULTI command is handled, all the subsequent commands will be queued rather than being executed immediately until the DISCARD or EXEC command is occurred.

The DISCARD command, just respects its literal meaning, discards all the queued commands and cancels the client’s transaction state; The EXEC command, just like the COMMIT statement of SQL-based DBMS, notifies Redis to execute the transaction.

But what about some of your commands are invalid? Say you just sent a command to your Redis server, if there is a syntactic error in your command (e.g. wrong number of arguments), Redis can detect it before queueing it, and error reply will be returned and this command wouldn’t be queued. Futhermore, current transaction will be marked as “dirty”, after the EXEC command arrived, Redis just replied “-EXECABORT Transaction discarded because of previous errors.”, and current client’s transaction state is also discarded. These kind of errors, we call it “programming faults”, because it’s visible in your development environment and can be rectified easily.

127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> get a 1111
(error) ERR wrong number of arguments for 'get' command
127.0.0.1:6379>

On the other hand, some invisible “runtime error” might arise in one of your commands, but only reported after the command is really executed, in such a case, command will be queued normally, because this command is right, at least in the syntax regard. When this command is being executed, Redis will detect this type of error and reply the corresponding message. But, what about the state of transaction? Does it abort? No, actually it will be executed without a break, Redis doesn’t “rollback” the transaction on this occasion.

127.0.0.1:6379> MULTI
OK
127.0.0.1:6379> INCR a
QUEUED
127.0.0.1:6379> HSET a b c
QUEUED
127.0.0.1:6379> EXEC
1) (integer) 2
2) (error) WRONGTYPE Operation against a key holding the wrong kind of value
127.0.0.1:6379> get a
"2"

So what the hell is the WATCH. You may know the optimistic locking mechanism, the WATCH command is really an implementation of optimistic locking, it allows you to monitor some keys, if any of them is changed before the transaction is executed, then the transaction would be aborted, and it’s clients’ responsibility to re-trigger the transaction.

The internal

OK, so now we have digested the behavior of Redis transaction, it’s time to analysis its implementations!

The source code is written in multi.c with hundreds of lines. Let’s start from the MULTI command, the command processor, multiCommand, just ORed the c->flags with CLIENT_MULTI. In addition, you cannot pass MULTI recursively, or you would get the error reply.

Now let’s focus on the behavior of function processCommand after the client into the transaction state, this is a high wrapper for the command calling. Looking through this function we find the last paragraph is related with transaction.

if (c->flags & CLIENT_MULTI &&
    c->cmd->proc != execCommand && c->cmd->proc != discardCommand &&
    c->cmd->proc != multiCommand && c->cmd->proc != watchCommand)
{
    queueMultiCommand(c);
    addReplay(c,shared.queued);
} else {
    ...
}

Conditions are concise, just check whether the client is in the transaction state and after excluding some transaction related commands, our command will be queued. So let’s summarize the queueMultiCommand by the way, it appends the current command to the c->mstate, which holds all the commands and their flags so far.

Now it’s DISCARD’s turn. The core part of discardCommand is calling discardTransaction, it frees all the queued commands and the relevant resouces, resets client’s transaction state and unwatches all the WATCHed keys (I will cover this in the later).

The most important command, EXEC, is implemented as the execCommand function, before really executing all the queued commands, this function checks the current transaction’s state, like whether this transaction is dirty (due to some previous errors), if so, transaction will be aborted, just like the DISCARD command’s behavior. Moreover, don’t forget the replication feature, if a transaction with some write operations is delivered in a Redis replica/slave instance, and the replica-read-only option is enabled, transaction will be discarded too. Only all the checks passed, could the commands be run, in the meanwhile, commands will be propagated into the append only file buffer and all the replicas (with the prologue command MULTI and epologue command EXEC).

Wait a minute, it seems that we neglect the WATCH command, right? The WATCH command handler watchCommand, is also simple enough, it builds the mapping between the target keys and the concerned clients (list) in c->db->watched_keys. Once one of keys is modified, the corresponding client list will be iterated and each of them will be marked with the flag CLIENT_DIRTY_CAS, which means this transaction now is dirty, and should be aborted while executing, this logic is boxed into the function touchWatchedKey.

void touchWatchedKey(redisDb *db, robj *key) {
    list *clients;
    listIter li;
    listNode *ln;

    if (dictSize(db->watched_keys) == 0) return;
    clients = dictFetchValue(db->watched_keys, key);
    if (!clients) return;

    listRewind(clients, &li);
    while ((ln = listNext(&li))) {
        client *c = listNodeValue(ln);

        c->flags |= CLIENT_DIRTY_CAS;
    }
}

So this is the basic descriptions of Redis transaction, it’s atomic, simple, but doesn’t support rollback. Using it properly may help us but frankly I even don’t have a chance to use it in the production environment… because the Lua script is preferable by constrast.