How is the message delivered in NSQD?

NSQ is a well-known Message Queue System, and one of the most important component, NSQD, takes charge of the message delivery. It accepts the message from outer publishers passively, stashs them either in memory (a fixed-size Golang Channel), or persisting it in the disk. In the meanwhile, it waits consumers’ arrival, pushes messages to these consumers as far as possible (tied with consumers’ RDY state).

This post introduces the “footprint” of a message when it was published from a publisher to an NSQD instance, until it was eaten by some consumers, whose channels are different.

Publishers can publish messages either through NSQD’s HTTP Restful APIs (/pub, /mpub), or over the TCP connection with NSQD’s client protocol (PUB, MPUB and DPUB commands). Whatever way you chose, after you received the normal response, your message had been saved (but not so safe). In the publishers’ side, messages are always tied with topic. Messages are always passed to topic’s gochan (or the disk queue if the memory channel is full) firstly, then they will be propagated to every channel, same put strategy is applied, actually each topic has such an exclusive goroutine to do this job (which is called messagePump in NSQD’s code base), it receives messages from the memory gochan or the backend gochan (supported by the disk queue facility), and iterates each channel, deliveries each message to them.

Consumers, however, is associated with the channel (not the Golang’s one). It subscribes a channel explicitly through the TCP protocol command SUB. With the constant coordination of RDY count, NSQD will push messages as far as possible, futhermore, once a message was sent to the client side, it would be saved in an “in-flight” queue, and there is some goroutines, named as “query scan workers”, always try to scan each topics’ each channel, what they search are timed out messages, those passed to client but without an timely FIN response, all of them would be re-pushed into the memory gochan or the disk queue. This feature, ensures messages’ safety in a way, although some risks still exist, like the NSQD instance was killed or the OS crashed, you can set the -mem-queue-size option to 0 to force NSQD puts messages in disk.

The total process, is really clear and simple, but powerful. Tries to use NSQD under the proper circumstances, it will help you alot!