Comprehensive guide to Redis. Part 4.

In this tutorial, we are going to overview other Redis commands and features.

Pub/Sub

Publish-Subscribe is a pattern where publishers send messages to channels, and subscribers receive these messages if they are listening to a given channel.

Pub/Sub use cases:
– chat apps
– push notifications
– command dashboards
– remote code execution

PUBLISH: sends a message to the Redis channel. Returns the number of clients that received that message.
SUBSCRIBE: subscribes a client to one or many channels
UNSUBSCRIBE: unsubscribes a client from one or many channels
PSUBSCRIBE: subscribes a client to one or many channels. Accepts glob-style patterns as channel names.
PUNSUBSCRIBE: unsubscribes a client from one or many channels. Accepts glob-style patterns as channel names.
PUBSUB: checks the state of the Redis Pub/Sub system. Accepts 3 subcommands: CHANNELS, NUMSUB, NUMPAT.
PUBSUB CHANNELS [pattern]: returns all channels with at least 1 subscriber. Accepts an optional glob-style pattern.
PUBSUB NUMSUB [channel1 … channelN]: returns the number of clients connected to channels via the SUBSCRIBE command. Accepts channel names as arguments.
PUBSUB NUMPAT: returns the number of clients connected to channels via the PSUBSCRIBE command.

Notice that when a Redis client executes the SUBSCRIBE or PSUBSCRIBE command, it stops accepting commands, except for SUBSCRIBE, PSUBSCRIBE, UNSUBSCRIBE, PUNSUBSCRIBE.

We are going to create a remote command execution system. In this system, a command is sent to a channel and the server that is subscribed to that channel executes the command.

Create a file called publisher.js with the following code:

Create a file called subscriber.js with the following code:

Now open 3 terminal windows and run the previous files. You should see the following:

img1

Transactions

A Redis transaction is a sequence of commands executed in order and atomically. The MULTI command marks the beginning of the transaction. The EXEC command marks the end. Any commands between the MULTI and EXEC commands are executed as an atomic operator. To prevent a transaction from being executed use the DISCARD command instead of EXEC.

Notice that transactions in Redis are not rolled back. If one of the commands fail, Redis proceeds to the next command.

The following example simulates a bank transfer. Money is transferred from the source account to a destination account.

Create a file called bank-transaction.js with the following code:

Now execute the file. You should see the following output:

img2

WATCH: implements an optimistic lock on a group of keys. Marks keys as being watched so that the EXEC command executes the transaction only if the keys were not changed. Otherwise, returns null and the operation needs to be repeated.
UNWATCH: removes keys from a watch list.

The following example implements a zpop function, which removes the first element of a Sorted Set and passes it to a callback function, using a transaction with WATCH.

Create a file called watch-transaction.js with the following code:

Now execute the file. You should see the following output:

img3

Pipelines

A pipeline is a way to send multiple commands together to the Redis server without waiting for replies. The replies are read all at once by a client. The time taken for a Redis client to send a command and receive a response from the Redis server is called RTT (Round Trip Time).

Redis without pipelines:
img4

Redis commands run sequentially in the server, but they are neither transactional nor atomic. By default, node_redis, the Node.js library, sends commands in pipelines. However, other Redis clients may not use pipelines by default.

Redis with pipelines:
img5

Lua scripting

Redis 2.6 introduced Lua scripting feature. Lua scripts are atomic, which means that the Redis server is blocked during script execution. Redis has a default timeout of 5 seconds to run any script. This value can ba changed through the configuration lua-time-limit.

When Lua script times out Redis will not automatically terminate it. The Redis server will start to reply with a BUSY message to every command. In this case, you should abort script execution with the command SCRIPT KILL or SHUTDOWN NOSAVE.

A Redis client must send Lua scripts as strings to the Redis server. There are 2 functions that execute Redis commands: redis.call and redis.pcall.

redis.call: requires the command name and all it parameters. Returns the result of the executed command. If there are errors, aborts the script.
redis.pcall: similar to redis.call, but when it is an error, this function returns the error as a Lua table and continues the script execution.

It is possible to pass Redis key names and parameters to a Lua script. They will be available through the KEYS and ARGV variables.

There are 2 commands to run Lua scripts: EVAL and EVALSHA.

Syntax:
EVAL script numkeys key [key …] arg [arg …]
script – the Lua script itself
numkeys – the number of Redis keys being passed
key – the key name that will be available through the KEYS variable inside the script
arg – an additional argument. It will be available through the ARGV variable.

The following example uses Lua to run the GET command and retrieve a key value. Create a file called luaget.js with the following code:

Then execute it. You should see the following:
img6

The next example will be the implementation of the zpop function as a Lua script. It will be atomic as Redis will always guarantee that there are no parallel changes to the Sorted Set during script execution.

Create a file called zpop-lua.js with the following code:

Run the above code. You should see the following console output:
img7

When executing the same script multiple times, you can save network bandwidth usage by using the commands SCRIPT LOAD and EVALSHA instead of EVAL. The SCRIPT LOAD command caches a Lua script and returns an identifier. The EVALSHA command executes a Lua script based on that identifier. With EVALSHA, over a small identifier is transferred over the network.

Create a file called evalsha-example.js with the following code:

Then execute the script. You should see the following output:
img8

Misc commands

INFO: returns all Redis server statistics, including the Redis version, OS, connected clients, memory usage, persistence, keyspace, and replication. By default, shows all available sections: memory, persistence, CPU, command, cluster, replication, and clients.
img9

DBSIZE: returns the number of existing keys in a Redis server.
DEBUG SEGFAULT: crashes the Redis server process by performing an invalid memory access.
MONITOR: shows all the commands processed by the Redis server in real time.
img10

CLIENT LIST: returns a list of all clients connected to the server.
CLIENT SETNAME: changes a client name.
CLIENT KILL: terminates a client connection. It is possible to terminate by IP, port, ID, or type.
img11

FLUSHALL: deletes all keys from Redis.
RANDOMKEY: returns a random existing key name.
EXPIRE: sets a timeout in seconds for a given key. The key will be deleted after the specified amount of seconds. A negative timeout will delete the key instantaneously.
EXPIREAT: sets a timeout for a given key based on a Unix timestamp.
TTL: returns the remaining time to live(in seconds) of a key that has an associated timeout. Returns -1 if the key does not have an associated timeout. Returns -2 if the key does not exist.
PTTL: the same as TTL, but the return value is in milliseconds.
SET: set a value to a given key.
Syntax:
SET key value [EX seconds|PX milliseconds] [NX|XX]
EX – set an expiration time in seconds
PX – set an expiration time in milliseconds
NX – only set the key if it does not exist
XX – only set the key if it already exists

img12

PERSIST: removes the existing timeout of a given key. Returns 1 if the timeout is removed of 0 if the key does not have an associated timeout.
SETEX: sets a value to a given key and an expiration.
DEL: removes one or many keys from Redis. Returns the number of removed keys.
EXISTS: returns 1 if a certain key exists and 0 if it does not.
PING: returns “PONG”. Useful for testing server/client connection.
MIGRATE: moves a given key to a destination Redis server. This command is atomic, and both Redis servers will be blocked during the key migration.
Syntax:
MIGRATE host port key destination-db timeout [COPY] [REPLACE]
COPY – keep the key in the local Redis server and create a copy in the destination server.
REPLACE – replace the existing key in the destination server.
SELECT: changes the current database that the client is connected to. Redis has 16 databases by default.
AUTH: is used to authorize a client to connect to Redis.
SCRIPT KILL: terminates the running Lua script if no write operations have been performed by the script. If the script has performed any write operations, the SHUTDOWN NOSAVE command must be executed. Returns OK, NOTBUSY, and UNKILLABLE.
SHUTDOWN: stops all client, causes data to persist if enabled, and shuts down the Redis server. Accepts optional parameters:
SAVE – forces Redis to save all of the data to a file called dump.rdb.
NOSAVE – prevents Redis from persisting data to the disk.
OBJECT ENCODING: returns the encoding used by a given key.

img13

Optimizations

All data types in Redis can use different encodings to improve performance or save memory. A String that has only digits (1234) uses less memory that a string of letters because they use different encodings. Data types use different encodings based on thresholds defined in the Redis configuration file (redis.conf).

Start a Redis server with low values for all configurations.
img14

String

Available String encodings:
int: is used when the string is represented by a 64-bit signed integer
embstr: is used for strings fewer that 40 bytes
raw: is used for strings more than 40 bytes

img15

List

Available encodings for Lists:
ziplist: is used when the List size has fewer elements than the configuration list-max-ziplist-entries and each List element has fewer bytes than the configuration list-max-ziplist-value
linkedlist: us used when the previous limits are exceeded

img16

Set

Available encodings for Sets:
intset: is used when all elements of a Set are integers and the Set cardinality is smaller than set-max-intset-entries
hashtable: is used when any element of a Set is not an integer or the Set cardinality exceeds set-max-intset-entries

img17

Hash

Available encodings for Hashes:
ziplist: is used when the number of fields in the Hash does to exceed the hash-max-ziplist-entries and each field name and value of the Hash is less(in bytes) that the hash-max-ziplist-value
hashtable: is used when a Hash size or any of its values exceed the hash-max-ziplist-entries and hash-max-ziplist-value

img18

Sorted Set

Available encodings:
ziplist: is used when a Sorted Set has fewer entries than the set-max-ziplist-entries and each of its values are smaller(in bytes) than zset-max-ziplist-value
skiplist: is used when the Sorted Set number of entries or size of any of its values exceeds the set-max-ziplist-entries and zset-max-ziplist-value

To sum up, if you have a large dataset and need to optimize for memory, you can tweak these configurations until you find a good trade-off between memory and performance. That’s all for today :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">