JSON Datatype overview
MemoryDB supports a number of Valkey and Redis OSS commands for working with the JSON datatype. Following is an overview of the JSON datatype and a detailed list of commands that are supported.
Terminology
Term | Description |
---|---|
JSON document |
refers to the value of a JSON key |
JSON value |
refers to a subset of a JSON Document, including the root that represents the entire document. A value could be a container or an entry within a container |
JSON element |
equivalent to JSON value |
Supported JSON standard
JSON format is compliant with RFC 7159
Root element
The root element can be of any JSON data type. Note that in earlier RFC 4627, only objects or arrays were allowed as root values. Since the update to RFC 7159, the root of a JSON document can be of any JSON data type.
Document size limit
JSON documents are stored internally in a format optimized for rapid access and modification. This format typically results in consuming somewhat more memory than does the equivalent serialized representation of the same document. The consumption of memory by a single JSON document is limited to 64MB, which is the size of the in-memory data structure, not the JSON string. The amount of memory consumed by a JSON document can be inspected by using the JSON.DEBUG MEMORY
command.
JSON ACLs
JSON datatype is fully integrated into the Valkey and Redis OSS Access Control Lists (ACL)
capability. Similar to the existing per-datatype categories (@string, @hash, etc.) a new category @json is added to simplify managing access to JSON commands and data. No other existing Valkey or Redis OSS commands are members of the @json category. All JSON commands enforce any keyspace or command restrictions and permissions. There are five existing ACL categories that are updated to include the new JSON commands: @read, @write, @fast, @slow and @admin. The table below indicates the mapping of JSON commands to the appropriate categories.
JSON Command | @read | @write | @fast | @slow | @admin |
---|---|---|---|---|---|
JSON.ARRAPPEND |
y |
y |
|||
JSON.ARRINDEX |
y |
y |
|||
JSON.ARRINSERT |
y |
y |
|||
JSON.ARRLEN |
y |
y |
|||
JSON.ARRPOP |
y |
y |
|||
JSON.ARRTRIM |
y |
y |
|||
JSON.CLEAR |
y |
y |
|||
JSON.DEBUG |
y |
y |
y |
||
JSON.DEL |
y |
y |
|||
JSON.FORGET |
y |
y |
|||
JSON.GET |
y |
y |
|||
JSON.MGET |
y |
y |
|||
JSON.NUMINCRBY |
y |
y |
|||
JSON.NUMMULTBY |
y |
y |
|||
JSON.OBJKEYS |
y |
y |
|||
JSON.OBJLEN |
y |
y |
|||
JSON.RESP |
y |
y |
|||
JSON.SET |
y |
y |
|||
JSON.STRAPPEND |
y |
y |
|||
JSON.STRLEN |
y |
y |
|||
JSON.STRLEN |
y |
y |
|||
JSON.TOGGLE |
y |
y |
|||
JSON.TYPE |
y |
y |
|||
JSON.NUMINCRBY |
y |
y |
Nesting depth limit
When a JSON object or array has an element that is itself another JSON object or array, that inner object or array is said to “nest” within the outer object or array. The maximum nesting depth limit is 128. Any attempt to create a document that contains a nesting depth greater than 128 will be rejected with an error.
Command syntax
Most commands require a Valkey or Redis OSS key name as the first argument. Some commands also have a path argument. The path argument defaults to the root if it is optional and not provided.
Notation:
Required arguments are enclosed in angle brackets, e.g. <key>
Optional arguments are enclosed in square brackets, e.g. [path]
Additional optional arguments are indicated by ..., e.g. [json ...]
Path syntax
JSON for Valkey and Redis OSS supports two kinds of path syntaxes:
Enhanced syntax – Follows the JSONPath syntax described by Goessner
, as shown in the table below. We've reordered and modified the descriptions in the table for clarity. Restricted syntax – Has limited query capabilities.
Note
Results of some commands are sensitive which type of path syntax is used.
If a query path starts with '$', it uses the enhanced syntax. Otherwise, the restricted syntax is used.
Enhanced Syntax
Symbol/Expression | Description |
---|---|
$ |
the root element |
. or [] |
child operator |
.. |
recursive descent |
* |
wildcard. All elements in an object or array. |
[] |
array subscript operator. Index is 0-based. |
[,] |
union operator |
[start:end:step] |
array slice operator |
?() |
applies a filter (script) expression to the current array or object |
() |
filter expression |
@ |
used in filter expressions referring to the current node being processed |
== |
equal to, used in filter expressions. |
!= |
not equal to, used in filter expressions. |
> |
greater than, used in filter expressions. |
>= |
greater than or equal to, used in filter expressions. |
< |
less than, used in filter expressions. |
<= |
less than or equal to, used in filter expressions. |
&& |
logical AND, used to combine multiple filter expressions. |
|| |
logical OR, used to combine multiple filter expressions. |
Examples
The below examples are built on Goessner's
{ "store": { "book": [ { "category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95, "in-stock": true, "sold": true }, { "category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99, "in-stock": false, "sold": true }, { "category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99, "in-stock": true, "sold": false }, { "category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99, "in-stock": false, "sold": false } ], "bicycle": { "color": "red", "price": 19.95, "in-stock": true, "sold": false } } }
Path | Description |
---|---|
$.store.book[*].author |
the authors of all books in the store |
$..author |
all authors |
$.store.* |
all members of the store |
$["store"].* |
all members of the store |
$.store..price |
the price of everything in the store |
$..* |
all recursive members of the JSON structure |
$..book[*] |
all books |
$..book[0] |
the first book |
$..book[-1] |
the last book |
$..book[0:2] |
the first two books |
$..book[0,1] |
the first two books |
$..book[0:4] |
books from index 0 to 3 (ending index is not inclusive) |
$..book[0:4:2] |
books at index 0, 2 |
$..book[?(@.isbn)] |
all books with isbn number |
$..book[?(@.price<10)] |
all books cheaper than $10 |
'$..book[?(@.price < 10)]' |
all books cheaper than $10. (The path must be quoted if it contains whitespaces) |
'$..book[?(@["price"] < 10)]' |
all books cheaper than $10 |
'$..book[?(@.["price"] < 10)]' |
all books cheaper than $10 |
$..book[?(@.price>=10&&@.price<=100)] |
all books in the price range of $10 to $100, inclusive |
'$..book[?(@.price>=10 && @.price<=100)]' |
all books in the price range of $10 to $100, inclusive. (The path must be quoted if it contains whitespaces) |
$..book[?(@.sold==true||@.in-stock==false)] |
all books sold or out of stock |
'$..book[?(@.sold == true || @.in-stock == false)]' |
all books sold or out of stock. (The path must be quoted if it contains whitespaces) |
'$.store.book[?(@.["category"] == "fiction")]' |
all books in the fiction category |
'$.store.book[?(@.["category"] != "fiction")]' |
all books in non-fiction categories |
More filter expression examples:
127.0.0.1:6379> JSON.SET k1 . '{"books": [{"price":5,"sold":true,"in-stock":true,"title":"foo"}, {"price":15,"sold":false,"title":"abc"}]}' OK 127.0.0.1:6379> JSON.GET k1 $.books[?(@.price>1&&@.price<20&&@.in-stock)] "[{\"price\":5,\"sold\":true,\"in-stock\":true,\"title\":\"foo\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?(@.price>1 && @.price<20 && @.in-stock)]' "[{\"price\":5,\"sold\":true,\"in-stock\":true,\"title\":\"foo\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?((@.price>1 && @.price<20) && (@.sold==false))]' "[{\"price\":15,\"sold\":false,\"title\":\"abc\"}]" 127.0.0.1:6379> JSON.GET k1 '$.books[?(@.title == "abc")]' [{"price":15,"sold":false,"title":"abc"}] 127.0.0.1:6379> JSON.SET k2 . '[1,2,3,4,5]' 127.0.0.1:6379> JSON.GET k2 $.*.[?(@>2)] "[3,4,5]" 127.0.0.1:6379> JSON.GET k2 '$.*.[?(@ > 2)]' "[3,4,5]" 127.0.0.1:6379> JSON.SET k3 . '[true,false,true,false,null,1,2,3,4]' OK 127.0.0.1:6379> JSON.GET k3 $.*.[?(@==true)] "[true,true]" 127.0.0.1:6379> JSON.GET k3 '$.*.[?(@ == true)]' "[true,true]" 127.0.0.1:6379> JSON.GET k3 $.*.[?(@>1)] "[2,3,4]" 127.0.0.1:6379> JSON.GET k3 '$.*.[?(@ > 1)]' "[2,3,4]"
Restricted syntax
Symbol/Expression | Description |
---|---|
. or [] |
child operator |
[] |
array subscript operator. Index is 0-based. |
Examples
Path | Description |
---|---|
.store.book[0].author |
the author of the first book |
.store.book[-1].author |
the author of the last book |
.address.city |
city name |
["store"]["book"][0]["title"] |
the title of the first book |
["store"]["book"][-1]["title"] |
the title of the last book |
Note
All Goessner
Common error prefixes
Each error message has a prefix. The following is a list of common error prefixes:
Prefix | Description |
---|---|
ERR |
a general error |
LIMIT |
size limit exceeded error. e.g., the document size limit or nesting depth limit exceeded |
NONEXISTENT |
a key or path does not exist |
OUTOFBOUNDARIES |
array index out of bounds |
SYNTAXERR |
syntax error |
WRONGTYPE |
wrong value type |
JSON related metrics
The following JSON info metrics are provided:
Info | Description |
---|---|
json_total_memory_bytes |
total memory allocated to JSON objects |
json_num_documents |
total number of documents in the Valkey or Redis OSS engine |
To query core metrics, run the command:
info json_core_metrics
How MemoryDB interacts with JSON
The following illustrates how MemoryDB interacts with the JSON datatype.
Operator precedence
When evaluating conditional expressions for filtering, &&s take precedence first, and then ||s are evaluated, as is common across most languages. Operations inside of parentheses will be executed first.
Maximum path nesting limit behavior
MemoryDB's maximum path nesting limit is 128. So a value like $.a.b.c.d...
can only reach 128 levels.
Handling numeric values
JSON does not have separate data types for integers and floating point numbers. They are all called numbers.
When a JSON number is received, it is stored in one of two formats. If the number fits into a 64-bit signed integer, then it is converted to that format; otherwise, it is stored as a string. Arithmetic operations on two JSON numbers (e.g. JSON.NUMINCRBY and JSON.NUMMULTBY) attempt to preserve as much precision as possible. If the two operands and the resulting value fit into a 64-bit signed integer, then integer arithmetic is performed. Otherwise, the input operands are converted into 64-bit IEEE double-precision floating point numbers, the arithmetic operation is performed, and the result is converted back into a string.
Arithmetic commands NUMINCRBY
and NUMMULTBY
:
If both numbers are integers, and the result is out of the range of int64, it will automatically become a double precision floating point number.
If at least one of the numbers is a floating point, the result will be a double precision floating point number.
If the result exceeds the range of double, the command will return an
OVERFLOW
error.
Note
Prior to Redis OSS engine version 6.2.6.R2 when a JSON number is received on input, it is converted into one of the two internal binary representations: a 64-bit signed integer or a 64-bit IEEE double precision floating point. The original string and all of its formatting are not retained. Thus, when a number is output as part of a JSON response, it is converted from the internal binary representation to a printable string that uses generic formatting rules. These rules might result in a different string being generated than was received.
If both numbers are integers and the result is out of the range of
int64
, it automatically becomes a 64-bit IEEE double precision floating point number.If at least one of the numbers is a floating point, the result is a 64-bit IEEE double precision floating point number.
If the result exceeds the range of 64-bit IEEE double, the command returns an
OVERFLOW
error.
For a detailed list of available commands, see Supported commands.
Strict syntax evaluation
MemoryDB does not allow JSON paths with invalid syntax, even if a subset of the path contains a valid path. This is to maintain correct behavior for our customers.