jq Recipes
Summary
Recipes and command examples for jq, a lightweight and flexible command-line JSON processor. Learn to delete and keep specific arrays and objects. Learn how to create an index of values.
What is jq? #
jq, written in all lowercase, is a command-line utility that is used for parsing and manipulating JSON data. It is written in C and has a command-line interface that allows users to filter, transform, and extract data from JSON documents.
jq is designed to be fast, flexible, and easy to use, making it a popular choice for working with JSON data in a Unix environment.
Arguments #
| Argument | Description |
|---|---|
--version | Print jq version and exit with zero. |
--sort-keys | Print the fields of each object with the keys in sorted order. |
Filters #
| Filter | Description |
|---|---|
. | Identity - Takes input and produces it unchanged as output. A useful way of formatting JSON output from, say, curl. |
.foo, .foo.bar | Object Identifier-Index - The simplest useful filter is .foo. When given a JSON object (aka dictionary or hash) as input, it produces the value at the key "foo", or null if there's none present. A filter of the form .foo.bar is equivalent to .foo|.bar. |
.foo? | Optional Object Identifier-Index - Just like .foo, but does not output even an error when . is not an array or an object. |
.[<string>] | Generic Object Index - Look up fields of an object using syntax like .["foo"] (.foo above is a shorthand version of this, but only for identifier-like strings). |
.[2] | Array Index - When the index value is an integer, .[<value>] can index arrays. Arrays are zero-based, so .[2] returns the third element. Negative indices are allowed, with -1 referring to the last element, -2 referring to the next to last element, and so on. |
.[10:15] | Array/String Slice - The .[10:15] syntax can be used to return a subarray of an array or substring of a string. The array returned by .[10:15] will be of length 5, containing the elements from index 10 (inclusive) to index 15 (exclusive). Either index may be negative (in which case it counts backwards from the end of the array), or omitted (in which case it refers to the start or end of the array). |
, | Comma - If two filters are separated by a comma, then the same input will be fed into both and the two filters' output value streams will be concatenated in order: first, all of the outputs produced by the left expression, and then all of the outputs produced by the right. |
| | Pipe - The | operator combines two filters by feeding the output(s) of the one on the left into the input of the one on the right. It's pretty much the same as the Unix shell's pipe. |
Delete objects #
Delete an object if it contains a particular key/value pair #
$ echo '
{
"theArray": [
{
"key": "foo",
"value": "0123"
},
{
"key": "bar",
"value": "4567"
}
]
}' | \
jq '. | del( ."theArray"[] | select(."value" == "4567") )'
Result:
{
"theArray": [
{
"key": "foo",
"value": "0123"
}
]
}
Delete keys from all JSON objects in document #
The walk(f) function, used in some of the examples, applies f recursively to every component of the input entity.
With matching #
Delete the key value pairs anywhere in the tree (at any level deep) depending on the value of the key.
jq 'walk(if type == "object" and .key0 == "value0" then del(.key0) else . end)'
Without matching #
Delete the key value pair anywhere in the tree (at any level deep) independent from the value of the key.
jq 'walk(if type == "object" then del(.key0, .key1) else . end)'
Alternative command:
jq '(.. | select(type == "object")) |= del(.key0, .key1)'
Delete empty keys #
Specific keys: #
jq 'walk(if type == "object" and length > 0 then del(.key0, .key1) else . end)'
Unspecified keys: #
jq 'walk(if type == "object" then with_entries(select(.value | (. != {} and . != []))) else . end)'
Strip empty strings from arrays #
jq 'walk(if type == "array" then map(select(length > 0)) else . end)'
Delete empty arrays #
Deletes all empty arrays by walking through all levels of the document tree using .. and selecting everything that is an empty array [] and set it to empty.
(..|select(.==[])) |= empty
Keep objects #
Keep objects or filter for specific objects and drop all others: #
$ echo '
{
"a":1,
"b":2,
"c":3,
"d":4
}' | \
jq '{b, d}'
Result:
{
"b": 2,
"d": 4
}
Index Values #
Create an index of the values of a defined property. Makes use of to_entries, which converts an object to an array of key/value objects.
$ echo '
{
"theArray": [
{
"key": "foo",
"id": "0123"
},
{
"key": "bar",
"id": "4567"
}
]
}' | \
jq '.theArray | to_entries | map( { (.value.id): .key} ) | add'
Result:
{
"0123": 0,
"4567": 1
}
Further readings #
Sources and recommended, further resources on the topic:
Author
Jonas Jared Jacek (J15k)
Jonas works as project manager, web designer, and web developer since 2001. On top of that, he is a Linux system administrator with a broad interest in things related to programming, architecture, and design. See: https://www.j15k.com/License
License: jq Recipes by Jonas Jared Jacek is licensed under CC BY-SA 4.0.
This license requires that reusers give credit to the creator. It allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, for noncommercial purposes only. To give credit, provide a link back to the original source, the author, and the license e.g. like this:
<p xmlns:cc="http://creativecommons.org/ns#" xmlns:dct="http://purl.org/dc/terms/"><a property="dct:title" rel="cc:attributionURL" href="https://www.ditig.com/publications/jq-recipes">jq Recipes</a> by <a rel="cc:attributionURL dct:creator" property="cc:attributionName" href="https://www.j15k.com/">Jonas Jared Jacek</a> is licensed under <a href="https://creativecommons.org/licenses/by-sa/4.0/" target="_blank" rel="license noopener noreferrer">CC BY-SA 4.0</a>.</p>For more information see the DITig legal page.
“There are only two hard things in Computer Science: cache invalidation and naming things.”
