jq

Very often you will get content in JSON format. If you want to process this in bash scripting you will soon find out, it is very hard to do. JSON can have lots of optional line breaks and whitespaces, and in order to find a specific element you often need to almost parse the structure. But worry no more:

jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

jq manual

https://stedolan.github.io/jq/manual/v1.6/

Identity

Just copies all that is there, but pretty print

echo '[{"name":"John","age":42, "colors": ["red", "green", "blue"]},{"name":"John","age":42, "colors": ["red", "green", "blue"]}]' \
| jq "."
[
  {
    "name": "John",
    "age": 42,
    "colors": [
      "red",
      "green",
      "blue"
    ]
  },
  {
    "name": "John",
    "age": 42,
    "colors": [
      "red",
      "green",
      "blue"
    ]
  }
]

Get a single field

echo '{  "name": "John",  "age": 42,  "address": {    "street": "Main street",    "zipcode": 12345  }}' \
| jq ".address.zipcode"

Arrays

You have an array / list of objects

Get all the elements of the array

echo '[ {"name":"John","age":42, "colors": ["red", "green", "blue"]}, {"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[]"
{
"name": "John",
"age": 42,
"colors": [
"red",
"green",
"blue"
]
}

{
"name": "Jane",
"age": 35,
"colors": [
"yellow",
"red",
"blue"
]
}

Get only the first element

echo '[ {"name":"John","age":42, "colors": ["red", "green", "blue"]}, {"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[1]"

Get the last element

echo '[ {"name":"John","age":42, "colors": ["red", "green", "blue"]}, {"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[-1]"

Get the first 3 element

echo '[ {"name":"John","age":42, "colors": ["red", "green", "blue"]}, {"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
jq ".[0:2]"

Multiple fields

echo '{"name":"John","age":42, "colors": ["red", "green", "blue"]}' \
| jq ".name, .age"
"John"
42

Somewhere in the object is an array

First copy some fields + the array (boring)

echo '{"name":"John","age":42, "colors": ["red", "green", "blue"]}' \
| jq "{name, age, colors: .colors }"

Now have for each entry in the array an extra output line

echo '{"name":"John","age":42, "colors": ["red", "green", "blue"]}' \
| jq "{name, age, colors: .colors[] }"

(:source lang=Bash:) [@
{
  "name": "John",
  "age": 42,
  "colors": "red"
}

{
  "name": "John",
  "age": 42,
  "colors": "green"
}

{
  "name": "John",
  "age": 42,
  "colors": "blue"
}

Pipe

Have several commands and the output of the first, goes to the second, ... Like a Unix "|"

Array 1 by 1, from the result only take 2 fields

echo '[{"name":"John","age":42, "colors": ["red", "green", "blue"]},{"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[] | .name, .age"

Build new objects

Take the fields with values and build a new object. This builds the same object again

echo '{"name":"John","age":42, "colors": ["red", "green", "blue"]}' \
| jq "{name, age}"

You can also take field name and field value seperately

echo '[{"name":"John","age":42}]' \
| jq "{name: .name, age: .age}"

And of course change them

echo '[{"name":"John","age":42, "colors": ["red", "green", "blue"]},{"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[] | { "TheName": .name, age: 43 }"

Filtering (like with grep)

echo '[{"name":"John","age":42, "colors": ["red", "green", "blue"]},{"name":"Jane","age":35, "colors": ["yellow", "red", "blue"]}]' \
| jq ".[]  | select(.name == \"Jane\")"
{
  "name": "Jane",
  "age": 35,
  "colors": [
    "yellow",
    "red",
    "blue"
  ]
}