How to Parse JSON With Shell Scripting in Linux?

JSON parsing is the procedure of converting Javascript Object Notation (JSON) into a format that the programming languages can interpret. When the bash scripting is dealing with APIs, it needs JSON parsing. 

To parse JSON in shell scripting, there is no native support, but the jq command is used for this.

This guide explains the parsing of JSON data with shell scripting on Linux.

  • What is JSON Data?
  • Does Shell Scripting Support JSON?
  • Parse JSON Data Using Shell Script on Linux
    • Using the jq Command
    • Parse JSON Data Using the curl Command
    • Parse JSON Data Using the grep Command

What is JSON Data?

JSON is a well-structured data format that is primarily utilized in modern APIs. It is supported and used by all major programming languages. Most data is transferred between your system and database using the JSON format.

It is easy for humans to read and more efficient for machines to parse and share because of its simplicity.

Does Shell Scripting Support JSON?

No, the shell scripting does not directly support JSON, but a command-line utility is known as JSON Processer or jq, using which is installed using these commands:

$ sudo apt install jq            #For Ubuntu/Debian
$ sudo dnf install jq            #For Fedora
$ sudo pacman -Sy jq           #For Manjaro

In the above image, the installation of jq is confirmed.

How to Parse JSON Data Using Shell Script on Linux Using the jq Command?

To parse JSON data using a shell script on Linux is explained in these steps:

Step 1: Create a JSON File

The JSON data file is created under the name “names.json” using the nano editor. The example code is provided below: 

{
  "message": "success",
  "number": 10,
  "people": [
{
  "id": "1",
  "name": "John"
},
{
  "id": "2",
  "name": "Jerry"
},
{
  "id": "3",
  "name": "Antonio"
},
{
  "id": "4",
  "name": "Peter"
},
{
  "id": "5",
  "name": "Chris"
},
{
  "id": "6",
  "name": "Stewie"
},
  ]
}

The above JSON file contains the data of people including their name and an ID

Step 2: Create a Shell Script to Parse JSON Data

Here, a shell script named “myscript.sh” is created to parse the JSON data:

#!/bin/sh

NAME[0]="John"
NAME[1]="Joe"
NAME[2]="Tom"
NAME[3]="Daisu"
NAME[4]="Doe"
echo "First Index: ${NAME[0]}"
echo "Second Index: ${NAME[1]}"

Here, the script uses the “jq” command, which is followed by a dot (.) to filter out the records from the file “names.json.”

This script first defines the JSON data in a variable named json and uses the jq command to extract the number of people and the names of all people. The -r option for the jq command is used to output the names as raw strings, without quotation marks. The script then prints out the results. 

Step 3: Execute the Script

To execute the above-created script, ensure it has the “execute” permission. To grant them, use this command:

$ sudo chmod +x myscript.sh

Now, the script is executable, which can be done using this command:

$ bash myscript.sh

As the above image shows, “myscript.sh” is successfully executed, and the JSON data is parsed.

How to Parse JSON Using the curl Command on Linux?

The curl command can be used to make a HTTP request to the server that serves the JSON data, and using the jq command, the data can be parsed.

Here is the explanation of the command before it is executed:

  • curl invokes the command
  • -s is an option of the curl command to suppress the error and progress
  • https://jsonplaceholder.typicode.com/todos/8 is the URL for the JSON data
  • The jq command is used to pretty-print the entire JSON data or object
$ curl -s https://jsonplaceholder.typicode.com/todos/8 | jq '.'

The above command displays the JSON data from the URL.

How to Parse JSON Data Using the grep Command on Linux?

The grep is primarily used for pattern matching within the given files and is not recommended to be used for JSON parsing. However, the JSON file can be filtered. Here is the explanation of the command used below to match the pattern:

  • grep invokes the command
  • -o is used to output only the matched pattern
  • “[^”]* searches all the data but ignores the double quotes “ “.
  • names.json is the input file name
  • grep -o ‘[^”]*$’ takes the output from the previous grep command and further extracts the values by removing everything up to the last double quote.

Let’s execute this command and see the results:

$ grep -o '"name": "[^"]*' names.json | grep -o '[^"]*$'

Conclusion

The bash scripting may not natively support parsing of JSON data (as of yet), but thanks to the “jq” command of Linux, it is possible. It can be combined with the curl command to make HTTP requests to the server that serves the JSON data. Python libraries can also be used for this purpose and are elaborated on in this detailed guide. Today, the whole procedure of parsing JSON with shell scripting is discussed.