How to parse JSON with shell scripting on Linux?

Question

I have a Json output from which I need to extract a few parameters in linux.

This is the json output:

{
  "OwnerId":"121456789127",
  "ReservationId":"r-48465168",
  "Groups":[

  ],
  "Instances":[
    {
      "Monitoring":{
        "State":"disabled"
      },
      "PublicDnsName":null,
      "RootDeviceType":"ebs",
      "State":{
        "Code":16,
        "Name":"running"
      },
      "EbsOptimized":false,
      "LaunchTime":"2014-03-19T09:16:56.000Z",
      "PrivateIpAddress":"10.250.171.248",
      "ProductCodes":[
        {
          "ProductCodeId":"aacglxeowvn5hy8sznltowyqe",
          "ProductCodeType":"marketplace"
        }
      ],
      "VpcId":"vpc-86bab0e4",
      "StateTransitionReason":null,
      "InstanceId":"i-1234576",
      "ImageId":"ami-b7f6c5de",
      "PrivateDnsName":"ip-10-120-134-248.ec2.internal",
      "KeyName":"Test_Virginia",
      "SecurityGroups":[
        {
          "GroupName":"Test",
          "GroupId":"sg-12345b"
        }
      ],
      "ClientToken":"VYeFw1395220615808",
      "SubnetId":"subnet-12345314",
      "InstanceType":"t1.micro",
      "NetworkInterfaces":[
        {
          "Status":"in-use",
          "SourceDestCheck":true,
          "VpcId":"vpc-123456e4",
          "Description":"Primary network interface",
          "NetworkInterfaceId":"eni-3619f31d",
          "PrivateIpAddresses":[
            {
              "Primary":true,
              "PrivateIpAddress":"10.120.134.248"
            }
          ],
          "Attachment":{
            "Status":"attached",
            "DeviceIndex":0,
            "DeleteOnTermination":true,
            "AttachmentId":"eni-attach-9210dee8",
            "AttachTime":"2014-03-19T09:16:56.000Z"
          },
          "Groups":[
            {
              "GroupName":"Test",
              "GroupId":"sg-123456cb"
            }
          ],
          "SubnetId":"subnet-31236514",
          "OwnerId":"109030037527",
          "PrivateIpAddress":"10.120.134.248"
        }
      ],
      "SourceDestCheck":true,
      "Placement":{
        "Tenancy":"default",
        "GroupName":null,
        "AvailabilityZone":"us-east-1c"
      },
      "Hypervisor":"xen",
      "BlockDeviceMappings":[
        {
          "DeviceName":"/dev/sda",
          "Ebs":{
            "Status":"attached",
            "DeleteOnTermination":false,
            "VolumeId":"vol-37ff097b",
            "AttachTime":"2014-03-19T09:17:00.000Z"
          }
        }
      ],
      "Architecture":"x86_64",
      "KernelId":"aki-88aa75e1",
      "RootDeviceName":"/dev/sda1",
      "VirtualizationType":"paravirtual",
      "Tags":[
        {
          "Value":"Server for testing RDS feature in us-east-1c AZ",
          "Key":"Description"
        },
        {
          "Value":"RDS_Machine (us-east-1c)",
          "Key":"Name"
        },
          {
          "Value":"1234",
          "Key":"Cost.centre"
        },
        {
          "Value":"Jyoti Bhanot",
          "Key":"Owner"
        }
      ],
      "AmiLaunchIndex":0
    }
  ]
}

Expected output :

 Instance id         Name                           cost centre             Owner
    i-1234576          RDS_Machine (us-east-1c)        1234                   Jyoti Bhanot

I want to write a file that contains headings like instance id, tag like name, cost centre, owner. Below that, certain values from the json output. The output here given is just a example.

How can I do that using sed and awk?

Any lead is appreciated.

Thanks

Refer stackoverflow.com/questions/20488315/… stackoverflow.com/questions/1955505/… — Rahul R Dhobi
– Rahul R Dhobi, Commented Mar 27, 2014 at 4:58
Why do you want to use awk for parsing JSON? You must use a dedicated json parsing tool like jq or jsawk. Below answer should work for you I believe. — anubhava
– anubhava, Commented Mar 27, 2014 at 6:52
ok for the sample above can you provide your expected output in question? — anubhava
– anubhava, Commented Mar 27, 2014 at 6:58
You're making same mistake. You expected output isn't matching with the input you provided. Input json has only one: "InstanceId":"i-1234576" — anubhava
– anubhava, Commented Mar 27, 2014 at 7:04

Community · Accepted Answer · 2017-05-23 11:45:15Z

14

Here's a sample using jsawk. Reference: Parsing JSON with Unix tools

Setup:

First download jsawk from https://github.com/micha/jsawk:

$ curl -L http://github.com/micha/jsawk/raw/master/jsawk > jsawk
$ chmod 755 jsawk && mv jsawk ~/bin/

You might want to install js-devel first before you can use jsawk. I'm using Fedora, so what I did was:

$ sudo yum install js-devel

The test:

I copied your JSON output sample to a text file. Called it sample.json. Here a sample to get a value from your JSON output sample:

$ jsawk 'return this.Instances[0].Monitoring.State' < sample.json
disabled
$ jsawk 'return this.Instances[0].VpcId' < sample.json
vpc-86bab0e4

For JSON data from a URL, you can use curl http://someserver.com/data.json instead of cat:

$ curl http://someserver.com/data.json | jsawk 'return this.Instances[0].VpcId'
vpc-86bab0e4

You can use these commands in your bash file to generate a new file that contains strings / text that you wanted. You can read more about jsawk from the GitHub link that I provided here.

Is this what you were looking for?

edited May 23, 2017 at 11:45

CommunityBot

11 silver badge

answered Mar 27, 2014 at 5:03

Muhammad Baja Aksha

48710 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Muhammad Baja Aksha Over a year ago

Looks like this is a duplicate from: stackoverflow.com/questions/22316714/… and stackoverflow.com/questions/1955505/…

Muhammad Baja Aksha Over a year ago

as Rahul R Dhobi said, you might want to see this: stackoverflow.com/questions/1955505/… for your specific question. jsawk might be your answer.

Muhammad Baja Aksha Over a year ago

please see my edited answer. I won't be able to write you a shell script to generate the file that you asked, but you should get how to write it yourself from here. let me know if it works.

icedwater Over a year ago

You could write a script that calls this one for each of the 100s of servers.

Muhammad Baja Aksha Over a year ago

Refer to this: A quick guide to writing scripts using the bash shell on how to write a shell script. Try it first on your own, then ask or search again for help.

lirik90 · Accepted Answer · 2023-04-17 03:22:56Z

0

Pure Bash 3.2+ without dependencies (such as jq, python, grep, etc.):

source <(curl -s -L -o- https://github.com/lirik90/bashJsonParser/raw/master/jsonParser.sh)
read -d '' JSON << EOF
// put your json here
EOF
JSON=$(minifyJson "$JSON")
id=$(parseJson "$JSON" Instances 0 InstanceId)
name=$(parseJson "$JSON" Instances 0 Tags 1 Value)
cost=$(parseJson "$JSON" Instances 0 Tags 2 Value)
owner=$(parseJson "$JSON" Instances 0 Tags 3 Value)
echo -e "Instance id |\t\t Name \t\t\t\t| cost centre | Owner"
echo -e "$id \t| $name  | $cost\t\t  | $owner"

Output:

Instance id |        Name               | cost centre | Owner
i-1234576   | RDS_Machine (us-east-1c)  | 1234        | Jyoti Bhanot

Try it.

edited Apr 17, 2023 at 3:22

answered Apr 17, 2023 at 1:44

lirik90

1751 silver badge7 bronze badges

Collectives™ on Stack Overflow

How to parse JSON with shell scripting on Linux?

2 Answers 2

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related