I am considering making a JSON parsing and generating command line tool. Started with looking around a bit. Below is a list of existing JSON command line tools. Numbers are [GitHub stars] at the time of writing this post. (… contributed by …) means that this post was updated with the item.
- jq [11126] – filter, extract, modify and output JSON or text using DSL
- jid [4426] – “You can drill down JSON interactively by using filtering queries like jq.” (item contributed by /u/Tacticus)
- gron [4103] – convert JSON or JSON lines (from file/stdin/url) to text (path=value) which can be processed with grep/sed/diff; the tool also supports converting back to JSON after such processing
- jo [2209] – generate JSON based on command line arguments and stdin; can read data from files and place it as base64 encoded values
- JSON.sh [1635] – written in shell/gawk; “traverses the JSON objects and prints out the path to the current object (as a JSON array) and then the object, without whitespace”
- underscore-cli [1588] ‘THE “Swiss Army knife” tool for processing JSON data – can be used as a simple pretty-printer, or as a full-powered JavaScript command-line’. Added on 2019-09-30 following comment from @joeytwiddle.
- jsawk [1239] – focused primarily on filtering and transforming a list (or an object). Update 2019-09-30: as @joeytwiddle suggested in comment, the project appears to be unmaintained and doesn’t work with recent Node.js versions. Latest commit and latest closed issue are from 2015.
- json (by trentm) [1218] – “massaging JSON on your Unix command line”; JS-like syntax for extracting values; in-place file editing
- rq [1007] – awk/sed-like tool for structured data; supports several formats, including JSON
- TickTick [469] – use JSON syntax directly in bash; “This is just a fun hack”
- jshon [309] – very CLI-ish way to extract, manipulate and output the data
- jl [308] – “a tiny functional language for querying and manipulating JSON”; visually reminds Haskell
- jsonpp [244] – JSON pretty printer (item contributed by /u/ferbass)
- fx [227] – conveniently run your JS code to manipulate JSON.
- RecordStream [224] – create, manipulate and output records; supports JSON; Perl-based so grep expressions for example are in Perl.
- JSON.awk [186] – JSON.sh fork in awk; after fork the projects added different features.
- jp [184] – “command line interface to JMESPath” (link contributed by Evgeny Zislis)
- json-command [143] – conveniently manipulate JSON using JS.
- jsonv.sh [130] – convert JSON to CSV; specify paths in JSON to
- jgrep (aka “JSON-grep”) [78] – “Command line tool and API for parsing JSON documents” in Ruby (item contributed by /u/tophlammiepie)
- jsed [48] – manipulate and extract data; somewhat similar to jsawk in mindset
- jayin [10] “Piping with js at terminal”. Added on 2019-09-30 following comment from @joeytwiddle.
- jsongrep [9] (by dsc) – extract data at given path using shell globs and output one per line
- jc [2] – “jc is used to JSONify the output of many standard linux cli tools”. Added on 2019-10-29 following comment from Kelly Brazil.
- jsongrep [0] (by terrycojones) – easily extract data at given path
Honorable mentions
python -m json.tool– the JSON Python module (mention contributed by Evgeny Zislis)
Update 2018-09-10:
I’ve added related post in which I argue that jq functionality belongs to a shell.
If you feel that some project is missing from the list, please let me know in comments below.
I just wanted to drop a thank you for that showstruct tool for jq. I’m always have problems with syntax and that has been very helpful. This list should be as well!
LikeLike
There’s another gawk script at https://gist.github.com/calraith/5c85d17b28e96f49e067f9b054a6fdbb for serializing / deserializing JSON. When fed a JSON string or file, the deserialize() function builds a multidimensional array from which values can be retrieved and manipulated as a hierarchical object. When fed an array (or an array of arrays), the serialize() function stringifies the array into JSON format.
LikeLike
Thanks!
LikeLike
Hi,
Also take a look at fx 🔥 and xx.
* NodeJS JSON cli tool https://github.com/antonmedv/fx
* Golang JSON cli tool https://github.com/antonmedv/xx
They have same api and really simple interface.
LikeLike
most likely it’s a misunderstanding. jtc (https://github.com/ldn-softdev/jtc) is a shell level cli tool, that takes input, validates, parses it, and let extract or manipulate resulting json – same thing that listed tools do. it just offers a novel way to select elements within the input json (via so called walk-path), but at the end, it’s still a tool to manipulate and transform input json at unix cli
LikeLike
For me it’s different enough but it’s arguable so approving the comments.
LikeLike
fx got a lot of stars recently)
LikeLike
There is also jj (https://github.com/tidwall/jj, 796 stars).
LikeLike
I can never get jq to do what I want, so I’m here looking for alternatives. Lots of interesting suggestions!
I think you could remove jsawk. It is 5 years old and doesn’t compile for recent versions of Node.
I have found underscore-cli somewhat useful: https://www.npmjs.com/package/underscore-cli
And (not JSON-specific) jayin lets us run JavaScript against any text file, with LoDash to hand: https://www.npmjs.com/package/jayin
LikeLike
Thanks! Updated.
LikeLike
Hi Ilya,
I wonder, why would not you include jtc (https://github.com/ldn-softdev/jtc) into your list of JSON tools? jtc is pretty much like jq (i.e. both are JSON processors), but has quite a few advantages over it:
1. both run on a metal, but unlike jq (written in C), jtc is written in the idiomatic C++(14 and above), which makes it free of memory/resource leaks – STL guarantee.
2. unlike jq, jtc support streaming seamlessly, which makes it effortless to utilize in between redirect (pipes).
3. due to a built-in cache engine jtc is times faster than jq (quite noticeable on processing big JSONs).
4. but the biggest advantage is that the same JSON queries are much more feasible to achieve with jtc (and it’s not my my opinion).
LikeLike
I created a tool called jc that JSONIfies the output of many popular gnu linux commands. Instead of sed/awk/grep/cut/tr, just pipe the output to jq.
Example:
ifconfig | jc –ifconfig | jq…
https://github.com/kellyjonbrazil/jc
or ‘pip3 install –upgrade jc’
LikeLike
I was thinking about creating something similar but in context of the Next Generation Shell. The language should be shipped with a library of parsers. NGS has syntax for run-and-parse: “my_command my args“. At the moment it can detect and parse JSON and output of find command. There is some additional processing (after JSON parsing) of AWS CLI to make the data a bit more palatable, for example Tags array is converted to Hash.
See
https://github.com/ngs-lang/ngs/blob/8d785a9907de7dcc496022170fecd889d8bc353f/lib/stdlib.ngs#L5284-L5289
https://github.com/ngs-lang/ngs/blob/8d785a9907de7dcc496022170fecd889d8bc353f/lib/stdlib.ngs#L5397-L5458
https://github.com/ngs-lang/ngs/blob/8d785a9907de7dcc496022170fecd889d8bc353f/lib/stdlib.ngs#L5460-L5469
Ideally the data should be parsed into more specific objects than just structured data (to fit the language best).
LikeLike
I completely agree – my tool should not need to exist in this day and age. The OS or she’ll should provide a machine readable view of every aspect of the OS and every command that outputs list or table output should have a json option. This is just a stopgap until that day comes. 🙂
LikeLike
JSON is minimum. Most of the information is tables. We need tools for tables. awk ‘{print $8}’ should not be a thing.
In my vision of shell UI, the tables on the screen should carry semantic information and be interactive – https://github.com/ngs-lang/ngs/wiki/UI-Design
LikeLike