NAME¶

jc - JSON Convert JSONifies the output of many CLI tools, file-types, and strings

SYNOPSIS¶

Standard syntax:

COMMAND | jc [OPTIONS] PARSER

cat FILE | jc [OPTIONS] PARSER

echo STRING | jc [OPTIONS] PARSER

Magic syntax:

jc [OPTIONS] COMMAND

jc [OPTIONS] /proc/<path-to-procfile>

DESCRIPTION¶

jc JSONifies the output of many CLI tools, file-types, and common strings for easier parsing in scripts. jc accepts piped input from STDIN and outputs a JSON representation of the previous command's output to STDOUT. Alternatively, the "Magic" syntax can be used by prepending jc to the command to be converted. Options can be passed to jc immediately before the command is given. (Note: "Magic" syntax does not support shell builtins or command aliases)

OPTIONS¶

Parsers:

--acpi

`acpi` command parser

--airport

`airport -I` command parser

--airport-s

`airport -s` command parser

--arp

`arp` command parser

--asciitable

ASCII and Unicode table parser

--asciitable-m

multi-line ASCII and Unicode table parser

--blkid

`blkid` command parser

--cbt

`cbt` (Google Bigtable) command parser

--cef

CEF string parser

--cef-s

CEF string streaming parser

--chage

`chage --list` command parser

--cksum

`cksum` and `sum` command parser

--clf

Common and Combined Log Format file parser

--clf-s

Common and Combined Log Format file streaming parser

--crontab

`crontab` command and file parser

--crontab-u

`crontab` file parser with user support

--csv

CSV file parser

--csv-s

CSV file streaming parser

--date

`date` command parser

--datetime-iso

ISO 8601 Datetime string parser

`df` command parser

`dig` command parser

`dir` command parser

`dmidecode` command parser

--dpkg-l

`dpkg -l` command parser

--du

`du` command parser

--email-address

Email Address string parser

--env

`env` command parser

--file

`file` command parser

--findmnt

`findmnt` command parser

--finger

`finger` command parser

--free

`free` command parser

--fstab

`/etc/fstab` file parser

--git-log

`git log` command parser

--git-log-s

`git log` command streaming parser

--git-ls-remote

`git ls-remote` command parser

--gpg

`gpg --with-colons` command parser

--group

`/etc/group` file parser

--gshadow

`/etc/gshadow` file parser

--hash

`hash` command parser

--hashsum

hashsum command parser (`md5sum`, `shasum`, etc.)

--hciconfig

`hciconfig` command parser

--history

`history` command parser

--hosts

`/etc/hosts` file parser

--id

`id` command parser

--ifconfig

`ifconfig` command parser

--ini

INI file parser

--ini-dup

INI with duplicate key file parser

--iostat

`iostat` command parser

--iostat-s

`iostat` command streaming parser

--ip-address

IPv4 and IPv6 Address string parser

--iptables

`iptables` command parser

--iso-datetime

Deprecated - please use datetime-iso

--iw-scan

`iw dev [device] scan` command parser

--iwconfig

`iwconfig` command parser

--jar-manifest

Java MANIFEST.MF file parser

--jobs

`jobs` command parser

--jwt

JWT string parser

--kv

Key/Value file and string parser

--last

`last` and `lastb` command parser

--ls

`ls` command parser

--ls-s

`ls` command streaming parser

--lsblk

`lsblk` command parser

--lsmod

`lsmod` command parser

--lsof

`lsof` command parser

--lspci

`lspci -mmv` command parser

--lsusb

`lsusb` command parser

--m3u

M3U and M3U8 file parser

--mdadm

`mdadm` command parser

--mount

`mount` command parser

--mpstat

`mpstat` command parser

--mpstat-s

`mpstat` command streaming parser

--netstat

`netstat` command parser

--nmcli

`nmcli` command parser

--ntpq

`ntpq -p` command parser

--openvpn

openvpn-status.log file parser

--os-prober

`os-prober` command parser

--passwd

`/etc/passwd` file parser

--pci-ids

`pci.ids` file parser

--pgpass

PostgreSQL password file parser

--pidstat

`pidstat -H` command parser

--pidstat-s

`pidstat -H` command streaming parser

--ping

`ping` and `ping6` command parser

--ping-s

`ping` and `ping6` command streaming parser

--pip-list

`pip list` command parser

--pip-show

`pip show` command parser

--plist

PLIST file parser

--postconf

`postconf -M` command parser

--proc

`/proc/` file parser

--proc-buddyinfo

`/proc/buddyinfo` file parser

--proc-consoles

`/proc/consoles` file parser

--proc-cpuinfo

`/proc/cpuinfo` file parser

--proc-crypto

`/proc/crypto` file parser

--proc-devices

`/proc/devices` file parser

--proc-diskstats

`/proc/diskstats` file parser

--proc-filesystems

`/proc/filesystems` file parser

--proc-interrupts

`/proc/interrupts` file parser

--proc-iomem

`/proc/iomem` file parser

--proc-ioports

`/proc/ioports` file parser

--proc-loadavg

`/proc/loadavg` file parser

--proc-locks

`/proc/locks` file parser

--proc-meminfo

`/proc/meminfo` file parser

--proc-modules

`/proc/modules` file parser

--proc-mtrr

`/proc/mtrr` file parser

--proc-pagetypeinfo

`/proc/pagetypeinfo` file parser

--proc-partitions

`/proc/partitions` file parser

--proc-slabinfo

`/proc/slabinfo` file parser

--proc-softirqs

`/proc/softirqs` file parser

--proc-stat

`/proc/stat` file parser

--proc-swaps

`/proc/swaps` file parser

--proc-uptime

`/proc/uptime` file parser

--proc-version

`/proc/version` file parser

--proc-vmallocinfo

`/proc/vmallocinfo` file parser

--proc-vmstat

`/proc/vmstat` file parser

--proc-zoneinfo

`/proc/zoneinfo` file parser

--proc-driver-rtc

`/proc/driver/rtc` file parser

--proc-net-arp

`/proc/net/arp` file parser

--proc-net-dev

`/proc/net/dev` file parser

--proc-net-dev-mcast

`/proc/net/dev_mcast` file parser

--proc-net-if-inet6

`/proc/net/if_inet6` file parser

--proc-net-igmp

`/proc/net/igmp` file parser

--proc-net-igmp6

`/proc/net/igmp6` file parser

--proc-net-ipv6-route

`/proc/net/ipv6_route` file parser

--proc-net-netlink

`/proc/net/netlink` file parser

--proc-net-netstat

`/proc/net/netstat` file parser

--proc-net-packet

`/proc/net/packet` file parser

--proc-net-protocols

`/proc/net/protocols` file parser

--proc-net-route

`/proc/net/route` file parser

--proc-net-unix

`/proc/net/unix` file parser

--proc-pid-fdinfo

`/proc/<pid>/fdinfo/<fd>` file parser

--proc-pid-io

`/proc/<pid>/io` file parser

--proc-pid-maps

`/proc/<pid>/maps` file parser

--proc-pid-mountinfo

`/proc/<pid>/mountinfo` file parser

--proc-pid-numa-maps

`/proc/<pid>/numa_maps` file parser

--proc-pid-smaps

`/proc/<pid>/smaps` file parser

--proc-pid-stat

`/proc/<pid>/stat` file parser

--proc-pid-statm

`/proc/<pid>/statm` file parser

--proc-pid-status

`/proc/<pid>/status` file parser

--ps

`ps` command parser

--route

`route` command parser

--rpm-qi

`rpm -qi` command parser

--rsync

`rsync` command parser

--rsync-s

`rsync` command streaming parser

--semver

Semantic Version string parser

--sfdisk

`sfdisk` command parser

--shadow

`/etc/shadow` file parser

--ss

`ss` command parser

--sshd-conf

sshd config file and `sshd -T` command parser

--stat

`stat` command parser

--stat-s

`stat` command streaming parser

--sysctl

`sysctl` command parser

--syslog

Syslog RFC 5424 string parser

--syslog-s

Syslog RFC 5424 string streaming parser

--syslog-bsd

Syslog RFC 3164 string parser

--syslog-bsd-s

Syslog RFC 3164 string streaming parser

--systemctl

`systemctl` command parser

--systemctl-lj

`systemctl list-jobs` command parser

--systemctl-ls

`systemctl list-sockets` command parser

--systemctl-luf

`systemctl list-unit-files` command parser

--systeminfo

`systeminfo` command parser

--time

`/usr/bin/time` command parser

--timedatectl

`timedatectl status` command parser

--timestamp

Unix Epoch Timestamp string parser

--toml

TOML file parser

--top

`top -b` command parser

--top-s

`top -b` command streaming parser

--tracepath

`tracepath` and `tracepath6` command parser

--traceroute

`traceroute` and `traceroute6` command parser

--udevadm

`udevadm info` command parser

--ufw

`ufw status` command parser

--ufw-appinfo

`ufw app info [application]` command parser

--uname

`uname -a` command parser

--update-alt-gs

`update-alternatives --get-selections` command parser

--update-alt-q

`update-alternatives --query` command parser

--upower

`upower` command parser

--uptime

`uptime` command parser

--url

URL string parser

--vmstat

`vmstat` command parser

--vmstat-s

`vmstat` command streaming parser

`w` command parser

`wc` command parser

`who` command parser

X.509 PEM and DER certificate file parser

--xml

XML file parser

--xrandr

`xrandr` command parser

--yaml

YAML file parser

--zipinfo

`zipinfo` command parser

Options:

-a, --about: About jc (JSON or YAML output)
-C, --force-color: Force color output even when using pipes (overrides -m and the NO_COLOR env variable)
-d, --debug: Debug - show traceback (use -dd for verbose traceback)
-h, --help: Help (--help --parser_name for parser documentation). Use twice to show hidden parsers (e.g. -hh)
-m, --monochrome: Monochrome output
-M, --meta-out: Add metadata to output including timestamp, parser name, magic command, magic command exit code, etc.
-p, --pretty: Pretty print output
-q, --quiet: Quiet mode. Suppresses parser warning messages (use -qq to ignore streaming parser errors)
-r, --raw: Raw output. Provides more literal output, typically with string values and no additional semantic processing
-u, --unbuffer: Unbuffer output (useful for slow streaming data with streaming parsers)
-v, --version: Version information
-y, --yaml-out: YAML output
-B, --bash-comp: Generate Bash shell completion script
-Z, --zsh-comp: Generate Zsh shell completion script

EXIT CODES¶

Any fatal errors within jc will generate an exit code of 100, otherwise the exit code will be 0.

When using the "magic" syntax (e.g. jc ifconfig eth0), jc will store the exit code of the program being parsed and add it to the jc exit code. This way it is easier to determine if an error was from the parsed program or jc.

Consider the following examples using ifconfig:

ifconfig exit code = 0, jc exit code = 0, combined exit code = 0 (no errors)

ifconfig exit code = 1, jc exit code = 0, combined exit code = 1 (error in ifconfig)

ifconfig exit code = 0, jc exit code = 100, combined exit code = 100 (error in jc)

ifconfig exit code = 1, jc exit code = 100, combined exit code = 101 (error in both ifconfig and jc)

When using the "magic" syntax you can also retrieve the exit code of the called program by using the --meta-out or -M option. This will append a _jc_meta object to the output that will include the magic command information, including the exit code.

Here is an example with ping:

$ jc --meta-out -p ping -c2 192.168.1.252
{


  "destination_ip": "192.168.1.252",


  "data_bytes": 56,


  "pattern": null,


  "destination": "192.168.1.252",


  "packets_transmitted": 2,


  "packets_received": 0,


  "packet_loss_percent": 100.0,


  "duplicates": 0,


  "responses": [


    {


      "type": "timeout",


      "icmp_seq": 0,


      "duplicate": false


    }


  ],


  "_jc_meta": {


    "parser": "ping",


    "timestamp": 1661357115.27949,


    "magic_command": [


      "ping",


      "-c2",


      "192.168.1.252"


    ],


    "magic_command_exit": 2


  }
}
$ echo $?
2

ENVIRONMENT¶

Custom Colors

You can specify custom colors via the JC_COLORS environment variable. The JC_COLORS environment variable takes four comma separated string values in the following format:

JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>

Where colors are: black, red, green, yellow, blue, magenta, cyan, gray, brightblack, brightred, brightgreen, brightyellow, brightblue, brightmagenta, brightcyan, white, or default

For example, to set to the default colors:

JC_COLORS=blue,brightblack,magenta,green

or

JC_COLORS=default,default,default,default

Disable Color Output

You can set the NO_COLOR environment variable to any value to disable color output in jc. Note that using the -C option to force color output will override both the NO_COLOR environment variable and the -m option.

STREAMING PARSERS¶

Most parsers load all of the data from STDIN, parse it, then output the entire JSON document serially. There are some streaming parsers (e.g. ls-s, ping-s, etc.) that immediately start processing and outputting the data line-by-line as JSON Lines (aka NDJSON) while it is being received from STDIN. This can significantly reduce the amount of memory required to parse large amounts of command output (e.g. ls -lR /) and can sometimes process the data more quickly. Streaming parsers have slightly different behavior than standard parsers as outlined below.

Note: Streaming parsers cannot be used with the "magic" syntax

Ignoring Errors

You may want to ignore parsing errors when using streaming parsers since these may be used in long-lived processing pipelines and errors can break the pipe. To ignore parsing errors, use the -qq cli option. This will add a _jc_meta object to the JSON output with a success attribute. If success is true, then there were no issues parsing the line. If success is false, then a parsing issue was found and error and line fields will be added to include a short error description and the contents of the unparsable line, respectively:

Successfully parsed line with -qq option:

{


  "command_data": "data",


  "_jc_meta": {


    "success": true


  }
}

Unsuccessfully parsed line with -qq option:

{


  "_jc_meta": {


    "success": false,


    "error": "error message",


    "line": "original line data"


  }
}

Unbuffering Output

Most operating systems will buffer output that is being piped from process to process. The buffer is usually around 4KB. When viewing the output in the terminal the OS buffer is not engaged so output is immediately displayed on the screen. When piping multiple processes together, though, it may seem as if the output is hanging when the input data is very slow (e.g. ping):

$ ping 1.1.1.1 | jc --ping-s | jq
<slow output>

This is because the OS engages the 4KB buffer between jc and jq in this example. To display the data on the terminal in realtime, you can disable the buffer with the -u (unbuffer) cli option:

$ ping 1.1.1.1 | jc --ping-s -u | jq
{"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
{"type":"reply","pattern":null,"timestamp":null,"bytes":"64",...}
etc...

Note: Unbuffered output can be slower for large data streams.

CUSTOM PARSERS¶

Custom local parser plugins may be placed in a jc/jcparsers folder in your local "App data directory":

- Linux/unix: $HOME/.local/share/jc/jcparsers
- macOS: $HOME/Library/Application Support/jc/jcparsers
- Windows: $LOCALAPPDATA\jc\jc\jcparsers

Local parser plugins are standard python module files. Use the jc/parsers/foo.py or jc/parsers/foo_s.py (streaming) parser as a template and simply place a .py file in the jcparsers subfolder.

Local plugin filenames must be valid python module names and therefore must start with a letter and consist entirely of alphanumerics and underscores. Local plugins may override default parsers.

Note: The application data directory follows the XDG Base Directory Specification

CAVEATS¶

Locale

For best results set the locale environment variables to C or en_US.UTF-8 by modifying the LC_ALL variable:

$ LC_ALL=C date | jc --date

You can also set the locale variables individually:

$ export LANG=C

$ export LC_NUMERIC=C

On some older systems UTF-8 output will be downgraded to ASCII with \u escape sequences if the C locale does not support UTF-8 encoding.

Timezones

Some parsers have calculated epoch timestamp fields added to the output. Unless a timestamp field name has a _utc suffix it is considered naive. (i.e. based on the local timezone of the system the jc parser was run on).

If a UTC timezone can be detected in the text of the command output, the timestamp will be timezone aware and have a _utc suffix on the key name. (e.g. epoch_utc) No other timezones are supported for aware timestamps.

EXAMPLES¶

Standard Syntax:

$ dig www.google.com | jc -p --dig

$ cat /proc/meminfo | jc --pretty --proc

Magic Syntax:

$ jc --pretty dig www.google.com

$ jc --pretty /proc/meminfo

For parser documentation:

$ jc --help --dig

AUTHOR¶

Kelly Brazil (kellyjonbrazil@gmail.com)

https://github.com/kellyjonbrazil/jc

COPYRIGHT¶

License: MIT License

Source file:	jc.1.en.gz (from jc 1.22.5-1)
Source last updated:	2023-02-25T12:05:45Z
Converted to HTML:	2024-02-24T03:53:14Z