## 1. Overview

jq is a lightweight and powerful command-line tool for parsing, filtering, transforming, and formatting JSON data.

In DevOps workflows, `jq` is commonly used to:

* Analyze logs (Docker, Kubernetes, application logs)
* Filter observability data (metrics/events in JSON format)
* Debug CI/CD pipelines
* Process API responses (AWS, GitHub, Terraform outputs)
* Transform JSON for automation scripts

It is essentially the “grep + awk + sed” equivalent for JSON.

---

## 2. Installation

### Linux (Debian/Ubuntu)

```bash
sudo apt-get update
sudo apt-get install jq
```

### RHEL/CentOS

```bash
sudo yum install jq
```

### macOS

```bash
brew install jq
```

### Verify installation

```bash
jq --version
```

---

## 3. Basic Syntax

```bash
jq '<filter>' file.json
```

Or pipe input:

```bash
cat file.json | jq '<filter>'
```

---

## 4. Core Concepts

### 4.1 Identity filter

Returns input as-is:

```bash
jq '.'
```

### 4.2 Access fields

```bash
jq '.name'
jq '.user.id'
```

### 4.3 Arrays

```bash
jq '.items[]'
```

### 4.4 Pretty print

```bash
jq '.'
```

---

## 5. Filtering Logs (DevOps Use Case)

### Example log entry

```json
{
  "level": "error",
  "service": "auth",
  "message": "invalid credentials",
  "status": 401,
  "timestamp": "2026-04-15T10:00:00Z"
}
```

### Filter only errors

```bash
jq 'select(.level == "error")'
```

### Filter by service

```bash
jq 'select(.service == "auth")'
```

### Extract specific fields

```bash
jq '{time: .timestamp, msg: .message}'
```

---

## 6. Working with Arrays (Common in Logs)

### Example: multiple log entries

### Count entries

```bash
jq 'length'
```

### Filter array elements

```bash
jq '.[] | select(.status >= 500)'
```

### Extract fields from array

```bash
jq '.[] | {service, status, message}'
```

---

## 7. Kubernetes Logs with jq

### Example:

```bash
kubectl logs pod-name -n default | jq
```

### Filter error logs

```bash
kubectl logs pod-name | jq 'select(.level=="error")'
```

### Extract container metadata logs

```bash
kubectl logs pod-name | jq '{time, container, message}'
```

---

## 8. Docker Logs with jq

### Streaming logs

```bash
docker logs container_name | jq
```

### Filter failures

```bash
docker logs container_name | jq 'select(.status != "success")'
```

---

## 9. AWS / Cloud Logs (JSON-based)

### Example CloudWatch JSON logs

```bash
aws logs filter-log-events --log-group-name my-app | jq
```

### Extract messages only

```bash
... | jq '.events[].message'
```

### Filter by keyword

```bash
... | jq '.events[] | select(.message | contains("ERROR"))'
```

---

## 10. Transforming JSON (Automation Use Cases)

### Rename fields

```bash
jq '{userId: .id, username: .name}'
```

### Add computed fields

```bash
jq '. + {isActive: true}'
```

### Build new structure

```bash
jq '{users: [.[] | {id, name}]}'
```

---

## 11. Advanced Filtering

### Logical conditions

```bash
jq 'select(.status == 200 and .service == "api")'
```

### Regex matching

```bash
jq 'select(.message | test("timeout|failed"))'
```

### Sorting

```bash
jq 'sort_by(.timestamp)'
```

### Unique values

```bash
jq 'unique_by(.service)'
```

---

## 12. Aggregations (DevOps Analytics)

### Count by status

```bash
jq 'group_by(.status) | map({status: .[0].status, count: length})'
```

### Error rate estimation

```bash
jq 'map(select(.status >= 400)) | length'
```

---

## 13. Formatting Output for Humans

### Compact JSON

```bash
jq -c '.'
```

### Raw output (no quotes)

```bash
jq -r '.message'
```

### Tabular-like output

```bash
jq -r '[.timestamp, .level, .message] | @tsv'
```

---

## 14. Debugging Pipelines

### Validate JSON

```bash
jq empty file.json
```

### Highlight structure

```bash
jq '. | type'
```

### Pretty inspect nested structures

```bash
jq 'paths'
```

---

## 15. DevOps Best Practices

### 1. Always validate JSON first

```bash
jq empty
```

### 2. Use `-c` in pipelines

Reduces log noise:

```bash
jq -c '.'
```

### 3. Use `-r` for scripting

```bash
jq -r '.field'
```

### 4. Combine with grep when needed

```bash
grep ERROR app.log | jq
```

### 5. Avoid unnecessary formatting in CI/CD

Keep output machine-readable.

---

## 16. Common Patterns Cheat Sheet

| Task            | Command                        |
| --------------- | ------------------------------ |
| Pretty print    | `jq '.'`                       |
| Filter by field | `jq 'select(.field=="value")'` |
| Extract field   | `jq '.field'`                  |
| Array iteration | `jq '.[]'`                     |
| Count items     | `jq 'length'`                  |
| Convert to text | `jq -r '.field'`               |
| Compact output  | `jq -c '.'`                    |

---

## 17. Real DevOps Example Pipeline

### Analyze application logs

```bash
cat app.log | jq -c 'select(.level=="error") | {time, service, message}'
```

### Kubernetes debugging

```bash
kubectl logs my-pod | jq -c 'select(.status>=500)'
```

### CI/CD artifact inspection

```bash
cat terraform-output.json | jq '.outputs'
```