added jq documents

Linux/Advanced-Administration/02-jq.md

@@ -0,0 +1,386 @@
+## 1. Overview
+
+jq is a lightweight and powerful command-line tool for parsing, filtering, transforming, and formatting JSON data.
+
+In DevOps workflows, `jq` is commonly used to:
+
+* Analyze logs (Docker, Kubernetes, application logs)
+* Filter observability data (metrics/events in JSON format)
+* Debug CI/CD pipelines
+* Process API responses (AWS, GitHub, Terraform outputs)
+* Transform JSON for automation scripts
+
+It is essentially the “grep + awk + sed” equivalent for JSON.
+
+---
+
+## 2. Installation
+
+### Linux (Debian/Ubuntu)
+
+```bash
+sudo apt-get update
+sudo apt-get install jq
+```
+
+### RHEL/CentOS
+
+```bash
+sudo yum install jq
+```
+
+### macOS
+
+```bash
+brew install jq
+```
+
+### Verify installation
+
+```bash
+jq --version
+```
+
+---
+
+## 3. Basic Syntax
+
+```bash
+jq '<filter>' file.json
+```
+
+Or pipe input:
+
+```bash
+cat file.json | jq '<filter>'
+```
+
+---
+
+## 4. Core Concepts
+
+### 4.1 Identity filter
+
+Returns input as-is:
+
+```bash
+jq '.'
+```
+
+### 4.2 Access fields
+
+```bash
+jq '.name'
+jq '.user.id'
+```
+
+### 4.3 Arrays
+
+```bash
+jq '.items[]'
+```
+
+### 4.4 Pretty print
+
+```bash
+jq '.'
+```
+
+---
+
+## 5. Filtering Logs (DevOps Use Case)
+
+### Example log entry
+
+```json
+{
+  "level": "error",
+  "service": "auth",
+  "message": "invalid credentials",
+  "status": 401,
+  "timestamp": "2026-04-15T10:00:00Z"
+}
+```
+
+### Filter only errors
+
+```bash
+jq 'select(.level == "error")'
+```
+
+### Filter by service
+
+```bash
+jq 'select(.service == "auth")'
+```
+
+### Extract specific fields
+
+```bash
+jq '{time: .timestamp, msg: .message}'
+```
+
+---
+
+## 6. Working with Arrays (Common in Logs)
+
+### Example: multiple log entries
+
+### Count entries
+
+```bash
+jq 'length'
+```
+
+### Filter array elements
+
+```bash
+jq '.[] | select(.status >= 500)'
+```
+
+### Extract fields from array
+
+```bash
+jq '.[] | {service, status, message}'
+```
+
+---
+
+## 7. Kubernetes Logs with jq
+
+### Example:
+
+```bash
+kubectl logs pod-name -n default | jq
+```
+
+### Filter error logs
+
+```bash
+kubectl logs pod-name | jq 'select(.level=="error")'
+```
+
+### Extract container metadata logs
+
+```bash
+kubectl logs pod-name | jq '{time, container, message}'
+```
+
+---
+
+## 8. Docker Logs with jq
+
+### Streaming logs
+
+```bash
+docker logs container_name | jq
+```
+
+### Filter failures
+
+```bash
+docker logs container_name | jq 'select(.status != "success")'
+```
+
+---
+
+## 9. AWS / Cloud Logs (JSON-based)
+
+### Example CloudWatch JSON logs
+
+```bash
+aws logs filter-log-events --log-group-name my-app | jq
+```
+
+### Extract messages only
+
+```bash
+... | jq '.events[].message'
+```
+
+### Filter by keyword
+
+```bash
+... | jq '.events[] | select(.message | contains("ERROR"))'
+```
+
+---
+
+## 10. Transforming JSON (Automation Use Cases)
+
+### Rename fields
+
+```bash
+jq '{userId: .id, username: .name}'
+```
+
+### Add computed fields
+
+```bash
+jq '. + {isActive: true}'
+```
+
+### Build new structure
+
+```bash
+jq '{users: [.[] | {id, name}]}'
+```
+
+---
+
+## 11. Advanced Filtering
+
+### Logical conditions
+
+```bash
+jq 'select(.status == 200 and .service == "api")'
+```
+
+### Regex matching
+
+```bash
+jq 'select(.message | test("timeout|failed"))'
+```
+
+### Sorting
+
+```bash
+jq 'sort_by(.timestamp)'
+```
+
+### Unique values
+
+```bash
+jq 'unique_by(.service)'
+```
+
+---
+
+## 12. Aggregations (DevOps Analytics)
+
+### Count by status
+
+```bash
+jq 'group_by(.status) | map({status: .[0].status, count: length})'
+```
+
+### Error rate estimation
+
+```bash
+jq 'map(select(.status >= 400)) | length'
+```
+
+---
+
+## 13. Formatting Output for Humans
+
+### Compact JSON
+
+```bash
+jq -c '.'
+```
+
+### Raw output (no quotes)
+
+```bash
+jq -r '.message'
+```
+
+### Tabular-like output
+
+```bash
+jq -r '[.timestamp, .level, .message] | @tsv'
+```
+
+---
+
+## 14. Debugging Pipelines
+
+### Validate JSON
+
+```bash
+jq empty file.json
+```
+
+### Highlight structure
+
+```bash
+jq '. | type'
+```
+
+### Pretty inspect nested structures
+
+```bash
+jq 'paths'
+```
+
+---
+
+## 15. DevOps Best Practices
+
+### 1. Always validate JSON first
+
+```bash
+jq empty
+```
+
+### 2. Use `-c` in pipelines
+
+Reduces log noise:
+
+```bash
+jq -c '.'
+```
+
+### 3. Use `-r` for scripting
+
+```bash
+jq -r '.field'
+```
+
+### 4. Combine with grep when needed
+
+```bash
+grep ERROR app.log | jq
+```
+
+### 5. Avoid unnecessary formatting in CI/CD
+
+Keep output machine-readable.
+
+---
+
+## 16. Common Patterns Cheat Sheet
+
+| Task            | Command                        |
+| --------------- | ------------------------------ |
+| Pretty print    | `jq '.'`                       |
+| Filter by field | `jq 'select(.field=="value")'` |
+| Extract field   | `jq '.field'`                  |
+| Array iteration | `jq '.[]'`                     |
+| Count items     | `jq 'length'`                  |
+| Convert to text | `jq -r '.field'`               |
+| Compact output  | `jq -c '.'`                    |
+
+---
+
+## 17. Real DevOps Example Pipeline
+
+### Analyze application logs
+
+```bash
+cat app.log | jq -c 'select(.level=="error") | {time, service, message}'
+```
+
+### Kubernetes debugging
+
+```bash
+kubectl logs my-pod | jq -c 'select(.status>=500)'
+```
+
+### CI/CD artifact inspection
+
+```bash
+cat terraform-output.json | jq '.outputs'
+```
+