Compare commits

..

10 Commits

Author SHA1 Message Date
RadinPirouz
f2c0cebd8a added jitsi replication doc 2026-05-30 20:35:11 +03:30
RadinPirouz
33399a8019 added jitsi plugin doc 2026-05-30 20:30:38 +03:30
a276e44338 Jitsi Introduction 2026-05-29 23:47:49 +03:30
4a421526c9 Added Innodb Docs 2026-04-29 00:34:46 +03:30
ff7e1fd246 Added MySQL Benchmark Doc 2026-04-26 00:27:42 +03:30
d9d59f570e Added MySQL Configuration Docs 2026-04-26 00:12:18 +03:30
30bae64e51 Added bind9 docs 2026-04-21 23:15:52 +03:30
457faf1989 Added bind9 docs 2026-04-21 23:09:34 +03:30
fa6bb1557d added jq documents 2026-04-15 00:45:23 +03:30
edea1fe9e8 Added Zombie Ps Docs 2026-04-14 18:02:10 +03:30
13 changed files with 3216 additions and 69 deletions

View File

@@ -0,0 +1,216 @@
# MySQL Performance and Administration Guide for DevOps
This document covers essential MySQL configuration parameters, monitoring practices, data integrity checks, slow query tuning, and useful command-line tools for database administration.
## Table of Contents
- [MySQL Performance and Administration Guide for DevOps](#mysql-performance-and-administration-guide-for-devops)
- [Table of Contents](#table-of-contents)
- [Configuration Parameters](#configuration-parameters)
- [max\_allowed\_packet](#max_allowed_packet)
- [Error and Slow Query Logs](#error-and-slow-query-logs)
- [skip\_name\_resolve](#skip_name_resolve)
- [Initial Root Password and Access Control](#initial-root-password-and-access-control)
- [Monitoring](#monitoring)
- [Performance Schema and Information Schema](#performance-schema-and-information-schema)
- [Percona Monitoring and Management (PMM)](#percona-monitoring-and-management-pmm)
- [Data Corruption Checking](#data-corruption-checking)
- [Slow Query Configuration Details](#slow-query-configuration-details)
- [Tools](#tools)
- [pt-stalk](#pt-stalk)
- [pt-diskstats](#pt-diskstats)
- [pt-summary](#pt-summary)
- [mysqlcheck](#mysqlcheck)
---
## Configuration Parameters
### max_allowed_packet
```ini
max_allowed_packet = 128M
```
- **Purpose**: Defines the maximum size of a single communication packet between the MySQL client and server.
- **Best Practice**: For large BLOB/ TEXT fields or large dumps, set to `1G`. Adjust according to workload and available memory.
### Error and Slow Query Logs
Place these directives under the `[mysqld]` section:
```ini
[mysqld]
log-error = /var/log/mysql/error.log
slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow.log
```
- `log-error`: Location of the error log file.
- `slow_query_log`: Enables slow query logging.
- `slow_query_log_file`: Path to the slow query log file.
### skip_name_resolve
```ini
skip_name_resolve
```
- **Effect**: Disables resolution of client hostnames to IP addresses.
- **Benefit**: Improves connection speed and reduces DNS overhead. Use when all users connect via IP addresses or CIDR ranges.
---
## Initial Root Password and Access Control
After the first initialization of MySQL, the temporary root password is stored in `/var/log/mysqld.log`. Use it to log in and change the password.
**Change root password:**
```sql
ALTER USER 'root'@'%' IDENTIFIED BY '123';
```
**Restrict access to a specific IP or range** (e.g., 192.168.1.0/24):
```sql
ALTER USER 'root'@'192.168.1.0/24' IDENTIFIED BY '123';
```
> Replace `'123'` with a strong password and adjust the subnet as needed.
---
## Monitoring
### Performance Schema and Information Schema
MySQL provides two built-in schemas for monitoring:
- **performance_schema**: Tracks server execution details at a low level (waits, events, statements, etc.).
- **information_schema**: Provides metadata about database objects (tables, columns, privileges, etc.).
### Percona Monitoring and Management (PMM)
PMM is an open-source monitoring solution that integrates with **Grafana** for dashboards and visualization. It collects metrics from MySQL, PostgreSQL, MongoDB, and system hosts.
**Key features**:
- Query analytics and slow query tracking.
- Realtime performance dashboards.
- Historical data retention.
**How to use**:
1. Install PMM Server (Docker or package) on a dedicated host.
2. Install PMM Client on each MySQL host.
3. Connect the client to the server:
`pmm-admin config --server-url=https://<pmm-server-ip>:443`
4. Add MySQL service:
`pmm-admin add mysql --username=root --password=<pwd>`
---
## Data Corruption Checking
Use `mysqlcheck` to verify table integrity.
**Check all databases:**
```bash
mysqlcheck --check --all-databases -u root -p
```
**Check a specific database:**
```bash
mysqlcheck --check <database_name> -u root -p
```
The command will report any corrupted tables. For deeper repair, use `--repair` after verifying the need.
---
## Slow Query Configuration Details
Extended slow query log configuration:
```ini
slow_query_log = 1
slow_query_log_file = /var/log/mysql/mysql-slow.log
long_query_time = 2
```
- `long_query_time`: Queries that take more than `2` seconds are logged. Fractional seconds allowed (e.g., `0.5`).
- Additional useful parameters:
- `log_queries_not_using_indexes = 1` logs queries that do not utilise indexes.
- `log_slow_admin_statements = 1` logs slow administrative statements (OPTIMIZE, ANALYZE, ALTER).
After changes, restart MySQL: `sudo systemctl restart mysql`
---
## Tools
This section covers command-line tools from the **Percona Toolkit** (commonly used by DBAs and DevOps). The names in the original notes (`py-stals`, `py-disktats`, `pt-summery`) likely refer to `pt-stalk`, `pt-diskstats`, and `pt-summary`.
### pt-stalk
**Description**: Watches for a MySQL problem (e.g., high load, long lock wait) and collects diagnostic data when the problem occurs.
**Installation** (Ubuntu/Debian):
```bash
sudo apt-get install percona-toolkit
```
**Basic usage**:
```bash
pt-stalk --user=root --password=<pwd> --dest=/var/log/pt-stalk -- --defaults-file=/etc/mysql/my.cnf
```
- `--user`, `--password`: MySQL credentials.
- `--dest`: Directory where collected data will be stored.
- The `--` separates pt-stalk options from MySQL options.
- By default, the script runs as a daemon. Use `--run-time=30s` for a single collection cycle.
**How to use**:
1. Configure thresholds (disk free, processlist size, etc.) to trigger data collection.
2. Review collected files (tarballs) after an incident to diagnose root causes.
### pt-diskstats
**Description**: Analyzes disk I/O performance interactively, similar to `iostat`, but with more detailed perdevice statistics and latency histograms.
**Basic usage**:
```bash
pt-diskstats --interval=5 --iterations=10
```
- `--interval`: Seconds between samples.
- `--iterations`: Number of samples (omit for infinite).
- You can specify devices: `pt-diskstats --devices=sda,sdb`
**How to use**:
- Monitor disk latency and IOPS in real time to identify storage bottlenecks for MySQL.
- Redirect output to a file for later analysis: `pt-diskstats --interval=2 > /tmp/io.log`.
### pt-summary
**Description**: Collects and prints a system overview CPU, memory, disk, network, and MySQL configuration.
**Basic usage**:
```bash
pt-summary
```
**How to use**:
- Run before and after changes to capture baseline system state.
- Combine with `pt-mysql-summary` for MySQLspecific detail.
- The output helps quickly understand the environment when debugging performance issues.
### mysqlcheck
Already covered in the [Data Corruption Checking](#data-corruption-checking) section.

View File

@@ -0,0 +1,161 @@
```markdown
# Benchmarking MySQL Performance
## Introduction
As a DevOps engineer, understanding MySQL performance under various workloads is critical for capacity planning, query optimization, and infrastructure tuning. Benchmarking provides repeatable, measurable insights into how your database behaves under stress. This document outlines standard methodologies, tools, and metrics for benchmarking MySQL effectively.
## Key Performance Metrics
Before running benchmarks, focus on these core metrics:
- **Throughput** - Transactions per second (TPS) or queries per second (QPS)
- **Latency** - Average, 95th, and 99th percentile response times
- **Concurrency** - How performance scales with increasing connections
- **Resource Utilization** - CPU, memory, disk I/O, and network usage on database host
- **Transaction Consistency** - Ensure ACID properties hold under load
## Benchmarking Tools
### Sysbench
The most common and flexible tool. Supports OLTP workloads, point selects, random reads/writes, and more.
Installation:
```bash
# Ubuntu/Debian
sudo apt install sysbench
# RHEL/CentOS
sudo yum install sysbench
```
### mysqlslap
Built-in MySQL utility for simulating client load. Simple but less customizable.
```bash
mysqlslap --host=localhost --user=root --password=secret \
--auto-generate-sql --concurrency=50 --iterations=3
```
### Other Tools
- **HammerDB** - Graphical TPC-C style benchmarking
- **tcpdump + pt-query-digest** - Analyze real production traffic
- **dbt2** - Open source TPC-C implementation
## Benchmark Methodology
### Prerequisites
1. **Isolate the environment** - Use a dedicated database server or cloud instance. Disable OS background services (backups, cron, monitoring) that interfere.
2. **Configure MySQL** - Match production settings (buffer pool, log file sizes, innodb_flush_log_at_trx_commit, etc.).
3. **Prepare data** - Use realistic data volumes. For sysbench, typically 10-100 million rows per table.
4. **Warm up the buffer pool** - Run a trial workload before measuring.
### Phases
1. **Plan** - Define workload type (read-heavy, write-heavy, mixed), duration, and concurrency levels.
2. **Prepare** - Create test tables and data.
3. **Run** - Execute benchmark with monitoring tools active (e.g., `htop`, `iostat`, `mysqladmin status`).
4. **Cleanup** - Remove test databases.
5. **Analyze** - Compare results against baseline.
## Example: Sysbench OLTP Benchmark
### 1. Prepare Data
Create 4 tables with 1 million rows each:
```bash
sysbench oltp_read_write \
--mysql-host=127.0.0.1 \
--mysql-port=3306 \
--mysql-user=sysbench \
--mysql-password=bench123 \
--mysql-db=testdb \
--tables=4 \
--table-size=1000000 \
prepare
```
### 2. Run the Benchmark
Execute with varying concurrency (e.g., 1, 4, 8, 16, 32, 64 threads):
```bash
sysbench oltp_read_write \
--mysql-host=127.0.0.1 \
--port=3306 \
--user=sysbench \
--password=bench123 \
--db=testdb \
--tables=4 \
--table-size=1000000 \
--threads=32 \
--time=300 \
--report-interval=10 \
run
```
Parameters explained:
- `--threads` - Number of concurrent clients
- `--time` - Benchmark duration in seconds (300 = 5 minutes)
- `--report-interval` - Print intermediate stats every N seconds
### 3. Clean Up
```bash
sysbench oltp_read_write \
--mysql-host=127.0.0.1 \
--mysql-user=sysbench \
--mysql-password=bench123 \
--mysql-db=testdb \
cleanup
```
## Analyzing Results
### Key Output from Sysbench
After a run, sysbench outputs:
```
SQL statistics:
queries performed:
read: 1091424
write: 311836
other: 155918
total: 1559178
transactions: 77958 (259.83 per sec.)
queries: 1559178 (5196.67 per sec.)
ignored errors: 0 (0.00 per sec.)
reconnects: 0 (0.00 per sec.)
General statistics:
total time: 300.0050s
total number of events: 77958
Latency (ms):
min: 4.01
avg: 123.07
max: 1152.19
95th percentile: 210.56
sum: 9591283.90
```
Critical metrics:
- **Transactions per second** - Primary throughput indicator
- **95th percentile latency** - Important for SLOs
- **Avg latency** - General responsiveness
### Interpreting Results
| Observation | Potential Cause |
|-------------|----------------|
| TPS scales linearly with threads up to a point | Healthy system, then bottleneck may shift |
| Latency spikes after certain concurrency | Contention on locks, mutexes, or I/O queue saturation |
| Dropping TPS at high concurrency | Context switching overhead or connection limits |
| High 95th vs avg latency | Occasional stalls (checkpointing, swapping, network latency) |

View File

@@ -0,0 +1,189 @@
# InnoDB Storage Engine:
This document provides an in-depth explanation of the InnoDB storage engine, its on-disk structures, memory management mechanisms (buffer pool), and change buffering. The target audience is database administrators and DevOps engineers who need to understand and tune InnoDB for performance and reliability.
## Table of Contents
1. What is InnoDB?
2. MySQL Data Directories Related to InnoDB
3. Pages in InnoDB
4. Index Pages
5. Tablespaces
6. Buffer and Buffer Pool
- Buffer Pool Metrics
- Configuration Example
7. Change Buffering
- Configuration Parameters
---
## 1. What is InnoDB?
InnoDB is a storage engine for MySQL that provides:
- ACID compliance (Atomicity, Consistency, Isolation, Durability)
- Row-level locking
- Foreign key constraints
- Crash recovery
- Multi-version concurrency control (MVCC)
It is the default storage engine for MySQL since version 5.5. InnoDB stores data in tablespaces, which are composed of pages.
---
## 2. MySQL Data Directories Related to InnoDB
In a typical MySQL installation, several directories are used to store InnoDBrelated files. Understanding their purpose helps with backup, recovery, and capacity planning.
| Directory | Description |
|---------------------|-------------------------------------------------------------------------------------------------|
| `innodb_redo` | Contains redo log files. Redo logs record changes made to InnoDB data to ensure durability. |
| `innodb_temp` | Stores temporary tablespaces used for internal temporary tables and ondisk temporary objects. |
| `mysql` | The system schema that holds metadata (database names, tables, privileges, etc.). |
Even though `mysql` is not exclusively InnoDB, many system tables now use InnoDB by default.
---
## 3. Pages in InnoDB
A page is the smallest unit of storage in InnoDB. All data (table rows, indexes, etc.) is stored in pages.
- **Default page size**: 16 KB (can be configured to 4 KB, 8 KB, 32 KB, or 64 KB via `innodb_page_size`).
- **Structure**: Each page contains a header, a trailer (checksum), and the actual data.
- When a page is full, InnoDB allocates a new page to hold more data.
Pages are read from disk into memory (the buffer pool) and written back to disk when modified.
---
## 4. Index Page in InnoDB
An **index page** is a special type of page that stores index entries. InnoDB uses a Btree data structure for both primary and secondary indexes.
- **Primary key index (clustered index)**: The leaf pages contain the actual row data for the table. The entire table is organised as a Btree based on the primary key.
- **Secondary index**: Leaf pages contain the indexed column value and the primary key value (which is used to look up the full row in the clustered index).
Index pages are also 16 KB by default. Each index page contains pointers to child pages (for nonleaf levels) or row pointers (for leaf levels).
---
## 5. Tablespace
A tablespace is a logical storage container that holds InnoDB data. There are several types of tablespaces:
| Tablespace Type | Description |
|--------------------------|------------------------------------------------------------------------------------|
| System tablespace | Contains the data dictionary, doublewrite buffer, change buffer, and undo logs. |
| Filepertable tablespace| Each table has its own `.ibd` file (controlled by `innodb_file_per_table`). |
| General tablespaces | Usercreated tablespaces that can hold multiple tables. |
| Undo tablespace | Stores undo logs for MVCC and transaction rollback. |
| Temporary tablespace | Stores temporary tables created during queries or sessions (nonpersistent). |
Each tablespace is divided into pages. The system tablespace (usually `ibdata1`) starts at 12 MB and grows as needed.
---
## 6. Buffer and Buffer Pool
### What is a Buffer?
A buffer is a memory area that temporarily holds data read from disk to reduce the number of direct disk I/O operations. In InnoDB, the main buffer is called the **buffer pool**.
### Buffer Pool
When a query requests data, InnoDB first checks whether the required pages are already present in the buffer pool:
- **If yes (cache hit)**: The data is returned directly from memory (extremely fast).
- **If no (cache miss)**: InnoDB reads the relevant pages from disk into the buffer pool, then serves the data from memory.
#### Recommended Size
A common best practice is to set the buffer pool size to approximately 75% of the available system memory on a dedicated database server. For shared servers, reduce the percentage accordingly.
### Configuration Example
In MySQL configuration file (`my.cnf` or `my.ini`):
```ini
[mysqld]
innodb_buffer_pool_size = 1G
```
Alternatively, change it dynamically at runtime (MySQL 8.0+):
```sql
SET PERSIST innodb_buffer_pool_size = 1073741824; -- value in bytes
```
### Buffer Pool Metrics
These status variables help monitor buffer pool efficiency. Query them with:
```sql
SHOW GLOBAL STATUS LIKE 'innodb_buffer_pool%';
```
| Metric | Description |
|-------------------------------------|------------------------------------------------------------------------------------------------------|
| `Innodb_buffer_pool_reads` | Number of times InnoDB had to read a page from disk because it was not available in the buffer pool. High values indicate a shortage of buffer pool memory. |
| `Innodb_buffer_pool_read_requests` | Total number of logical read requests (page accesses) made to the buffer pool. |
| `Innodb_buffer_pool_wait_free` | Count of times a thread had to wait for a clean page to become available. Nonzero values suggest the buffer pool is under pressure (e.g., dirty page flushing is slow). |
| `Innodb_buffer_pool_pages_free` | Number of free pages currently in the buffer pool. Low values mean the buffer pool is nearly full. |
#### Interpreting Metrics
- **Cache hit ratio** = `(Innodb_buffer_pool_read_requests - Innodb_buffer_pool_reads) / Innodb_buffer_pool_read_requests`. Aim for >99%.
- If `Innodb_buffer_pool_wait_free` keeps increasing, consider increasing the buffer pool size or tuning flushing behaviour (`innodb_io_capacity`, `innodb_max_dirty_pages_pct`).
- Low `Innodb_buffer_pool_pages_free` alone is not a problem; it just shows the buffer pool is actively used.
---
## 7. Change Buffering
Change buffering is a feature that delays writing changes to secondary index pages. Instead of immediately updating the index pages on disk when a nonunique secondary index is modified, InnoDB records the change in a special area called the **change buffer** (which is part of the system tablespace). Later, when the index pages are read into the buffer pool by other queries, the buffered changes are merged (applied) to the pages.
This reduces random disk I/O and improves performance for workloads with many Data Manipulation Language (DML) operations (INSERT, UPDATE, DELETE) that affect secondary indexes.
### Configuration Parameters
Both parameters are set in the MySQL configuration file.
#### `innodb_change_buffering`
Controls which operations are buffered. Possible values:
| Value | Description |
|-----------|--------------------------------------------------------------------------|
| `none` | Do not buffer any changes. |
| `inserts` | Buffer only insert operations. |
| `deletes` | Buffer only delete operations (including purge operations). |
| `changes` | Buffer inserts and deletemarking operations (but not actual purges). |
| `purges` | Buffer only the physical deletion of rows that occur during background purge. |
| `all` | Buffer inserts, deletemarking, and purges (default value). |
Example configuration:
```ini
[mysqld]
innodb_change_buffering = all
```
#### `innodb_change_buffer_max_size`
Specifies the maximum size of the change buffer as a percentage of the total buffer pool size. The default is 25 (meaning 25% of the buffer pool). Valid range is 0 to 50.
Increasing this value allows more space for buffered changes, which can help workloads with heavy DML on secondary indexes, but it reduces the space available for cached data pages.
Example:
```ini
[mysqld]
innodb_change_buffer_max_size = 30
```
### When to Tune Change Buffering
- **Writeheavy OLTP**: Keep `innodb_change_buffering = all` and possibly increase `innodb_change_buffer_max_size` to 3040.
- **Readonly or mostly reads**: Set `innodb_change_buffering = none` to avoid wasting buffer pool memory.
- **Unique indexes**: Change buffering does not apply to unique secondary indexes because uniqueness checks require immediate disk access.

View File

View File

@@ -0,0 +1,46 @@
## Zombie Processes
### What is a Zombie Process?
In Linux/Unix operating systems, when a process ends, its execution is halted, but it leaves behind an entry in the process table. This entry contains the process's exit status, which needs to be read by its parent process.
A **zombie process** (or defunct process, indicated by the `Z` state in `ps` output) is a child process that has completed its execution, but its parent process has not yet called the `wait()` or `waitpid()` system calls to read its exit status. Because the parent hasn't acknowledged the death, the OS keeps the child's entry in the process table.
### The Effect of Zombie Processes
At first glance, a zombie process seems harmless:
* It consumes **$0$** CPU resources.
* It consumes **$0$** Memory (RAM).
**The Danger: PID Exhaustion**
The only resource a zombie consumes is an entry in the OS process table and a Process ID (PID). Operating systems have a maximum limit of PIDs available (often $32768$ by default, though tunable in `sysctl`). If a poorly written parent process continuously spawns children and never reaps them, the system will eventually run out of available PIDs.
When PID exhaustion occurs, the OS cannot create any new processes. You won't be able to SSH into the server, execute basic commands, or spawn new application threads, effectively bringing the system down.
### How to Identify Zombies
* **Using `top`:** The header will explicitly show a counter for zombie processes.
* **Using `ps`:** List the PIDs of all processes with a `Z` (Zombie) state:
```bash
ps aux | awk '{ print $8 " " $2 }' | grep -w Z
```
### How to "Kill" a Zombie Process
**Important Rule:** You cannot kill a zombie process directly. Even `kill -9 <zombie_pid>` (SIGKILL) will not work because the process is already dead. To clear a zombie, you must deal with its **parent process**.
**Step 1: Find the Parent Process ID (PPID)**
Find out which process spawned the zombie:
```bash
ps -o ppid= -p <zombie_pid>
```
**Step 2: Ask the parent to reap the child**
Send a `SIGCHLD` signal to the parent process. This acts as a gentle reminder for the parent to execute the `wait()` system call and clean up its children.
```bash
kill -s SIGCHLD <parent_pid>
```
**Step 3: Kill the Parent Process (If Step 2 fails)**
If the parent process is poorly programmed, hung, or ignoring the `SIGCHLD` signal, your only operational choice is to kill the parent process:
```bash
kill -9 <parent_pid>
```
*Note on Step 3:* When the parent dies, the zombie process becomes an "orphan". The OS kernel automatically reassigns all orphan processes to the init system (usually `systemd` or `init`, which is PID $1$). PID $1$ is specifically designed to routinely execute `wait()` and will instantly reap the zombie, finally clearing it from the process table.

View File

@@ -0,0 +1,386 @@
## 1. Overview
jq is a lightweight and powerful command-line tool for parsing, filtering, transforming, and formatting JSON data.
In DevOps workflows, `jq` is commonly used to:
* Analyze logs (Docker, Kubernetes, application logs)
* Filter observability data (metrics/events in JSON format)
* Debug CI/CD pipelines
* Process API responses (AWS, GitHub, Terraform outputs)
* Transform JSON for automation scripts
It is essentially the “grep + awk + sed” equivalent for JSON.
---
## 2. Installation
### Linux (Debian/Ubuntu)
```bash
sudo apt-get update
sudo apt-get install jq
```
### RHEL/CentOS
```bash
sudo yum install jq
```
### macOS
```bash
brew install jq
```
### Verify installation
```bash
jq --version
```
---
## 3. Basic Syntax
```bash
jq '<filter>' file.json
```
Or pipe input:
```bash
cat file.json | jq '<filter>'
```
---
## 4. Core Concepts
### 4.1 Identity filter
Returns input as-is:
```bash
jq '.'
```
### 4.2 Access fields
```bash
jq '.name'
jq '.user.id'
```
### 4.3 Arrays
```bash
jq '.items[]'
```
### 4.4 Pretty print
```bash
jq '.'
```
---
## 5. Filtering Logs (DevOps Use Case)
### Example log entry
```json
{
"level": "error",
"service": "auth",
"message": "invalid credentials",
"status": 401,
"timestamp": "2026-04-15T10:00:00Z"
}
```
### Filter only errors
```bash
jq 'select(.level == "error")'
```
### Filter by service
```bash
jq 'select(.service == "auth")'
```
### Extract specific fields
```bash
jq '{time: .timestamp, msg: .message}'
```
---
## 6. Working with Arrays (Common in Logs)
### Example: multiple log entries
### Count entries
```bash
jq 'length'
```
### Filter array elements
```bash
jq '.[] | select(.status >= 500)'
```
### Extract fields from array
```bash
jq '.[] | {service, status, message}'
```
---
## 7. Kubernetes Logs with jq
### Example:
```bash
kubectl logs pod-name -n default | jq
```
### Filter error logs
```bash
kubectl logs pod-name | jq 'select(.level=="error")'
```
### Extract container metadata logs
```bash
kubectl logs pod-name | jq '{time, container, message}'
```
---
## 8. Docker Logs with jq
### Streaming logs
```bash
docker logs container_name | jq
```
### Filter failures
```bash
docker logs container_name | jq 'select(.status != "success")'
```
---
## 9. AWS / Cloud Logs (JSON-based)
### Example CloudWatch JSON logs
```bash
aws logs filter-log-events --log-group-name my-app | jq
```
### Extract messages only
```bash
... | jq '.events[].message'
```
### Filter by keyword
```bash
... | jq '.events[] | select(.message | contains("ERROR"))'
```
---
## 10. Transforming JSON (Automation Use Cases)
### Rename fields
```bash
jq '{userId: .id, username: .name}'
```
### Add computed fields
```bash
jq '. + {isActive: true}'
```
### Build new structure
```bash
jq '{users: [.[] | {id, name}]}'
```
---
## 11. Advanced Filtering
### Logical conditions
```bash
jq 'select(.status == 200 and .service == "api")'
```
### Regex matching
```bash
jq 'select(.message | test("timeout|failed"))'
```
### Sorting
```bash
jq 'sort_by(.timestamp)'
```
### Unique values
```bash
jq 'unique_by(.service)'
```
---
## 12. Aggregations (DevOps Analytics)
### Count by status
```bash
jq 'group_by(.status) | map({status: .[0].status, count: length})'
```
### Error rate estimation
```bash
jq 'map(select(.status >= 400)) | length'
```
---
## 13. Formatting Output for Humans
### Compact JSON
```bash
jq -c '.'
```
### Raw output (no quotes)
```bash
jq -r '.message'
```
### Tabular-like output
```bash
jq -r '[.timestamp, .level, .message] | @tsv'
```
---
## 14. Debugging Pipelines
### Validate JSON
```bash
jq empty file.json
```
### Highlight structure
```bash
jq '. | type'
```
### Pretty inspect nested structures
```bash
jq 'paths'
```
---
## 15. DevOps Best Practices
### 1. Always validate JSON first
```bash
jq empty
```
### 2. Use `-c` in pipelines
Reduces log noise:
```bash
jq -c '.'
```
### 3. Use `-r` for scripting
```bash
jq -r '.field'
```
### 4. Combine with grep when needed
```bash
grep ERROR app.log | jq
```
### 5. Avoid unnecessary formatting in CI/CD
Keep output machine-readable.
---
## 16. Common Patterns Cheat Sheet
| Task | Command |
| --------------- | ------------------------------ |
| Pretty print | `jq '.'` |
| Filter by field | `jq 'select(.field=="value")'` |
| Extract field | `jq '.field'` |
| Array iteration | `jq '.[]'` |
| Count items | `jq 'length'` |
| Convert to text | `jq -r '.field'` |
| Compact output | `jq -c '.'` |
---
## 17. Real DevOps Example Pipeline
### Analyze application logs
```bash
cat app.log | jq -c 'select(.level=="error") | {time, service, message}'
```
### Kubernetes debugging
```bash
kubectl logs my-pod | jq -c 'select(.status>=500)'
```
### CI/CD artifact inspection
```bash
cat terraform-output.json | jq '.outputs'
```

View File

@@ -1,18 +1,18 @@
# ⚙️ PS Command
# PS Command
The `ps` (process status) command is used to **view running processes** on a Linux system. Its useful for monitoring and troubleshooting tasks.
---
## 🧾 Basic Usage
## Basic Usage
### 🔍 Show tasks in the current shell
### Show tasks in the current shell
```bash
ps
```
### 🔍 Show tasks in the current shell with **full info**
### Show tasks in the current shell with **full info**
```bash
ps -f
@@ -20,9 +20,9 @@ ps -f
---
## 🌍 View System-Wide Processes
## View System-Wide Processes
### 📋 Show **all** processes
### Show **all** processes
```bash
ps -A
@@ -32,17 +32,17 @@ ps -e
---
### 👤 Show tasks by **specific user**
### Show tasks by **specific user**
```bash
ps -u <username>
```
📌 Replace `<username>` with the actual user name.
Replace `<username>` with the actual user name.
---
### 📊 Show **detailed info for all** tasks
### Show **detailed info for all** tasks
```bash
ps aux
@@ -50,7 +50,7 @@ ps aux
---
## 📘 Output Fields Explained
## Output Fields Explained
| Column | Description |
| --------- | -------------------------------------------------- |
@@ -58,22 +58,20 @@ ps aux
| `PID` | Process ID |
| `%CPU` | CPU usage percentage |
| `%MEM` | Memory usage percentage |
| `STAT` | Process state: `R` (running), `S` (sleeping), etc. |
| `STAT` | Process state: `R` (running), `S` (sleeping), `Z` (zombie), etc. |
| `START` | Time when the process started |
| `TIME` | Total CPU time used |
| `COMMAND` | Command that started the process |
### 📑 Show List Jobs
### Show List Jobs
```bash
jobs
```
### 🔄Move Process From Background To Forground
### Move Process From Background To Foreground
```bash
fg
```

188
README.md
View File

@@ -1,57 +1,111 @@
# 🐧 DevOps Documents
# 🐧 DevOps Knowledge Base
A curated collection of scripts, configuration files, and guides for managing and configuring Linux-based systems. This personal repository serves as a comprehensive knowledge base to simplify deployment, automation, monitoring, security, and much more.
> 🚀 *Your centralized hub for Linux, DevOps, and Infrastructure mastery*
A structured and ever-growing collection of **scripts, configurations, and hands-on guides** designed to simplify:
* ⚙️ Automation
* 🐳 Containerization
* 📊 Monitoring
* 🔐 Security
* ☁️ Cloud & Infrastructure
---
## 📂 Repository Structure
## 🧭 Quick Navigation
### ⚙️ Configuration Management & Automation
- [Ansible](./Configuration%20Management%20&%20Automation/Ansible)
- [CronJob](./Configuration%20Management%20&%20Automation/CronJob)
### ⚙️ Configuration & Automation
### 🐳 Containerization & Orchestration
- [Docker](./Containerization%20&%20Orchestration/Docker)
- [Kubernetes (In Progress)](./Containerization%20&%20Orchestration/Kubernetes)
* 🔹 Ansible
* 🔹 CronJobs
### 🐳 Containers & Orchestration
* 🔹 Docker
* 🔹 Kubernetes *(Work in Progress)*
* 🔹 Dozzle
### ☁️ Cloud
* 🔹 AWS
### 🗄️ Databases
- [PostgreSQL](./Databases/Postgresql)
* 🔹 PostgreSQL
* 🔹 MariaDB
### ⚡ Caching
- [Redis](./Caching/redis)
* 🔹 Redis
### 💻 Code Management
- [Git](./Code%20Management/Git)
* 🔹 Git
* 🔹 GitLab (CI/CD, Cache, Baremetal Setup)
### 🔀 High Availability
- [HAProxy](./High%20Availability/Ha-Proxy)
* 🔹 HAProxy
### 📊 Monitoring & Logging
- [Grafana](./Monitoring%20&%20Logging/Grafana)
- [LibreNMS](./Monitoring%20&%20Logging/Librenms)
- [Netdata](./Monitoring%20&%20Logging/Netdata)
- [Zabbix](./Monitoring%20&%20Logging/Zabbix)
### 🔐 Networking & Security
- [iptables](./Security%20&%20Networking/Iptables)
- [Nmap](./Security%20&%20Networking/Nmap)
- [Nginx](./Security%20&%20Networking/Nginx)
- [File Sharing](./Security%20&%20Networking/FileSharing)
* 🔹 Grafana
* 🔹 Zabbix
* 🔹 Netdata
* 🔹 LibreNMS
* 🔹 ELK Stack
### 📦 Storage
- [NFS](./Storage/NFS)
### 🔐 Security & Networking
### 🧠 System & Kernel Management
- [Kernel](./System%20&%20Kernel%20Management/Kernel)
* 🔹 iptables
* 🔹 Nmap
* 🔹 tcpdump
* 🔹 hping3
* 🔹 File Sharing (SMB)
### 📦 Storage & Object Systems
* 🔹 NFS
* 🔹 MinIO
* 🔹 S5CMD
### 🧠 Linux & System Administration
* 🔹 Bash Scripting
* 🔹 System Administration
* 🔹 File Synchronization (rsync)
* 🔹 Terminal Multiplexers (screen)
### 🔁 Web Servers & Reverse Proxies
- [Nginx (Web)](./Web%20Servers%20&%20Reverse%20Proxies/Nginx)
### 🤖 Bots & Automation Tools
- [Telegram Bot](./Bots%20&%20Automation%20Tools/TelegramBot)
* 🔹 Nginx
* 🔹 Certbot
* 🔹 Nextcloud
### 📝 Miscellaneous
- [Info](./Info)
### 🔑 Password Management
* 🔹 Vaultwarden
### 🖥️ Virtualization & Dev Environments
* 🔹 Vagrant
### 🤖 Automation & Bots
* 🔹 Telegram Bot
---
## 🗂️ Documentation Structure
This repository is organized into **topic-based directories**, each containing:
* 📘 Step-by-step guides
* ⚡ Real-world configurations
* 🧪 Practical examples
* 🧾 Ready-to-use scripts
> 💡 Each section is self-contained—start anywhere based on your needs.
---
@@ -60,47 +114,73 @@ A curated collection of scripts, configuration files, and guides for managing an
```bash
git clone https://github.com/RadinPirouz/linux-documents.git
cd linux-documents
````
```
* Explore each folder for setup guides, scripts, and configuration examples.
* Follow individual README or documentation files inside each directory before running any scripts.
📌 Then:
1. Navigate to the relevant category
2. Open the `.md` documentation files
3. Follow instructions step-by-step
---
## 📌 Notes
## 🧪 Philosophy
* Tested on **Debian/Ubuntu** and **CentOS/RHEL**-based distributions.
* ⚠️ Always review and test configurations in a staging environment before applying to production.
This knowledge base is built on:
* ✅ Practical, real-world usage
* ✅ Minimal theory, maximum application
* ✅ Copy-paste friendly configs
* ✅ Modular learning approach
---
## ⚠️ Important Notes
* 🐧 Tested on:
* Debian / Ubuntu
* CentOS / RHEL
* 🚨 Always:
* Review configs before running
* Test in staging environments
* Understand before deploying to production
---
## 🤝 Contributing
Contributions are welcome! 🛠️
Want to improve this knowledge base? You're welcome!
1. Fork the repository.
2. Create a new branch:
`git checkout -b feature/YourFeature`
3. Commit your changes:
`git commit -m "Add new config for X"`
4. Push to the branch:
`git push origin feature/YourFeature`
5. Open a Pull Request 🙌
```bash
# 1. Fork the repo
# 2. Create your feature branch
git checkout -b feature/your-feature
Please ensure your code is tested and well-documented.
# 3. Commit changes
git commit -m "Add: your feature"
# 4. Push to GitHub
git push origin feature/your-feature
```
Then open a Pull Request 🙌
---
## 📬 Contact
## 📬 Contact & Support
Questions or feedback? Reach out:
* 💬 Telegram: [@RadinPirouz](https://t.me/RadinPirouz)
* 🐛 GitHub Issues: [Open an Issue](https://github.com/RadinPirouz/linux-documents/issues)
* 💬 Telegram: [https://t.me/RadinPirouz](https://t.me/RadinPirouz)
* 🐛 Issues: [https://github.com/RadinPirouz/linux-documents/issues](https://github.com/RadinPirouz/linux-documents/issues)
---
## ⭐ Support
## ⭐ Support the Project
If you find this repository useful, please give it a ⭐ and share it with others!
If this helped you:
* ⭐ Star the repository
* 🔁 Share it with others
* 🧠 Use it, improve it, contribute back

View File

@@ -0,0 +1,233 @@
# BIND9 DNS Forwarder Configuration Guide
## 1. Installing BIND9
```bash
sudo apt install bind9
```
### Explanation
BIND9 (Berkeley Internet Name Domain) is one of the most widely used DNS servers. In this setup, it will act as a **DNS forwarder**, meaning it forwards DNS queries to upstream servers instead of resolving them recursively from root servers.
---
## 2. Configuration Overview
The configuration snippet defines how BIND9 behaves as a DNS server. It is typically located in:
```
/etc/bind/named.conf.options
```
---
## 3. Detailed Configuration Breakdown
### Global Options Block
```conf
options {
directory "/var/cache/bind";
```
* `directory`: Specifies where BIND stores cache and zone files.
* `/var/cache/bind`: Default working directory for cached DNS data.
---
### Forwarders
```conf
forwarders {
192.168.1.10;
8.8.8.8;
1.1.1.1;
};
```
* Defines upstream DNS servers to which queries are forwarded.
* `192.168.1.10`: Likely an internal DNS server (e.g., corporate or local network).
* `8.8.8.8`: Public DNS server provided by Google.
* `1.1.1.1`: Public DNS server provided by Cloudflare.
**Behavior:**
* Queries that BIND cannot resolve locally are sent to these servers.
---
### DNSSEC Validation
```conf
dnssec-validation no;
```
* Disables DNSSEC (DNS Security Extensions) validation.
* DNSSEC ensures DNS responses are authentic and not tampered with.
**Why disable it?**
* Simplicity in lab or internal environments.
* Avoid issues if upstream servers or zones are misconfigured.
**Production note:**
* It is generally recommended to enable DNSSEC in secure environments.
---
### Listening Interfaces
```conf
#listen-on { any; };
# listen-on-v6 { any; };
listen-on port 53 { 127.0.0.1; };
listen-on-v6 { none; };
```
* `listen-on port 53 { 127.0.0.1; };`
* BIND listens only on the loopback interface (localhost).
* This means only the local machine can query this DNS server.
* `listen-on-v6 { none; };`
* Disables IPv6 listening.
* Commented lines:
* `#listen-on { any; };` would allow all IPv4 interfaces.
* `#listen-on-v6 { any; };` would enable IPv6 support.
**Implication:**
* This configuration is suitable for a **local DNS resolver**, not a network-wide DNS server.
---
### Forwarding Mode
```conf
forward only;
```
* Forces BIND to **only use forwarders**.
* It will not attempt full recursive resolution if forwarders fail.
**Behavior:**
* If all forwarders fail → DNS resolution fails.
---
### Query Access Control
```conf
allow-query { any; };
```
* Allows any client to query the DNS server.
**Note:**
* Safe here because the server only listens on `127.0.0.1`.
---
### Recursion Settings
```conf
recursion yes;
allow-recursion { any; };
```
* `recursion yes;`
* Enables recursive DNS resolution (required for a caching resolver).
* `allow-recursion { any; };`
* Allows all clients to use recursion.
**Important:**
* In public-facing servers, unrestricted recursion can lead to abuse (e.g., DNS amplification attacks).
* In this case, it is safe due to localhost restriction.
---
## 4. Summary of Behavior
This configuration sets up BIND9 as:
* A **local DNS forwarder**
* Listening only on **localhost (127.0.0.1)**
* Forwarding queries to:
* Internal DNS: `192.168.1.10`
* Public DNS: `8.8.8.8`, `1.1.1.1`
* Performing recursion via forwarders only
* Not using DNSSEC validation
* Not exposed to external clients
---
## 5. Typical Use Cases
* Local development environments
* Caching DNS resolver for a single machine
* Forwarding DNS queries inside containers or VMs
* Acting as a DNS proxy for internal services
---
## 6. Recommendations for Production
* Enable DNSSEC validation:
```conf
dnssec-validation auto;
```
* Restrict recursion:
```conf
allow-recursion { trusted_network; };
```
* Bind to specific internal interfaces instead of localhost if needed:
```conf
listen-on { 192.168.1.0/24; };
```
* Implement logging for observability
---
## 7. Restarting the Service
After making changes:
```bash
sudo systemctl restart bind9
```
To check status:
```bash
sudo systemctl status bind9
```
---
## 8. Testing DNS Resolution
```bash
dig google.com @127.0.0.1
```
* Confirms that the local BIND server is resolving queries correctly via forwarders.

View File

@@ -0,0 +1,293 @@
# BIND9 Zone File and SOA Configuration Guide
## 1. What is a Zone File
A **zone file** defines DNS records for a specific domain. It maps domain names to IP addresses and other resources.
In this example, we are configuring a zone for:
```
test.com
```
---
## 2. SOA (Start of Authority) Record
### Example
```conf id="soa-example"
$TTL 120
@ IN SOA test.com. admin.test.com (
1;
86400;
7200;
57600;
3600);
```
### Explanation
#### `$TTL 120`
* Default Time To Live for all records in this zone.
* Value is in seconds (120 seconds = 2 minutes).
* Controls how long DNS responses are cached.
---
### SOA Record Structure
```
@ IN SOA <primary-ns> <admin-email> (
<serial>
<refresh>
<retry>
<expire>
<minimum>
)
```
#### Fields Breakdown
* `@`
* Refers to the root of the zone (`test.com`).
* `IN`
* Internet class (standard for DNS).
* `SOA`
* Start of Authority record. Defines the authoritative source for the zone.
---
### SOA Parameters
* **Primary Nameserver**
```
test.com.
```
* The authoritative DNS server for this zone.
* Must be a fully qualified domain name (FQDN).
* **Admin Email**
```
admin.test.com
```
* Represents `admin@test.com`.
* The `@` is replaced with a dot in DNS format.
---
### Timing Parameters
* **Serial**
```
1;
```
* Version number of the zone.
* Must be incremented on every change.
* Secondary DNS servers use this to detect updates.
* **Refresh (86400 seconds = 24 hours)**
* How often secondary servers check for updates.
* **Retry (7200 seconds = 2 hours)**
* Retry interval if refresh fails.
* **Expire (57600 seconds = 16 hours)**
* Time after which secondary servers discard the zone if they cannot reach the primary.
* **Minimum TTL (3600 seconds = 1 hour)**
* Default negative caching time (NXDOMAIN responses).
---
## 3. DNS Records in the Zone
### Example Zone File
```conf id="zone-file"
@ IN NS test.com.
@ IN A 10.10.30.1
www IN CNAME docs.test.com
docs IN A 10.10.20.1
```
---
### NS Record
```conf id="ns-record"
@ IN NS test.com.
```
* Defines the authoritative nameserver for the domain.
* `test.com.` must resolve to an IP (via an A record).
---
### A Record
```conf id="a-record-root"
@ IN A 10.10.30.1
```
* Maps `test.com` → `10.10.30.1`.
---
### CNAME Record
```conf id="cname-record"
www IN CNAME docs.test.com
```
* `www.test.com` becomes an alias of `docs.test.com`.
* DNS queries for `www` will resolve to the IP of `docs`.
---
### Additional A Record
```conf id="a-record-docs"
docs IN A 10.10.20.1
```
* Maps `docs.test.com` → `10.10.20.1`.
---
## 4. The Trailing Dot in DNS
### Example
```
test.com.
```
### Explanation
* The trailing dot (`.`) indicates a **fully qualified domain name (FQDN)**.
* Without the dot, BIND appends the current zone name.
#### Example Behavior
* `docs.test.com` (no dot)
→ interpreted as `docs.test.com.test.com`
* `docs.test.com.` (with dot)
→ interpreted correctly as `docs.test.com`
**Rule:**
* Always use a trailing dot for absolute domain names in zone files.
---
## 5. Zone Configuration in BIND
### File: `/etc/bind/named.conf.local`
```conf id="named-conf-local"
zone 'test.com' IN {
type master;
file "/etc/bind/zones/test.com.zone";
};
```
### Explanation
* `zone 'test.com'`
* Declares the domain being managed.
* `type master`
* This server is the authoritative source for the zone.
* `file`
* Path to the zone file.
---
## 6. Validating the Zone File
```bash id="check-zone"
named-checkzone test.com /etc/bind/zones/test.com.zone
```
### Purpose
* Validates syntax and logic of the zone file.
* Detects:
* Missing dots
* Invalid records
* Formatting errors
---
## 7. Applying Configuration Changes
### Reconfigure BIND
```bash id="rndc-reconfig"
rndc reconfig
```
* Reloads BIND configuration files.
* Detects new or modified zones.
---
### Reload Specific Zone
```bash id="rndc-reload"
rndc reload test.com
```
* Reloads only the `test.com` zone.
* Faster and more efficient than restarting the entire service.
---
## 8. Key Operational Notes
* Always increment the **serial number** after modifying the zone.
* Use `named-checkzone` before applying changes.
* Prefer `rndc reload` over full service restart for production systems.
* Ensure proper file permissions for `/etc/bind/zones/`.
---
## 9. Summary
This setup defines:
* A **master DNS zone** for `test.com`
* Authoritative records:
* Root domain (`test.com`)
* `docs.test.com`
* Alias `www.test.com`
* Proper SOA configuration for synchronization
* DNS validation and reload workflow using BIND tools

View File

@@ -0,0 +1,839 @@
---
title: "Jitsi Production Component Guide"
subtitle: "Component-by-component explanation for production DevOps design"
author: "Prepared for production planning"
date: "2026-05-29"
---
# Jitsi Production Component Guide
## 1. Purpose of this document
This document explains the main Jitsi components, what each one does, how they communicate, what ports they use, how they scale, and how to operate them in a production environment.
The focus is a production Jitsi Meet deployment that can handle more than 1000 concurrent participants across many different meetings. This is not the same as one single 1000-person interactive room. A single huge room should normally be treated as a webinar or livestream design, while many simultaneous rooms are handled by horizontal scaling of the media layer.
## 2. Core idea: signaling and media are separate
A Jitsi system has two main traffic planes:
1. Signaling plane: users and backend components exchange control messages. This includes joining a room, creating a conference, presence, mute state, chat, permissions, lobby, and room metadata. Jitsi uses XMPP for this signaling layer.
2. Media plane: audio, video, screen share, RTP/RTCP, bandwidth estimation, packet routing, and WebRTC transport. This is handled mainly by Jitsi Videobridge.
A production deployment is successful when these two planes are treated separately:
- Prosody and Jicofo are the control/signaling brain.
- Jitsi Videobridge is the high-bandwidth media router.
- Nginx serves the web app and proxies WebSocket/BOSH traffic.
- TURN helps users behind restrictive networks.
- Jibri records or livestreams conferences.
- Jigasi connects SIP/PSTN-style audio systems.
## 3. High-level architecture
```text
Browser / Mobile App
|
| HTTPS 443, WebSocket, BOSH
v
Nginx / Jitsi Meet Web
|
| XMPP signaling
v
Prosody XMPP Server <----> Jicofo Conference Focus
^ |
| | COLIBRI / bridge control
| v
| Jitsi Videobridge Pool
| |
| | WebRTC media: UDP 10000, SRTP/RTP/RTCP
v v
Participants <---------------> JVB media routing
Optional components:
- Coturn: STUN/TURN relay for difficult NAT/firewall cases
- Jibri: recording and livestreaming worker
- Jigasi: SIP audio gateway
- Etherpad: collaborative document integration
- Monitoring: Prometheus, Grafana, logs, alerts
```
Official Jitsi documentation describes the main components as Jitsi Meet, Jitsi Videobridge, Jicofo, Jigasi, Jibri, and Prosody. It also defines Prosody as the XMPP server used for signaling, JVB as the WebRTC server that routes video streams, and Jicofo as the server-side focus component that manages media sessions and acts as a load-balancing controller between participants and videobridges. [1]
## 4. Component summary table
| Component | Main job | Traffic type | Scale method | Production note |
|---|---|---|---|---|
| Jitsi Meet Web | Browser UI and frontend application | HTTPS, WebSocket, BOSH | Horizontally with stateless web nodes or shards | Keep config consistent across nodes |
| Nginx | TLS termination, static files, reverse proxy | TCP 80/443 | Horizontal behind load balancer | Must correctly proxy WebSocket/BOSH paths |
| Prosody | XMPP signaling and authentication | XMPP, internal modules | Usually per shard; not the main media bottleneck | Protect internal XMPP ports |
| Jicofo | Conference focus, room orchestration, bridge selection | XMPP, COLIBRI control | Usually per shard; one active focus per deployment/shard | Critical control-plane component |
| Jitsi Videobridge | SFU media routing | WebRTC, UDP/RTP/SRTP | Add more JVB nodes | Main scaling point for 1000+ users |
| Coturn | STUN/TURN relay | UDP/TCP/TLS relay | Add more TURN nodes | Can consume large bandwidth |
| Jibri | Recording/livestream worker | Joins as special participant, encodes output | One worker per simultaneous recording | Heavy CPU/RAM/disk usage |
| Jigasi | SIP audio gateway | SIP/RTP/XMPP | Add instances if SIP demand grows | Audio-only SIP bridge |
| Etherpad | Shared notes/document editing | HTTP/WebSocket | Optional app scaling | Not required for video calls |
| Prometheus/Grafana/Loki | Metrics, dashboarding, logs | Metrics/log collection | Scale by observability need | Required for production operation |
## 5. Jitsi Meet Web
### What it is
Jitsi Meet Web is the user-facing web application. It is a WebRTC-compatible JavaScript application built with React and React Native concepts. In a browser deployment, users load this app from the Jitsi web server, usually through Nginx. The same product family also supports mobile applications.
### What it does
Jitsi Meet Web handles:
- Room URL and initial page load.
- User interface for camera, microphone, screen share, chat, reactions, tiles, moderator controls, lobby, settings, and device selection.
- WebRTC client logic in the browser.
- Signaling connection to Prosody through BOSH or WebSocket.
- Interaction with the Jitsi Meet External API when embedded inside another application.
- Configuration from files such as `config.js` and `interface_config.js` in package-based deployments.
### How it works in a call
1. User opens `https://meet.example.com/room-name`.
2. Nginx serves the static Jitsi Meet web application.
3. The web app reads configuration such as domain, anonymous domain, BOSH/WebSocket URLs, video constraints, prejoin behavior, lobby, and authentication settings.
4. The browser connects to Prosody for signaling.
5. The browser starts WebRTC negotiation and exchanges transport/media information through the signaling layer.
6. Actual audio/video packets go to Jitsi Videobridge, not to the web app.
### Production handling
- Keep web configuration version-controlled.
- Use the same `config.js` values across all web nodes in a shard.
- Put web nodes behind a load balancer only if the signaling paths and domain/shard routing are designed correctly.
- Do not overload the web component with recording, media routing, or TURN duties.
- For application integration, prefer JWT authentication and controlled room creation rather than public anonymous room creation.
## 6. Nginx or reverse proxy
### What it is
Nginx is normally used to serve the Jitsi Meet frontend, terminate TLS, redirect HTTP to HTTPS, and proxy special routes for signaling and bridge communication.
### What it does
Nginx handles:
- Port 80 for HTTP redirects and Let's Encrypt validation.
- Port 443 for HTTPS access.
- Static web assets.
- Reverse proxying for XMPP over WebSocket or BOSH.
- Reverse proxying for Colibri WebSocket paths used by JVB.
- Optional TLS routing or stream multiplexing when TURN over 443 is used.
### Important routes
Common important routes include:
```text
/ Jitsi Meet web application
/http-bind BOSH fallback for XMPP signaling
/xmpp-websocket XMPP over WebSocket
/colibri-ws JVB Colibri WebSocket path
```
### Production handling
- Use a trusted TLS certificate.
- Enable HTTP to HTTPS redirect.
- Forward WebSocket upgrade headers correctly.
- Do not expose internal admin or metrics endpoints through public Nginx.
- If using Cloudflare or another proxy, ensure WebRTC and WebSocket behavior is compatible with the deployment.
- Keep Nginx logs integrated with centralized logging.
## 7. Prosody
### What it is
Prosody is the XMPP server in Jitsi. It is the signaling backbone. Jitsi documentation describes Prosody as the XMPP server used for signaling. [1]
### What it does
Prosody handles:
- User XMPP sessions.
- Presence inside rooms.
- Multi-user chat rooms for conferences.
- Authentication domains.
- Guest domains.
- Lobby rooms and waiting behavior.
- JWT token verification modules.
- Internal accounts used by Jicofo, JVB, Jibri, and Jigasi.
- XMPP service discovery.
- TURN credential advertisement through XMPP when configured.
### Important virtual hosts and components
A typical deployment can include these logical domains:
```text
meet.example.com Main user-facing XMPP virtual host
auth.meet.example.com Internal authenticated domain
conference.meet.example.com MUC component for conference rooms
internal.auth.meet.example.com Internal component/auth domain
focus.meet.example.com Jicofo focus identity
recorder.meet.example.com Jibri recorder domain, if recording is enabled
guest.meet.example.com Anonymous guest domain, if guest access is enabled
```
Exact names depend on package, Docker, and authentication design.
### How Prosody works in a call
1. A browser client connects to Prosody through Nginx using XMPP over WebSocket or BOSH.
2. Prosody authenticates the user or treats the user as a guest depending on configuration.
3. The user joins a MUC room such as `room-name@conference.meet.example.com`.
4. Jicofo observes room creation and coordinates the conference.
5. JVB and Jicofo also connect through XMPP service accounts.
6. Signaling messages continue through Prosody while media flows through JVB.
### Production handling
- Do not expose Prosody's internal ports publicly.
- Restrict XMPP component/client ports to internal networks or known JVB/Jicofo/Jibri hosts.
- Use JWT authentication for app-based deployments.
- Use guest domain only when you want authenticated moderators and unauthenticated attendees.
- Monitor Prosody CPU, memory, connection count, and logs.
- Be aware that Prosody is usually not the media bottleneck, but it is a critical control-plane dependency.
## 8. Jicofo
### What it is
Jicofo means Jitsi Conference Focus. It is the conference coordinator. Official Jitsi documentation describes Jicofo as the server-side focus component used in Jitsi Meet conferences that manages media sessions and acts as a load balancer between participants and the videobridge. [1]
### What it does
Jicofo handles:
- Conference creation and lifecycle.
- Selecting a Jitsi Videobridge for a conference.
- Managing participants at the signaling level.
- Managing the relationship between conference rooms and JVBs.
- Controlling JVBs through the COLIBRI protocol.
- Coordinating Jibri sessions for recording or livestreaming.
- Moderator and feature coordination with Prosody modules.
- Bridge health and load-aware bridge selection.
### How Jicofo works in a call
1. A user joins a room through Prosody.
2. Jicofo detects or is assigned to manage the room.
3. Jicofo checks available JVBs.
4. Jicofo selects an appropriate JVB for the conference.
5. Jicofo instructs JVB to create or update the conference state.
6. Participants exchange WebRTC offers/answers and ICE data through signaling.
7. Jicofo continues coordinating participant joins/leaves, bridge state, and optional services.
### Scaling behavior
In the official scalable setup, the central Jitsi Meet instance includes Nginx, Prosody, and Jicofo, while multiple JVB nodes are attached separately. The documentation states that when a new conference starts, Jicofo picks a videobridge and schedules the conference on it. [3]
### Production handling
- Treat Jicofo as a critical control-plane service.
- Keep Jicofo logs centralized.
- Monitor bridge selection behavior and conference allocation.
- If using sharding, run separate Jicofo/Prosody stacks per shard rather than trying to make one huge control plane without design.
- Restart Jicofo carefully because existing conferences can be affected depending on deployment behavior and reconnect handling.
## 9. Jitsi Videobridge
### What it is
Jitsi Videobridge, usually called JVB, is the media router. It is a Selective Forwarding Unit, or SFU. Official Jitsi documentation describes it as a WebRTC-compatible server designed to route video streams among conference participants. [1]
### What it does
JVB handles:
- WebRTC media transport.
- ICE connectivity.
- DTLS-SRTP media security.
- RTP and RTCP packet routing.
- Audio/video forwarding.
- Screen-share forwarding.
- Simulcast and scalable video routing.
- Bandwidth estimation.
- Receiver constraints and LastN behavior.
- Packet loss recovery features such as retransmissions, depending on configuration and browser support.
- Colibri WebSocket communication.
- Media-related metrics.
### How JVB is different from an MCU
JVB normally does not mix or transcode every user's video into one combined stream. Instead, it selectively forwards streams. This is why Jitsi can scale better than a traditional MCU design, but it also means that bandwidth planning and client CPU are still important.
### How media flows
```text
User A camera/mic -> encrypted WebRTC stream -> JVB
JVB decides which participants should receive User A
JVB forwards selected encrypted media packets -> User B, User C, User D
```
JVB is not the same as TURN. TURN simply relays traffic when endpoints cannot connect normally. JVB understands the conference and makes routing decisions.
### Why JVB is the main scaling component
The official scalable setup says the first limiting factor in a single-server Jitsi installation is the Videobridge, because it handles the actual video and audio traffic, and that videobridges are easy to scale horizontally by adding as many as needed. [3]
### Production handling
- Put JVB nodes on servers with strong network capacity.
- Open UDP 10000 from the public internet to each JVB unless your deployment uses a different media design.
- Ensure the advertised public IP is correct, especially with Docker, NAT, or private cloud networks.
- Keep 25-35 percent spare capacity.
- Monitor network throughput, packet loss, CPU, memory, conferences, endpoints, and bridge stress.
- Add JVB nodes to scale concurrent meetings.
- Do not assume one JVB can safely carry a whole production deployment.
## 10. Coturn, STUN, and TURN
### What it is
Coturn is a TURN/STUN server commonly used with Jitsi. STUN helps clients discover how they are seen from the public internet. TURN relays media when direct UDP connectivity is blocked or impossible.
### What it does
TURN helps users behind:
- Symmetric NAT.
- Corporate firewalls.
- UDP-blocking networks.
- Mobile carrier restrictions.
- Networks that only allow TCP 443.
### How TURN works with Jitsi
In a simple case, users connect to JVB over UDP 10000. In a restrictive network, the browser may be unable to send media directly to the JVB. TURN then relays the traffic through an allowed port such as TCP/TLS 443 or TCP 5349.
Official Jitsi TURN documentation says peer-to-peer calls should avoid going through JVB when possible, but direct connection is not always possible, and in those cases a TURN server can relay traffic. It also notes that default TURN ports include UDP 3478 and TCP/TLS 5349, and that TCP 443 can be useful for corporate networks that allow only HTTPS-like traffic. [7]
### Production handling
- Run at least two TURN servers for production.
- Treat TURN as a bandwidth-heavy media service.
- Avoid static TURN credentials exposed to browsers for long periods; prefer ephemeral credentials when possible.
- Monitor TURN bandwidth separately from JVB bandwidth.
- Do not put TURN on the same overloaded machine as Jitsi unless traffic is tiny.
- Use valid TLS certificates for TLS TURN services.
## 11. Jibri
### What it is
Jibri means Jitsi Broadcasting Infrastructure. It is the component used for server-side recording and livestreaming.
Official Jitsi architecture documentation describes Jibri as tools for recording and/or streaming a conference by launching a Chrome instance in a virtual framebuffer and capturing/encoding the output with ffmpeg. [1]
### What it does
Jibri handles:
- Joining a Jitsi room as a special recorder participant.
- Rendering the conference in Chrome.
- Capturing the rendered output.
- Encoding with ffmpeg.
- Saving a recording file or streaming to a service such as YouTube/RTMP.
- Reporting recording state back through XMPP.
### How Jibri works in a call
```text
Moderator clicks Record or Livestream
|
v
Jicofo requests an available Jibri
|
v
Jibri joins the room as a hidden/special participant
|
v
Chrome renders the conference layout
|
v
ffmpeg captures and encodes the output
|
v
Recording file or livestream output is created
```
### Capacity rule
Jibri does not scale like JVB. One Jibri instance supports one recording or livestream session at a time. Jitsi requirements documentation states that Jibri needs one system per recording: one Jibri instance equals one meeting, and five simultaneous recordings require five Jibri instances. [5]
The Jibri repository also states that only one recording at a time is supported on a single Jibri. [6]
### Production handling
- Do not run Jibri on the main Jitsi controller node for serious production.
- Do not run Jibri on JVB nodes.
- Use a separate Jibri pool.
- Size one worker per simultaneous recording or livestream.
- Monitor CPU, memory, disk throughput, disk usage, Chrome/Chromedriver health, and ffmpeg errors.
- Store recordings on durable storage, object storage, or a post-processing pipeline.
## 12. Jigasi
### What it is
Jigasi means Jitsi Gateway to SIP. It allows regular SIP clients to join Jitsi Meet conferences. Official Jitsi architecture documentation describes Jigasi as a server-side application that allows regular SIP clients to join Jitsi Meet conferences. [1]
### What it does
Jigasi handles:
- SIP call-in or call-out integration.
- Audio bridge between SIP endpoints and a Jitsi room.
- Connection to Prosody/XMPP as a component or service account.
- SIP registration to a SIP provider or PBX.
- Optional transcription-related workflows in some deployments.
### How Jigasi works
```text
SIP phone / PBX / provider
|
| SIP/RTP
v
Jigasi
|
| XMPP signaling + media bridge behavior
v
Jitsi room
```
### Production handling
- Deploy only if you need SIP/PSTN integration.
- Isolate SIP credentials and PBX connectivity.
- Plan port ranges for SIP media if enabled.
- Monitor call setup failures, SIP registration state, audio quality, and provider errors.
- Keep SIP access restricted to expected providers or internal PBX networks.
## 13. Etherpad
### What it is
Etherpad is an optional collaborative text editing service that can be integrated with Jitsi Meet for shared notes.
### What it does
Etherpad handles:
- Shared collaborative documents.
- Meeting notes.
- Text collaboration beside the video call.
### Production handling
- Do not deploy Etherpad unless users need shared notes.
- Put it behind authentication or controlled access if documents contain sensitive content.
- Back up its database if meeting notes matter.
- Monitor it separately from Jitsi media services.
## 14. Authentication components
### Available authentication models
Common production authentication models include:
- Internal Prosody users.
- JWT/token authentication.
- LDAP authentication.
- Guest domain with authenticated moderators.
- Application-controlled room creation.
Official token authentication documentation states that Jitsi can allow only users with a valid token to create new conference rooms, while others can join from an anonymous domain after the room exists. [8]
### Recommended production model
For a custom application or platform:
```text
Use JWT auth.
Only your backend creates valid meeting tokens.
Moderators receive tokens with room permissions.
Guests join through controlled room links or guest domain.
Lobby is enabled for sensitive rooms.
Anonymous public room creation is disabled.
```
### Why JWT is usually best for production
JWT makes Jitsi part of your application security model:
- Your app decides who can create rooms.
- Your app decides who is moderator.
- Your app can restrict access by room name.
- Your app can expire tokens.
- Your app can map users, avatars, display names, and roles.
### Production handling
- Store JWT secrets safely.
- Rotate secrets carefully.
- Use short token lifetimes where possible.
- Do not expose app secrets in frontend code.
- Disable anonymous room creation.
- Enable lobby and moderator controls.
## 15. Web clients and mobile clients
### What they are
Clients are the browsers and mobile apps used by participants.
### What they do
Clients handle:
- Camera and microphone capture.
- WebRTC encryption and transport.
- Encoding local media.
- Decoding remote media.
- UI interactions.
- Sending receiver constraints to request fewer or lower-quality remote streams.
### Production handling
Client performance is part of your infrastructure capacity. Even if JVB has enough bandwidth, weak phones or old laptops may fail in large rooms.
Use:
- Start with audio muted for large rooms.
- Start with video muted for large rooms.
- Limit default resolution.
- Limit visible remote videos with LastN/receiver constraints.
- Recommend Chrome/Chromium-based browsers or tested clients.
- Monitor client-side error reports if you control the application.
## 16. Monitoring and logging components
### What they are
Production Jitsi needs observability. Without monitoring, you cannot know whether the problem is JVB bandwidth, TURN fallback, Prosody signaling, client CPU, or bad network conditions.
### Recommended stack
```text
Prometheus Metrics collection
Grafana Dashboards
Loki or ELK Logs
Node exporter Server metrics
Blackbox exporter External health checks
Alertmanager Alerts
```
### Metrics to watch
| Area | Important metrics |
|---|---|
| JVB | endpoints, conferences, packet loss, bitrate, CPU, memory, network in/out |
| Prosody | connections, auth failures, MUC behavior, module errors |
| Jicofo | bridge selection, conference allocation, errors |
| Nginx | 4xx/5xx, WebSocket upgrade failures, TLS expiry |
| TURN | relay bandwidth, allocation count, failed allocations |
| Jibri | active sessions, failed recordings, CPU, memory, disk, ffmpeg errors |
| System | CPU steal, disk usage, disk IO, network saturation, process restarts |
### Production handling
- Alert before saturation, not after users complain.
- Keep dashboards per shard and per JVB.
- Log conference IDs and bridge allocation events where possible.
- Track TURN usage percentage. A sudden increase means users cannot reach UDP directly.
- Track certificate expiry.
## 17. Ports and network paths
### Standard ports
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 80 | TCP | Nginx/Web | HTTP redirect and Let's Encrypt validation |
| 443 | TCP | Nginx/Web | HTTPS web app and WebSocket/BOSH proxy |
| 10000 | UDP | JVB | Main WebRTC media path |
| 5222 | TCP | Prosody | XMPP client/component communication, usually internal/restricted |
| 3478 | UDP | Coturn | STUN/TURN |
| 5349 | TCP/TLS | Coturn | TURN over TLS fallback |
| 20000-20050 | UDP | Jigasi | Optional SIP media range depending on deployment |
The official Docker deployment documentation lists external ports 80/tcp, 443/tcp, and 10000/udp, and also notes 20000-20050/udp for Jigasi SIP access if that component is deployed. [2]
The official Debian/Ubuntu self-hosting guide lists 80 TCP, 443 TCP, 10000 UDP, SSH, 3478 UDP, and 5349 TCP as relevant firewall ports for a typical server with coturn support. [4]
### Production rule
Open only what must be public. Keep the rest private.
```text
Public:
- 80/tcp
- 443/tcp
- 10000/udp on JVB nodes
- 3478/udp and 5349/tcp on TURN nodes, if used
Private/restricted:
- 5222/tcp Prosody
- metrics ports
- admin ports
- SSH
- Docker/Kubernetes APIs
```
## 18. How a participant joins a meeting
```text
1. User opens room URL.
2. Browser downloads Jitsi Meet Web from Nginx.
3. Browser opens XMPP signaling over WebSocket or BOSH.
4. Prosody authenticates or accepts guest access.
5. Browser joins the MUC room.
6. Jicofo coordinates the conference.
7. Jicofo selects a JVB.
8. Browser and JVB exchange WebRTC transport information through signaling.
9. Browser sends encrypted media to JVB.
10. JVB forwards selected media streams to other participants.
11. Prosody/Jicofo continue managing room state while JVB handles media.
```
## 19. Production topologies
### Single-server deployment
```text
One server:
- Nginx
- Jitsi Meet Web
- Prosody
- Jicofo
- JVB
- optional coturn
```
Good for testing, small internal teams, and proof of concept.
Not recommended for 1000+ production users.
### Split JVB deployment
```text
Controller node:
- Nginx
- Jitsi Meet Web
- Prosody
- Jicofo
Media nodes:
- JVB 1
- JVB 2
- JVB 3
- JVB N
```
This is the first serious production architecture. Jitsi's scalable setup recommends splitting the central Jitsi Meet instance from videobridges as the first scaling step. [3]
### Sharded deployment
```text
Shard A:
- Web
- Prosody
- Jicofo
- JVB pool
Shard B:
- Web
- Prosody
- Jicofo
- JVB pool
Application router:
- routes rooms/users to a shard
```
Good for high availability and large scale.
### Regional deployment
```text
EU region:
- EU web/control shard
- EU JVBs
US region:
- US web/control shard
- US JVBs
Asia region:
- Asia web/control shard
- Asia JVBs
```
Good when users are globally distributed and latency matters.
## 20. Scaling guide by component
| Component | Bottleneck | How to scale |
|---|---|---|
| Web/Nginx | TLS, static traffic, WebSocket proxying | Add web nodes or shards |
| Prosody | XMPP connections, modules, room state | Usually scale by shard, not by simply adding random replicas |
| Jicofo | Conference orchestration, bridge control | Usually scale by shard; design active focus carefully |
| JVB | Bandwidth, packet forwarding, CPU | Add more JVB nodes |
| TURN | Relay bandwidth | Add more TURN nodes, use geo-distribution |
| Jibri | Encoding CPU/RAM/disk | One Jibri per simultaneous recording |
| Jigasi | SIP calls and audio bridge load | Add more Jigasi instances if SIP use grows |
| Monitoring | Metrics/log volume | Scale storage and retention separately |
## 21. Production best practices
### Infrastructure
- Use separate controller and JVB nodes.
- Use separate TURN nodes for serious production.
- Use separate Jibri nodes for recording.
- Use configuration management such as Ansible, Terraform, or GitOps.
- Use pinned versions, not random latest images in production.
- Keep staging and production separate.
### Security
- Use trusted TLS certificates.
- Disable anonymous room creation.
- Prefer JWT for application integration.
- Enable lobby for guest access.
- Restrict Prosody, metrics, SSH, and admin ports.
- Rotate secrets.
- Do not expose Docker socket or internal service ports.
### Performance
- Keep UDP 10000 working for JVB nodes.
- Use TURN only as fallback, not as the normal path for everyone.
- Limit default video quality.
- Limit LastN/visible remote video count in large rooms.
- Start audio/video muted for large public rooms.
- Keep enough JVB spare capacity.
### Operations
- Monitor before production launch.
- Load test with realistic room patterns.
- Keep rollback packages or images ready.
- Upgrade JVB nodes one by one.
- Drain traffic before restarting busy components when possible.
- Keep clear incident runbooks.
## 22. Failure modes and where to look
| Symptom | Likely component | What to check |
|---|---|---|
| Page does not load | Nginx, DNS, TLS | DNS, certificate, Nginx logs, firewall 443 |
| Users can join but no audio/video | JVB, firewall, NAT | UDP 10000, JVB advertised IP, browser ICE candidates |
| Two users work but three or more fail | JVB path | JVB public IP, UDP 10000, NAT, Docker advertise IP |
| Users behind corporate networks fail | TURN | coturn health, 443/5349, credentials, certificates |
| Rooms are not created | Prosody/Jicofo | XMPP logs, auth config, Jicofo service account |
| JWT users cannot join | Prosody auth | app_id, app_secret, token claims, room claim, time skew |
| Recording fails | Jibri | Chrome, Chromedriver, ffmpeg, ALSA loopback, disk, Jibri account |
| SIP call-in fails | Jigasi | SIP credentials, PBX routing, firewall, media range |
| High packet loss | JVB/network | NIC saturation, cloud network, UDP drops, region distance |
| Random disconnects | Client/network/JVB | WebSocket stability, JVB stress, browser logs |
## 23. Recommended architecture for 1000+ concurrent users
For more than 1000 concurrent participants across many calls, a practical starting design is:
```text
2 Jitsi control shards
8-10 JVB nodes total
2 TURN nodes
1 monitoring/logging stack
Jibri pool only if recording/livestreaming is required
```
Each shard:
```text
1 controller node:
- Nginx
- Jitsi Meet Web
- Prosody
- Jicofo
4-5 JVB nodes:
- Jitsi Videobridge only
```
Shared or per-region:
```text
2 TURN nodes
Monitoring/logging
Optional Jibri worker pool
Optional Jigasi worker pool
```
This design allows:
- Horizontal media scaling by adding JVBs.
- Failure isolation by shard.
- Easier upgrades.
- Better observability.
- Controlled authentication and room routing.
## 24. Component responsibility map
```text
User interface problem -> Jitsi Meet Web / browser
TLS or static file problem -> Nginx / reverse proxy
Login or room auth problem -> Prosody / JWT / LDAP
Room orchestration problem -> Jicofo
Audio/video routing problem -> JVB
Strict firewall problem -> TURN
Recording/livestream problem -> Jibri
SIP/PSTN problem -> Jigasi
Scale problem -> Usually JVB, then sharding
```
## 25. Final production checklist
Before launch:
- Domain and DNS are correct.
- TLS certificate is trusted and auto-renewing.
- UDP 10000 reaches every JVB.
- JVB advertised IPs are correct.
- Prosody internal ports are not publicly exposed.
- JWT or chosen authentication is working.
- Guests cannot create rooms unless intentionally allowed.
- TURN works for restricted networks.
- Monitoring and alerts are active.
- Load test has been run with realistic room distribution.
- Jibri is deployed only if recording/livestreaming is needed.
- Backups exist for configuration and secrets.
- Upgrade and rollback procedure exists.
## References
[1] Jitsi Meet Handbook, Architecture: https://jitsi.github.io/handbook/docs/architecture/
[2] Jitsi Meet Handbook, Self-Hosting Guide - Docker: https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-docker/
[3] Jitsi Meet Handbook, DevOps Guide - Scalable setup: https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-scalable/
[4] Jitsi Meet Handbook, Self-Hosting Guide - Debian/Ubuntu server: https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-quickstart/
[5] Jitsi Meet Handbook, Requirements - Recording/Jibri: https://jitsi.github.io/handbook/docs/devops-guide/devops-guide-requirements/
[6] Jitsi Jibri GitHub repository: https://github.com/jitsi/jibri
[7] Jitsi Meet Handbook, Setting up TURN: https://jitsi.github.io/handbook/docs/devops-guide/turn/
[8] Jitsi Meet Handbook, Token Authentication: https://jitsi.github.io/handbook/docs/devops-guide/token-authentication/

View File

@@ -0,0 +1,179 @@
# Jitsi Docker Plugins and Third-Party Software Catalog
This is a practical DevOps checklist for a self-hosted Jitsi Meet deployment running with Docker Compose. The official Docker stack is based around `web`, `prosody`, `jicofo`, and `jvb`, with optional Compose overlays for services like `jibri`, `jigasi`, `etherpad`, `whiteboard`, `transcriber`, `grafana`, `prometheus`, `rtcstats`, and log analysis. ([GitHub][1])
## 1. Core Jitsi Docker Components
| Component | Purpose | Docker Service |
| ----------------- | --------------------------------------------------- | -------------- |
| Jitsi Meet Web | Frontend web UI, Nginx, static assets, external API | `web` |
| Prosody | XMPP server used for signaling, auth, room control | `prosody` |
| Jicofo | Conference focus, room/session orchestration | `jicofo` |
| Jitsi Videobridge | SFU media bridge for audio/video routing | `jvb` |
| Jibri | Recording and live streaming worker | `jibri` |
| Jigasi | SIP gateway and dial-in/dial-out support | `jigasi` |
| Jitsi Transcriber | Speech-to-text transcription support | `transcriber` |
| JaaS Components | Hosted Jigasi-style components from 8x8/JaaS | optional |
## 2. Official Optional Docker Overlays
| Overlay File | Feature | Use Case |
| ------------------ | ----------------------- | --------------------------------------- |
| `jibri.yml` | Recording and streaming | Record meetings, stream to YouTube/RTMP |
| `jigasi.yml` | SIP gateway | Connect SIP PBX, PSTN, VoIP users |
| `etherpad.yml` | Shared documents | Collaborative meeting notes |
| `whiteboard.yml` | Excalidraw whiteboard | Collaborative drawing/whiteboard |
| `transcriber.yml` | Transcription | Meeting captions/transcripts |
| `grafana.yml` | Grafana dashboard | Metrics visualization |
| `prometheus.yml` | Metrics scraping | Monitoring Jitsi services |
| `rtcstats.yml` | WebRTC analytics | Client-side WebRTC quality data |
| `log-analyser.yml` | Log analysis | Loki/OpenTelemetry/Grafana log view |
The official Docker guide shows these overlays being started with commands like `docker compose -f docker-compose.yml -f jibri.yml up -d`, and similar combinations for Jigasi, Etherpad, whiteboard, transcriber, Grafana, and log analysis. ([Jitsi][2])
## 3. Reverse Proxy and TLS Software
| Software | Purpose | Docker-Friendly | Notes |
| ------------- | -------------------------------------------- | --------------- | --------------------------------------------------------------- |
| Nginx | Reverse proxy, TLS termination, HTTP routing | Yes | Common production choice |
| Traefik | Dynamic reverse proxy for Docker labels | Yes | Good for multi-service Docker hosts |
Jitsi Docker requires a real `PUBLIC_URL` for production deployments, and the official `.env` includes Lets Encrypt-related settings such as domain, email, staging mode, and ACME server selection. ([Jitsi][2])
## 4. NAT, STUN, and TURN
| Software | Purpose | When to Use |
| ------------------ | ---------------------- | ------------------------------------------------------- |
| coturn | TURN/STUN relay server | Required for reliable calls behind strict NAT/firewalls |
| Google STUN | Public STUN service | Basic NAT discovery, not enough for all networks |
| Custom STUN | Your own STUN endpoint | Controlled infrastructure |
| TURN over TCP 443 | Firewall bypass | Corporate networks that block UDP |
| TURN over TLS 5349 | Secure TURN relay | Better for enterprise deployments |
Jitsi can use a TURN server for cases where direct peer-to-peer connectivity fails; the official TURN guide discusses coturn, XMPP-delivered TURN credentials, UDP 3478, TCP/TLS 5349, and using port 443 for restrictive networks. ([Jitsi][3])
## 5. Authentication and SSO
| Tool | Integration Type | Notes |
| ------------------------------ | -------------------------------- | ------------------------------------------------- |
| Internal Prosody Auth | Username/password inside Prosody | Simple small deployment |
| JWT Auth | Token-based authentication | Best for custom apps and portals |
| LDAP | Directory authentication | Enterprise user directories |
| Active Directory | LDAP/SASL integration | Corporate auth |
| OpenLDAP | LDAP backend | Self-hosted directory |
| Keycloak | OIDC/SAML identity provider | Usually integrated through JWT adapters |
| authentik | OIDC/SAML identity provider | Good self-hosted SSO option |
| Authelia | SSO and access control | Usually used in front of apps |
| Dex | Lightweight OIDC provider | Kubernetes-friendly |
| OAuth2 Proxy | Auth gateway | Can protect Jitsi landing pages or custom portals |
| jitsi-OIDC-adapter | OIDC to Jitsi JWT bridge | Community integration |
| jitsi-OIDC-SAML-adapter | OIDC/SAML to Jitsi JWT bridge | Community integration |
| nordeck/jitsi-keycloak-adapter | Keycloak adapter | Dockerized Jitsi integration |
The official Docker `.env` supports `AUTH_TYPE=internal`, `jwt`, `ldap`, or `matrix`, and includes JWT and LDAP configuration fields. Jitsis JWT auth plugin verifies client connections using JWT and supports shared-secret or public-key validation. ([GitHub][4])
## 6. SIP, VoIP, and Telephony
| Software | Purpose | Works With |
| --------------------------- | ---------------------- | ------------------------ |
| Jigasi | Jitsi SIP gateway | SIP providers, PBX, PSTN |
| Asterisk | PBX server | Jigasi |
| FreePBX | Asterisk management UI | Jigasi |
| FreeSWITCH | PBX/media server | Jigasi |
| Kamailio | SIP proxy | Large SIP routing |
| OpenSIPS | SIP proxy | Large SIP routing |
| SIP provider account | External calling | Jigasi |
| Twilio Elastic SIP Trunking | SIP trunk | Jigasi/Asterisk |
| Telnyx SIP | SIP trunk | Jigasi/Asterisk |
| VoIP.ms | SIP trunk | Jigasi/Asterisk |
| SignalWire | SIP/telephony | Jigasi/Asterisk |
Jitsi Dockers `.env` includes Jigasi SIP settings such as SIP URI, SIP password, SIP server, SIP port, and SIP transport. ([GitHub][4])
## 7. Recording, Streaming, and Storage
| Software | Purpose | Notes |
| ---------------------- | ---------------------------- | ------------------------------------- |
| Jibri | Recording and streaming | Official Jitsi recording component |
| FFmpeg | Media processing | Used in recording/streaming workflows |
| Google Chrome/Chromium | Headless capture for Jibri | Required by Jibri |
| ALSA/PulseAudio | Audio capture stack | Used by Jibri |
| YouTube Live | RTMP streaming target | Jibri can stream to RTMP |
| Twitch | RTMP streaming target | Possible with stream key |
| Facebook Live | RTMP streaming target | Possible with stream key |
| Nginx RTMP Module | Self-hosted RTMP endpoint | Internal streaming pipeline |
| Owncast | Self-hosted live streaming | RTMP target |
| Restream | Multi-platform streaming | RTMP target |
| MinIO | S3-compatible object storage | Store recordings |
| AWS S3 | Object storage | Store recordings |
| Wasabi | S3-compatible storage | Store recordings |
| Backblaze B2 | Object storage | Store recordings |
| rclone | Upload/sync recordings | Post-recording automation |
## 8. Collaboration Add-ons
| Software | Purpose | Integration Style |
| ---------------------- | ---------------------------- | ------------------------------ |
| Etherpad | Shared document editing | Official Docker overlay |
| Excalidraw | Whiteboard | Official whiteboard overlay |
| Nextcloud | Files, calendar, office docs | External integration |
| OnlyOffice | Document editing | With Nextcloud or standalone |
| Collabora Online | Document editing | With Nextcloud |
The official Docker setup has direct support for Etherpad document sharing and an Excalidraw-based virtual collaborative whiteboard. ([Jitsi][2])
## 9. Chat and Team Platform Integrations
| Platform | Integration Method | Notes |
| -------------------------- | ----------------------------------------- | ----------------------------------- |
| Matrix / Element | Matrix auth or meeting integration | Jitsi can be used from Matrix rooms |
| Mattermost | Jitsi plugin/integration | Team chat video calls |
| Rocket.Chat | Jitsi integration | Team chat video calls |
| Nextcloud Talk / Nextcloud | External meeting links or app integration | Good self-hosted suite |
| Moodle | Jitsi plugin | Education/LMS |
## 10. Web and App Embedding
| Tool | Purpose | Notes |
| ----------------- | ------------------------------- | ------------------------------ |
| Jitsi IFrame API | Embed meetings in websites/apps | Official supported method |
| External API JS | Browser-side meeting control | Loaded from `/external_api.js` |
| lib-jitsi-meet | Low-level JS library | Build custom video apps |
The official IFrame API lets you embed Jitsi Meet into your own application, and the event API allows listening to meeting events through `JitsiMeetExternalAPI`. ([Jitsi][5])
## 11. Prosody Plugins and XMPP Modules
| Plugin / Module Type | Purpose |
| ---------------------------- | ------------------------------- |
| Custom Prosody modules | Add custom XMPP behavior |
| JWT auth module | Token authentication |
| LDAP/SASL auth module | Enterprise directory auth |
| MUC modules | Room behavior customization |
| Lobby modules | Guest waiting room behavior |
| MUC size module | Room participant metrics |
| MUC domain mapper | Multi-domain setups |
| Token moderation | Moderator control from JWT |
| Room metadata modules | Store extra room info |
| Reservation modules | Room booking or room validation |
| External services module | TURN credential delivery |
| Rate limiting modules | Abuse protection |
| Anti-spam modules | Public server protection |
| Webhook-style custom module | Send events to external backend |
| Custom access control module | Per-room or per-user policy |
For Docker deployments, custom Prosody plugins are usually mounted into the Prosody config/plugin path and enabled through Prosody/Jitsi configuration. The official Docker guide creates a `prosody/prosody-plugins-custom` directory for custom plugin use. ([Jitsi][2])
## 12. Monitoring and Observability
| Software | Purpose | Notes |
| ------------------- | ------------------------------- | -------------------------------------- |
| Prometheus | Metrics collection | Official Docker overlay exists |
| Grafana | Dashboards | Official Docker overlay exists |
| Jitsi Meet Exporter | Prometheus exporter | Exposes Jitsi metrics |
| Loki | Log aggregation | Used in log analyzer stack |
| OpenTelemetry | Telemetry/log pipeline | Used in log analyzer stack |
The Jitsi Docker repository includes `prometheus.yml`, `grafana.yml`, `rtcstats.yml`, and `log-analyser.yml`; the log analyser uses Grafana Loki and OpenTelemetry for log management and analysis. ([GitHub][1])

View File

@@ -0,0 +1,527 @@
# Replicating Jitsi Videobridge in Docker
## 1. What JVB replication means
In Jitsi, the component that normally becomes the bottleneck is **Jitsi Videobridge**, also called **JVB**. JVB is the SFU/media router that handles RTP audio and video traffic. The official Jitsi scalable setup is based on **one Jitsi Meet core node** running web, Prosody, and Jicofo, plus **multiple JVB nodes** handling media traffic. Jitsis own guide says the videobridge is usually the first limiting factor and that bridges can be scaled horizontally by adding more of them. ([Jitsi][1])
Important: this is not classic load balancing with HAProxy or Nginx in front of UDP media. JVBs register into the bridge pool, and **Jicofo selects a bridge when a new conference starts**. ([Jitsi][1])
## 2. Target architecture
```text
Users
|
80/443 TCP
|
+----------------+
| Jitsi Core |
| web/nginx |
| prosody |
| jicofo |
+----------------+
| 5222 TCP
private / firewall-restricted XMPP
|
+----------------+----------------+
| | |
+---------------+ +---------------+ +---------------+
| JVB node 1 | | JVB node 2 | | JVB node 3 |
| Docker jvb | | Docker jvb | | Docker jvb |
| 10000/udp | | 10000/udp | | 10000/udp |
+---------------+ +---------------+ +---------------+
| | |
+------ media RTP to clients -----+
```
Jitsis scalable guide shows this same pattern: one central Jitsi Meet server with nginx, Prosody, and Jicofo, plus multiple videobridges connected over XMPP. It also documents `80/tcp`, `443/tcp`, `5222/tcp`, and `10000/udp` as the key ports in this architecture. ([Jitsi][1])
## 3. What replication improves
JVB replication improves:
* Total concurrent meetings
* Total concurrent users
* Media CPU capacity
* Network bandwidth capacity
* Fault isolation between conferences
* Easier horizontal scaling by adding more bridge hosts
It does not automatically make one very large conference split across all bridges. By default, Jicofo schedules a conference onto a selected bridge. If you need one conference distributed across multiple bridges, that becomes an **Octo / cascading JVB** design and should be treated as a separate advanced architecture.
## 4. Recommended deployment model
For production, use:
```text
1 core Jitsi node:
web
prosody
jicofo
optionally one local jvb
N remote JVB nodes:
jvb only
```
Running many JVB containers on the same host is possible for testing, but it is not the best production model because each JVB needs UDP media ports, CPU, memory, kernel UDP buffers, and public reachability. The official sizing guide also notes that videobridges carry more load than the main Jitsi Meet server and suggests larger CPU allocation for JVB hosts. ([Jitsi][1])
## 5. Required ports
### Core Jitsi node
| Port | Direction | Purpose |
| ---------- | -----------------------------: | ---------------------------------- |
| `80/tcp` | public inbound | HTTP redirect or ACME challenge |
| `443/tcp` | public inbound | Jitsi web UI and WebSocket traffic |
| `5222/tcp` | private inbound from JVB nodes | Prosody XMPP client connection |
| `5347/tcp` | internal only | XMPP component connections |
| `5280/tcp` | internal or reverse-proxied | BOSH/WebSocket depending on setup |
The Docker self-hosting guide lists `80/tcp`, `443/tcp`, and `10000/udp` as the main external ports, and the scalable guide says `5222/tcp` should be open only to videobridges. ([Jitsi][2])
### Each JVB node
| Port | Direction | Purpose |
| ------------------------------- | ---------------------: | --------------------------------------- |
| `10000/udp` | public inbound | WebRTC RTP media |
| `8080/tcp` | localhost/private only | Colibri REST API |
| `443/tcp` or reverse proxy path | optional | Colibri WebSocket if exposed separately |
The Docker guide defines `JVB_PORT` as the UDP media port, defaulting to `10000`, and `JVB_COLIBRI_PORT` as the local Colibri API port, defaulting to `8080`. ([Jitsi][2])
## 6. Core node Docker configuration
Start with the normal `docker-jitsi-meet` stack.
```bash
git clone https://github.com/jitsi/docker-jitsi-meet
cd docker-jitsi-meet
cp env.example .env
./gen-passwords.sh
mkdir -p ~/.jitsi-meet-cfg/{web,transcripts,prosody/config,prosody/prosody-plugins-custom,jicofo,jvb,jigasi,jibri}
```
The Docker guide recommends copying `env.example`, generating strong internal passwords with `./gen-passwords.sh`, and creating the required config directories before starting the stack. ([Jitsi][2])
Example core `.env`:
```env
CONFIG=~/.jitsi-meet-cfg
TZ=UTC
PUBLIC_URL=https://meet.example.com
HTTP_PORT=80
HTTPS_PORT=443
ENABLE_LETSENCRYPT=1
LETSENCRYPT_DOMAIN=meet.example.com
LETSENCRYPT_EMAIL=admin@example.com
ENABLE_HTTP_REDIRECT=1
JVB_AUTH_USER=jvb
JVB_AUTH_PASSWORD=use_the_generated_password_from_core_env
JVB_BREWERY_MUC=jvbbrewery
XMPP_DOMAIN=meet.jitsi
XMPP_AUTH_DOMAIN=auth.meet.jitsi
XMPP_INTERNAL_MUC_DOMAIN=internal-muc.meet.jitsi
XMPP_MUC_DOMAIN=muc.meet.jitsi
XMPP_SERVER=xmpp.meet.jitsi
XMPP_PORT=5222
```
Expose Prosody `5222/tcp` from the core node to the JVB nodes. Do not expose it to the entire Internet.
Example `docker-compose.override.yml` on the core node:
```yaml
services:
prosody:
ports:
- "10.0.0.10:5222:5222"
```
Use a private network address if possible. If your JVBs are on separate public servers, restrict this port with firewall rules.
Example firewall logic:
```bash
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow from JVB1_PUBLIC_OR_PRIVATE_IP to any port 5222 proto tcp
ufw allow from JVB2_PUBLIC_OR_PRIVATE_IP to any port 5222 proto tcp
ufw deny 5222/tcp
```
Start the core stack:
```bash
docker compose up -d
```
## 7. Remote JVB node Docker Compose
On every remote JVB server, run only the `jvb` container.
Directory layout:
```bash
mkdir -p /opt/jitsi-jvb
cd /opt/jitsi-jvb
mkdir -p ~/.jitsi-meet-cfg/jvb
```
Create `.env`:
```env
CONFIG=~/.jitsi-meet-cfg
TZ=UTC
JITSI_IMAGE_VERSION=stable
PUBLIC_URL=https://meet.example.com
XMPP_SERVER=10.0.0.10
XMPP_PORT=5222
XMPP_DOMAIN=meet.jitsi
XMPP_AUTH_DOMAIN=auth.meet.jitsi
XMPP_INTERNAL_MUC_DOMAIN=internal-muc.meet.jitsi
JVB_AUTH_USER=jvb
JVB_AUTH_PASSWORD=the_same_JVB_AUTH_PASSWORD_from_core
JVB_BREWERY_MUC=jvbbrewery
JVB_PORT=10000
JVB_ADVERTISE_IPS=JVB_PUBLIC_IP
JVB_MUC_NICKNAME=jvb-node-01
JVB_INSTANCE_ID=jvb-node-01
COLIBRI_REST_ENABLED=true
SHUTDOWN_REST_ENABLED=true
VIDEOBRIDGE_MAX_MEMORY=3072m
```
`JVB_ADVERTISE_IPS` is critical. The Docker guide says it controls which IPs and ports the bridge advertises for WebRTC media, and it must be set correctly when behind NAT or on the public Internet. If it is wrong, calls can fail when more than two users join. ([Jitsi][2])
Create `docker-compose.yml`:
```yaml
services:
jvb:
image: jitsi/jvb:${JITSI_IMAGE_VERSION:-stable}
restart: unless-stopped
ports:
- "${JVB_PORT:-10000}:${JVB_PORT:-10000}/udp"
- "127.0.0.1:8080:8080"
volumes:
- ${CONFIG}/jvb:/config
environment:
- TZ
- PUBLIC_URL
- XMPP_SERVER
- XMPP_PORT
- XMPP_DOMAIN
- XMPP_AUTH_DOMAIN
- XMPP_INTERNAL_MUC_DOMAIN
- JVB_AUTH_USER
- JVB_AUTH_PASSWORD
- JVB_BREWERY_MUC
- JVB_PORT
- JVB_ADVERTISE_IPS
- JVB_MUC_NICKNAME
- JVB_INSTANCE_ID
- COLIBRI_REST_ENABLED
- SHUTDOWN_REST_ENABLED
- VIDEOBRIDGE_MAX_MEMORY
```
Start the remote JVB:
```bash
docker compose up -d
```
Check logs:
```bash
docker compose logs -f jvb
```
On the core node:
```bash
docker compose logs -f prosody
docker compose logs -f jicofo
```
You should see the new bridge join the brewery MUC, and Jicofo should detect it. The scalable setup guide says you can verify bridge connection in Prosody and Jicofo logs, and that Jicofo picks a videobridge when a new conference starts. ([Jitsi][1])
## 8. Same-host multi-JVB setup
Use this only for testing or small deployments.
Problem: two containers cannot both bind host port `10000/udp`.
Example:
```yaml
services:
jvb1:
image: jitsi/jvb:${JITSI_IMAGE_VERSION:-stable}
restart: unless-stopped
ports:
- "10000:10000/udp"
- "127.0.0.1:8081:8080"
environment:
- JVB_PORT=10000
- JVB_ADVERTISE_IPS=PUBLIC_IP#10000
- JVB_MUC_NICKNAME=jvb1
- JVB_INSTANCE_ID=jvb1
- JVB_AUTH_USER=jvb
- JVB_AUTH_PASSWORD=${JVB_AUTH_PASSWORD}
- JVB_BREWERY_MUC=jvbbrewery
- XMPP_SERVER=xmpp.meet.jitsi
- XMPP_DOMAIN=meet.jitsi
- XMPP_AUTH_DOMAIN=auth.meet.jitsi
- XMPP_INTERNAL_MUC_DOMAIN=internal-muc.meet.jitsi
jvb2:
image: jitsi/jvb:${JITSI_IMAGE_VERSION:-stable}
restart: unless-stopped
ports:
- "10001:10000/udp"
- "127.0.0.1:8082:8080"
environment:
- JVB_PORT=10000
- JVB_ADVERTISE_IPS=PUBLIC_IP#10001
- JVB_MUC_NICKNAME=jvb2
- JVB_INSTANCE_ID=jvb2
- JVB_AUTH_USER=jvb
- JVB_AUTH_PASSWORD=${JVB_AUTH_PASSWORD}
- JVB_BREWERY_MUC=jvbbrewery
- XMPP_SERVER=xmpp.meet.jitsi
- XMPP_DOMAIN=meet.jitsi
- XMPP_AUTH_DOMAIN=auth.meet.jitsi
- XMPP_INTERNAL_MUC_DOMAIN=internal-muc.meet.jitsi
```
The `#port` syntax is used when the advertised external port differs from the internal JVB port. The Docker guide documents this pattern for `JVB_ADVERTISE_IPS`. ([Jitsi][2])
## 9. Colibri WebSocket considerations
Modern Jitsi deployments commonly use Colibri WebSockets for bridge-channel communication. Jitsi Videobridge documents that WebSocket URLs include a `server-id` path such as:
```text
/colibri-ws/server-id/conf-id/endpoint-id
```
When multiple bridges are behind one HTTP proxy, the proxy must route each `server-id` to the correct JVB. Jitsis Videobridge WebSocket documentation explicitly shows separate proxy routes for `jvb1` and `jvb2`. ([GitHub][3])
For a simple Docker deployment, the easiest options are:
1. Keep JVBs directly reachable by UDP and avoid custom WebSocket routing unless needed.
2. If using Colibri WebSocket through the main domain, assign unique bridge IDs and configure reverse-proxy routing.
3. If using remote JVBs with their own public hostnames, make each JVB advertise the correct public WebSocket domain.
For production behind a reverse proxy, review these variables:
```env
ENABLE_COLIBRI_WEBSOCKET=1
COLIBRI_WEBSOCKET_REGEX=
COLIBRI_WEBSOCKET_JVB_LOOKUP_NAME=
DISABLE_COLIBRI_WEBSOCKET_JVB_LOOKUP=
JVB_WS_DOMAIN=
JVB_WS_SERVER_ID=
JVB_WS_TLS=
```
The Docker guide states that `COLIBRI_WEBSOCKET_REGEX` controls proxy matching to JVBs and recommends overriding it in production with values matching the possible JVB IP ranges. ([Jitsi][2])
## 10. Health checks
### Check JVB container
```bash
docker compose ps
docker compose logs --tail=200 jvb
```
### Check UDP listening
```bash
ss -lunp | grep 10000
```
### Check Colibri REST locally
```bash
curl -s http://127.0.0.1:8080/colibri/stats | jq
```
Useful fields:
```text
conferences
participants
endpoints
bit_rate_download
bit_rate_upload
packet_rate_download
packet_rate_upload
stress_level
version[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
```
### Check Jicofo sees bridges
On the core node:
```bash
docker compose logs jicofo | grep -i bridge
```
Expected idea:
```text
Added new videobridge
Bridge selected for conference
```
### Check Prosody connection
```bash
docker compose logs prosody | grep -i jvb
```
## 11. Monitoring
Recommended stack:
```text
Prometheus
Grafana
Loki or centralized Docker logs
Node Exporter
cAdvisor
Blackbox Exporter
```
Monitor at least:
| Metric | Why it matters |
| ------------------------- | -------------------------------- |
| CPU usage per JVB | SFU forwarding is CPU-sensitive |
| NIC bandwidth | Media traffic is bandwidth-heavy |
| UDP packet drops | Causes audio/video instability |
| JVB stress level | Used for bridge load decisions |
| Conferences per JVB | Confirms distribution |
| Participants per JVB | Capacity planning |
| Jicofo bridge count | Detects missing bridges |
| Prosody 5222 availability | Remote JVB registration |
| Packet loss / jitter | User quality indicator |
## 12. Autoscaling approach
Basic autoscaling logic:
```text
if average JVB stress_level > 0.75 for 5 minutes:
add one JVB node
if average JVB stress_level < 0.25 for 30 minutes:
drain one JVB node
wait until conferences = 0
remove node
```
Safe scale-down process:
```bash
curl -X POST http://127.0.0.1:8080/colibri/shutdown
```
Then wait until:
```bash
curl -s http://127.0.0.1:8080/colibri/stats | jq '.conferences'
```
returns:
```text
0
```
Do not kill a busy JVB unless you accept dropping active conferences.
## 13. Common failure modes
### Calls work with two users but fail with three or more
Most likely cause:
```text
JVB_ADVERTISE_IPS is wrong
UDP 10000 is blocked
NAT is not forwarding UDP correctly
```
The Docker guide specifically warns([Jitsi][2])IP advertisement can cause calls to crash when more than two users join. citeturn115407view3
### Remote JVB never appears in Jicofo
Check:
```text
JVB_AUTH_PASSWORD mismatch
Prosody 5222 blocked
Wrong XMPP_SERVER
Wrong XMPP_AUTH_DOMAIN
Wrong XMPP_INTERNAL_MUC_DOMAIN
Wrong JVB_BREWERY_MUC
Firewall allows only public interface but JVB uses private route
```
### Multiple JVBs appear, but traffic only goes to one
Possible causes:
```text
Very few conferences
Bridge stress threshold not reached
Jicofo bridge selection strategy
One bridge has better region/locality
Other bridges are unhealthy
```
Remember: distribution is usually per conference, not per packet.
### Browser console shows Colibri WebSocket errors
Check:
```text
ENABLE_COLIBRI_WEBSOCKET
COLIBRI_WEBSOCKET_REGEX
JVB_WS_SERVER_ID
JVB_WS_DOMAIN
Reverse proxy websocket Upgrade headers
Routing /colibri-ws/<server-id>/ to the correct JVB
```
Jitsi Videobridges WebSocket documentati([GitHub][3]) support WebSocket and route the `server-id` path to the correct bridge. citeturn206078view0