diff --git a/README.md b/README.md index 6c86428..c2786f0 100644 --- a/README.md +++ b/README.md @@ -7,11 +7,11 @@

- + - + All releases @@ -25,12 +25,12 @@ of your web applications. RoadRunner includes [PSR-7](https://www.php-fig.org/psr/psr-7), [PSR-17](https://www.php-fig.org/psr/psr-17) compatible HTTP and `HTTP(S)/2/3/fCGI` servers and can be used to replace classic Nginx+FPM setups with much greater performance and flexibility. `HTTP(S)/2/3/fCGI` servers as just one of its many available plugins, but its capabilities extend far beyond: -- Queue drivers: RabbitMQ, Kafka, SQS, Beanstalk, NATS, In-Memory. +- Queue drivers: RabbitMQ, Kafka, SQS, Beanstalk, NATS, Google Pub/Sub, BoltDB, In-Memory, and [NSQ](queues/nsq.md). - KV drivers: Redis, Memcached, BoltDB, In-Memory. -- OpenTelemetry protocol support (`gRPC`, `http`, `jaeger`). +- OTLP trace export over gRPC or HTTP. - Workflows engine via [Temporal](https://temporal.io) -- `gRPC` server. For increased speed, the `protobuf` extension can be used. -- `HTTP(S)/2/3` and `fCGI` server features **automatic TLS management**, **103 Early Hints** support and middleware like: Static, Headers, gzip, prometheus (metrics), send (x-sendfile), OTEL, proxy_ip_parser, etc. +- [gRPC](grpc/grpc.md) server with reflection and a [protobuf registry](grpc/protoreg.md). +- `HTTP(S)/2/3` and `fCGI` servers with automatic TLS management, 103 Early Hints, static files, headers, gzip, [Zstd](http/zstd.md), [rate limiting](http/rate-limiter.md), metrics, X-Sendfile, tracing, and proxy IP parsing. - Embedded distribute lock plugin which manages access to shared resources. - Metrics server (you might easily expose your own). - WebSockets and Broadcast via [Centrifugo](https://centrifugal.dev) server. @@ -39,6 +39,8 @@ RoadRunner includes [PSR-7](https://www.php-fig.org/psr/psr-7), [PSR-17](https:/ - Compatible with Windows, WSL2, FreeBSD, GNU/Linux, etc. - And more πŸ˜‰ +These pages describe RoadRunner v3. See the [v3 release notes](releases/v3-0-0.md) and [upgrade steps from v2025](intro/v3-migration.md). + If you have a feature request in mind, you can check out [GitHub issues](https://github.com/roadrunner-server/roadrunner/issues) page. Here you'll find a list of open feature requests. The RoadRunner community is active and responsive, so feel free to join the discussion on diff --git a/SUMMARY.md b/SUMMARY.md index 100fa1d..0bf9a3b 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -9,6 +9,7 @@ - [Configuration](intro/config.md) - [Contributing](intro/contributing.md) - [Upgrade and Compatibility](intro/compatibility.md) +- [Upgrade from v2025 to v3](intro/v3-migration.md) ## πŸ‘· PHP Worker @@ -76,6 +77,7 @@ - [BoltDB](queues/boltdb.md) - [Kafka](queues/kafka.md) - [NATS](queues/nats.md) +- [NSQ](queues/nsq.md) - [SQS](queues/sqs.md) ## πŸ•ΈοΈ HTTP @@ -83,14 +85,17 @@ - [Intro into HTTP](http/http.md) - [Headers and CORS](http/headers.md) - [Proxy IP parser](http/proxy.md) +- [Rate limiter](http/rate-limiter.md) - [Static files](http/static.md) - [X-Sendfile](http/sendfile.md) - [Streaming](http/resp-streaming.md) - [gzip](http/gzip.md) +- [zstd](http/zstd.md) ## ⚑️ gRPC - [Intro into gRPC](grpc/grpc.md) +- [Interceptors](grpc/interceptors.md) - [Protoreg](grpc/protoreg.md) ## πŸ“ˆ Logging and Observability @@ -110,7 +115,6 @@ ## 🧩 Integrations -- [Migration from RRv1 to RRv2](integration/migration.md) - [Spiral Framework](integration/spiral.md) - [Yii](integration/yii.md) - [Symfony](integration/symfony.md) @@ -128,6 +132,7 @@ ## πŸ“š Releases +- [v3.0.0](releases/v3-0-0.md) - [v2025.1.14](releases/v2025-1-14.md) - [v2025.1.13](releases/v2025-1-13.md) - [v2025.1.12](releases/v2025-1-12.md) diff --git a/app-server/aws-lambda.md b/app-server/aws-lambda.md index 05cbc15..f2236fd 100644 --- a/app-server/aws-lambda.md +++ b/app-server/aws-lambda.md @@ -4,17 +4,12 @@ RoadRunner can run PHP as an AWS Lambda function. ## PHP Worker -The PHP worker does not require any specific configuration to run inside a Lambda function. We can use the default snippet with -an internal counter to demonstrate how workers are reused: +Use PHP `8.5` with the `sockets` extension and Composer 2. This worker returns an HTTP response for each invocation: {% code title="handler.php" %} ```php = 600 { return events.APIGatewayV2HTTPResponse{Body: "", StatusCode: 500}, nil } + + response := events.APIGatewayV2HTTPResponse{ + StatusCode: int(responseMetadata.Status), + Headers: make(map[string]string, len(responseMetadata.Headers)), + Body: base64.StdEncoding.EncodeToString(r.Body), + IsBase64Encoded: true, + } + for name, header := range responseMetadata.Headers { + values := make([]string, 0, len(header.GetValue())) + for _, value := range header.GetValue() { + values = append(values, string(value)) + } + if strings.EqualFold(name, "Set-Cookie") { + response.Cookies = append(response.Cookies, values...) + } else { + response.Headers[name] = strings.Join(values, ", ") + } + } return response, nil } } @@ -351,18 +437,34 @@ endure: Here you can take full advantage of RoadRunner: you can include any plugin here and configure it with the embedded config (within reasonable limits). -To build and package your Lambda function, run: +Use Go `1.27.1`. If your project has no `go.mod`, run `go mod init example.com/lambda` from the project root. Select the dependencies before building: + +```bash +go get github.com/aws/aws-lambda-go@v1.55.0 \ + github.com/roadrunner-server/api-go/v6@latest \ + github.com/roadrunner-server/config/v6@latest \ + github.com/roadrunner-server/endure/v2@v2.6.2 \ + github.com/roadrunner-server/errors@v1.5.0 \ + github.com/roadrunner-server/goridge/v4@latest \ + github.com/roadrunner-server/logger/v6@latest \ + github.com/roadrunner-server/pool/v2@latest \ + github.com/roadrunner-server/server/v6@latest \ + google.golang.org/protobuf@v1.36.12 +``` + +The build uses `-mod=readonly` because Composer's `vendor` directory does not contain Go modules. AWS Lambda requires an executable named [`bootstrap`](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html#runtimes-custom-bootstrap) at the root of the deployment package. Include a Linux PHP `8.5` executable named `php` and its shared libraries in `lib/`, built for Amazon Linux 2023 and `x86_64`. Run these commands from the project root: {% code title="build.sh" %} ```bash -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags "-s" -o bootstrap-amd64 main.go plugin.go -zip main.zip * -r +go mod tidy +CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -mod=readonly -trimpath -ldflags "-s" -o bootstrap main.go plugin.go +zip -r main.zip bootstrap php lib handler.php vendor ``` {% endcode %} -You can now upload and invoke your handler using a simple string event. +Use the Lambda `provided.al2023` runtime and `x86_64` architecture. Upload the package and connect it to an API Gateway HTTP API with payload format `2.0`. For a direct Lambda test, use an API Gateway v2 HTTP request event, not a string event. API Gateway v2 omits the custom-domain API mapping prefix from `rawPath`; this adapter uses the path supplied in the event. ## Repository with the full example diff --git a/app-server/cli.md b/app-server/cli.md index 8e08fb9..f51faf6 100644 --- a/app-server/cli.md +++ b/app-server/cli.md @@ -2,6 +2,17 @@ RoadRunner offers a convenient CLI to start and manage the server. +| Command | Purpose | +| --- | --- | +| `rr serve` | Start RoadRunner. | +| `rr stop` | Signal the process recorded in `.pid`. | +| `rr reset [plugin...]` | Reset worker pools through RPC. | +| `rr workers [plugin...]` | Show worker state through RPC. | +| `rr jobs --list` | List Jobs pipelines through RPC. | +| `rr jobs --pause`, `--resume`, or `--destroy` | Manage Jobs pipelines through RPC. | + +Use `rr --help` or `rr --help` to list the options in your binary. + ## Version To display the version of RoadRunner, you can use the `-v` or `--version` option: @@ -14,13 +25,12 @@ To display the version of RoadRunner, you can use the `-v` or `--version` option {% endcode %} -This command will display the version of RoadRunner, as well as information about the build time, operating system, and -architecture. +The output includes the RoadRunner version, build time, Go version, operating system, and architecture. The build time below is a placeholder. {% code %} ```output -rr version 2023.1.2 (build time: 2023-05-04T13:19:13+0000, go1.20.4), OS: linux, arch: amd64 +rr version 3.0.0 (build time: , go1.27.1), OS: linux, arch: amd64 ``` {% endcode %} @@ -32,13 +42,12 @@ To start the server, you can use the following command: {% code %} ```terminal -./rr serve +./rr serve ``` {% endcode %} -By default, RoadRunner looks for a `.rr.yaml` file in the current working directory where the binary is placed. However, -you can also specify the configuration file from a different location using the `-c` option: +RoadRunner looks for `.rr.yaml` in the current working directory. Use `-c` to select a different configuration file: {% code %} @@ -48,7 +57,7 @@ you can also specify the configuration file from a different location using the {% endcode %} -You can also specify the working directory using the -w option: +Use `-w` to select a working directory. Without `-w`, RoadRunner uses the directory that contains the selected configuration file: {% code %} @@ -58,8 +67,7 @@ You can also specify the working directory using the -w option: {% endcode %} -RoadRunner also supports `.env` files. To read environment variables from the `.env` file, use the `--dotenv` CLI -command: +Use `--dotenv` to load an environment file before the command reads configuration. `DOTENV_PATH` takes precedence over this option: {% code %} @@ -69,10 +77,9 @@ command: {% endcode %} -## Background mode +## PID file -Additionally, you can start the server in background mode using the `-p` option. In this mode, RoadRunner creates -a `.pid` file: +Use `-p` to write the process ID to `.pid` in the working directory. The process stays in the foreground. Use a process manager such as [systemd](systemd.md) to run it as a service. {% code %} @@ -118,18 +125,13 @@ To enable experimental features, use the `--enable-experimental` or `-e` option: ## Options -- `-c` - specifies the path to the configuration file. By default, RoadRunner looks for a `.rr.yaml` file in the current - working directory. However, you can specify a different file using this option. -- `-w` - sets the working directory for the server. By default, RoadRunner uses the current working directory. -- `--dotenv` - populates the process with environment variables from the `.dotenv` file. This can be useful when you - want to set environment variables for your application. -- `-d` - starts a Golang profiling server. This allows you to analyze the performance of your application and - identify potential bottlenecks. (Read more [here](https://pkg.go.dev/net/http/pprof).) -- `-s` - enables silent mode. In this mode, RoadRunner does not display any output in the console. -- `-o` - allows you to override configuration keys with your values. For example, `-o=http.address=8080` will override - the `http.address` configuration key from the `.rr.yaml` file. -- `-p` - creates a `.pid` file to use with the rr stop command later. This can be useful when you want to run RoadRunner - in the background mode. +- `-c` - selects the configuration file. The default is `.rr.yaml` in the current working directory. +- `-w` - sets the working directory. Without this option, RoadRunner uses the configuration file's directory. +- `--dotenv` - loads an environment file. Existing process environment values take precedence over values in the file. +- `-d` - starts a Go profiling server. See [pprof](https://pkg.go.dev/net/http/pprof). +- `-s` - suppresses command status messages. Configure plugin logs through the `logs` section. +- `-o` - overrides a configuration key, such as `-o http.address=127.0.0.1:8080`. Included files can override these values. +- `-p` - creates a `.pid` file for `rr stop`. - `-e` or `--enable-experimental` - enables experimental features. This option is useful when you want to test new features that are not yet available in the stable release. @@ -147,8 +149,7 @@ To stop RoadRunner, you have a few options: {% endhint %} {% hint style="info" %} -By default, the grace period is 30 seconds. You can change it in the configuration file using `endure.grace_period` -setting. You can find an example [here](https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml#L2115). +The default grace period is 30 seconds. Set `endure.grace_period` to change it. See [plugin shutdown](../customization/plugin.md#serving). {% endhint %} You can also use the following command to stop the server: @@ -182,26 +183,26 @@ The `rr stop` command can only be used to stop a RoadRunner server that was star ## Restarting the Server -RoadRunner supports graceful restarts using the `SIGUSR2` signal on Unix-like systems. This allows you to restart the RoadRunner process without downtime, which is particularly useful in deployment scenarios. +Use `SIGUSR2` on Unix-like systems to reload RoadRunner configuration. RoadRunner stops its plugins, then starts the same executable with the same arguments and environment. Listeners close during this restart, so requests need another available instance if continuous service is required. To restart RoadRunner, send the `SIGUSR2` signal to the main RoadRunner process: {% code %} ```bash -kill -USR2 +kill -USR2 1234 ``` {% endcode %} -Where `` is the process ID of the main RoadRunner process. +Replace `1234` with the process ID of the main RoadRunner process. When RoadRunner receives the `SIGUSR2` signal, it performs the following steps: -1. Validates the executable path -2. Notifies systemd (if running under systemd) -3. Gracefully stops all workers and plugins -4. Replaces the current process with a new one using the same executable path +1. Resolves the executable path. +2. Stops workers and plugins. +3. Replaces the process with the same executable, arguments, and environment. +4. Reads the configuration during startup. {% hint style="info" %} If the configuration file is a symlink, RoadRunner will properly re-read the configuration from the symlink target after restart. This allows you to update the symlink to point to a new configuration file for deployment workflows. @@ -223,9 +224,7 @@ RoadRunner allows you to reload all workers. {% endcode %} -This command reloads all RoadRunner workers, including HTTP, gRPC, and others, and starts them with a new worker pool. -This can be useful when you make changes to your application code and want to see the changes reflected in the running -server. +This command calls `resetter.Reset` for plugins that support worker reset. It loads new PHP code with the running server's configuration. Use [SIGUSR2](#restarting-the-server) to apply changes to `.rr.yaml`. By default, this command displays output in the console. However, if you want to reload the workers silently, you can use the `--silent` option: @@ -253,11 +252,7 @@ Additionally, you can reload only particular plugins by specifying their names: {% endcode %} {% hint style="info" %} -RoadRunner will wait for any active requests to finish processing before reloading the worker pool. This ensures that -all active requests are completed before the new worker pool is started. -Any new incoming requests that arrive during the reloading process will be queued and processed once the new worker -pool is started. This means that there may be a temporary delay in processing new requests while the worker pool is -being reloaded, but no requests should be lost. +The pool waits up to `pool.reset_timeout` for active work before it starts stopping workers. Requests waiting for a worker can still reach their allocation timeout. See [worker pool timeouts](../php/pool.md#timeouts-and-admission). {% endhint %} ### Options @@ -338,7 +333,7 @@ workers at the time the command is executed. - `-c` - specifies the path to the configuration file. By default, RoadRunner looks for a `.rr.yaml` file in the current working directory. However, you can specify a different file using this option. -- `-w` - sets the working directory for the server. By default, RoadRunner uses the current working directory. +- `-w` - sets the working directory. Without this option, RoadRunner uses the configuration file's directory. - `-i` - interactive mode (updates statistics every second). ## Jobs commands @@ -413,8 +408,12 @@ For example: {% code %} ```yaml +version: "3" + rpc: - listen: "127.0.0.1:6001" + listen: "tcp://127.0.0.1:6001" ``` {% endcode %} + +If the RPC address uses values from a dotenv file, pass `--dotenv` to the client command. These commands do not load the root `envfile` setting. See [environment files](../php/environment.md#dotenv). diff --git a/app-server/docker.md b/app-server/docker.md index 53d8b15..6f62c9a 100644 --- a/app-server/docker.md +++ b/app-server/docker.md @@ -1,12 +1,33 @@ # Docker Images -The following Docker images are available: +Use the [official RoadRunner image](https://github.com/roadrunner-server/roadrunner/pkgs/container/roadrunner) at `ghcr.io/roadrunner-server/roadrunner:3.0.0`. The application, Nginx, and debugging examples use this release tag. -| Description | Links | Status | -|------------------------------------------|-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Official RR image** | [Link](https://github.com/roadrunner-server/roadrunner/pkgs/container/roadrunner) | ![Latest Stable Version](https://img.shields.io/github/v/release/roadrunner-server/roadrunner.svg?maxAge=30) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) | -| **Third-party image from `n1215`** | [Link](https://github.com/n1215/roadrunner-docker-skeleton) | [![License](https://poser.pugx.org/n1215/roadrunner-docker-skeleton/license)](https://packagist.org/packages/n1215/roadrunner-docker-skeleton) | -| **Third-party image from `spacetab-io`** | [Link](https://github.com/spacetab-io/docker-roadrunner-php) | ![Latest Stable Version](https://img.shields.io/github/v/release/spacetab-io/docker-roadrunner-php) ![License](https://img.shields.io/github/license/spacetab-io/docker-roadrunner-php) | +The `3` and `3.0` tags follow releases in their respective version lines. Use `3.0.0` to select a specific release. Copy the RoadRunner binary into an application image with PHP. + +## Build the RoadRunner Image + +For a custom binary, follow the [source installation guide](../intro/install.md#build-from-source). Build a static Linux `rr` binary for the target architecture with `CGO_ENABLED=0`. Put the binary and `Dockerfile.rr` in the same build directory: + +{% code title="Dockerfile.rr" %} + +```dockerfile +FROM scratch AS roadrunner +COPY rr /usr/bin/rr +``` + +{% endcode %} + +Build the image for the same target platform as your PHP application image: + +```bash +docker build -f Dockerfile.rr -t roadrunner:v3-local . +``` + +`roadrunner:v3-local` contains the compiled binary. Copy this binary into an application image with PHP. The binary, this image, and the application image must use the same target architecture. Setting Docker `--platform` does not recompile a binary copied from the host. + +To use this custom image, set the first `FROM` below to `roadrunner:v3-local`. + +## Build the Application Image Here is an example of a `Dockerfile` that can be used to build a Docker image with RoadRunner for a PHP application: @@ -17,9 +38,9 @@ Note that this example utilizes a folder named `app` for your application. If yo {% code title="Dockerfile" %} ```dockerfile -FROM ghcr.io/roadrunner-server/roadrunner:2024 as roadrunner +FROM ghcr.io/roadrunner-server/roadrunner:3.0.0 AS roadrunner -FROM php:8.3-alpine +FROM php:8.5-cli-alpine # https://github.com/mlocati/docker-php-extension-installer # https://github.com/docker-library/docs/tree/0fbef0e8b8c403f581b794030f9180a68935af9d/php#how-to-install-more-php-extensions @@ -43,7 +64,7 @@ RUN composer install --optimize-autoloader --no-dev COPY ./app . # Run the RoadRunner server -CMD ./rr serve -c .rr.yaml +CMD ["/usr/local/bin/rr", "serve", "-c", ".rr.yaml"] ``` {% endcode %} diff --git a/app-server/kubernetes-argocd.md b/app-server/kubernetes-argocd.md index 4813386..0d92f4a 100644 --- a/app-server/kubernetes-argocd.md +++ b/app-server/kubernetes-argocd.md @@ -10,7 +10,7 @@ You can deploy RoadRunner to Kubernetes via GitOps using Argo CD with the offici - Kubernetes `>= 1.26` - Argo CD installed - `kubectl` access to your cluster -- A runnable RoadRunner container image (or the example image from `k8s-examples`) +- A RoadRunner v3 application image in a registry that the cluster can access. See [Build the Application Image](docker.md#build-the-application-image). ## Use the Official Example @@ -22,11 +22,17 @@ Use these files from the upstream repository: ## Recommended Values (MetalLB-Friendly) -This profile avoids dependency on a Gateway controller and works well on clusters using MetalLB for external service exposure: +Set `image.repository` and `image.tag` to your published v3 application image. The probe paths below require v3. + +This profile uses MetalLB for external service access: {% code title="values.yaml" %} ```yaml +image: + repository: registry.example.com/your-team/rr-app + tag: "3.0.0" + service: type: LoadBalancer @@ -39,9 +45,29 @@ ingress: {% endcode %} +## Configure Probes + +Set the chart probe paths to the RoadRunner v3 status endpoints: + +{% code title="values.yaml" %} + +```yaml +probes: + liveness: + path: /livez?plugin=http + port: status + readiness: + path: /readyz?plugin=http + port: status +``` + +{% endcode %} + +The chart binds the status server to port `2114` in the pod. Liveness accepts active HTTP workers, including busy workers. Readiness requires at least one idle HTTP worker. During graceful shutdown, liveness remains successful and readiness fails. See [Health and Readiness checks](../lab/health.md). + ## Apply with Argo CD -Apply the Argo CD `Application` manifest: +Commit your values to the repository used by Argo CD. Set `repoURL` and `targetRevision` in `application.yaml` to that repository and revision. Then apply the Argo CD `Application` manifest: {% code %} @@ -51,8 +77,6 @@ kubectl apply -f deploy/argocd/application.yaml {% endcode %} -If needed, customize `repoURL` and `targetRevision` in `application.yaml` before applying. - ## Verify Sync and Health {% code %} @@ -60,11 +84,23 @@ If needed, customize `repoURL` and `targetRevision` in `application.yaml` before ```bash kubectl -n roadrunner get svc roadrunner -w curl -sS http:/// -curl -sS http:///health ``` {% endcode %} +To check the status server, forward its pod port: + +```bash +kubectl -n roadrunner port-forward deployment/roadrunner 2114:2114 +``` + +In another terminal, request the probes: + +```bash +curl -sS 'http://127.0.0.1:2114/livez?plugin=http' +curl -sS 'http://127.0.0.1:2114/readyz?plugin=http' +``` + ## Troubleshooting - If Argo CD shows `Progressing` and `Waiting for controller`, Gateway resources are enabled but no Gateway controller/GatewayClass is reconciling them. diff --git a/app-server/nginx-with-rr.md b/app-server/nginx-with-rr.md index d462a65..de8fe41 100644 --- a/app-server/nginx-with-rr.md +++ b/app-server/nginx-with-rr.md @@ -55,6 +55,33 @@ Consider using `fastcgi_pass` instead of `proxy_pass`: Using the `fastcgi_pass` performance in certain configurations. {% endhint %} +#### Unix Socket + +The HTTP plugin can give Nginx group access to a FastCGI socket without changing application file permissions: + +{% code title=".rr.yaml fragment" %} + +```yaml +http: + fcgi: + address: "unix:///run/roadrunner/fcgi.sock" + unix_socket: + mode: "0660" + gid: 2000 +``` + +{% endcode %} + +Replace the `fastcgi_pass` directive in the Nginx example with: + +```nginx +fastcgi_pass unix:/run/roadrunner/fcgi.sock; +``` + +Replace `2000` with the numeric group shared by RoadRunner and the Nginx worker processes. A non-root RoadRunner process must belong to this group to set the socket group. Nginx workers must also belong to this group to connect through the group permissions. The socket owner stays unchanged because `uid` is omitted. + +Create the parent directory first. RoadRunner needs permission to create the socket there. Nginx needs search permission on every parent directory. See [Unix socket attributes](../intro/config.md#unix-socket-attributes) for configuration. The Docker example below uses TCP connections. A Unix socket between containers requires a shared socket directory. + ### Proxy RoadRunner can be configured to listen for HTTP requests on a specific port. @@ -153,11 +180,13 @@ In this example, we will demonstrate how to use RoadRunner with Nginx in a Docke ### Dockerfile +This Dockerfile uses the [official RoadRunner v3 image](docker.md). + {% code title="docker/app/Dockerfile" %} -```docker -FROM --platform=${TARGETPLATFORM:-linux/amd64} ghcr.io/roadrunner-server/roadrunner:latest as roadrunner -FROM --platform=${TARGETPLATFORM:-linux/amd64} php:8.3-alpine +```dockerfile +FROM ghcr.io/roadrunner-server/roadrunner:3.0.0 AS roadrunner +FROM php:8.5-cli-alpine COPY --from=roadrunner /usr/bin/rr /usr/local/bin/rr COPY --from=mlocati/php-extension-installer:2 /usr/bin/install-php-extensions /usr/local/bin/ @@ -177,12 +206,6 @@ ENTRYPOINT ["rr"] {% endcode %} -{% hint style="warning" %} - -Consider using the direct version in production. The `latest` image tag might be used in development environments only. - -{% endhint %} - ### RoadRunner configuration Create a `.rr.yaml` configuration file to specify how RoadRunner should interact with your PHP application. @@ -258,10 +281,9 @@ Do not forget the `composer.json` file: ```json { - "minimum-stability": "dev", - "prefer-stable": true, "require": { - "spiral/roadrunner-http": "^3.0", + "nyholm/psr7": "^1.8", + "spiral/roadrunner-http": "^4.1", "spiral/goridge": "^4.0" } } diff --git a/app-server/systemd.md b/app-server/systemd.md index 8024ee8..7e3dc21 100644 --- a/app-server/systemd.md +++ b/app-server/systemd.md @@ -86,7 +86,7 @@ Example output: ``` ● rr.service - High-performance PHP application server Loaded: loaded (/lib/systemd/system/rr.service; enabled; vendor preset: enabled) - Active: active (running) since Tue 2024-03-12 19:20:55 UTC; 35min ago + Active: active (running) since Tue 2026-10-06 19:20:55 UTC; 35min ago Docs: https://docs.roadrunner.dev/ CGroup: /system.slice/rr.service β”œβ”€2835793 /usr/local/bin/rr serve -c /var/www/.rr.yaml @@ -105,13 +105,13 @@ journalctl -f -u rr.service {% endcode %} -Example output with `info`-level logs (see [levels](../lab/logger.md#level)): +Example startup output (see [startup logs](../lab/logger.md#startup-logs)). The build time below is a placeholder: -```bash -Mar 12 20:12:53 server systemd[1]: Starting PHP application server... -Mar 12 20:12:55 server rr[2936506]: [INFO] RoadRunner server started; version: 2023.3.10, buildtime: 2024-02-01T22:33:17+0000 -Mar 12 20:12:55 server rr[2936506]: [INFO] sdnotify: notified -Mar 12 20:12:55 server systemd[1]: Started PHP application server. +```log +Oct 06 20:12:53 server systemd[1]: Starting PHP application server... +Oct 06 20:12:55 server rr[2936506]: [INFO] RoadRunner server started; version: 3.0.0, buildtime: +Oct 06 20:12:55 server rr[2936506]: [INFO] sdnotify: notified +Oct 06 20:12:55 server systemd[1]: Started PHP application server. ``` ## `sd_notify` protocol diff --git a/community-plugins/cache.md b/community-plugins/cache.md index 9080c2a..ade26fe 100644 --- a/community-plugins/cache.md +++ b/community-plugins/cache.md @@ -23,27 +23,9 @@ information. ## Building RoadRunner with Cache -As it's based on the [Souin](https://github.com/darkweak/souin) HTTP cache library, we can directly use the RoadRunner -middleware implementation. We have to set the `folder` property because this middleware is located in a subdirectory. +RoadRunner v3 requires a Souin revision that implements the [v6 plugin interfaces](../customization/plugin.md#v6-migration), including `NamedLogger(string) *slog.Logger` for logging. -{% code title="configuration.toml" %} - -```ini -[github] -[github.token] -token = "YOUR_GH_TOKEN" - -[github.plugins] -# Use the Souin third-party middleware. -cache = { ref = "master", owner = "darkweak", repository = "souin", folder = "/plugins/roadrunner" } -# others ... - -[log] -level = "debug" -mode = "development" -``` - -{% endcode %} +Follow the [Velox build guide](../customization/build.md). Add `[plugins.cache]` with `module_name = "github.com/darkweak/souin/plugins/roadrunner"` and set `tag` to a compatible revision. The module path includes the plugin subdirectory. **Available storages**: In-memory/Filesystem diff --git a/community-plugins/intro.md b/community-plugins/intro.md index b4b4164..ad14b32 100644 --- a/community-plugins/intro.md +++ b/community-plugins/intro.md @@ -5,7 +5,7 @@ Community plugins are plugins that are not officially supported by the RoadRunner team. They are maintained by the community and are not part of the RoadRunner distribution. They can be built using the tool called [Velox](https://github.com/roadrunner-server/velox). Not being official does not mean they are not good. In fact, some community plugins are very useful and can be used in production. -A community plugin does not necessarily need to be in the `roadrunner-server` organization. It can be in any other organization or even a personal repository. The only requirement is that the plugin has a `velox.toml` configuration file and can be built with the latest version of RoadRunner. +A community plugin can be in any organization or personal repository. For RoadRunner v3, its Go module must export a `Plugin` type and implement the [v6 plugin interfaces](../customization/plugin.md#v6-migration). Add its module path and revision to the [Velox configuration](../customization/build.md). Community plugins have their own issues and PRs. If you have any questions or issues with a community plugin, you should create an issue or discussion in the plugin's repository. diff --git a/customization/build.md b/customization/build.md index cc39dd4..556a2ec 100644 --- a/customization/build.md +++ b/customization/build.md @@ -1,122 +1,42 @@ # Building a Server -Developers can take advantage of the customization options available with RoadRunner to create a server optimized -for their particular project. - -**This can include:** - -- Adding custom plugins. -- Forking existing ones to make changes. -- Building a lightweight server with only the necessary plugins. - -We created a tool called **Velox** that lets developers build a RoadRunner server binary. It uses a configuration file -to determine which plugins and repositories are required for building a RoadRunner server binary. - -## Configuration - -The configuration file is written in TOML format and contains a list of repositories to add to the build. For each -repository, you can specify the owner and version. You can also add private repositories from GitHub or GitLab and -authenticate with access tokens. +Velox builds a RoadRunner binary from the plugins listed in `velox.toml`. Use it to select plugins or build with a custom plugin or fork. {% hint style="info" %} -To download all the required plugins for RoadRunner, you need a GitHub token. If you try to download plugins without a -token, anonymous access is limited to 50 requests per hour. You can read more about these limits on -the [Rate limits for GitHub Apps](https://docs.github.com/en/apps/creating-github-apps/setting-up-a-github-app/rate-limits-for-github-apps) -page. +This guide uses Velox v3 to build RoadRunner v3 with v6 plugins. {% endhint %} -**Here is an example of a configuration file:** +## Configuration + +This TOML configuration selects a minimal HTTP server. Keep all RoadRunner plugins on the `/v6` module line. Use exact plugin tags and an RR commit SHA when you need repeatable builds. {% code title="velox.toml" %} ```toml [roadrunner] -# ref -> reference, tag, commit or branch -ref = "master" - -# the debug option is used to build RR with debug symbols to profile it with pprof -[debug] -enabled = true +ref = "v3.0.0" [github] +base_url = "https://github.com" + [github.token] -token = "${RT_TOKEN}" - -# ref -> master, commit or tag -[github.plugins] -# LOGS -appLogger = { ref = "v5.0.2", owner = "roadrunner-server", repository = "app-logger" } -logger = { ref = "v5.0.2", owner = "roadrunner-server", repository = "logger" } -lock = { ref = "v5.0.2", owner = "roadrunner-server", repository = "lock" } -rpc = { ref = "v5.0.2", owner = "roadrunner-server", repository = "rpc" } - -# CENTRIFUGE BROADCASTING PLATFORM -centrifuge = { ref = "v5.0.2", owner = "roadrunner-server", repository = "centrifuge" } - -# WORKFLOWS ENGINE -temporal = { ref = "v5.1.0", owner = "temporalio", repository = "roadrunner-temporal" } - -# METRICS -metrics = { ref = "v5.0.2", owner = "roadrunner-server", repository = "metrics" } - -# HTTP + MIDDLEWARE -http = { ref = "v5.0.2", owner = "roadrunner-server", repository = "http" } -gzip = { ref = "v5.0.2", owner = "roadrunner-server", repository = "gzip" } -prometheus = { ref = "v5.0.1", owner = "roadrunner-server", repository = "prometheus" } -headers = { ref = "v5.0.2", owner = "roadrunner-server", repository = "headers" } -static = { ref = "v5.0.1", owner = "roadrunner-server", repository = "static" } -proxy = { ref = "v5.0.2", owner = "roadrunner-server", repository = "proxy_ip_parser" } -send = { ref = "v5.0.1", owner = "roadrunner-server", repository = "send" } - -# OpenTelemetry -otel = { ref = "v5.0.1", owner = "roadrunner-server", repository = "otel" } - -# SERVER -server = { ref = "v5.0.2", owner = "roadrunner-server", repository = "server" } - -# SERVICE aka lightweight systemd -service = { ref = "v5.0.2", owner = "roadrunner-server", repository = "service" } - -# JOBS -jobs = { ref = "v5.0.2", owner = "roadrunner-server", repository = "jobs" } -amqp = { ref = "v5.0.2", owner = "roadrunner-server", repository = "amqp" } -sqs = { ref = "v5.0.2", owner = "roadrunner-server", repository = "sqs" } -beanstalk = { ref = "v5.0.2", owner = "roadrunner-server", repository = "beanstalk" } -nats = { ref = "v5.0.2", owner = "roadrunner-server", repository = "nats" } -kafka = { ref = "v5.0.2", owner = "roadrunner-server", repository = "kafka" } -googlepubsub = { ref = "v5.0.2", owner = "roadrunner-server", repository = "google-pub-sub" } - -# KV -kv = { ref = "v5.0.2", owner = "roadrunner-server", repository = "kv" } -boltdb = { ref = "v5.0.2", owner = "roadrunner-server", repository = "boltdb" } -memory = { ref = "v5.0.2", owner = "roadrunner-server", repository = "memory" } -redis = { ref = "v5.0.2", owner = "roadrunner-server", repository = "redis" } -memcached = { ref = "v5.0.2", owner = "roadrunner-server", repository = "memcached" } - -# FILESERVER (static files) -fileserver = { ref = "v5.0.1", owner = "roadrunner-server", repository = "fileserver" } - -# gRPC plugin -grpc = { ref = "v5.0.2", owner = "roadrunner-server", repository = "grpc" } - -# HEALTHCHECKS + READINESS CHECKS -status = { ref = "v5.0.2", owner = "roadrunner-server", repository = "status" } - -# TCP for the RAW TCP PAYLOADS -tcp = { ref = "v5.0.2", owner = "roadrunner-server", repository = "tcp" } - -[gitlab] -[gitlab.token] -# api, read-api, read-repo -token = "${GL_TOKEN}" - -[gitlab.endpoint] -endpoint = "https://gitlab.com" - -[gitlab.plugins] -# ref -> master, commit or tag -test_plugin_1 = { ref = "main", owner = "rustatian", repository = "36405203" } -test_plugin_2 = { ref = "main", owner = "rustatian", repository = "36405235" } +token = "${GITHUB_TOKEN}" + +[plugins.logger] +module_name = "github.com/roadrunner-server/logger/v6" +tag = "latest" + +[plugins.server] +module_name = "github.com/roadrunner-server/server/v6" +tag = "latest" + +[plugins.rpc] +module_name = "github.com/roadrunner-server/rpc/v6" +tag = "latest" + +[plugins.http] +module_name = "github.com/roadrunner-server/http/v6" +tag = "latest" [log] level = "info" @@ -125,215 +45,114 @@ mode = "production" {% endcode %} -{% hint style="info" %} -You can find the latest version of the example configuration file in -the [official repository](https://github.com/roadrunner-server/velox/blob/master/velox.toml). -{% endhint %} - -{% hint style="warning" %} -When using official plugins for RoadRunner, it is recommended avoid using the `master` branch as it may contain -unstable code. Instead, use tags with the same major version (e.g., `logger:v4.x.x` + `amqp:v4.x.x`, but -not `logger:v4.0.0` + `amqp:v3.0.5`). Please note that the currently supported plugin version is `v5.x.x`, and the -supported RoadRunner version is `>=v2024.2.x`. - -Failure to follow these guidelines may result in compatibility issues and -other problems. Please pay close attention to your configuration file to ensure proper use of plugins. -{% endhint %} - -You can use environment variables in the configuration file. This is useful when you want to keep the configuration file -in the repository, but you don't want to expose your tokens or just want to pass them as arguments to the `vx` command. - -Here is the list of environment variables from the example above: +Velox includes `informer` and `resetter` automatically from the downloaded RR `go.mod`. Do not add them to `[plugins]`. Velox warns and ignores those entries. -| Variable | Description | -|---------------|----------------------------------------------------------------------------| -| `${GL_TOKEN}` | GitLab token. | -| `${RT_TOKEN}` | GitHub token. | -| `${VERSION}` | RR version to write into the binary (will be shown with `./rr --version`). | -| `${TIME}` | Build time (will be shown with `./rr --version`). | - -{% hint style="info" %} -Keep in mind to set the latest stable version in the `${VERSION}` environment variable. You may also use the `${TIME}` environment -variable to write the build time in the output binary. -{% endhint %} +List each plugin module once. Custom plugins must export a `Plugin` type from the module root. ### Options -| Option | Description | -|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **ref** | Tag, commit hash or branch name. | -| **owner** | Repository owner (might be the user or organization). | -| **repository** | Repository name. | -| **folder** | If the plugin is in some folder in your repository, you may specify it via this configuration option.
For example: `cache = { ref = "v1.6.18", owner = "darkweak", repository = "souin", folder="plugins/roadrunner" }` | -| **replace** | Go.mod [replace directive](https://go.dev/ref/mod#go-mod-file-replace). | +| Key | Meaning | +| --- | --- | +| `roadrunner.ref` | RR semver tag, branch, or 40-character hexadecimal commit SHA. Defaults to `master`. | +| `plugins..module_name` | Full Go module path, including its major-version suffix. | +| `plugins..tag` | Plugin version or ref. Use an exact semver tag to enable version checks. | +| `target_platform.os` | Target `GOOS`. Defaults to the host OS. | +| `target_platform.arch` | Target `GOARCH`. Defaults to the host architecture. | +| `debug.enabled` | Set to `true` to disable optimization and inlining, retain debug symbols, and enable the `debug` build tag. Defaults to `false`. | +| `debug.race` | Set to `true` to build with `-race` and `CGO_ENABLED=1`. Defaults to `false`. | -{% hint style="info" %} +Each missing target platform value defaults independently. Race builds need a C compiler and C libraries for the target platform. Without `debug.race`, Velox sets `CGO_ENABLED=0`. -To replace the module with the local copy or some remote module, use the following `velox.toml` configuration: +### Module replacements + +Append `[[replaces]]` and `[[excludes]]` sections when you need Go module overrides. This example uses a compatible local HTTP fork at `../http` and excludes one dependency version: {% code title="velox.toml" %} -```ini -your_module = { ref = "master", owner = "owner", repository = "repo", replace="github.com/owner2/repo2" } +```toml +[[replaces]] +new = "../http" +old = "github.com/roadrunner-server/http/v6" + +[[excludes]] +module = "github.com/redis/go-redis/v9" +version = "v9.15.0" ``` {% endcode %} +Relative replacement paths resolve against the working directory of `vx`, not the configuration file or downloaded RR directory. A container build must mount the local module at a path available inside the container. -Or with your local copy: +For a remote replacement, use `module@version` in `new`. The `old` value can include `@version` to restrict the replacement to that version. A local path must not have a version suffix. Each `old` value must be unique. -{% code title="velox.toml" %} +An excluded version must be canonical semver and match the module path major. Velox applies these directives before Go resolves dependencies. It retains the downloaded RR `go.mod` as the starting point. -```ini -your_module = { ref = "master", owner = "owner", repository = "repo", replace="../path/to/a/local/dir" } -``` +### Older configuration -{% endcode %} +Velox v3 ignores the old `[github.plugins.*]` and `[gitlab.*]` tables. It does not convert them. Move plugin entries to `[plugins.]` with `module_name` and `tag`. -{% endhint %} +The old per-plugin `ref`, `owner`, `repository`, `folder`, and inline `replace` fields are also ignored. Use the module path declared in the plugin's `go.mod`, including for plugins in repository subdirectories. Move inline replacements to `[[replaces]]`. +The `build_args` setting is removed. Use `debug.enabled`, `debug.race`, and `target_platform` for build options. Velox sets version metadata from `roadrunner.ref` and the build timestamp. ### Private repositories -- Make sure the `ssh-agent` is running and the ssh key has been - added: [link](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) -- Exclude your organization package prefix from the Go environment variables: - -{% code %} +Go downloads plugin modules, including modules hosted on GitLab. Configure SSH or HTTPS credentials for Go module downloads. Set `GOPRIVATE` for your private module prefixes. Replace the organization names in this example: ```bash -go env -w GOPRIVATE="github.com//*,gitlab.com//*" -go env -w GONOSUMDB="github.com//*,gitlab.com//*" +export GOPRIVATE="github.com/your-org/*,gitlab.com/your-org/*" ``` -{% endcode %} - -## Building +Velox inherits the Go environment, including `GOPRIVATE`, `GOPROXY`, and `GOFLAGS`. The GitHub archive token does not authenticate plugin module downloads. -{% tabs %} +### RR archive access -{% tab title="Docker" %} +The `[github.token]` section is optional for public RR source. Its `token` value expands environment variables and is sent as a bearer token on the archive request. With the example above, export `GITHUB_TOKEN` when authentication is needed. Exporting the variable without the token configuration does not enable authentication. -Using the Docker image simplifies the build process by automatically building the RoadRunner binary and storing it in -the `/usr/bin/` folder. This eliminates the need to install Golang or other dependencies on your computer. Once the -build is complete, Docker will automatically start the RoadRunner server. +To download from GitHub Enterprise, set `[github] base_url` to your host, such as `https://ghe.example.com`. The RR mirror must be at `/roadrunner-server/roadrunner`. This setting does not change plugin module hosts. Archive downloads accept redirects and direct HTTP 200 responses. -**Here is an example of Dockerfile:** - -{% code title="Dockerfile" %} - -```dockerfile -# https://docs.docker.com/buildx/working-with-buildx/ -# TARGETPLATFORM if not empty OR linux/amd64 by default -FROM --platform=${TARGETPLATFORM:-linux/amd64} ghcr.io/roadrunner-server/velox:latest as velox - -# app version and build date must be passed during image building (version without any prefix). -# e.g.: `docker build --build-arg "APP_VERSION=1.2.3" --build-arg "BUILD_TIME=$(date +%FT%T%z)" .` -ARG APP_VERSION="undefined" -ARG BUILD_TIME="undefined" - -# copy your configuration into the docker -COPY velox.toml . - -# we don't need CGO -ENV CGO_ENABLED=0 - -# RUN build -RUN vx build -c velox.toml -o /usr/bin/ - -FROM --platform=${TARGETPLATFORM:-linux/amd64} php:8.3-cli - -# copy required files from builder image -COPY --from=velox /usr/bin/rr /usr/bin/rr - -# use roadrunner binary as image entrypoint -CMD ["/usr/bin/rr"] -``` - -{% endcode %} - -{% endtab %} - -{% tab title="Go" %} - -You can use the `go install` command to download Velox. +## Building -{% hint style="warning" %} -To download Velox and build an application server, you need [Golang 1.22+](https://golang.org/dl/) on your local machine. -{% endhint %} +Use Go `1.27.1` for this example. Install Velox v3: {% code title="go install" %} ```bash -go install github.com/roadrunner-server/velox/v2025/cmd/vx@latest +go install github.com/roadrunner-server/velox/v3/cmd/vx@latest ``` {% endcode %} -After the binary has been downloaded, you can build the application server: +Add the Go binary installation directory to `PATH`. Build from the directory that contains `velox.toml`: {% code title="vx build" %} ```bash -vx build -c velox.toml -o ~/Downloads +vx build -c velox.toml -o . ``` {% endcode %} -| Option | Description | -|--------|---------------------------------| -| `-c` | path to the configuration | -| `-o` | path where to put the RR binary | - -{% endtab %} - -{% tab title="Downloading Binary" %} - -To build the application server, you need to download the Velox binary from -the [GitHub releases page](https://github.com/roadrunner-server/velox/releases) and unpack it to your `PATH`. - -{% hint style="warning" %} -To build an application server, you need [Golang 1.22+](https://golang.org/dl/) on your local machine. -{% endhint %} - -After the binary has been downloaded, you can build the application server: - -{% code title="vx build" %} - -```bash -vx build -c velox.toml -o ~/Downloads -``` - -{% endcode %} - -| Option | Description | -|--------|---------------------------------| -| `-c` | path to the configuration | -| `-o` | path where to put the RR binary | - -{% endtab %} - -{% endtabs %} - -## Video tutorials - -### How to write a plugin +| Option | Meaning | +| --- | --- | +| `-c`, `--config` | Configuration file. Defaults to `velox.toml`. | +| `-o`, `--out` | Output directory. Defaults to the current directory. | -{% embed url="https://www.youtube.com/watch?v=h5PPvc_YOtg" %} +The command produces `./rr`. When the target matches the host OS and architecture, Velox checks `rr --version` before it replaces the output binary. For a cross-build, run that check on the target system. The output directory can be on another filesystem, including a container volume mount. -### `v2023.x.x` update +Build errors include the last 8 KB of the Go command's standard error output. Set `log.level = "debug"` to see that output while the command runs. -{% embed url="https://www.youtube.com/watch?v=w_uxFhdinvU" %} +### Reproducible builds -### Velox configuration +`SOURCE_DATE_EPOCH` sets the binary build timestamp in Unix seconds. Without a valid value, Velox uses the current time. The reported binary version comes from `roadrunner.ref`; the old `VERSION` and `TIME` environment variables are not used. -{% embed url="https://www.youtube.com/watch?v=sddi_lh7ePo" %} +Keep the Velox revision, RR commit, plugin tags, Go toolchain, target platform, build flags, and environment fixed for repeated builds. Keep local replacement contents fixed. -## Third-party and deprecated plugins +After Go resolves dependencies, Velox checks semver plugin pins. A different resolved version causes the build to fail. Use `[[replaces]]` to force a version only after you check its compatibility. -- [souin, third-party](https://github.com/darkweak/souin/tree/master/plugins/roadrunner) -- [reload, deprecated](https://github.com/roadrunner-server/reload) +Non-semver plugin refs, including `latest`, branch names, and commit SHAs, skip this version comparison. A same-module replacement compares its replacement version with the requested plugin tag. Local and different-module replacements skip the comparison. ## Known limitations -- At the moment, only GitHub and GitLab repositories are supported. +- Windows targets are rejected, including `target_platform.os = "windows"`. +- The Connect/gRPC build server, `vx server`, and `--address`/`-a` are removed. Use `vx build`. diff --git a/customization/embedding.md b/customization/embedding.md index 01ca077..a9f6b24 100644 --- a/customization/embedding.md +++ b/customization/embedding.md @@ -7,16 +7,17 @@ program. Here's an example of how to embed RoadRunner into a Go program with an HTTP handler: -Import the RoadRunner library via the `go get` command into your Go project: +Use Go `1.27.1`. Import the RoadRunner library. Use the same commit SHA as your [source build](../intro/install.md#build-from-source) when you need a fixed revision: -{% code title="main.go" %} +{% code title="Install the library" %} ```bash -go get -u github.com/roadrunner-server/roadrunner/v2025/lib +go get github.com/roadrunner-server/roadrunner/v3/lib@latest ``` {% endcode %} +The library imports v6 plugins and uses the [v6 plugin contracts](plugin.md#v6-migration). `NewRR` returns `(*RR, error)`. ## Create an RR instance @@ -25,13 +26,13 @@ go get -u github.com/roadrunner-server/roadrunner/v2025/lib ```go import ( - "github.com/roadrunner-server/roadrunner/v2025/lib" + "github.com/roadrunner-server/roadrunner/v3/lib" ) func main() { - overrides := []string{} // List of configuration overrides - plugins := lib.DefaultPluginsList() // List of RR plugins to enable - rr, err := lib.NewRR(".rr.yaml", overrides, plugins) + overrides := []string{} // List of configuration overrides + plugins := lib.DefaultPluginsList() // List of RR plugins to enable + rr, err := lib.NewRR(".rr.yaml", overrides, plugins) } ``` @@ -41,41 +42,39 @@ func main() { Here we use the default list of plugins. This is the same list of plugins you would get if you were to run `rr serve` with a stock RoadRunner binary. -You can, however, choose only the plugins you want and add your own private plugins as well: +You can select plugins and add your own plugin. Replace `example.com/my-plugin` with your module path. The omitted entries must include the dependencies required by the selected plugins: {% code title="main.go" %} ```go import ( - "github.com/roadrunner-server/roadrunner/v2025/lib" - httpPlugin "github.com/roadrunner-server/http/v5" - "github.com/roadrunner-server/resetter/v5" - "github.com/roadrunner-server/informer/v5" - "github.com/yourCompay/yourCusomPlugin" // compilation error here, used only as an example. + custom "example.com/my-plugin" + httpPlugin "github.com/roadrunner-server/http/v6" + "github.com/roadrunner-server/informer/v6" + "github.com/roadrunner-server/resetter/v6" + "github.com/roadrunner-server/roadrunner/v3/lib" ) func main() { - overrides := []string{ - "http.address=127.0.0.1:4444", // example override to set the HTTP address - "http.pool.num_workers=4", // example override of how to set the number of PHP workers - } // List of configuration overrides - - plugins := []interface{}{ - &informer.Plugin{}, - &resetter.Plugin{}, - // ... - &httpPlugin.Plugin{}, - // ... - &yourCustomPlugin.Plugin{}, - } - plugins := lib.DefaultPluginsList() // List of RR plugins to enable - rr, err := lib.NewRR(".rr.yaml", overrides, plugins) + overrides := []string{ + "http.address=127.0.0.1:4444", + "http.pool.num_workers=4", + } + + plugins := []any{ + &informer.Plugin{}, + &resetter.Plugin{}, + // ... + &httpPlugin.Plugin{}, + // ... + &custom.Plugin{}, + } + rr, err := lib.NewRR(".rr.yaml", overrides, plugins) } ``` {% endcode %} - ## Starting & stopping embedded RoadRunner Once everything is ready, we can start the RoadRunner instance: diff --git a/customization/events-bus.md b/customization/events-bus.md index 67766fe..23cc12d 100644 --- a/customization/events-bus.md +++ b/customization/events-bus.md @@ -31,34 +31,33 @@ import ( func foo() { // Get the (global) instance of the event bus. Make sure to // unsubscribe the event handler when you don't need it anymore: eh.Unsubscribe(id). - eh, id := events.Bus() + eh, id := events.NewEventBus() defer eh.Unsubscribe(id) // Create an events channel. ch := make(chan events.Event, 100) - // Subscribe to the events that fit your pattern (e.g., `http.EventJobOK`). - err := eh.SubscribeP(id, "http.EventJobOK", ch) + // Subscribe to worker errors from the HTTP plugin. + err := eh.SubscribeP(id, "http.EventWorkerError", ch) if err != nil { panic(err) } // Send an event to the channel. - eh.Send(events.NewEvent(events.EventJobOK, "http", "foo")) + eh.Send(events.NewEvent(events.EventWorkerError, "http", "worker failed")) // Receive an event from the channel. evt := <-ch - // evt.Message() -> "foo" + // evt.Message() -> "worker failed" // evt.Plugin() -> "http" - // evt.Type().String() -> "EventJobOK" + // evt.Type().String() -> "EventWorkerError" } ``` {% endcode %} {% hint style="info" %} -If you use only `eh.Send` events bus function, you don't need to unsubscribe, so, you may simplify the declaration to -the `eh, _ := events.Bus()`. +If you only send events, use `eh, _ := events.NewEventBus()`. You do not need to unsubscribe when you have no subscriptions. {% endhint %} ### Event Payload @@ -70,9 +69,9 @@ Let's take a closer look at each of these properties: | Property | Description | |-------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Message** | The message is a custom, user-defined string that can be used to convey additional information about the event. This can be useful for logging purposes or providing extra context to the subscriber when an event is received. In the examples provided earlier, the message is set to "foo" when sending the event. | +| **Message** | The message is a custom, user-defined string that can be used to convey information about the event. This can be useful for logging purposes or providing extra context to the subscriber when an event is received. In the examples provided earlier, the message is set to "worker failed" when sending the event. | | **Plugin** | The plugin property indicates the source plugin that raised the event. This information can be helpful in identifying the origin of the event and can be used for filtering or processing events based on their source. In the examples, the plugin is set to "http" when creating the event. | -| **Type** | The event type is a custom or RoadRunner (RR) defined identifier that categorizes the event. This identifier is used when subscribing to events and can help subscribers determine how to process the event or decide if they are interested in it. In the examples, the event type is set to `events.EventJobOK`. | +| **Type** | The event type is a custom or RoadRunner defined identifier that categorizes the event. This identifier is used when subscribing to events and can help subscribers determine how to process the event or decide if they are interested in it. In the examples, the event type is set to `events.EventWorkerError`. | When receiving an event, you can access these properties using the following methods: @@ -99,7 +98,7 @@ import ( ) func foo() { - eh, id := events.Bus() + eh, id := events.NewEventBus() defer eh.Unsubscribe(id) ch := make(chan events.Event, 100) @@ -110,18 +109,17 @@ func foo() { panic(err) } - eh.Send(events.NewEvent(events.EventJobOK, "http", "foo")) + eh.Send(events.NewEvent(events.EventWorkerError, "http", "worker failed")) evt := <-ch - // evt.Message() -> "foo" + // evt.Message() -> "worker failed" // evt.Plugin() -> "http" - // evt.Type().String() -> "EventJobOK" + // evt.Type().String() -> "EventWorkerError" } ``` {% endcode %} -In this example, we've changed the subscription pattern from `http.EventJobOK` to `http.*`, allowing the subscription to -match any event from the HTTP plugin. +The `http.*` subscription matches any event from the HTTP plugin, including `http.EventWorkerError`. ## How to implement a custom event @@ -212,7 +210,7 @@ import ( ) func foo() { - eh, id := events.Bus() + eh, id := events.NewEventBus() defer eh.Unsubscribe(id) ch := make(chan events.Event, 100) diff --git a/customization/jobs-driver.md b/customization/jobs-driver.md index 641390f..22d7a69 100644 --- a/customization/jobs-driver.md +++ b/customization/jobs-driver.md @@ -4,7 +4,7 @@ JOBS drivers are mini-plugins that are connected to the main JOBS plugin and ini ## Architecture -While initializing, the JOBS plugin searches for registered drivers by the `Constructor` interface. The `Constructor` and `Driver` (described below) interfaces are declared in the [RR API repository](https://github.com/roadrunner-server/api/blob/master/plugins/v4/jobs/driver.go). +The Jobs plugin discovers drivers through the `Constructor` interface. The [Jobs contracts](https://github.com/roadrunner-server/api-plugins/blob/master/jobs/driver.go) are in `github.com/roadrunner-server/api-plugins/v6/jobs`. See [plugin migration](plugin.md#v6-migration) for the shared import and logging changes. Constructor interface: @@ -16,9 +16,9 @@ type Constructor interface { // Name returns the name of the driver Name() string // DriverFromConfig constructs a driver (e.g. kafka, amqp) from the configuration using the provided configKey - DriverFromConfig(configKey string, queue Queue, pipeline Pipeline) (Driver, error) + DriverFromConfig(ctx context.Context, configKey string, queue Queue, pipeline Pipeline) (Driver, error) // DriverFromPipeline constructs a driver (e.g. kafka, amqp) from the pipeline. All configuration is provided by the pipeline - DriverFromPipeline(pipe Pipeline, queue Queue) (Driver, error) + DriverFromPipeline(ctx context.Context, pipe Pipeline, queue Queue) (Driver, error) } ``` @@ -51,8 +51,8 @@ type Driver interface { So every driver should implement the `Constructor` interface to be found by the JOBS plugin. Let's have a look at the methods included in the `Constructor` interface: 1. `Name() string`: This method should return a user-friendly name for the driver. It'll be used later in the pipelines `.driver` option. **It is an important option. The name here and name in the pipeline options should match.** -2. `DriverFromConfig(configKey string, queue Queue, pipeline Pipeline) (Driver, error)`: Returns a `Driver` implementation declared via configuration. RoadRunner, in turn, provides a configuration key (such as `jobs.pipelines.pipeline-name.driver-name.config`), the queue implementation to which messages are pushed, and the pipeline with all information about the pipeline. Later we will look at how to use this. -3. `DriverFromPipeline(pipe Pipeline, queue Queue) (Driver, error)`: Returns a `Driver` implementation declared via the RPC `jobs.Declare` call. It doesn't have configuration, but all info and configuration options are stored in the `pipeline` method argument. +2. `DriverFromConfig(ctx context.Context, configKey string, queue Queue, pipeline Pipeline) (Driver, error)`: Creates a driver from configuration. RoadRunner supplies the context, configuration key, queue, and pipeline. +3. `DriverFromPipeline(ctx context.Context, pipe Pipeline, queue Queue) (Driver, error)`: Creates a driver for an RPC `jobs.Declare` call. The pipeline contains its configuration. Pass the context to backend connection and setup operations. ### Initialization @@ -67,7 +67,7 @@ If required, you may use the `Configurer` plugin to unmarshal global driver conf ### How to create a driver for JOBS -All code from the tutorial is here: [link](https://github.com/roadrunner-server/samples/blob/master/plugins/jobs_driver/) +The [sample driver](https://github.com/roadrunner-server/samples/blob/master/plugins/jobs_driver/) shows the backend structure. The examples below use the v6 contracts. Use your own module path in place of `example.com/jobs-driver`. To create a driver for jobs, you need to create a plugin instance: @@ -77,51 +77,53 @@ To create a driver for jobs, you need to create a plugin instance: package jobs_driver //nolint:revive,stylecheck import ( - "github.com/roadrunner-server/api/v4/plugins/v4/jobs" - "github.com/roadrunner-server/errors" - "github.com/roadrunner-server/samples/plugins/jobs_driver/driver" - "go.uber.org/zap" + "context" + "log/slog" + + "example.com/jobs-driver/driver" + "github.com/roadrunner-server/api-plugins/v6/jobs" + "github.com/roadrunner-server/errors" ) const pluginName string = "my_awesome_driver" +var _ jobs.Constructor = (*Plugin)(nil) + type Configurer interface { - // UnmarshalKey takes a single key and unmarshal it into a Struct. - UnmarshalKey(name string, out any) error - // Has checks if a config section exists. - Has(name string) bool + UnmarshalKey(name string, out any) error + Has(name string) bool } type Logger interface { - NamedLogger(name string) *zap.Logger + NamedLogger(name string) *slog.Logger } type Plugin struct { - log *zap.Logger - cfg Configurer + log *slog.Logger + cfg Configurer } func (p *Plugin) Init(log Logger, cfg Configurer) error { - if !cfg.Has(pluginName) { - return errors.E(errors.Disabled) - } + if !cfg.Has(pluginName) { + return errors.E(errors.Disabled) + } - p.log = log.NamedLogger(pluginName) - p.cfg = cfg - return nil + p.log = log.NamedLogger(pluginName) + p.cfg = cfg + return nil } func (p *Plugin) Name() string { - return pluginName + return pluginName } -func (p *Plugin) DriverFromConfig(configKey string, pq jobs.Queue, pipeline jobs.Pipeline) (jobs.Driver, error) { - return driver.FromConfig(configKey, p.log, p.cfg, pipeline, pq) +func (p *Plugin) DriverFromConfig(ctx context.Context, configKey string, pq jobs.Queue, pipeline jobs.Pipeline) (jobs.Driver, error) { + return driver.FromConfig(ctx, configKey, p.log, p.cfg, pipeline, pq) } -func (p *Plugin) DriverFromPipeline(pipe jobs.Pipeline, pq jobs.Queue) (jobs.Driver, error) { - return driver.FromPipeline(pipe, p.log, p.cfg, pq) -} +func (p *Plugin) DriverFromPipeline(ctx context.Context, pipe jobs.Pipeline, pq jobs.Queue) (jobs.Driver, error) { + return driver.FromPipeline(ctx, pipe, p.log, p.cfg, pq) +} ``` {% endcode %} @@ -130,9 +132,10 @@ This is a simple representation of the RR plugin. It is called driver because it Keep in mind the plugin's name. JOBS plugin will send the following data to the `Constructor` interface methods: + 1. If declared via configuration (`.rr.yaml`) - `configKey`, you may use that key to unmarshal the configuration section related solely to this driver. If the driver was declared via `jobs.Declare` RPC method, all configuration options would be stored in the `jobs.Pipeline` interface. -2. `jobs.Queue`: Priority-Queue, used to push the messages and later process by the PHP workers. +2. `jobs.Queue`: Priority-Queue, used to push the messages and later process by the PHP workers. 3. All other things like logger, `Configurer` plugin which will be used to get the values from the `.rr.yaml` configuration you may pass if you need them from the driver's root (e.g.: `p.log`). Now, let's see the simplified `Driver` implementation: @@ -143,55 +146,54 @@ Now, let's see the simplified `Driver` implementation: package driver import ( - "context" + "context" + "log/slog" - "github.com/roadrunner-server/api/v4/plugins/v4/jobs" - "go.uber.org/zap" + "github.com/roadrunner-server/api-plugins/v6/jobs" ) var _ jobs.Driver = (*Driver)(nil) type Configurer interface { - // UnmarshalKey takes a single key and unmarshal it into a Struct. - UnmarshalKey(name string, out any) error - // Has checks if a config section exists. - Has(name string) bool + UnmarshalKey(name string, out any) error + Has(name string) bool } type Driver struct { + queue jobs.Queue } -func FromConfig(configKey string, log *zap.Logger, cfg Configurer, pipeline jobs.Pipeline, pq jobs.Queue) (*Driver, error) { - return &Driver{}, nil +func FromConfig(ctx context.Context, configKey string, log *slog.Logger, cfg Configurer, pipeline jobs.Pipeline, pq jobs.Queue) (*Driver, error) { + return &Driver{queue: pq}, nil } // FromPipeline initializes consumer from pipeline -func FromPipeline(pipeline jobs.Pipeline, log *zap.Logger, cfg Configurer, pq jobs.Queue) (*Driver, error) { - return &Driver{}, nil +func FromPipeline(ctx context.Context, pipeline jobs.Pipeline, log *slog.Logger, cfg Configurer, pq jobs.Queue) (*Driver, error) { + return &Driver{queue: pq}, nil } func (d *Driver) Push(ctx context.Context, job jobs.Message) error { - return nil + return nil } func (d *Driver) Run(ctx context.Context, p jobs.Pipeline) error { - return nil + return nil } func (d *Driver) State(ctx context.Context) (*jobs.State, error) { - return nil, nil + return &jobs.State{}, nil } func (d *Driver) Pause(ctx context.Context, p string) error { - return nil + return nil } func (d *Driver) Resume(ctx context.Context, p string) error { - return nil + return nil } func (d *Driver) Stop(ctx context.Context) error { - return nil + return nil } ``` @@ -204,38 +206,38 @@ Remember the following things: 3. For pipelines declared via the `jobs.Declare` RPC call, the `jobs.Resume` method should be called instead. ### Pushing jobs into the priority queue -To push a job into the priority queue, you need to slightly transform it to add `Ack`, `Nack`, etc. methods to it. -All interfaces are in the [RR API repository](https://github.com/roadrunner-server/api/blob/master/plugins/v4/jobs/job.go). Let's have a look at the `Job` interface. +To push a job into the priority queue, you need to slightly transform it to add `Ack`, `Nack`, etc. methods to it. +The [Job interface](https://github.com/roadrunner-server/api-plugins/blob/master/jobs/job.go) includes `jobs.Item`: {% code title="job.go" %} ```go +import "github.com/roadrunner-server/api-plugins/v6/jobs" + // Job represents a binary heap item type Job interface { - pq.Item - // Ack acknowledges the item after processing - Ack() error - // Nack discards the item - Nack() error - // NackWithOptions discards the item with an optional requeue flag - NackWithOptions(requeue bool, delay int) error - // Requeue puts the message back to the queue with an optional delay - Requeue(headers map[string][]string, delay int) error - // Body returns the payload associated with the item - Body() []byte - // Context returns any meta-information associated with the item - Context() ([]byte, error) - // Headers return the metadata for the item - Headers() map[string][]string + jobs.Item + // Ack acknowledges the item after processing + Ack() error + // Nack discards the item + Nack() error + // NackWithOptions discards the item with an optional requeue flag + NackWithOptions(requeue bool, delay int) error + // Requeue puts the message back to the queue with an optional delay + Requeue(headers map[string][]string, delay int) error + // Body returns the payload associated with the item + Body() []byte + // Context returns any meta-information associated with the item + Context() ([]byte, error) + // Headers return the metadata for the item + Headers() map[string][]string } ``` {% endcode %} -The `Job` interface also includes the `pq.Item` interface to satisfy a minimal priority queue requirement. -You may add this (`pq.Item`) interface to any interface and benefit from RoadRunner's priority queue. -So, our driver's `Push` method would be updated as follows: +`jobs.Item` requires `ID() string`, `GroupID() string`, and `Priority() int64`. Update the driver's `Push` method to insert a `jobs.Job` into its queue: {% code title="driver.go" %} diff --git a/customization/middleware.md b/customization/middleware.md index 51bca1b..f24fc40 100644 --- a/customization/middleware.md +++ b/customization/middleware.md @@ -63,9 +63,7 @@ func (p *Plugin) Name() string { {% endcode %} {% hint style="info" %} -Middleware must correspond to the -following [interface](https://github.com/roadrunner-server/http/blob/master/common/interfaces.go#L33) and -be [named](https://github.com/roadrunner-server/endure/blob/master/container.go#L47). +The plugin must implement the [HTTP middleware interface](https://github.com/roadrunner-server/http/blob/master/api/interfaces.go#L36-L40), including `Name() string`. See [plugin migration](plugin.md#v6-migration) for the v6 imports and shared contracts. {% endhint %} ## gRPC @@ -75,12 +73,9 @@ authentication, rate limiting, and logging. To create a custom interceptor for gRPC requests in RoadRunner, follow these steps: -1. Define a struct that implements the `Init()`, `Interceptor()`, and `Name()` methods. The `Init()` method is called - when the plugin is initialized, the `Interceptor()` method is called for each incoming gRPC request, and the `Name()` - method returns the name of the middleware/plugin. +1. Define a struct with `Init()`, `UnaryServerInterceptor()`, and `Name()` methods. `UnaryServerInterceptor()` returns the interceptor used for incoming requests. -2. In the `Interceptor()` method, perform any necessary processing on the incoming gRPC request, and then call the next - interceptor in the pipeline using the `handler(ctx, req)` method. +2. Process the request in the returned function, then call `handler(ctx, req)` to continue execution. {% hint style="warning" %} RoadRunner supports `gRPC` interceptors since version `v2023.2.0`. @@ -94,7 +89,9 @@ Here is an example: package middleware import ( - "net/http" + "context" + + "google.golang.org/grpc" ) const PluginName = "interceptor" @@ -107,7 +104,9 @@ func (p *Plugin) Init() error { } func (p *Plugin) UnaryServerInterceptor() grpc.UnaryServerInterceptor { - // Do something and return interceptor + return func(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) { + return handler(ctx, req) + } } // Middleware/plugin name. @@ -119,37 +118,41 @@ func (p *Plugin) Name() string { {% endcode %} {% hint style="info" %} -Interceptor must correspond to the -following [interface](https://github.com/roadrunner-server/grpc/blob/master/common/interfaces.go#L14) and -be [named](https://github.com/roadrunner-server/endure/blob/master/container.go#L47). +The plugin must implement the [gRPC interceptor interface](https://github.com/roadrunner-server/grpc/blob/master/api/interfaces.go#L16-L19), including `Name() string`. {% endhint %} -You can find a lot of examples here: [link](https://github.com/grpc-ecosystem/go-grpc-middleware). Keep in mind that, at -the moment, RR supports only `UnaryServerInterceptor` gRPC interceptors. +See [unary gRPC interceptors](../grpc/interceptors.md) for configuration and [go-grpc-middleware](https://github.com/grpc-ecosystem/go-grpc-middleware) for interceptor examples. ## PSR7 Attributes -PSR7 attributes are a way of attaching metadata to an incoming HTTP request or response. The PSR7 specification defines -a standard interface for HTTP messages, which includes the ability to set and retrieve attributes on both requests and -responses. +PSR-7 server request attributes hold metadata for request processing. They are not response attributes. Attributes can be used to store any kind of metadata that might be useful for processing the request or response. For example, you might use attributes to store information about the authenticated user, the user's IP address, or any other custom data that you want to attach to the request. -The `Psr\Http\Message\ServerRequestInterface->getAttributes()` method can be used to retrieve attributes from an incoming HTTP request, while the `ResponseInterface->withAttribute()` method can be used to set attributes on an outgoing HTTP response. +Use `Psr\Http\Message\ServerRequestInterface::getAttributes()` to read the attributes in PHP. You can safely pass values to a PHP application and retrieve attributes on the PHP side using the `Psr\Http\Message\ServerRequestInterface->getAttributes()` method through the [attributes](https://github.com/roadrunner-server/http/blob/master/attributes/attributes.go) package: {% code title="middleware.go" %} ```go -func (s *Service) Middleware(next http.HandlerFunc) http.HandlerFunc { - return func(w http.ResponseWriter, r *http.Request) { +import ( + "net/http" + + "github.com/roadrunner-server/http/v6/attributes" +) + +func (p *Plugin) Middleware(next http.Handler) http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { r = attributes.Init(r) - attributes.Set(r, "key", "value") + if err := attributes.Set(r, "key", "value"); err != nil { + http.Error(w, "cannot set request attribute", http.StatusInternalServerError) + return + } next.ServeHTTP(w, r) - } + }) } ``` @@ -162,37 +165,11 @@ the `getAttributes()` method. For example, the `nyholm/psr7` package provides a ## Registering middleware -You must register this service in the -[container/plugins.go](https://github.com/roadrunner-server/roadrunner/blob/master/container/plugins.go) file to -properly resolve dependencies: +Include the middleware plugin in your RoadRunner binary. Follow [Building RoadRunner](build.md) for the Velox configuration and build steps. -{% code title="plugin.go" %} +If you maintain the Go entry point yourself, import the middleware module and add `&middleware.Plugin{}` to the existing plugin list in [container/plugins.go](https://github.com/roadrunner-server/roadrunner/blob/master/container/plugins.go). Keep the other required plugins in that list. -```go -package roadrunner - -import ( - "middleware" -) - -func Plugins() []any { - return []any { - // ... - - // middleware - &middleware.Plugin{}, - - // ... -} -``` - -{% endcode %} - -Or you can use the Velox tool to [build the RR binary](./build.md). - -You should also make sure you configure the middleware to be used via -the [config or the command line](../intro/config.md). Otherwise, the plugin will be loaded, but the middleware will not -be used with incoming requests. +Then add the value returned by `Name()` to `http.middleware`. A plugin included in the binary does not handle HTTP requests until it is selected in this list. {% code title=".rr.yaml" %} @@ -209,4 +186,3 @@ http: ### Writing a middleware for HTTP {% embed url="https://www.youtube.com/watch?v=f5fUSYaDKxo" %} - diff --git a/customization/plugin.md b/customization/plugin.md index 6ca18f8..e9c0ec9 100644 --- a/customization/plugin.md +++ b/customization/plugin.md @@ -7,18 +7,60 @@ implementation with automatic method injection. **To create a custom plugin, you can follow these steps:** - Define a struct with a public `Init` method that returns an error value. -- Implement the `Service` interface in your struct to provide the `Serve` and `Stop` methods. +- Implement `Serve` and `Stop` only if the plugin starts a service. - Request dependencies using their respective interfaces and inject them using the Endure container. -- Register your plugin with RoadRunner by creating a custom version of the `main.go` file and [building it](build.md). +- Register the plugin in the RoadRunner container and [build the binary](build.md). Below you can find more information about the plugin interface, how to define a plugin, and how to access other plugins. +## v6 migration + +Use the module versions selected by the RoadRunner build. RoadRunner v3 uses v6 plugin modules and `endure/v2`. The Endure lifecycle and dependency injection interfaces stay the same. The RoadRunner library module is `roadrunner/v3`. + +The API repositories now have separate roles. [api](https://github.com/roadrunner-server/api) contains protobuf source, not a Go module. [api-go](https://github.com/roadrunner-server/api-go) contains generated Go bindings. [api-plugins](https://github.com/roadrunner-server/api-plugins) contains Go plugin contracts, not RPC messages. + +All import paths in this table start with `github.com/roadrunner-server/`: + +| Previous import | v6 plugin import | +| --- | --- | +| `roadrunner/v2025/` | `roadrunner/v3/` | +| `/v5` | `/v6` | +| `pool/` | `pool/v2/` | +| `goridge/v3/pkg/` | `goridge/v4/pkg/` | +| `api/v4/build//v1` | `api-go/v6//v1` | +| `api/v4/build/lock/v1beta1` | `api-go/v6/lock/v1` | +| `api/v4/build/status/v1beta1` | `api-go/v6/status/v1` | +| `api/v4/plugins/v4/jobs` | `api-plugins/v6/jobs` | +| `api/v4/plugins/v1/{kv,lock,logger,status}` | `api-plugins/v6/{kv,lock,logger,status}` | +| `api/v4/plugins/v4/priority_queue` | `api-plugins/v6/priority_queue` | + +Generated imports have no `build/` segment. For example: + +```go +import jobsv1 "github.com/roadrunner-server/api-go/v6/jobs/v1" +``` + +Update implementations, local interfaces, and call sites together: + +- **Logging:** use `logger.Named` from `api-plugins/v6/logger` or a local interface with `NamedLogger(string) *slog.Logger`. The old `logger.Log` interface is removed. Pool constructors, worker factories, and logger options also take `*slog.Logger`. Replace `log.Info("started", zap.String("plugin", name))` with `log.Info("started", "plugin", name)`. Slog has no `Fatal`, `Panic`, or `DPanic` methods. +- **Jobs:** add a leading `context.Context` to `DriverFromConfig` and `DriverFromPipeline`. Calls become `constructor.DriverFromConfig(ctx, key, queue, pipeline)` and `constructor.DriverFromPipeline(ctx, pipeline, queue)`. Existing `Driver` methods already take contexts. See the [Jobs driver tutorial](jobs-driver.md). +- **KV:** every `Storage` method now takes a leading context, including `Stop`. Update calls such as `storage.Get(ctx, key)`, `storage.Set(ctx, items...)`, and `storage.Stop(ctx)`. Construction becomes `constructor.KvFromConfig(ctx, key)`. Pass the context to backend operations. See the [KV contracts](https://github.com/roadrunner-server/api-plugins/blob/master/kv/interface.go). +- **Queues:** lock queue signatures use `lock.Item` and `[]lock.Item`, not the old priority-queue package's named interface. Jobs defines its own `jobs.Item`. Both retain `ID`, `GroupID`, and `Priority`. Update queue type arguments and method signatures. The `priority_queue` package now declares the Go package name `priorityqueue`. +- **Pool defaults:** direct calls to `DynamicAllocationOpts.InitDefaults()` must pass the base worker count: `InitDefaults(cfg.NumWorkers)`. `WorkerWatcher.Watch(workers)` has no return value. Remove checks for its former error result. See the [worker pool](../php/pool.md) for execution and shutdown behavior. +- **Goridge frames:** `Header()` and `Payload()` return frame-owned slices. Copy data that must survive frame reset or reuse. `From`, `ReadFrame`, and `ReadHeader` retain the supplied buffers; later writes can change those buffers. See the [frame API](https://github.com/roadrunner-server/goridge/blob/master/pkg/frame/frame.go). +- **Removed helpers:** replace `proxy.Cidrs` from `proxy_ip_parser` with `[]*net.IPNet`. Resetter no longer exposes `Plugin.Reset(string)`; its `resetter.Reset` RPC remains available. OTEL removes `HTTPHandler` and `TemporalHandler`; use `Plugin.Middleware` and `Plugin.WorkerInterceptor`. Temporal no longer exposes `ResetAP`; normal activity-worker replacement is handled by the pool. + +### DTO compatibility + +`api-go/v6` uses the v1 message set. The `lock/v1` package defines `Request` and `Response`. Regenerated PHP lock DTOs use `RoadRunner\Lock\DTO\V1`. The lock field numbers and types are unchanged; custom descriptor or protobuf `Any` users must account for the package-name change. + +Relocation alone does not change the retained HTTP or Jobs wire fields and does not require a PHP worker-loop rewrite. RPC still uses Goridge and Go `net/rpc`, not Connect. See [RPC compatibility](../php/rpc.md#v6-compatibility) for the MessagePack change. + +Direct Centrifugo DTO users have separate changes. Use typed fields instead of `Command.id/method/params` and `Reply.id/result`, which are removed. The `RateLimit` RPC and its types are removed. `UpdatePushStatusRequest.uid` becomes `analytics_uid` at the same string field number 1; update generated accessors and JSON names. The proxy bindings add experimental `NotifyCacheEmpty`; a custom server must implement it or embed the generated unimplemented server. The RoadRunner plugin forwards this event to PHP; update handlers and DTOs as described in [Centrifuge](../plugins/centrifuge.md). + ## Interface -RoadRunner plugins are implemented using the `Service` interface, which provides the `Serve` and `Stop` methods for -starting and stopping the plugin. Additionally, plugins can implement other optional interfaces -like `Named`, `Provider`, `Weighted`, and `Collector`. These interfaces enable plugins to provide dependencies to other -plugins, define their weight in the plugin's topology, and collect plugins that implement specific interfaces. +A plugin that starts a service implements `Service`, which provides `Serve` and `Stop`. Middleware and other plugins that do not start a service do not need those methods. Optional interfaces such as `Named`, `Provider`, `Weighted`, and `Collector` provide names, dependencies, initialization weights, and dependency collection. **Here is an example:** @@ -144,7 +186,7 @@ interfaces, and a plugin implementing this interface should be registered in RR' package custom import ( - "go.uber.org/zap" + "log/slog" ) type Configurer interface { // <-- config plugin implements @@ -155,7 +197,7 @@ type Configurer interface { // <-- config plugin implements } type Logger interface { // <-- logger plugin implements - NamedLogger(name string) *zap.Logger + NamedLogger(name string) *slog.Logger } type Service struct{} @@ -191,7 +233,8 @@ custom: package custom import ( - "go.uber.org/zap" + "log/slog" + "github.com/roadrunner-server/errors" ) @@ -205,11 +248,12 @@ type Configurer interface { // <-- config plugin implements } type Logger interface { // <-- logger plugin implements - NamedLogger(name string) *zap.Logger + NamedLogger(name string) *slog.Logger } type Plugin struct { cfg *Config + log *slog.Logger } // Init plugin @@ -220,6 +264,8 @@ func (s *Plugin) Init(cfg Configurer, log Logger) error { return errors.E(op, errors.Disabled) } + s.log = log.NamedLogger(PluginName) + // unmarshal initial configuration err := cfg.UnmarshalKey(PluginName, &s.cfg) if err != nil { @@ -236,7 +282,7 @@ func (s *Plugin) Init(cfg Configurer, log Logger) error { {% endcode %} -### Configuration +### Configuration type {% code title="config.go" %} @@ -259,8 +305,9 @@ func (cfg *Config) InitDefaults() { ## Serving -Create `Serve` and `Stop` methods in your structure to let RoadRunner start and stop your service. You may also use the -context from the `Stop` method to let RR force your plugin to stop after a specified timeout in the configuration. +Endure calls `Serve()` synchronously. Start blocking work in a goroutine owned by the plugin, then return an error channel promptly. A blocking `Serve()` prevents the remaining plugins from starting and prevents container shutdown. + +`Stop(ctx)` must stop the service cooperatively and respect the context deadline. The `endure.grace_period` setting determines this deadline. Endure does not terminate plugin goroutines when the deadline expires. {% code title=".rr.yaml" %} @@ -277,6 +324,8 @@ endure: ### Plugin +This example starts a local HTTP server. Its `Stop` method uses `http.Server.Shutdown(ctx)` to wait for active requests until the context expires. + {% code title="plugin.go" %} ```go @@ -284,37 +333,41 @@ package custom import ( "context" + "net/http" ) -type Plugin struct{} +type Plugin struct { + server *http.Server +} + +func (s *Plugin) Init() error { + s.server = &http.Server{ + Addr: "127.0.0.1:8088", + Handler: http.NotFoundHandler(), + } + return nil +} func (s *Plugin) Serve() chan error { - const op = errors.Op("custom_plugin_serve") errCh := make(chan error, 1) - err := s.DoSomeWork() - if err != nil { - errCh <- errors.E(op, err) - return errCh - } + go func() { + if err := s.server.ListenAndServe(); err != nil && err != http.ErrServerClosed { + errCh <- err + } + }() - return nil + return errCh } func (s *Plugin) Stop(ctx context.Context) error { - return s.stopServing() -} - -func (s *Plugin) DoSomeWork() error { - return nil + return s.server.Shutdown(ctx) } ``` {% endcode %} -The `Serve` method is thread-safe. It runs in a separate goroutine managed by the `Endure` container. -One note is that you should unblock it when calling `Stop` on the container. -Otherwise, the service will be killed after the timeout (which can be set in Endure). +`http.ErrServerClosed` is the normal result of HTTP shutdown, so the example does not send it to Endure. Other errors are sent through the buffered channel. Endure does not make plugin code thread-safe; the plugin must synchronize access to shared state. ## Collecting dependencies at runtime @@ -323,7 +376,7 @@ This is very useful for middlewares or extending plugins with additional functio Let's create an HTTP middleware: -1. Declare a required interface +Declare the required interface: {% code title="middleware.go" %} @@ -336,13 +389,14 @@ import ( // Middleware interface type Middleware interface { - Middleware(f http.Handler) http.HandlerFunc + Middleware(f http.Handler) http.Handler + Name() string } ``` {% endcode %} -2. Implement the `Collects` Endure interface in the plugin where you want to have these dependencies at runtime. +Implement the `Collects` interface in the plugin that accepts the middleware: {% code title="middleware.go" %} @@ -372,15 +426,13 @@ Important notes: ## RPC Methods -Extending your plugin with RPC methods does not change the plugin at all. The only thing you have to do is to create a -file with RPC methods (let's call it `rpc.go`) and add all RPC methods for the plugin without modifying the plugin -itself. +Expose an RPC receiver through `RPC() any`. Its exported methods use the Go `net/rpc` signature: an input argument, a reply pointer, and an `error` result. Do not add a context argument to these RPC methods when updating the Jobs or KV contracts. **Example based on the `informer` plugin:** Suppose we have created a file `rpc.go`. The next step is to create a structure: -1. Create a structure: (logger is optional) +Create the receiver type: {% code title="rpc.go" %} @@ -388,18 +440,20 @@ Suppose we have created a file `rpc.go`. The next step is to create a structure: package custom import ( - "go.uber.org/zap" + "log/slog" ) type rpc struct { plugin *Plugin - log *zap.Logger + log *slog.Logger } ``` {% endcode %} -2. Create a method that you want to expose: +Add an exported method: + +{% code title="rpc.go" %} ```go package custom @@ -414,7 +468,7 @@ func (s *rpc) Hello(input string, output *string) error { {% endcode %} -3. Create a method called `RPC` that accepts nothing and returns `any`: +Add `RPC()` to the plugin: {% code title="rpc.go" %} @@ -422,7 +476,7 @@ func (s *rpc) Hello(input string, output *string) error { package custom func (p *Plugin) RPC() any { - return &rpc{srv: p, log: p.log} + return &rpc{plugin: p, log: p.log} } ``` diff --git a/experimental/experimental.md b/experimental/experimental.md index d89df57..f45523d 100644 --- a/experimental/experimental.md +++ b/experimental/experimental.md @@ -26,27 +26,13 @@ Or: {% endcode %} -## List of experimental features +## Configuration Files -### Support for loading [`envfiles`](https://github.com/roadrunner-server/roadrunner/issues/1077) in the `.rr.yaml`: `[>= v2023.3.5]` +The `envfile` and `include` settings work without the experimental flag. See [Configuration](../plugins/config.md) for these settings. -In `v2023.3.5`, we added experimental support for loading `envfiles` in the `.rr.yaml` configuration file. -`.env` file should be in the same directory as the `.rr.yaml` file. +## HTTP/3 Server -Sample `.rr.yaml` file: - -{% code title=".rr.yaml" %} - -```yaml .rr.yaml -version: "3" -envfile: .env -``` - -{% endcode %} - -### Support for the HTTP/3 server: `[>=2023.3.8]` - -In `v2023.3.8`, we added experimental support for an HTTP/3 server. It can work with the ACME provider to generate certificates for the HTTP/3 server automatically. +Set `http.http3.cert` and `http.http3.key` to existing certificate and private key files. The HTTP/3 listener requires these files at startup. Start RoadRunner with `-e` to enable this listener. Sample `.rr.yaml` file: @@ -71,39 +57,6 @@ http: {% endcode %} -Or if you use an ACME provider: - -{% code title=".rr.yaml" %} - -```yaml -version: "3" - -server: - command: "php worker.php" - relay: pipes - -http: - address: 127.0.0.1:15389 - pool: - num_workers: 2 - http3: - address: 127.0.0.1:34555 - key: "localhost+2-key.pem" - cert: "localhost+2.pem" - ssl: - acme: - certs_dir: rr_le_certs - email: you-email-here@email - alt_http_port: 80 - alt_tlsalpn_port: 443 - challenge_type: http-01 - use_production_endpoint: false - domains: - - your-cool-domains.here -``` - -{% endcode %} - You can also generate test certificates manually and use them in the configuration file. To do that, use [mkcert](https://github.com/FiloSottile/mkcert) or [certbot](https://certbot.eff.org/): {% code %} diff --git a/grpc/grpc.md b/grpc/grpc.md index b86d2d8..d8bfb45 100644 --- a/grpc/grpc.md +++ b/grpc/grpc.md @@ -7,6 +7,8 @@ It consists of two main parts: 1. **protoc-plugin `protoc-gen-php-grpc`:** This is a plugin for the protoc compiler that generates PHP code from a gRPC service definition file (`.proto`). It generates PHP classes that correspond to the service definition and message types. These classes provide an interface for handling incoming gRPC requests and sending responses back to the client. 2. **gRPC server:** This is a server that starts PHP workers and listens for incoming gRPC requests. It receives requests from gRPC clients, proxies them to the PHP workers, and sends the responses back to the client. The server is responsible for managing the lifecycle of the PHP workers and ensuring that they are available to handle requests. +For custom gRPC interceptor plugins, see [Interceptors](./interceptors.md). + ## Protoc-plugin The first step is to define a `.proto` file that describes the gRPC service and messages that your PHP application will handle. @@ -49,61 +51,22 @@ The `php_namespace` and `php_metadata_namespace` options allow you to specify th ### Generating PHP code -After defining the proto file, you need to generate the PHP files using the `protoc` compiler and the `protoc-gen-php-grpc` plugin. You can install the plugin binary using Composer or download a pre-built binary from the GitHub releases page. - -{% tabs %} - -{% tab title="Prebuilt Binary" %} - -The simplest way to get the latest version of `protoc-gen-php-grpc` plugin is to download one of the pre-built release binaries on the GitHub [releases page](https://github.com/roadrunner-server/roadrunner/releases). - -Just download the appropriate archive from the release page and extract it into your desired application directory. - -{% endtab %} - -{% tab title="Composer" %} - -If you use Composer to manage your PHP dependencies, you can install the `spiral/roadrunner-cli` package to download the latest version of `protoc-gen-php-grpc` plugin to your project's root directory. - -**Install the package** - -{% code %} +Use `protoc` `36.1` and Go `1.27.1`. Install the RoadRunner generator from its pinned source revision: ```bash -composer require spiral/roadrunner-cli +go install github.com/roadrunner-server/grpc/protoc_plugins/v5/protoc-gen-php-grpc@4965bf6d7e43 ``` -{% endcode %} - -And run the following command to download the latest version of the plugin - -{% code %} - -```bash -./vendor/bin/rr download-protoc-binary -``` - -{% endcode %} - -Server binary will be available at the root of your project. - -{% hint style="warning" %} -PHP's extensions `php-curl` and `php-zip` are required. Check with `php --modules` your installed extensions. -{% endhint %} +Add the Go binary installation directory to `PATH`. Create the `generated` directory before running `protoc`. -{% endtab %} - -{% endtabs %} - -Once the plugin is installed, you can use the `protoc` command to compile the proto file into PHP files. +The PHP generator uses the separate `protoc_plugins/v5` module. Its module version is independent of the gRPC plugin v6. **Here's an example command:** {% code %} ```bash -protoc --plugin=protoc-gen-php-grpc \ - --php_out=./generated \ +protoc --php_out=./generated \ --php-grpc_out=./generated \ proto/helloworld.proto ``` @@ -147,31 +110,19 @@ Here's an example of a `buf.yaml` file: ```yaml version: v2 -deps: - - buf.build/googleapis/googleapis:fb98f92554c17ec159a0b35ea8ffca71bac14385 - -name: buf.build// +modules: + - path: proto lint: use: - - DEFAULT - except: - - FIELD_NOT_REQUIRED - - PACKAGE_NO_IMPORT_CYCLE + - STANDARD breaking: use: - FILE - except: - - EXTENSION_NO_DELETE - - FIELD_SAME_DEFAULT ``` {% endcode %} -Note that you need to optionally replace `` and `` with your organization and project names created on the [BUF](https://login.buf.build/u/signup) website. - -In the `deps` section, you can specify the dependencies that your `.proto` file relies on. In this example, we're using a dependency from the Google APIs repository. - -Also, you may configure the linting and breaking changes rules. +The module contains the `.proto` files in `proto/`. If your files import external schemas, add their Buf modules under `deps`, run `buf dep update`, and keep the resulting `buf.lock` with your source files. Here's an example of a `buf.gen.yaml` file: @@ -180,31 +131,31 @@ Here's an example of a `buf.gen.yaml` file: ```yaml version: v2 plugins: - - remote: buf.build/protocolbuffers/php:v26.1 + - remote: buf.build/protocolbuffers/php:v36.1 out: generated/php - - remote: buf.build/community/roadrunner-server-php-grpc:v4.8.0 + - remote: buf.build/community/roadrunner-server-php-grpc:v5.3.0 out: generated/php - - remote: buf.build/protocolbuffers/go:v1.32.0 + - remote: buf.build/protocolbuffers/go:v1.36.12 out: generated/go opt: paths=source_relative - - remote: buf.build/grpc/go:v1.3.0 + - remote: buf.build/grpc/go:v1.6.2 out: generated/go opt: - paths=source_relative - require_unimplemented_servers=false - - remote: buf.build/grpc/python:v1.63.0 + - remote: buf.build/grpc/python:v1.83.1 out: generated/python - - remote: buf.build/protocolbuffers/python + - remote: buf.build/protocolbuffers/python:v36.1 out: generated/python - - remote: buf.build/protocolbuffers/pyi + - remote: buf.build/protocolbuffers/pyi:v36.1 out: generated/python ``` {% endcode %} -As you can see, the `buf.gen.yaml` file specifies the plugins that will be used to generate the code. In this example, we're using the `buf.build/community/roadrunner-server-php-grpc` plugin to generate PHP gRPC services. +Run `buf generate` from the directory that contains `buf.yaml` and `buf.gen.yaml`. The configuration pins each generator version and uses the RoadRunner plugin to generate PHP gRPC services. -Also, you may generate code for other languages like Go and Python. +For Buf output, set the `GRPC\\` Composer autoload path to `generated/php/GRPC`. The example also generates Go and Python code. ## PHP Client @@ -343,7 +294,7 @@ grpc: {% endtabs %} {% hint style="info" %} -You can define multiple proto files in the `proto` section. From [>=2025.1.9], one can use glob patterns to specify multiple proto files. +You can list multiple proto files or glob patterns in the `proto` section. Glob patterns have been supported since RoadRunner v2025.1.9. {% endhint %} After configuring the server, you can start it using the following command: @@ -384,11 +335,11 @@ Message json (type ? to see defaults): {"name":"Roadrunner"} } ``` -On the PHP side, you should see that the successful request has been logged: +With debug logging enabled, RoadRunner writes records such as: -``` -2025-03-28T10:04:06+0000 DEBUG server req-resp mode {"pid": 81551} -2025-03-28T10:04:06+0000 DEBUG grpc method was called successfully {"method": "/helloworld.Greeter/SayHello", "start": "2025-03-28T10:04:06+0000", "elapsed": 0} +```log +time=2026-10-06T12:00:00.000Z level=DEBUG msg="req-resp mode" logger=server pid=81551 +time=2026-10-06T12:00:00.000Z level=DEBUG msg="method was called successfully" logger=grpc method=/helloworld.Greeter/SayHello start=2026-10-06T12:00:00.000Z elapsed=0 ``` Congratulations on being able to communicate over gRPC to PHP using Roadrunner! @@ -419,11 +370,13 @@ grpc: key: "server-key.pem" cert: "server-cert.pem" root_ca: "rootCA.pem" - client_auth_type: request_client_cert + client_auth_type: require_and_verify_client_cert ``` {% endcode %} +`require_and_verify_client_cert` requires a client certificate signed by a trusted CA. `request_client_cert` only requests a certificate; it does not require or verify one. + Options for the `client_auth_type` are: - `request_client_cert` @@ -432,9 +385,29 @@ Options for the `client_auth_type` are: - `require_and_verify_client_cert` - `no_client_certs` +## Server reflection + +The gRPC plugin enables server reflection on the gRPC listen port. Both the v1 and v1alpha reflection APIs are available without an enable flag. + +Without a descriptor registry, reflection lists registered services but cannot return the file and message descriptors for PHP services. Configure the bundled [Protoreg plugin](./protoreg.md#server-reflection) with the same service definitions used by `grpc.proto` to provide those descriptors. See [gRPC PR #141](https://github.com/roadrunner-server/grpc/pull/141). + +For a listener without TLS, list services with: + +{% code %} + +```bash +grpcurl -plaintext 127.0.0.1:9001 list +``` + +{% endcode %} + +{% hint style="warning" %} +Reflection uses streaming RPCs. Authentication in `grpc.interceptors` applies only to unary RPCs and does not protect reflection. Restrict network access to the gRPC port or require [verified client certificates](#mtls). The plugin has no configuration option to disable reflection. +{% endhint %} + ## Health Checking -RoadRunner automatically registers a [`grpc.health.v1.Health`](https://github.com/grpc/grpc/blob/master/doc/health-checking.md) service on the same gRPC listen port. This is the standard gRPC health checking protocol β€” no additional configuration is required. +RoadRunner automatically registers a [`grpc.health.v1.Health`](https://github.com/grpc/grpc/blob/master/doc/health-checking.md) service on the same gRPC listen port. This is the standard gRPC health checking protocol. No additional configuration is required. ### Available methods @@ -489,7 +462,7 @@ grpc: # This option is required listen: "tcp://127.0.0.1:9001" - # Proto file to use, multiply files supported [SINCE 2.6]. As of [2023.1.4], wildcards are allowed in the proto field and in version [>=2025.1.9] glob patterns are supported. + # Multiple proto files: since v2.6. Wildcards: since v2023.1.4. Glob patterns: since v2025.1.9. # # This option is required proto: @@ -548,8 +521,8 @@ grpc: # This option is optional. Default value: infinity. max_connection_age: 0s - # MaxConnectionAgeGrace is an additive period after MaxConnectionAge after - # which the connection will be forcibly closed. + # Time allowed for active RPCs to finish after max_connection_age. + # Zero or omitted means unlimited grace. max_connection_age_grace: 0s # Maximal concurrent streams count. @@ -611,8 +584,72 @@ grpc: {% endcode %} +### Unix Socket + +Configure optional `mode`, `uid`, and `gid` fields under `grpc.unix_socket`. See [Unix socket attributes](../intro/config.md#unix-socket-attributes) for their values. Keep the other gRPC settings from the preceding example: + +{% code title=".rr.yaml fragment" %} + +```yaml +grpc: + listen: "unix:///run/roadrunner/grpc.sock" + unix_socket: + mode: "0660" +``` + +{% endcode %} + +Configure clients to use the same Unix socket. These settings apply to the listener socket. They were added in [gRPC PR #154](https://github.com/roadrunner-server/grpc/pull/154). + +### Connection age grace + +`max_connection_age_grace` controls how long active RPCs can continue after the connection reaches `max_connection_age`. Zero or omitted grace means unlimited time. Set a finite grace to close the connection after that period. + +{% code title=".rr.yaml" %} + +```yaml +grpc: + max_connection_age: 5m + max_connection_age_grace: 30s +``` + +{% endcode %} + +The v5 plugin used `max_connection_age` as the grace period and ignored `max_connection_age_grace`. To preserve that behavior, explicitly set both values to the same duration. + +## Error details in logs + +For a failed unary call, the `grpc` logger adds a `details` field when the error contains supported `google.rpc.Status` details. This field contains an array of type names and one-line message values. It is omitted when the error has no supported details. + +The supported types from [`google/rpc/error_details.proto`](https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto) are `BadRequest`, `ErrorInfo`, `DebugInfo`, `Help`, `LocalizedMessage`, `PreconditionFailure`, `QuotaFailure`, `RequestInfo`, `ResourceInfo`, and `RetryInfo`. Other detail types are omitted. + +Use a structured log mode to retain the field: + +{% code title=".rr.yaml" %} + +```yaml +logs: + channels: + grpc: + mode: production + level: error +``` + +{% endcode %} + +For example, a field validation error can include: + +```json +{ + "details": [ + "google.rpc.BadRequest: field_violations:{field:\"email\" description:\"is required\"}" + ] +} +``` + +See [logger configuration](../lab/logger.md) and [gRPC PR #148](https://github.com/roadrunner-server/grpc/pull/148). -## OTLP support in the `gRPC` plugin: `[>=2023.3.8]` +## OTLP support in the `gRPC` plugin In the `v2023.3.8` we added experimental support for the `OTLP` protocol in the `gRPC` plugin. Stable since `v2024.1.1`. To enable it, you need to activate `otel` plugin by adding the following lines to the `.rr.yaml` file: diff --git a/grpc/interceptors.md b/grpc/interceptors.md new file mode 100644 index 0000000..1198c59 --- /dev/null +++ b/grpc/interceptors.md @@ -0,0 +1,104 @@ +# gRPC Interceptors + +The RoadRunner gRPC plugin supports custom unary interceptors in both v5.3.0 and v6. + +Use an interceptor to add processing before or after an RPC call, such as: + +- request/response logging, +- authentication checks, +- rate limiting, +- custom metrics. + +{% hint style="info" %} +The plugin supports only unary interceptors (`grpc.UnaryServerInterceptor`). Streaming RPCs, including [server reflection](./grpc.md#server-reflection), do not call these interceptors. +{% endhint %} + +## Interceptor contract + +Your plugin must implement this interface: + +{% code title="grpc/api/interfaces.go" %} + +```go +type Interceptor interface { + UnaryServerInterceptor() grpc.UnaryServerInterceptor + Name() string +} +``` + +{% endcode %} + +The `Name()` return value is used in the `grpc.interceptors` configuration list. + +## Configuration + +Add interceptor names under the `grpc.interceptors` section: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +server: + command: "php grpc-worker.php" + +grpc: + listen: "tcp://127.0.0.1:9001" + + proto: + - "proto/helloworld.proto" + + interceptors: + - "custom-grpc-interceptor" +``` + +{% endcode %} + +{% hint style="warning" %} +Each name in `grpc.interceptors` must match the `Name()` of a registered interceptor plugin. RR fails to start if a configured interceptor is missing. +{% endhint %} + +## Execution order + +RoadRunner applies configured interceptors in the same order as the config list. + +Example: + +{% code title=".rr.yaml" %} + +```yaml +grpc: + interceptors: ["first", "second", "third"] +``` + +{% endcode %} + +Execution order will be: + +`first -> second -> third -> handler` + +## Build a custom binary + +Include your interceptor plugin in a [custom RR build](../customization/build.md). Use plugin versions compatible with v6. Listing an interceptor in `.rr.yaml` does not add its code to the binary. + +For an interceptor that reads protobuf descriptors, see the [registry-based example](./protoreg.md#example-grpc-interceptor). Use the plugin's `Name()` in `grpc.interceptors`. + +### Run and verify + +Start RoadRunner: + +{% code %} + +```bash +./rr serve -c .rr.yaml +``` + +{% endcode %} + +Send a request using your preferred gRPC client (for example, `grpc-client-cli`) and verify interceptor logs in RR output. + +## What's next? + +1. [Intro into gRPC](./grpc.md) +2. [Writing a Middleware](../customization/middleware.md) +3. [Building RR with a custom plugin](../customization/build.md) diff --git a/grpc/protoreg.md b/grpc/protoreg.md index 89641ea..fbddf06 100644 --- a/grpc/protoreg.md +++ b/grpc/protoreg.md @@ -1,6 +1,6 @@ # Protoreg Plugin -`protoreg` provides a shared protobuf registry, enabling other plugins to access proto definitions at runtime, making it ideal for building custom gRPC interceptors, middleware, and dynamic message handling. +The `protoreg` plugin provides a shared protobuf registry. It reads `.proto` definitions for [gRPC server reflection](#server-reflection), custom interceptors, and dynamic message handling. The RoadRunner v3 bundle includes this plugin. ## Features @@ -20,15 +20,19 @@ ## Configuration -{% hint style="warning" %} -The `protoreg` plugin must be enabled alongside the `grpc` plugin in your configuration. Without the `grpc` plugin enabled, the `protoreg` plugin will not be started. -{% endhint %} +Both `grpc` and `protoreg` configuration sections are required to start the plugin. Keep the service definitions in both sections consistent. The plugin is configured under the `protoreg` section in your `.rr.yaml` configuration file: ```yaml version: '3' +grpc: + listen: "tcp://127.0.0.1:9001" + proto: + - "proto/serviceapis/service/v1/service.proto" + - "proto/serviceapis/user/v1/user.proto" + protoreg: proto_path: - proto/commonapis @@ -38,6 +42,8 @@ protoreg: - user/v1/user.proto ``` +Set `server.command` to your gRPC worker as shown in the [gRPC configuration](grpc.md#usage). + ### Configuration Options #### `proto_path` (required) @@ -59,8 +65,9 @@ proto_path: #### `files` (required) -A list of proto files to parse and register. Paths are relative to the `proto_path` directories. -This works similarly to passing [variadic proto files argument in `protoc`](https://www.mankier.com/1/protoc#Options-@%3Cfilename%3E). +A list of proto files to parse and register. Paths are relative to the `proto_path` directories. Use explicit file names. This list does not expand the glob patterns supported by `grpc.proto`. + +Both `proto_path` and `files` must contain at least one entry. Blank entries fail initialization. Each file must exist under at least one configured import path. **Why it's needed**: This explicitly declares which proto files define the services and messages you want to access at runtime. The plugin will: @@ -74,6 +81,42 @@ files: - user/v1/user.proto # Additional service definitions ``` +## Server reflection + +RR automatically uses the `protoreg` registry for server reflection. Configure the same service files in `grpc.proto` and `protoreg.files`. Paths in `protoreg.files` are relative to `proto_path`; they do not replace `grpc.proto`. + +For the [Hello World service](./grpc.md#protoc-plugin), use: + +{% code title=".rr.yaml" %} + +```yaml +grpc: + listen: "tcp://127.0.0.1:9001" + proto: + - "proto/helloworld.proto" + +protoreg: + proto_path: + - "proto" + files: + - "helloworld.proto" +``` + +{% endcode %} + +Reflection clients can retrieve file and message descriptors for the PHP service. For the preceding listener without TLS, use `grpcurl` without a local `.proto` file: + +{% code %} + +```bash +grpcurl -plaintext 127.0.0.1:9001 describe helloworld.Greeter +grpcurl -plaintext -d '{"name":"RoadRunner"}' 127.0.0.1:9001 helloworld.Greeter/SayHello +``` + +{% endcode %} + +Reflection streams do not call unary authentication interceptors. See [reflection access controls](./grpc.md#server-reflection). + ## Example: Project Structure A typical project structure when using this plugin: @@ -128,6 +171,11 @@ service MyService { ```yaml version: '3' +grpc: + listen: "tcp://127.0.0.1:9001" + proto: + - "proto/serviceapis/service/v1/service.proto" + protoreg: proto_path: - proto/commonapis @@ -138,116 +186,86 @@ protoreg: ## Using the Registry in Your Plugin -Other plugins can depend on `protoreg` to access the registry. Here's how to use it in a custom plugin: +Declare `protoreg.Registry` as an initialization dependency: ```go package myplugin -import ( - "github.com/jhump/protoreflect/desc" - "github.com/jhump/protoreflect/v2/protoresolve" -) - -type Registry interface { - Registry() *protoresolve.Registry - Services() map[string]*desc.ServiceDescriptor - FindMethodByFullPath(method string) (*desc.MethodDescriptor, error) -} +import "github.com/roadrunner-server/protoreg/v6" type Plugin struct { - registry Registry + registry protoreg.Registry } -func (p *Plugin) Init(registry Registry) error { - +func (p *Plugin) Init(registry protoreg.Registry) error { p.registry = registry - - // Access the underlying registry - reg := p.registry.Registry() - - // Get all registered services - services := p.registry.Services() - - // Find a specific method by its full path - method, err := p.registry.FindMethodByFullPath("/service.v1.MyService/Process") - if err != nil { - return err - } - - // Use the method descriptor for reflection - inputType := method.GetInputType() - outputType := method.GetOutputType() - return nil } ``` ### Registry Interface -The plugin exposes the following interface: +The interface uses `protoresolve.Registry` from `github.com/jhump/protoreflect/v2/protoresolve` and descriptors from `github.com/jhump/protoreflect/desc`: ```go type Registry interface { - // Registry returns the underlying protoresolve.Registry that - // contains all the parsed descriptors + // Registry returns the parsed descriptors. Registry() *protoresolve.Registry - // Services returns a map of all registered service descriptors - // keyed by their fully qualified name + // Services maps fully qualified service names to descriptors. Services() map[string]*desc.ServiceDescriptor - // FindMethodByFullPath finds a method descriptor by its full gRPC path - // Format: "/package.Service/Method" or "package.Service/Method" + // The path format is "/package.Service/Method" or "package.Service/Method". FindMethodByFullPath(method string) (*desc.MethodDescriptor, error) } ``` ## Example: gRPC Interceptor -Here's an example of using the registry in a custom gRPC interceptor to log request details: +This unary interceptor logs method descriptors without logging request bodies. Include it with `grpc` and `protoreg` in your custom build. Set `grpc.interceptors: ["descriptor-logger"]` to activate it. ```go package interceptor import ( "context" - "log" + "log/slog" - "github.com/roadrunner-server/protoreg/v5" + "github.com/roadrunner-server/protoreg/v6" "google.golang.org/grpc" - "google.golang.org/protobuf/proto" - "google.golang.org/protobuf/types/dynamicpb" ) +type Logger interface { + NamedLogger(name string) *slog.Logger +} + type Plugin struct { registry protoreg.Registry + log *slog.Logger } -func (i *Plugin) UnaryInterceptor( - ctx context.Context, - req interface{}, - info *grpc.UnaryServerInfo, - handler grpc.UnaryHandler, -) (interface{}, error) { - // Find the method descriptor - method, err := i.registry.FindMethodByFullPath(info.FullMethod) - if err != nil { - log.Printf("Method not found in registry: %s", info.FullMethod) - return handler(ctx, req) - } +func (p *Plugin) Init(registry protoreg.Registry, logger Logger) error { + p.registry = registry + p.log = logger.NamedLogger(p.Name()) + return nil +} - // Log method information - log.Printf("Method: %s", method.GetFullyQualifiedName()) - log.Printf("Input type: %s", method.GetInputType().GetFullyQualifiedName()) - log.Printf("Output type: %s", method.GetOutputType().GetFullyQualifiedName()) +func (p *Plugin) Name() string { + return "descriptor-logger" +} - // You can also dynamically inspect the message fields - inputDesc := method.GetInputType() - for _, field := range inputDesc.GetFields() { - log.Printf(" Field: %s (type: %s)", field.GetName(), field.GetType().String()) +func (p *Plugin) UnaryServerInterceptor() grpc.UnaryServerInterceptor { + return func(ctx context.Context, req any, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (any, error) { + method, err := p.registry.FindMethodByFullPath(info.FullMethod) + if err == nil && method != nil { + p.log.Info("grpc method", + "method", method.GetFullyQualifiedName(), + "input_type", method.GetInputType().GetFullyQualifiedName(), + "output_type", method.GetOutputType().GetFullyQualifiedName(), + ) + } + return handler(ctx, req) } - - return handler(ctx, req) } ``` diff --git a/http/gzip.md b/http/gzip.md index d09e6b4..299bae4 100644 --- a/http/gzip.md +++ b/http/gzip.md @@ -1,11 +1,10 @@ -# HTTP β€” Gzip middleware +# HTTP - Gzip middleware -The gzip middleware supports the `Accept-Encoding: gzip` header and compresses or decompresses the contents of -outgoing and incoming requests. +The gzip middleware can compress HTTP responses for clients that send `Accept-Encoding: gzip`. It does not decompress incoming request bodies. ## Documentation -- MDN [link](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding) +- MDN: [Accept-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding) ## Configuration @@ -25,4 +24,8 @@ http: {% endcode %} +Put `gzip` before `static` or `sendfile` to apply compression to their responses. See [middleware order](./http.md#middleware-order). + +This middleware provides gzip compression only. Use the separate [Zstd middleware](zstd.md) for clients that accept `zstd`. + The gzip middleware supports OpenTelemetry header propagation. diff --git a/http/headers.md b/http/headers.md index 22d6a01..83a24d0 100644 --- a/http/headers.md +++ b/http/headers.md @@ -1,4 +1,4 @@ -# HTTP β€” Headers and CORS +# HTTP - Headers and CORS The headers middleware sets request/response headers and controls CORS for your application. @@ -17,11 +17,11 @@ http: # ... headers: cors: - allowed_origin: "*" + allowed_origin: "https://foo.example.com" # If `allowed_origin_regex` option is set, the content of `allowed_origin` is ignored - allowed_origin_regex: "^http://foo" + allowed_origin_regex: "^https://foo[.]example[.]com$" allowed_headers: "*" - allowed_methods: "GET,POST,PUT,DELETE" + allowed_methods: "GET, POST, PUT, DELETE" allow_credentials: true exposed_headers: "Cache-Control,Content-Language,Content-Type,Expires,Last-Modified,Pragma" max_age: 600 @@ -35,6 +35,10 @@ http: > Make sure to declare "headers" middleware. +Replace the example origin in both settings with your trusted origin. Keep `^` and `$` in the regex. Use `[.]` to match literal dots. Do not use `allowed_origin: "*"` with `allow_credentials: true`. + +`allowed_origin`, `allowed_methods`, `allowed_headers`, and `exposed_headers` ignore whitespace around each comma-separated value. For example, `"GET, POST"` selects both methods. + {% hint style="info" %} Since RoadRunner v2023.2.0, the following changes were made: @@ -48,6 +52,8 @@ Since RoadRunner v2023.2.0, the following changes were made: You can control additional headers for outgoing responses and headers to be added to requests sent to your application. +Put `headers` before `static` in the v6 middleware list to apply headers to static responses. See [middleware order](./http.md#middleware-order). + {% code title=".rr.yaml" %} ```yaml diff --git a/http/http.md b/http/http.md index 36c30b2..d16b9e4 100644 --- a/http/http.md +++ b/http/http.md @@ -2,6 +2,8 @@ HTTP plugin is used to pass `HTTP`/`HTTPS`/`fCGI`/`HTTP2(h2c)`/`HTTP3` requests to the PHP worker. +The RoadRunner v3 bundle includes [HTTP rate limiting](rate-limiter.md) with global, IP, or header keys and [Zstd response compression](zstd.md). + ## Configuration reference {% code title=".rr.yaml" %} @@ -13,7 +15,7 @@ version: "3" http: # Host and port to listen on (e.g.: `127.0.0.1:8080`). # - # This option is required. + # Required for plain HTTP. Omit to use only HTTPS or FastCGI. address: 127.0.0.1:8080 # Override HTTP error code for internal RR errors @@ -26,9 +28,9 @@ http: # Default: false access_logs: false - # Maximum incoming request size in megabytes. Zero means no limit. + # Maximum incoming request size in MiB. Zero selects the default limit. # - # Default: 0 + # Default: 1000 max_request_size: 256 # Send raw body (unescaped) to the PHP worker for the application/x-www-form-urlencoded content type @@ -36,23 +38,23 @@ http: # Optional, default: false raw_body: false - # Middleware for the HTTP plugin; order is important. Allowed values are: "headers", "gzip", "static", "sendfile", [SINCE 2.6] -> "new_relic", [SINCE 2.6] -> "http_metrics", [SINCE 2.7] -> "cache" + # Middleware names depend on the plugins in the build. Requests run left to right in v6. + # The "zstd" middleware compresses responses for clients that accept Zstd. + # The "rate_limiter" middleware uses http.rate_limiter settings. # # Default value: [] middleware: [ "headers", "gzip" ] - # Allow incoming requests only from the following subnets (https://en.wikipedia.org/wiki/Reserved_IP_addresses). + # Trust HTTP forwarding headers from these proxy addresses. + # Requires "proxy_ip_parser" in middleware. This is not a network access filter. # - # Default: ["10.0.0.0/8", "127.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "::1/128", "fc00::/7", "fe80::/10"] - trusted_subnets: [ - "10.0.0.0/8", - "127.0.0.0/8", - "172.16.0.0/12", - "192.168.0.0/16", - "::1/128", - "fc00::/7", - "fe80::/10", - ] + # Default: [] (forwarding headers are not trusted) + trusted_subnets: [ "127.0.0.1/32" ] + + # Forwarding headers to read, in priority order. + # Requires "proxy_ip_parser" and a trusted_subnets match. + # An omitted or empty list uses the default order shown here. + trusted_headers: [ "Forwarded", "X-Forwarded-For", "X-Real-IP", "True-Client-IP", "CF-Connecting-IP" ] # File uploading settings. uploads: @@ -61,9 +63,11 @@ http: # Default: "" dir: "/tmp" - # Deny files with the following extensions to upload. + # Deny uploads with these file extensions. + # Configure forbid or allow for your application's extension restrictions. + # Without either list, RR accepts all file extensions. # - # Default: [".php", ".exe", ".bat"] + # Default: [] forbid: [ ".php", ".exe", ".bat", ".sh" ] # [SINCE 2.6] Allow files with the following extensions to upload @@ -78,9 +82,10 @@ http: # feature disabling. cors: # Controls "Access-Control-Allow-Origin" header value (docs: https://mzl.la/2OgD4Qf). + # Replace this example with your application's trusted origin. # # Default: "" - allowed_origin: "*" + allowed_origin: "https://foo.example.com" # Controls "Access-Control-Allow-Headers" header value (docs: https://mzl.la/2OzDVvk). # @@ -121,30 +126,45 @@ http: # Settings for "static" middleware. static: - # Path to the directory to serve + # Existing directory to serve. # - # Default: "." (current) + # Required when static middleware is enabled. dir: "." - # File patterns to forbid + # File extensions to forbid # # Default: empty - forbid: [ "" ] + forbid: [ ".php", ".htaccess" ] - # ETag calculation (based on the body CRC32) + # Calculate ETags. Strong ETags use CRC32C of the file contents. # # Default: false calculate_etag: false - # Weak ETag calculation (based only on the content-length CRC32) + # Weak ETags use the file size and modification time. # # Default: false weak: false - # Patterns to allow + # File extensions to allow # # Default: empty - allow: [ ".txt", ".php" ] + allow: [ ".txt", ".css", ".js" ] + + # URL path prefixes to serve. An empty list accepts all paths. + prefixes: [] + + # Cache file metadata when calculate_etag is true. Zero disables this cache. + # Default: 10s + cache_ttl: 10s + + # Cache missing files and directories. Zero disables this cache. + # Default: 10s + cache_miss_ttl: 10s + + # Maximum entries in each cache. Zero selects the default. + # Default: 16384 + cache_max_entries: 16384 # Request headers # @@ -239,7 +259,7 @@ http: ssl: # Host and port to listen on (e.g.: `127.0.0.1:443`). # - # Default: ":443" + # Default: "127.0.0.1:443" address: "127.0.0.1:443" # Use ACME certificates provider (Let's encrypt) @@ -334,6 +354,8 @@ http: You can enable HTTPS support by adding the `ssl` section to the `http` config. +Use brackets around IPv6 addresses, such as `"[::1]:8443"`. The port must be an unsigned integer from 0 through 65535. HTTP-to-HTTPS redirects preserve IPv6 brackets. + {% code title=".rr.yaml" %} ```yaml @@ -343,7 +365,7 @@ http: address: 127.0.0.1:8080 ssl: - # host and port separated by semicolon (default :443) + # Host and port separated by a colon. address: :8892 redirect: false cert: fixtures/server.crt @@ -482,18 +504,6 @@ http: {% endcode %} -### Upgrade connection from `HTTP/1.1` to `H2C` [`v2.10.2`] - -Connection might be upgraded from the `http/1.1` -to `h2c`: [rfc7540](https://datatracker.ietf.org/doc/html/rfc7540#section-3.4) - -**Headers, which should be sent to upgrade connection:** - -1. `Upgrade`: `h2c` -2. `Connection`: `HTTP2-Settings` -3. `Connection`: `Upgrade` -4. `HTTP2-Settings`: `AAMAAABkAARAAAAAAAIAAAAA` [RFC](https://datatracker.ietf.org/doc/html/rfc7540#section-3.2.1) - ### HTTP/2 Push Resources RoadRunner supports [HTTP/2 push](https://en.wikipedia.org/wiki/HTTP/2_Server_Push) via virtual headers provided by the PHP @@ -502,11 +512,13 @@ response. {% code title="script.php" %} ```php -return $response->withAddedHeader('http2-push', '/test.js'); +return $response->withAddedHeader('Http2-Push', '/test.js'); ``` {% endcode %} +This virtual header requires the exact name `Http2-Push`. + Note that the path of the resource must be related to the public application directory and must include `/` at the beginning. @@ -516,7 +528,7 @@ HTTP/2 push only works under HTTPS with the `static` service enabled. ### H2C -You can enable HTTP/2 support over non-encrypted TCP connection using H2C: +H2C provides HTTP/2 over an unencrypted connection. The client must start with HTTP/2 prior knowledge. HTTP/1.1 requests with `Upgrade: h2c` are handled as HTTP/1.1; RR does not upgrade them. {% code title=".rr.yaml" %} @@ -524,7 +536,19 @@ You can enable HTTP/2 support over non-encrypted TCP connection using H2C: version: "3" http: - http2.h2c: true + address: 127.0.0.1:8080 + http2: + h2c: true +``` + +{% endcode %} + +For example, use a curl build with HTTP/2 support: + +{% code %} + +```bash +curl --http2-prior-knowledge http://127.0.0.1:8080/ ``` {% endcode %} @@ -546,6 +570,72 @@ http: {% endcode %} +## Unix Sockets + +Configure independent [Unix socket attributes](../intro/config.md#unix-socket-attributes) for plain HTTP and FastCGI. Each listener accepts optional `mode`, `uid`, and `gid` fields: + +{% code title=".rr.yaml fragment" %} + +```yaml +http: + address: "unix:///run/roadrunner/http.sock" + unix_socket: + mode: "0600" + fcgi: + address: "unix:///run/roadrunner/fcgi.sock" + unix_socket: + mode: "0660" +``` + +{% endcode %} + +`http.unix_socket` also applies when H2C uses the plain HTTP listener. It does not configure FastCGI, HTTPS, HTTP/3, or ACME challenge listeners. Keep `http.fcgi.unix_socket` separate. PROXY protocol still requires TCP and cannot be used on these Unix listeners. + +A socket options object does not enable a listener. Set its `address` to a filesystem Unix socket. Omit its socket options when the listener is disabled. + +During shutdown, the HTTP plugin closes the FastCGI listener. This stops new connections. It does not close FastCGI connections that the server already accepted. + +See [Nginx configuration](../app-server/nginx-with-rr.md) for a FastCGI socket shared with a web server. These settings were added in [HTTP PR #284](https://github.com/roadrunner-server/http/pull/284). + +## PROXY protocol + +PROXY protocol v1 and v2 let a TCP proxy supply client addresses. Configure each application listener separately. `http.proxy_protocol` controls plain HTTP, including H2C. `http.ssl.proxy_protocol` controls HTTPS. Omit the corresponding section to disable it. + +The following example enables PROXY protocol on both listeners: + +{% code title=".rr.yaml" %} + +```yaml +http: + address: "0.0.0.0:8080" + proxy_protocol: + trusted_proxies: [ "10.20.0.10" ] + read_header_timeout: 5s + ssl: + address: "0.0.0.0:8443" + cert: "server.crt" + key: "server.key" + proxy_protocol: + trusted_proxies: [ "10.20.0.10" ] + read_header_timeout: 5s +``` + +{% endcode %} + +- `trusted_proxies` must contain at least one IP address or CIDR for an immediate TCP peer. Hostnames are not accepted. Replace the example address with the address of your proxy. +- RR rejects peers outside this list. Trusted peers must send a valid PROXY header. Direct HTTP requests without that header are rejected. +- `read_header_timeout` limits the time to read the PROXY header. The default is `5s`. Zero selects the default; negative values are invalid. This setting does not control HTTP or TLS timeouts. +- For HTTPS, the proxy must send the PROXY header before the TLS handshake. The HTTPS listener still requires certificates or ACME configuration. Temporary ACME challenge listeners are not wrapped. +- Only TCP application listeners support this setting. It does not apply to HTTP/3 or FastCGI. Headers with TCP4 or TCP6 addresses replace the connection addresses; v1 `UNKNOWN` and v2 `LOCAL` retain the socket addresses. + +The supplied client address applies to the whole connection, including all HTTP/2 streams. The proxy must use separate backend connections for different client identities. PROXY metadata does not change the TLS state or URL scheme. + +This trust list is separate from [HTTP forwarding-header trust](./proxy.md). If `proxy_ip_parser` also runs, it checks `trusted_subnets` against the client address supplied by PROXY protocol, not the original TCP peer. + +Route HTTP readiness checks through a trusted proxy that sends a PROXY header. For direct checks, use the status plugin's separate [health and readiness endpoints](../lab/health.md). A successful TCP connection alone does not prove that RR accepted the PROXY header or that a worker is ready. + +The parser can reject fragmented v1 headers. It limits the v2 address and metadata payload to 4,096 bytes. Use v2 when the proxy supports it. See [HTTP PR #283](https://github.com/roadrunner-server/http/pull/283). + ## HTTP/3 HTTP/3 support is experimental and might change in the future. Docs are available in the [experimental](../experimental/experimental.md) section. @@ -566,24 +656,54 @@ http: The `http.internal_error_code` is used for `SoftJob`, allocation, TTL, network, and similar errors. For example, a load balancer might require a different code, so you may override the default. +Malformed request bodies that RR cannot parse return `400 Bad Request`. Requests that exceed `max_request_size` return `413 Request Entity Too Large`. `internal_error_code` does not override these responses. + +{% hint style="warning" %} +With `http.pool.debug: true`, internal error responses can contain HTML-escaped error text. Keep debug mode disabled on public production servers. +{% endhint %} + ## Middleware order -Since all middleware components are independent, they can remove or update headers set by -the [previous one](https://github.com/roadrunner-server/roadrunner/issues/1501). +Requests enter middleware from left to right in the configuration list. Responses return through the same middleware in reverse order. A middleware can return a response without calling the remaining handlers. -**Note that the request (imagine) comes from the right:** +The order was reversed in v5. Reverse an existing list to preserve its v5 behavior. -{% code title=".rr.yaml" %} +{% code title="v5 configuration" %} ```yaml http: - middleware: # RESPONSE FROM HERE --> [ "static", "gzip", "sendfile" ] # <-- REQUEST COMES FROM HERE + middleware: [ "static", "headers", "gzip" ] ``` {% endcode %} -So in this case, the request gets into the `sendfile` middleware, then `gzip`, and `static`. And vice versa from the -response. +{% code title="Equivalent v6 configuration" %} + +```yaml +http: + middleware: [ "gzip", "headers", "static" ] +``` + +{% endcode %} + +In the v6 example, requests enter `gzip`, then `headers`, then `static`. Put `headers` and `gzip` before `static` to apply them to static responses. Middleware can replace headers set by an earlier handler. + +## Tracing + +Configure [OpenTelemetry](../lab/otel.md) and put `otel` before the middleware to trace: + +{% code title=".rr.yaml fragment" %} + +```yaml +http: + middleware: [ "otel", "headers", "gzip" ] +``` + +{% endcode %} + +The `headers`, `gzip`, `zstd`, `proxy_ip_parser`, and `rate_limiter` spans end before the next handler starts. They cover work before that call. Compression spans do not measure the full response compression time. The `static` span includes file serving when it handles the request. When it forwards the request, its span ends before the next handler starts. + +The `sendfile` middleware uses a second span, `sendfile:post`, for work after the PHP response. The HTTP plugin's internal `http` span includes worker and response processing. The outer HTTP server span measures the request through the handlers that follow `otel`. ## Request queues diff --git a/http/proxy.md b/http/proxy.md index beb9855..7a5472f 100644 --- a/http/proxy.md +++ b/http/proxy.md @@ -1,18 +1,12 @@ # Proxy IP parser -This middleware resolves the real client IP from proxy headers when a request arrives -through a trusted subnet. By default it consults, in order: `Forwarded`, `X-Forwarded-For`, -`X-Real-IP`, `True-Client-IP`, and `CF-Connecting-IP`. The set and order of headers can be -customized with `trusted_headers`. +This middleware gets the client address from HTTP forwarding headers when the request comes from a trusted proxy. In v6, `trusted_headers` selects the headers and their order. ## Description -When the immediate peer is within one of the `trusted_subnets`, the middleware resolves the -client IP from the configured headers β€” the first non-empty match wins β€” and sets it as -`RemoteAddr`. Otherwise `RemoteAddr` is left unchanged. +When the immediate peer is within `trusted_subnets`, the middleware uses the first nonempty parsed header value as `RemoteAddr`. Otherwise, it leaves `RemoteAddr` unchanged. This setting controls trust in forwarding headers; it does not block incoming connections. -The middleware is active only when `trusted_subnets` is configured; without it, forwarding -headers are never trusted. +Add `proxy_ip_parser` to `http.middleware` and configure a nonempty `http.trusted_subnets` list to enable it. An omitted or empty subnet list disables it. Each subnet must use CIDR notation, such as `127.0.0.1/32` or `::1/128`. ## Usage @@ -25,16 +19,8 @@ http: middleware: [ "proxy_ip_parser" ] # Middleware uploads: forbid: [ ".php", ".exe", ".bat" ] - trusted_subnets: # Trusted addresses in CIDR format - [ - "10.0.0.0/8", - "127.0.0.0/8", - "172.16.0.0/12", - "192.168.0.0/16", - "::1/128", - "fc00::/7", - "fe80::/10" - ] + # Replace this with the immediate proxy's actual CIDR. + trusted_subnets: [ "127.0.0.1/32" ] pool: num_workers: 2 allocate_timeout: 60s @@ -45,27 +31,31 @@ http: ## Trusted headers -`trusted_headers` is an ordered allowlist of the headers used to resolve the client IP. The -middleware checks them in order and uses the first non-empty value; headers that are not -listed are ignored, and custom headers are supported. When `trusted_headers` is omitted, the -default order is used: `Forwarded`, `X-Forwarded-For`, `X-Real-IP`, `True-Client-IP`, -`CF-Connecting-IP`. +`http.trusted_headers` is an ordered allowlist. The middleware ignores headers that are not listed. It removes whitespace around configured header names, compares names without case sensitivity, and removes duplicate names. -For example, to trust only `X-Real-IP` and Cloudflare's `CF-Connecting-IP` while ignoring -`X-Forwarded-*`: +An omitted, empty, or all-blank list uses the default order: `Forwarded`, `X-Forwarded-For`, `X-Real-IP`, `True-Client-IP`, `CF-Connecting-IP`. An empty header list does not disable trust. If a header produces no parsed value, the middleware tries the next header. For example, a `Forwarded` value without `for=` does not prevent use of `X-Forwarded-For`. This setting was added in [proxy_ip_parser PR #120](https://github.com/roadrunner-server/proxy_ip_parser/pull/120). + +The following example trusts only `X-Real-IP` and `CF-Connecting-IP`: {% code title=".rr.yaml" %} ```yaml http: - trusted_subnets: [ "10.0.0.0/8", "127.0.0.0/8" ] + middleware: [ "proxy_ip_parser" ] + trusted_subnets: [ "10.20.0.10/32" ] trusted_headers: [ "X-Real-IP", "CF-Connecting-IP" ] ``` {% endcode %} {% hint style="info" %} -`X-Forwarded-For` uses the left-most address from its comma-separated list, and `Forwarded` -is parsed per [RFC 7239](https://datatracker.ietf.org/doc/html/rfc7239) (`for=`). All other -headers, including custom ones, are taken verbatim. +`X-Forwarded-For` uses the first value before a comma. `Forwarded` uses the first `for=` value and removes its surrounding quotes. All other headers, including custom headers, are used without changing their values. +{% endhint %} + +{% hint style="warning" %} +The parser does not validate that a selected header value is an IP address. Trust only proxy addresses you control. Configure each trusted proxy to overwrite the selected headers so a client cannot supply the address used by the application. {% endhint %} + +## PROXY protocol + +HTTP forwarding headers are separate from the TCP [PROXY protocol](./http.md#proxy-protocol). With PROXY protocol enabled, `trusted_subnets` checks the client address supplied by the TCP proxy. Configure forwarding-header trust for that address. diff --git a/http/rate-limiter.md b/http/rate-limiter.md new file mode 100644 index 0000000..bc09a56 --- /dev/null +++ b/http/rate-limiter.md @@ -0,0 +1,97 @@ +# HTTP - Rate Limiter + +The `rate_limiter` middleware limits requests before they reach PHP. It uses one process-local token bucket policy with a global, IP, or header key. + +## Availability + +The RoadRunner v3 bundle includes this middleware. See [rate-limiter PR #1](https://github.com/roadrunner-server/rate-limiter/pull/1) and [RoadRunner PR #2389](https://github.com/roadrunner-server/roadrunner/pull/2389). + +## Configuration + +Add `rate_limiter` to `http.middleware`. Define its policy under `http.rate_limiter`: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +server: + command: "php worker.php" + relay: pipes + +http: + address: "127.0.0.1:8080" + middleware: ["rate_limiter"] + rate_limiter: + key: ip + rate: 10 + interval: 1s + burst: 20 + max_entries: 10000 +``` + +{% endcode %} + +| Field | Default | Meaning | +| --- | --- | --- | +| `key` | `ip` | One of `global`, `ip`, or `header`. | +| `rate` | Required | Positive integer. Tokens added per `interval`. | +| `interval` | `1s` | Positive Go duration, such as `250ms`, `1s`, or `1m`. | +| `burst` | `1` | Positive integer. Bucket capacity and initial token count. | +| `max_entries` | `10000` | Positive integer. Maximum number of stored buckets. | +| `header` | Empty | A valid HTTP header name is required for `key: header`. It must be empty in other modes. | + +Omitted fields use their defaults. An explicit zero or negative value for `rate`, `interval`, `burst`, or `max_entries` is invalid. Invalid configuration fails initialization. An absent `http.rate_limiter` section disables the plugin. A configured policy has no effect unless the middleware list selects it. + +## Token Accounting + +The middleware uses the [`golang.org/x/time/rate` token bucket](https://pkg.go.dev/golang.org/x/time/rate#Limiter). Each new bucket starts with `burst` tokens. Each allowed request uses one token. Tokens refill continuously at `rate / interval`, up to `burst`. Rejected requests do not reserve future tokens. + +For example, `rate: 60`, `interval: 1m`, and `burst: 10` allow 10 requests immediately and add one token per second. This is not a fixed-window limit of 60 requests per minute. The bucket does not reset at minute boundaries, and `burst` can exceed `rate`. + +## Client Identity + +| Key | Bucket identity | +| --- | --- | +| `global` | All requests share one bucket. | +| `ip` | The normalized IP from `RemoteAddr`, without its port. IPv4-mapped IPv6 addresses share the IPv4 bucket. | +| `header` | The trimmed value of the configured header. Values are case-sensitive. The plugin stores a SHA-256 hash, not the raw value. For `Host`, it reads Go's `Request.Host` field. | + +For header mode, set `key: header` and `header: X-Client-ID`. A header value is not proof of identity. For per-user limits, a trusted upstream must supply a stable, verified identity and overwrite client-supplied values. A client that controls the selected value can obtain new buckets and fill the map. + +IP mode reads `RemoteAddr`. It does not read `Forwarded` or `X-Forwarded-For` directly. Behind a proxy, configure the [Proxy IP parser](proxy.md) and place it before the limiter. The proxy must overwrite the selected forwarding headers. Without this setup, IP mode can limit the proxy address or use a forged client identity. Use `global` or `header` mode when the transport provides no client IP. + +## Responses + +| Condition | Response | +| --- | --- | +| A token is available | Call the next handler without changing its response. | +| The selected header is missing or empty after trimming | `400 Bad Request`. | +| `RemoteAddr` has no valid IP in IP mode | `400 Bad Request`. | +| The bucket has no token | `429 Too Many Requests` with `Retry-After`. | +| A new identity arrives when the map is full | `503 Service Unavailable`. Existing identities still use their buckets. | + +The middleware sets `Cache-Control: no-store` on its error responses. `Retry-After` gives the delay until the next token, rounded up to whole seconds, with a minimum of `1`. It does not reserve capacity; other requests can consume the token first. Rejected requests do not reach PHP. Rejected HEAD requests have no response body. + +See [RFC 6585, section 4](https://www.rfc-editor.org/rfc/rfc6585.html#section-4) for `429` and [RFC 9110, section 10.2.3](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3) for `Retry-After`. + +## State And Cleanup + +All HTTP listeners and PHP workers in one RoadRunner process share the policy state. Separate RoadRunner processes have independent quotas. A process restart clears the state. An HTTP worker reset does not clear it. + +`max_entries` bounds the number of buckets. Cleanup runs during requests, at most once per minute. It removes only fully refilled buckets. The plugin does not evict depleted buckets to admit new identities. Idle entries can remain until a later request starts cleanup. There is no background cleanup worker. + +The middleware provides HTTP admission control only. It has no RPC API, PHP-supplied policy updates, database lookups, path rules, or distributed storage. + +## Middleware Order + +In v6, requests enter middleware from left to right. Place `proxy_ip_parser` before the limiter to resolve client IPs from trusted proxy headers. Place `headers` before it when error responses need CORS headers. Place `http_metrics` and `otel` before it to include rejected requests in metrics and tracing. + +```yaml +http: + middleware: ["otel", "proxy_ip_parser", "headers", "http_metrics", "rate_limiter"] +``` + +A handler that answers before the limiter consumes no token. This includes CORS preflight responses when `headers` comes first. Place the limiter before `static` to count static-file requests. Place `static` first to let its responses bypass the limit. + +The built-in HTTP access logger runs inside the configured middleware chain. It does not record limiter rejections. The limiter's internal OpenTelemetry span ends before the next handler starts. The HTTP server span includes downstream request time. diff --git a/http/resp-streaming.md b/http/resp-streaming.md index 67c9d37..460ee14 100644 --- a/http/resp-streaming.md +++ b/http/resp-streaming.md @@ -1,5 +1,7 @@ -# HTTP response streaming `[>=2023.3]` +# HTTP response streaming +Response streaming has been available since RoadRunner v2023.3. + RoadRunner supports HTTP response streaming, which allows responses to be sent to the client in chunks. This is useful when you need to send a large amount of data to the client. You don't need to update the configuration to enable this feature; it is enabled by default and controlled by the PHP worker. @@ -65,7 +67,11 @@ try { ### Sending headers and status codes -You can send headers and status codes (`1XX` multiple times, or others once) to the client during streaming. +You can send multiple informational responses before the final response. Send an empty body and set `endOfStream: false` for each informational response. Send the final status and headers before streaming its body. + +{% hint style="warning" %} +RR ignores worker responses with status `101 Switching Protocols`. A PHP worker cannot upgrade the connection by sending this status. RR also drops bodies attached to informational responses. Do not send informational responses after the final response has started. +{% endhint %} {% code title="worker.php" %} @@ -101,7 +107,6 @@ $read = static function (): Generator { try { while ($req = $http->waitRequest()) { $http->respond(100, '', headers: ['X-100' => ['100']], endOfStream: false); - $http->respond(101, '', headers: ['X-101' => ['101']], endOfStream: false); $http->respond(102, '', headers: ['X-102' => ['102']], endOfStream: false); $http->respond(103, '', headers: ['Link' => ['; rel=preload; as=style'], 'X-103' => ['103']], endOfStream: false); $http->respond(200, $read(), headers: ['X-200' => ['200']], endOfStream: true); @@ -113,4 +118,4 @@ try { {% endcode %} -In this example, we send five status codes and five headers to the client. You may send a `103 Early Hints` status code (or any `1XX` status code) at any time during streaming (do not forget about `$endOfStream`). +This example sends `100`, `102`, and `103` before the final `200` response. Headers supplied only in an informational response are not copied to the final response. Repeat any headers needed in the final `respond()` call. diff --git a/http/sendfile.md b/http/sendfile.md index 14fc3f9..9d106c7 100644 --- a/http/sendfile.md +++ b/http/sendfile.md @@ -1,11 +1,9 @@ -# HTTP β€” X-Sendfile middleware +# HTTP - X-Sendfile middleware The `Send` HTTP middleware and the `X-Sendfile` HTTP response header are used to stream large files using RoadRunner. While the file is being streamed with the help of RoadRunner, the PHP worker may accept the next request. -Original issue: [link](https://github.com/roadrunner-server/roadrunner-plugins/issues/9) -The middleware reads the file in 10 MB chunks. For example, for a 5 GB file, only 10 MB of RSS memory is used. If the file -is smaller than 10 MB, the middleware adjusts the buffer to fit the file size. +The middleware reads the file with a buffer of up to 10 MiB. For smaller files, the buffer matches the file size. See the [X-Sendfile proposal](https://github.com/roadrunner-server/roadrunner-plugins/issues/9). ## Similar approaches @@ -33,3 +31,11 @@ http: ``` {% endcode %} + +## File responses + +A response with `X-Sendfile` uses `Content-Type: application/octet-stream`. This replaces any content type supplied by the PHP worker. Check clients that depend on a specific media type for inline display. Use `Content-Disposition: attachment` for downloads. + +An empty file returns `200 OK` with no body. Paths are normalized before file access. Use paths controlled by the application: this middleware does not restrict access to a configured root directory. + +To apply gzip to file responses, put `gzip` before `sendfile` in the v6 middleware list. See [middleware order](./http.md#middleware-order). diff --git a/http/static.md b/http/static.md index 19a3e1a..12c70e0 100644 --- a/http/static.md +++ b/http/static.md @@ -1,10 +1,9 @@ -# HTTP β€” Serving static content +# HTTP - Serving static content -The `static` HTTP middleware serves static content using RoadRunner on the main HTTP plugin endpoint. Using this middleware -can slow down the overall performance by up to `~10%`, because RoadRunner has to check the path for each file request. +The `static` HTTP middleware serves static content using RoadRunner on the main HTTP plugin endpoint. {% hint style="info" %} -If there is no file to serve, RR will redirect the request back to the PHP worker. +The middleware serves files for `GET` and `HEAD` requests. It forwards other requests to PHP. See [cache and prefixes](#cache-and-prefixes) to limit filesystem checks and cache file metadata. {% endhint %} ## Enable HTTP middleware @@ -17,15 +16,15 @@ To enable static content serving, use the configuration inside the HTTP section: version: "3" http: - # host and port separated by semicolon + # Host and port separated by a colon. address: 127.0.0.1:44933 middleware: [ "static" ] # Add static to the list of middleware static: dir: "." - forbid: [ "" ] + forbid: [ ".php", ".htaccess" ] calculate_etag: false weak: false - allow: [ ".txt", ".php" ] + allow: [ ".txt", ".css", ".js" ] request: input: "custom-header" response: @@ -36,14 +35,14 @@ http: Where: -1. `dir`: path to the directory. +1. `dir`: required path to an existing directory. 2. `forbid`: file extensions that should not be served. 3. `allow`: extensions that should be served (empty = serve all except forbidden). If an extension is present in both lists (allow and forbid), it is treated as forbidden. -4. `calculate_etag`: enable etag calculation for the static file. -5. `weak`: use a weak generator (/W); it uses only the filename to generate a CRC32 sum. If false, the entire file content is used to generate the CRC32 sum. +4. `calculate_etag`: enable [ETag calculation](#etags) for the static file. +5. `weak`: use file size and modification time for a weak ETag (`W/`). With `weak: false`, RR calculates a strong CRC32C ETag from the file contents. This setting requires `calculate_etag: true`. 6. `request/response`: custom headers for the static files. -To combine static content with other middleware, use the following sequence (static last, then headers and gzip): +Put `static` after `gzip` and `headers` so they also apply to static responses. See [middleware order](./http.md#middleware-order) when migrating a v5 configuration. {% code title=".rr.yaml" %} @@ -51,13 +50,13 @@ To combine static content with other middleware, use the following sequence (sta version: "3" http: - # host and port separated by semicolon + # Host and port separated by a colon. address: 127.0.0.1:44933 - middleware: [ "static", "headers", "gzip" ] + middleware: [ "gzip", "headers", "static" ] # Settings for "headers" middleware. headers: cors: - allowed_origin: "*" + allowed_origin: "https://app.example.com" allowed_headers: "*" allowed_methods: "GET,POST,PUT,DELETE" allow_credentials: true @@ -66,10 +65,10 @@ http: # Settings for "static" middleware. static: dir: "." - forbid: [ "" ] + forbid: [ ".php", ".htaccess" ] calculate_etag: false weak: false - allow: [ ".txt", ".php" ] + allow: [ ".txt", ".css", ".js" ] request: input: "custom-header" response: @@ -78,15 +77,58 @@ http: {% endcode %} +## Cache and prefixes + +The middleware normalizes the URL path before checking prefixes and file extensions. Requests for missing files, directories, or paths without a file extension go to PHP. + +{% code title=".rr.yaml" %} + +```yaml +http: + address: 127.0.0.1:44933 + middleware: [ "static" ] + static: + dir: "." + forbid: [ ".php", ".htaccess" ] + allow: [ ".txt", ".css", ".js" ] + calculate_etag: true + weak: false + prefixes: [ "/assets/", "/build/" ] + cache_ttl: 10s + cache_miss_ttl: 10s + cache_max_entries: 16384 +``` + +{% endcode %} + +- `prefixes`: serve only normalized paths that start with a listed prefix. An empty list considers every path. Each prefix must start with `/`. Use a trailing slash, such as `/assets/`, to match a directory. Prefixes are not removed from the file path. +- `cache_ttl`: cache file metadata, including the ETag and content type. This cache is active only when `calculate_etag` is enabled. The default is `10s`. An explicit `0s` disables it. +- `cache_miss_ttl`: cache missing-file and directory results. The default is `10s`. An explicit `0s` disables it. +- `cache_max_entries`: entry limit for each cache. The default is `16384`; zero selects the default. A full cache attempts to remove expired entries. If it cannot free space, RR serves the request without adding a new entry. + +Negative TTLs and negative entry limits are invalid. + +A positive cache hit still opens the file and reads its metadata. RR reuses cached metadata only when the file size and modification time match. It detects deleted files and changed metadata on the next request. A cached miss avoids the filesystem lookup. RR can continue to send requests for a newly created file to PHP until `cache_miss_ttl` expires. + +If a deployment preserves both file size and modification time, RR can reuse an old ETag until `cache_ttl` expires. Run `rr reset static` after such a deployment to clear both caches. This command requires [RPC](../php/rpc.md). Set both TTLs to `0s` to disable caching. + +These options were added in [static PR #147](https://github.com/roadrunner-server/static/pull/147). + +## ETags + +Set `calculate_etag: true` to add ETags. Strong ETags use CRC32C of the file contents. RR does not generate a strong ETag for an empty file or a file larger than 32 MiB. Set `weak: true` to use file size and modification time for a weak ETag. + +Clients can send `If-None-Match` with a stored ETag. An unchanged file can return `304 Not Modified` without its body. Store and send ETags as opaque values. Do not calculate them in the client. + ## Fileserver plugin The Fileserver plugin serves static files. It works similarly to the `static` HTTP middleware and has extended functionality. -Static HTTP middleware slows down request processing by `~10%` because RR has to check each request for the -corresponding file. -The file server plugin uses a different port and serves only static files. +The `static` middleware runs on the main HTTP endpoint. The Fileserver plugin uses a separate listener and serves only static files. ## File server configuration +Startup fails if `address` is empty or `serve` has no entries. Each `prefix` must be nonempty and start with `/`. + {% code title=".rr.yaml" %} ```yaml @@ -95,7 +137,7 @@ fileserver: # # Error on empty address: 127.0.0.1:10101 - # Etag calculation. Request body CRC32. + # ETag calculation from the response body. # # Default: false calculate_etag: true @@ -105,7 +147,7 @@ fileserver: # Default: false weak: false - # Enable body streaming for files more than 4KB + # Stream incoming request bodies. # # Default: false stream_request_body: true @@ -113,7 +155,7 @@ fileserver: serve: # HTTP prefix # - # Error on empty + # Required. Must start with a forward slash. - prefix: "/foo" # Directory to serve @@ -133,7 +175,7 @@ fileserver: # The value for the Cache-Control HTTP-header. Units: seconds # - # Default: 10 seconds + # Default: 0 (no Cache-Control header) max_age: 10 # Enable range requests @@ -151,3 +193,23 @@ fileserver: ``` {% endcode %} + +### Unix Socket + +The Fileserver plugin supports [Unix socket attributes](../intro/config.md#unix-socket-attributes). Set optional `mode`, `uid`, and `gid` fields under `fileserver.unix_socket`: + +{% code title=".rr.yaml fragment" %} + +```yaml +fileserver: + address: "unix:///run/roadrunner/files.sock" + unix_socket: + mode: "0660" + serve: + - prefix: "/" + root: "public" +``` + +{% endcode %} + +These settings apply to the Fileserver listener. See [Fileserver PR #95](https://github.com/roadrunner-server/fileserver/pull/95). diff --git a/http/zstd.md b/http/zstd.md new file mode 100644 index 0000000..bdedd8c --- /dev/null +++ b/http/zstd.md @@ -0,0 +1,53 @@ +# HTTP - Zstd middleware + +The `zstd` middleware compresses HTTP response bodies when the client sends `Accept-Encoding: zstd`. It uses a pure Go implementation and does not require CGO. It does not decompress request bodies. + +## Availability + +The RoadRunner v3 bundle includes the Zstd plugin. Add `zstd` to `http.middleware` to enable it. + +## Configuration + +Add `zstd` to the middleware list in your existing HTTP configuration: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +http: + address: 127.0.0.1:8080 + middleware: [ "zstd" ] +``` + +{% endcode %} + +The middleware does not require a separate configuration section. Eligible responses use `Content-Encoding: zstd`. + +Put `zstd` before `static` or `sendfile` to compress their responses. See [middleware order](http.md#middleware-order). + +## Compression behavior + +- Clients must request `zstd` explicitly with a nonzero quality value. Missing or unsupported encodings and `zstd;q=0` leave the response uncompressed. +- The default compression level is `zstd.SpeedFastest`. The implementation reuses encoders through a pool. +- The normal minimum response size is 1,024 bytes. A flush can start compression below this limit. +- The middleware skips HEAD requests, empty bodies, responses with an existing `Content-Encoding` or `Content-Range`, and content types excluded by the compression library. +- The middleware adds `Vary: Accept-Encoding`. It removes the original `Content-Length` when it compresses a response. +- The compression library does not change ETags by default. + +The middleware supports OpenTelemetry header propagation when RoadRunner tracing is active. + +## Using gzip and zstd + +The zstd middleware does not provide gzip fallback. The [gzip middleware](gzip.md) is a separate plugin. + +{% hint style="warning" %} +If both gzip and zstd middleware are enabled, their order can determine the selected encoding. The separate plugins do not compare quality values with each other. Do not expect them to select the encoding with the highest quality value across both plugins. +{% endhint %} + +## Documentation + +- [Zstd middleware plugin](https://github.com/roadrunner-server/zstd) +- [Zstd middleware PR #1](https://github.com/roadrunner-server/zstd/pull/1) +- [MDN Accept-Encoding header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Accept-Encoding) +- [HTTP response streaming](resp-streaming.md) diff --git a/integration/migration.md b/integration/migration.md deleted file mode 100644 index c938c2d..0000000 --- a/integration/migration.md +++ /dev/null @@ -1,118 +0,0 @@ -# Migration from v1.0 to v2.0/v2023+ - -To migrate integration from RoadRunner v1.* to v2.*/v2023+, follow these steps. - -## Update Configuration - -The second version of RoadRunner uses a single worker factory for all of its plugins. This means that you must include a new -`server` section in your config, which is responsible for worker creation. The limit service is no longer presented as a separate -entity but rather as part of specific service configuration. - -{% code title=".rr.yaml" %} - -```yaml -rpc: - listen: tcp://127.0.0.1:6001 - -server: - command: "php tests/psr-worker-bench.php" - -http: - address: "0.0.0.0:8080" - pool: - num_workers: 4 -``` - -{% endcode %} - -{% hint style="warning" %} -Read more in the [config reference](/intro/config.md). -{% endhint %} - -## No longer worry about echoing - -RoadRunner 2.0+ intercepts all output to STDOUT. This means you can start using default var_dump and other echo -functions without breaking the communication. Yay! - -## Explicitly declare PSR-15 dependency - -We no longer ship a default PSR implementation with RoadRunner; make sure to include one you like the most -yourself. -For example: - -```bash -$ composer require nyholm/psr7 -``` - -## Update Worker Code - -RoadRunner simplifies worker creation: use the static `create()` method to automatically configure your worker: - -{% code title="worker.php" %} - -```php -waitRequest()) { - try { - $rsp = new Psr7\Response(); - $rsp->getBody()->write('Hello world!'); - - $worker->respond($rsp); - } catch (\Throwable $e) { - $worker->getWorker()->error((string)$e); - } -} -``` - -{% endcode %} - -## Update RPCs - -To create an RPC client, use the new Goridge API: - -{% code title="worker.php" %} - -```php -=2023.x.x** | **3** | -| **>=2.8** | **2.7** | -| **2.7.x** | **2.7** `OR` Unversioned (treated as `v2.6.0`, will be auto-updated to `v2.7`) | -| **<=2.6.x** | Doesn't support versions | +Keep `version: "3"` at the root of `.rr.yaml`. The configuration version is separate from the RoadRunner release and the Go plugin versions. -{% hint style="info" %} -*non-versioned: configuration used in the 2.0.x-2.6.x releases. -{% endhint %} +## PHP Applications -## Changelog +Use compatible RoadRunner PHP packages for each enabled plugin. The [RPC reference](../php/rpc.md) describes codec compatibility. The [plugin API reference](../customization/plugin.md#dto-compatibility) describes changes to generated DTOs. -### v3.0 Configuration and RR v2023.x.x +## Custom Builds -#### Reload plugin update - -{% hint style="warning" %} -The `reload` plugin has been removed from the default plugins list. Please use `*.pool.debug=true` instead. -{% endhint %} - -#### OpenTelemetry middleware update - -Starting from version **v2023.1.0**, the OpenTelemetry (OTEL) middleware configuration has been moved out of the HTTP -plugin to support its usage across multiple plugins, including HTTP, gRPC, jobs, and temporal. The OTEL middleware is now -configured using a top-level YAML key. - -**RoadRunner 2.x:** - -{% code title=".rr.yaml" %} - -```yaml -# HTTP plugin settings. -http: - ... - middleware: [ "otel" ] - otel: - insecure: true - compress: false - client: http - exporter: otlp - custom_url: "" - service_name: "rr_test" - service_version: "1.0.0" - endpoint: "127.0.0.1:4318" -``` - -{% endcode %} - -**RoadRunner v2023.x.x:** - -{% code title=".rr.yaml" %} - -```yaml -http: - ... - middleware: [ "otel" ] - -otel: - insecure: true - compress: false - client: http - exporter: otlp - custom_url: "" - service_name: "rr_test" - service_version: "1.0.0" - endpoint: "127.0.0.1:4318" -``` - -{% endcode %} - -## Updating from `version: 2.7` to `version: 3` - -To update your configuration from version 2.7 to version 3, follow these steps: - -1. **Update the version number:** Change the `version` value from `2.7` to `3`. -2. **Relocate the `otel` middleware configuration:** If your configuration uses the `otel` middleware configuration - within the `http` plugin, move it to the configuration root by cutting it from the `http` plugin and pasting it at - the root level. -3. **Remove** the `reload` plugin configuration and if needed, use the `*.pool.debug=true` option instead. - -### Upgrading to RoadRunner v2024.1.x - -**There are no breaking changes in the userland API.** - -#### Configuration - -1. **`server.relay_timeout`** was deprecated and replaced internally with the Go context timeout. Starting from `v2024.1.0`, this option is a no-op. - -#### ⚠️ HTTP plugin ⚠️ - -{% hint style="warning" %} -Starting from `v2024.1.0` RR uses the protobuf encoded messages to send the payloads to the PHP workers via pipes and our protocol called `goridge`. -That means you'll need to install the `protobuf` PHP extension to benefit from increased performance. The PHP userland API remains the same. -{% endhint %} - -{% hint style="info" %} -RR uses `pipes` and our custom protocol called `goridge` to communicate with the PHP worker. -However, the user payload was usually encoded with a `JSON` codec. -This approach led to issues, -such as broken raw binary payloads due to `JSON's` limitations and fields reordering, even in raw mode. -With the new protobuf codec, these problems have been resolved. -Now, by using the `http.raw_body=true` option, you'll receive the payload completely untouched as it is. -Additionally, encoded images or raw binary payloads will no longer be escaped -(which previously led to broken images in payloads not encoded in base64). -{% endhint %} - -#### Compatibility with RoadRunner PHP packages - -1. The `spiral/roadrunner-http` and `spiral/roadrunner-worker` packages should be upgraded to version **3.5**. Old RR versions up to `v2023.3.12` are also supported with the latest versions of the PHP packages. - -### Upgrading to RoadRunner v2024.2.x - -{% hint style="info" %} -**There are no breaking changes in the userland API and no configuration changes.** -{% endhint %} - -#### Information for plugin developers - -{% hint style="warning" %} - -Starting from `v2024.2.0`, all plugins were updated to version 5 due to the deprecation of the RoadRunner [SDK](https://github.com/roadrunner-server/sdk). -Pool, Workers, and Context tools are now available separately: - -- The `util.CreateListener` API was moved to the [TcpListen](https://github.com/roadrunner-server/tcplisten) package with the same functionality. -- `util.Context` (and various context keys) was moved to the [Context](https://github.com/roadrunner-server/context) package. -- `Pool`, `Workers API`, and all `IPC` related stuff was moved to the [Pool](https://github.com/roadrunner-server/pool) package. You only need to replace `sdk/v4` with `pool` in your imports. For example, `github.com/roadrunner-server/sdk/v4/pool/static_pool/config` -> `github.com/roadrunner-server/pool/static_pool/config`. -- `events` (events bus package from `github.com/roadrunner-server/sdk/v4/events`) was moved to the [Events](https://github.com/roadrunner-server/events) package. - -{% endhint %} - -### Upgrading to RoadRunner v2025 - -- For users of the RoadRunner library, the only required update is to change the Go package version from `v2024` to `v2025` in your `go.mod` file and package imports. -- There are no breaking changes in the userland API and no configuration changes. +Build custom Go plugins with the v6 interfaces and module paths. See [Writing a Plugin](../customization/plugin.md) and [Building RR with a Custom Plugin](../customization/build.md). Source builds require Go 1.27 or later. diff --git a/intro/config.md b/intro/config.md index e108972..0643c92 100644 --- a/intro/config.md +++ b/intro/config.md @@ -1,4 +1,4 @@ -# RoadRunner β€” Configuration +# RoadRunner Configuration RoadRunner supports both **YAML** and **JSON** configuration formats. The examples in our documentation use YAML, but you can use JSON as well. @@ -10,18 +10,59 @@ as . ## Configuration reference -The most recent configuration reference with all available options can be found in the `.rr.yaml` file in the RoadRunner -GitHub repository: +Use the configuration reference for the RoadRunner revision you build: -- [**.rr.yaml**](https://github.com/roadrunner-server/roadrunner/blob/master/.rr.yaml) +- [**.rr.yaml for v3.0.0**](https://github.com/roadrunner-server/roadrunner/blob/v3.0.0/.rr.yaml) {% hint style="warning" %} We use dots as level separators, e.g.: `http.pool`, you can't use dots in section names, queue names, etc. You can find out more about it [here](https://github.com/roadrunner-server/roadrunner/issues/1529). {% endhint %} +## Unix Socket Attributes + +{% hint style="info" %} +RoadRunner v3 supports these options through its v6 plugins. See the [v3 release notes](../releases/v3-0-0.md). +{% endhint %} + +Configure each filesystem Unix listener separately. Omit its options object to keep the existing socket defaults. + +The configuration provider can discard an empty options object (`{}`). It then behaves like omission, including on TCP addresses or the `pipes` worker relay. + +| Options object | Listener address | +| --- | --- | +| `http.unix_socket` | `http.address`, including H2C | +| `http.fcgi.unix_socket` | `http.fcgi.address` | +| `rpc.unix_socket` | `rpc.listen` | +| `grpc.unix_socket` | `grpc.listen` | +| `centrifuge.proxy_socket` | `centrifuge.proxy_address`, the incoming proxy listener | +| `fileserver.unix_socket` | `fileserver.address` | +| `server.relay_socket` | `server.relay`, the worker communication listener | + +All objects use the same optional fields: + +| Field | Meaning | +| --- | --- | +| `mode` | Quoted octal string from `"0000"` through `"0777"`. Omitted or empty means no mode change. | +| `uid` | Numeric socket owner ID. Omit it to keep the default owner. | +| `gid` | Numeric socket group ID. Omit it to keep the default group. | + +Use numeric UID and GID values from `0` through `4294967294` that fit the platform's Go `int` type. Zero is an explicit ID, not an omitted value. Account names are not supported. + +The configuration provider converts values before socket validation. Viper can convert booleans and fractional numbers to integer IDs. An unset or empty environment variable can become ID `0`, rather than cause an error. Set ID variables explicitly, or omit the fields to keep ownership unchanged. + +These options require a filesystem `unix://` address on Linux, macOS, or FreeBSD. They do not apply to TCP addresses such as `0.0.0.0:8000`, the `pipes` worker relay, Windows, or Linux abstract sockets. They do not change RoadRunner or PHP worker credentials, the process umask, or application file permissions. + +The plugin validates the decoded socket options during initialization, before it opens that listener. Invalid modes, out-of-range IDs, and incompatible listener addresses fail initialization. This validation does not check filesystem permissions. + +Create the parent directory before startup. RoadRunner needs permission to create the socket there. Clients need search (`x`) permission on every parent directory. The process also needs permission to apply the requested ownership changes. An unprivileged socket owner can change the socket group only to a group to which its process belongs. + +Ownership is applied before mode, after the socket starts listening. Clients can connect before these operations finish. The settings specify final attributes, not access control during startup. Existing directory permissions and umask must restrict initial access. Parent directories must prevent untrusted path replacement. If an attribute change fails, RoadRunner closes that listener and reports the error. + +See [Nginx group access](../app-server/nginx-with-rr.md#unix-socket) for a FastCGI example. + ## Configuration file -The RoadRunner looks for a configuration file named `.rr.yaml` in the same directory as the server binary. +RoadRunner looks for `.rr.yaml` in the current working directory. If your configuration file and other application files are located in a different directory than the binary, you can use the `-w` option to specify the working directory. @@ -55,6 +96,8 @@ Or you can combine the `-c` and `-w` options to specify both the configuration f {% endcode %} +Without `-w`, RoadRunner changes its working directory to the configuration file's directory. With `-w`, it uses that directory and resolves a relative `-c` path from there. + {% hint style="info" %} Read more about starting the server in the [**Server Commands**](../app-server/cli.md) section. @@ -79,7 +122,7 @@ exec /var/www/rr \ -w /var/www \ -o http.pool.num_workers=${RR_NUM_WORKERS:-8} \ -o http.pool.max_jobs=${RR_MAX_JOBS:-16} \ - -o http.pool.supervisor.max_worker_memory=${RR_MAX_WORKER_MEMORY:-512} + -o http.pool.supervisor.max_worker_memory=${RR_MAX_WORKER_MEMORY:-512} \ serve ``` @@ -101,9 +144,9 @@ In this example, the following options are used: | **/var/www/config/.env** | File that contains the required environment variables. | | **${RR_NUM_WORKERS:-8}** | Sets the number of workers to `RR_NUM_WORKERS` from the `.env` file or uses the default value of `8` if the variable is not present. | -## Support for the nested configurations: `[>=2024.2.0]`, [issue](https://github.com/roadrunner-server/roadrunner/issues/935) +## Include configuration files -Using the following syntax, you may include other configuration files into the main one: +List additional configuration files under `include`. RoadRunner reads the list in order after it reads the main file and applies CLI overrides. Includes do not require experimental mode. {% code title=".rr.yaml" %} @@ -111,77 +154,66 @@ Using the following syntax, you may include other configuration files into the m version: "3" include: - - .rr.include1-sub1.yaml - - ../foo/bar/rr.include1-sub2.yaml - -reload: - interval: 1s - patterns: [".php"] + - config/http.yaml + - config/local.yaml ``` {% endcode %} -{% hint style="warning" %} - -Starting from version `v2024.2.0`, the `includes` configuration option no longer has restrictions on where the included config can be placed. However, please note that the path for the included configurations is calculated based on the working directory of the RoadRunner process. +Paths are relative to the RoadRunner process working directory. Each included file must contain the same string `version` as the main file. A missing file, invalid YAML, or a different version stops initialization. -{% endhint %} - -Includes override the main configuration file. For example, if you have the following nested configuration: +For example, put the HTTP settings in this file: -{% code title=".rr.include1-sub1.yaml" %} +{% code title="config/http.yaml" %} ```yaml version: "3" server: - command: "php php_test_files/psr-worker-bench.php" + command: "php worker.php" relay: pipes http: - address: 127.0.0.1:15389 - middleware: - - "sendfile" + address: 127.0.0.1:8080 pool: - allocate_timeout: 10s num_workers: 2 ``` {% endcode %} -It will override the `server` and `http` sections of the main configuration file. -You may use env variables in the included configuration files, but you can't use overrides for the nested configuration. For example: +An included value overrides a matching value from the main file or an earlier include. Values found only in the main file remain available. Included sections can also override values supplied with `-o`. Use complete sections when replacing an earlier include. Lists are replaced, not appended. Includes have no delete operation and do not process another `include` list recursively. -{% hint style="info" %} +Environment expansion also applies to included files: -The next 'include' will override values set by the previous 'include'. Values in the root `.rr.yaml` will also be overwritten by the includes. Feel free to send us feedback on this feature. - -{% endhint %} - -{% code title=".rr.include1-sub1.yaml" %} +{% code title="config/local.yaml" %} ```yaml version: "3" -server: - command: "${PHP_COMMAND:-php_test_files/psr-worker-bench.php}" - relay: pipes +logs: + level: "${RR_LOG_LEVEL:-debug}" ``` {% endcode %} -You may use any number of the included configuration files via CLI command, in quotas and separated by whitespace. For example: +To set the include list from the CLI, pass a quoted, space-separated list: {% code %} ```bash -./rr serve -e -c .rr.yaml -o include=".rr.yaml .rr2.yaml" +./rr serve -c .rr.yaml -o 'include=config/http.yaml config/local.yaml' ``` {% endcode %} +### Local environment values + +Every include is required. The `include` setting accepts file paths, not objects with `file` and `optional` fields. For developer-specific values, use [environment defaults and `envfile`](../php/environment.md#dotenv). An `envfile` must also exist when configured. + +Use `version: "3"` in the main file and all includes. The configuration version is separate from the RoadRunner release number. See the [Config plugin](../plugins/config.md#configuration-file-structure). + ## What's Next? 1. [Server Commands](../app-server/cli.md) - learn how to start the server. -2. [PHP Workers β€” Environment variables](../php/environment.md) - learn how to configure PHP workers environment. -3. [Config plugin](../plugins/config.md) - learn more about the Config plugin. \ No newline at end of file +2. [PHP worker environment variables](../php/environment.md) - learn how to configure the worker environment. +3. [Config plugin](../plugins/config.md) - learn more about the Config plugin. diff --git a/intro/install.md b/intro/install.md index ce034c4..9f060fb 100644 --- a/intro/install.md +++ b/intro/install.md @@ -1,106 +1,73 @@ -# RoadRunner β€” Installation +# RoadRunner Installation -There are several ways to install RoadRunner, depending on your needs and preferences. +These guides describe RoadRunner v3 with v6 plugins. -## Pre-built Binaries - -The simplest way to get the latest version of RoadRunner is to download one of the pre-built release binaries, which are -available for various operating systems, including macOS, Linux, FreeBSD, and Windows. You can find these binaries on -the GitHub [releases page](https://github.com/roadrunner-server/roadrunner/releases). - -To install RoadRunner, just download the appropriate archive from the releases page and extract it into your desired -application directory. - -## Docker - -If you prefer to use RoadRunner inside a Docker container, you can use the official RoadRunner Docker -image `ghcr.io/roadrunner-server/roadrunner:latest`. - -{% hint style="info" %} -More information about available tags can be -found [here](https://github.com/roadrunner-server/roadrunner/pkgs/container/roadrunner). -{% endhint %} +## Requirements -**Here is an example of usage:** +- Go `1.27.1` for the source build. +- PHP `8.5` with the `sockets` extension for the examples. +- Composer 2 for PHP dependencies. +- `curl` and `tar` to download and extract archives. -```dockerfile -FROM php:8.x-cli -COPY --from=ghcr.io/roadrunner-server/roadrunner:2025.X.X /usr/bin/rr /usr/local/bin/rr +Check the installed versions and PHP extensions: -# Install and configure your application -# ... - -CMD rr serve -c .rr.yaml +```bash +go version +php --version +php --modules +composer --version ``` -{% hint style="warning" %} -Don't forget to replace `2025.X.X` with the desired version of RoadRunner. -{% endhint %} - -## Composer - -If you use Composer to manage your PHP dependencies, you can install the `spiral/roadrunner-cli` package to download the -latest version of RoadRunner to your project's root directory. - -**Install the package** +## Pre-built Binaries -```terminal -composer require spiral/roadrunner-cli -``` +Download RoadRunner `v3.0.0` from the [GitHub release page](https://github.com/roadrunner-server/roadrunner/releases/tag/v3.0.0). Select the archive for your operating system and architecture. Extract `rr` into your application directory. -Run the following command to download the latest version of RoadRunner: +For Linux AMD64, run: -```terminal -./vendor/bin/rr get-binary +```bash +curl --fail --location --output roadrunner-3.0.0-linux-amd64.tar.gz \ + https://github.com/roadrunner-server/roadrunner/releases/download/v3.0.0/roadrunner-3.0.0-linux-amd64.tar.gz +tar -xzf roadrunner-3.0.0-linux-amd64.tar.gz --strip-components=1 \ + roadrunner-3.0.0-linux-amd64/rr +./rr --version ``` -The server binary will be available at the root of your project. - -{% hint style="warning" %} -The `php-curl` and `php-zip` extensions are required to download RoadRunner automatically. -The `php-sockets` extension needs to be installed to run RoadRunner. -Check your installed extensions with `php --modules`. -{% endhint %} +## Build From Source -## Debian Package - -For Debian-based operating systems such as **Ubuntu**, **Mint**, and **MX**, you can download the `.deb` package from -the RoadRunner GitHub releases page and install it using dpkg. - -**Just run the following commands:** +Run these commands from your application directory. The build uses the plugin versions in the downloaded `go.mod` and produces `./rr` for the host operating system and architecture. Set `RR_REF` to a full commit SHA when you need a repeatable build. ```bash -wget https://github.com/roadrunner-server/roadrunner/releases/download/v2024.X.X/roadrunner-2024.X.X-linux-amd64.deb -sudo dpkg -i roadrunner-2024.X.X-linux-amd64.deb +RR_REF=v3.0.0 +curl --fail --location --output rr-source.tar.gz \ + "https://github.com/roadrunner-server/roadrunner/archive/${RR_REF}.tar.gz" +mkdir rr-source +tar -xzf rr-source.tar.gz --strip-components=1 -C rr-source +CGO_ENABLED=0 go -C rr-source build -mod=readonly -trimpath \ + -ldflags "-s -X github.com/roadrunner-server/roadrunner/v3/internal/meta.version=${RR_REF}" \ + -o ../rr ./cmd/rr +./rr --version ``` -{% hint style="warning" %} -Don't forget to replace `2024.X.X` with the desired version of RoadRunner. -{% endhint %} +Inspect the compiled module versions with `go version -m ./rr`. Keep the source revision and `go.sum` with your build inputs. To select a different set of plugins, use the [Velox build guide](../customization/build.md). -## macOS package using [Homebrew](https://brew.sh/): -```terminal -brew install roadrunner -``` +## Docker -## Windows using [Chocolatey](https://community.chocolatey.org/): -```bash -choco install roadrunner -``` +Use the official image `ghcr.io/roadrunner-server/roadrunner:3.0.0`. The [Docker guide](../app-server/docker.md#build-the-application-image) shows how to copy its binary into a PHP application image. The v3 image tags are `3`, `3.0`, and `3.0.0`. -## CURL +For a custom source build, set `CGO_ENABLED=0` and build for Linux and the container architecture. For example, add `GOOS=linux GOARCH=amd64` before `go` for a Linux AMD64 image. Run the version check on the target platform when you cross-compile. See [Build the RoadRunner Image](../app-server/docker.md#build-the-roadrunner-image) to package a custom binary. -You can also install RoadRunner using curl and the `download-latest.sh` script from the RoadRunner GitHub repository. +## Composer -**Just run the following commands:** +Install the PHP packages needed by your worker. For an HTTP worker, run: ```bash -curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/roadrunner-server/roadrunner/master/download-latest.sh | sh +composer require spiral/roadrunner-http nyholm/psr7 ``` -## What's Next? +Composer installs the PHP libraries. Use a pre-built binary or a source build for the RoadRunner server. -After you have installed RoadRunner, you can proceed to the next steps and configure it for your needs. +## What's Next? -1. [RoadRunner β€” Configuration](./config.md). -2. [Developer Mode](../php/developer.md). +1. [Quick Start Guide](quick-start.md). +2. [RoadRunner Configuration](config.md). +3. [Developer Mode](../php/developer.md). diff --git a/intro/quick-start.md b/intro/quick-start.md index 5f717b1..dd5d107 100644 --- a/intro/quick-start.md +++ b/intro/quick-start.md @@ -2,14 +2,20 @@ This guide walks you through getting started with RoadRunner. You'll learn how to install RoadRunner and configure it for your project. -## Step 1: Download RR for your platform +## Step 1: Install RoadRunner -To begin, you need to download RR for your platform. Visit the [RR installation guide](install.md) and download the appropriate version for your operating system. +Follow the [installation guide](install.md) to install RoadRunner v3. Keep the `rr` binary in your project directory. ## Step 2: Install PHP RR requires PHP to run. If you don't have PHP installed, you can download it from the [official PHP website](https://www.php.net/downloads.php) and follow the installation instructions for your operating system. +Install [Composer](https://getcomposer.org/download/). Enable the PHP `sockets` extension. Run this command in the project directory to install the worker dependencies: + +```bash +composer require spiral/roadrunner-http nyholm/psr7 +``` + ## Step 3: Create a simple configuration Next, you need to create a simple configuration file for RR. Open a text editor and create a new file called `.rr.yaml`. Add the following content to the file: @@ -87,6 +93,9 @@ Now start the server. You should have the following files in the current folder: - `.rr.yaml` - `psr-worker.php` +- `composer.json` +- `composer.lock` +- `vendor/` - `rr` binary Then, open a terminal window in the current folder and run the following command: diff --git a/intro/v3-migration.md b/intro/v3-migration.md new file mode 100644 index 0000000..eea3363 --- /dev/null +++ b/intro/v3-migration.md @@ -0,0 +1,33 @@ +# Upgrade from v2025 to v3 + +This guide lists the major upgrade steps from RoadRunner v2025 with v5 plugins to RoadRunner v3 with v6 plugins. The linked reference pages contain the configuration examples and full behavior descriptions. See the [v3 release notes](../releases/v3-0-0.md) for new features. + +Keep `version: "3"` at the root of `.rr.yaml`. Source builds require Go 1.27 or later. + +## Application and Configuration Changes + +| Area | Upgrade action | +| --- | --- | +| HTTP middleware | Requests enter middleware from left to right. Reverse an existing v5 list to keep its execution order. See [Middleware order](../http/http.md#middleware-order). | +| HTTP/2 | Use HTTP/2 prior knowledge for cleartext connections. HTTP/1.1 `Upgrade: h2c` is no longer supported. See [HTTP/2](../http/http.md#http2). | +| AMQP | Move broker settings to named connections. Set `config.connection` for each pipeline. Move exchange and queue settings into their nested sections and remove AMQP `config.version`. See [RabbitMQ](../queues/amqp.md). | +| Logging | Replace `file_logger_options` with `output`. Use an external tool for log rotation. Update JSON parsers for `time` and uppercase levels. Output is asynchronous; a full queue drops new records. See [Logger](../lab/logger.md). | +| Tracing | Replace the removed Zipkin exporter with OTLP. Use the server span to measure request latency; middleware spans end before the next handler starts. See [OpenTelemetry](../lab/otel.md). | +| RPC | Replace MessagePack with a supported codec. Goridge v4 supports JSON, protobuf, Gob, and raw payloads. See [RPC](../php/rpc.md). | +| Jobs | Reserve the `pool` job header for named-pool routing. Configure either `jobs.pool` or `jobs.pools`. The default `pipeline_size` is `100000`. Set `pipeline_size: 1000000` to keep the v5 limit. See [Jobs](../queues/overview-queues.md). | +| NATS | Review stream retention and repeated-job handling. Stopping a pipeline now preserves messages unless stream deletion is enabled. A new consumer can receive retained messages again after restart. See [NATS retention](../queues/nats.md#delete-stream-on-stop). | +| Centrifuge | Remove calls to `centrifuge.RateLimit`. Update direct clients for the supported DTOs and events. See [Centrifuge](../plugins/centrifuge.md). | +| Environment files | A root `envfile` is loaded without experimental mode. Supply the configured file or remove the setting. See [Environment](../php/environment.md). | +| Server initialization | Values in `server.on_init.env` override inherited environment values. Use a command sequence for arguments that contain spaces. See [Server](../plugins/server.md). | +| Metrics | Update Jobs dashboards for counter types and separate requeue totals. See [Metrics](../lab/metrics.md). | +| TCP plugin | RoadRunner v3 does not support the TCP plugin. Applications that use the `tcp:` plugin stay on RoadRunner v2025. This does not affect `tcp://` transport for RPC or worker relays. See [TCP](../plugins/tcp.md). | + +## Custom Go Plugins + +- Update RoadRunner library imports to `github.com/roadrunner-server/roadrunner/v3`. +- Replace imports from `api/v4` with generated bindings from `api-go/v6` or contracts from `api-plugins/v6`. The `api` repository contains protobuf schemas. +- Update plugin loggers to `*slog.Logger` and use the context-aware Jobs and KV contracts. +- Update worker pool imports to `pool/v2` and Goridge imports to `goridge/v4`. +- Update custom builds to the Velox v3 module configuration. + +See [Writing a Plugin](../customization/plugin.md), [Writing a Jobs Driver](../customization/jobs-driver.md), and [Building RR with a Custom Plugin](../customization/build.md) for imports, signatures, and build examples. diff --git a/known-issues/stdout-crc.md b/known-issues/stdout-crc.md index c876e3d..87ae19b 100644 --- a/known-issues/stdout-crc.md +++ b/known-issues/stdout-crc.md @@ -7,6 +7,6 @@ How to fix that? 1. Check your application. All dependencies should send their messages to `STDERR` instead of `STDOUT`. You may see the output after the `invalid message`. It might be truncated on Windows. 2. Check any `dd` or `echo` statements you inserted. PHP workers can redirect `echo` automatically from `STDOUT` to `STDERR`, but only after the worker is fully initialized. -3. A worker from RR v1 was used. To update, see: [link](../integration/migration.md) +3. Check that the application uses a compatible [PHP worker](../php/worker.md). 4. OPcache is enabled with JIT, but some extensions don't support it, which leads to warnings. Adjust the `error_reporting` configuration (use only errors): [issue](https://github.com/roadrunner-server/roadrunner/issues/1306) 5. If you use a Symfony runtime, add `APP_RUNTIME` to the server environment variables, as described here: https://github.com/php-runtime/roadrunner-symfony-nyholm. diff --git a/kv/boltdb.md b/kv/boltdb.md index 443b860..37dc80a 100644 --- a/kv/boltdb.md +++ b/kv/boltdb.md @@ -27,10 +27,6 @@ kv: # Default: 0777 permissions: 0777 - # Optional section. - # Default: "rr" - bucket: "rr" - # Optional section. # Default: 60 interval: 60 @@ -49,20 +45,25 @@ own at startup. Note that this must be an existing directory, otherwise a "The s error will occur, indicating that the full database pathname is invalid. Might be a full path with file: `/foo/bar/rr1.db`. Default: `rr.db`. +Use a separate database file for each KV storage. Each storage opens the file with an exclusive lock. Sharing a file between storage instances causes startup to fail. + +The driver uses the fixed internal bucket `default`. It does not support a `bucket` configuration option. + ### Permissions `permissions`: The file permissions in UNIX format of the database file, set at the time of its creation. If the file already exists, the permissions will not be changed. -### Bucket +### Interval -`bucket`: The bucket name. You can create several boltdb connections by specifying different buckets and in this case -the data stored in one bucket will -not intersect with the data stored in the other, even if the database file and other settings are completely -identical. +`interval`: The time in seconds between expiration checks. Expired entries can remain readable until the next check. -### Interval +## Persistence + +Values remain in the database after a RoadRunner restart. Expiration timestamps are kept only in memory and are lost at restart. Reapply expiration timestamps from application data after startup if needed. Use [Redis](./redis.md) when expiration must survive a RoadRunner restart. + +Upgrading from v5 does not require a database format conversion. Stop RoadRunner before backing up the database file. + +## Write Errors -`interval`: The interval (in seconds) between checks for the lifetime of the -value in the cache. The meaning and behavior is similar to that used in the -case of the memory driver. +`kv.Set` and `kv.Delete` return database commit errors through RPC. Handle these errors before treating a write or deletion as successful. diff --git a/kv/memory.md b/kv/memory.md index 598c074..50aa39e 100644 --- a/kv/memory.md +++ b/kv/memory.md @@ -3,8 +3,7 @@ This type of driver is already supported by the RoadRunner and does not require any additional installations. {% hint style="warning" %} -This type of storage, all data is contained in memory and will be destroyed when the RoadRunner Server is restarted. -If you need persistent storage without additional dependencies, then it is recommended to use the boltdb driver. +Restarting RoadRunner removes all data from this storage. For persistent values, see [BoltDB](./boltdb.md#persistence), which does not persist expiration metadata. {% endhint %} ## Configuration @@ -27,4 +26,14 @@ kv: {% endcode %} -There are no additional configuration options for this driver. The `in-memory` driver will automatically create callbacks for items with TTL. \ No newline at end of file +There are no additional configuration options for this driver. The `in-memory` driver will automatically create callbacks for items with TTL. + +## Expiration + +The `kv.MExpire` RPC method takes an absolute RFC 3339 `timeout` timestamp. + +{% hint style="warning" %} +`kv.MExpire` truncates the remaining time to whole seconds. A result of zero or less includes past deadlines and deadlines less than one second ahead when RoadRunner handles the request. In this case, the driver replaces the entry with the request's value and removes its TTL. A request with only a key and timeout can replace an existing value with empty bytes. +{% endhint %} + +Use `kv.Delete` for immediate removal. Do not use past or subsecond deadlines with `kv.MExpire`. diff --git a/kv/overview-kv.md b/kv/overview-kv.md index 08508b8..cbbaff1 100644 --- a/kv/overview-kv.md +++ b/kv/overview-kv.md @@ -59,6 +59,8 @@ To use the RoadRunner KV plugin, you need to define multiple key-value storages configuration file. Each storage must have a `driver` that indicates the type of connection used by those storages. At the moment, four different types of drivers are available: `boltdb`, `redis`, `memcached`, and `memory`. +An unknown driver name in `kv..driver` stops RoadRunner startup with a `no such constructor was registered` error. Check the driver name. Confirm that your RoadRunner binary includes the driver. + {% hint style="info" %} The `memory` and `boltdb` drivers do not require additional binaries and are available immediately, while the others require additional setup. Please see the appropriate documentation for installing [Redis Server](https://redis.io/) @@ -289,9 +291,11 @@ use Spiral\Goridge\RPC\RPC; use Spiral\RoadRunner\KeyValue\Factory; use Spiral\RoadRunner\KeyValue\Serializer\IgbinarySerializer; -$storage = (new Factory($rpc) +$rpc = RPC::create('tcp://127.0.0.1:6001'); + +$storage = (new Factory($rpc)) ->withSerializer(new IgbinarySerializer()) - ->select('storage'); + ->select('example'); ``` {% endcode %} @@ -329,13 +333,14 @@ After generating the keypair, you can use it to encrypt and decrypt the data. use Spiral\Goridge\RPC\RPC; use Spiral\RoadRunner\KeyValue\Factory; use Spiral\RoadRunner\KeyValue\Serializer\SodiumSerializer; -use Spiral\RoadRunner\KeyValue\Serializer\DefaultSerializer; -$storage = new Factory($rpc); - ->select('storage'); +$rpc = RPC::create('tcp://127.0.0.1:6001'); + +$storage = (new Factory($rpc)) + ->select('example'); // Encrypted serializer -$key = file_get_contents(__DIR__ . '/path/to/keypair.key'); +$key = file_get_contents(__DIR__ . '/keypair.key'); $encrypted = new SodiumSerializer($storage->getSerializer(), $key); // Storing public data @@ -356,7 +361,7 @@ To make it easy to use the KV proto API in PHP, we provide a [GitHub repository](https://github.com/roadrunner-php/roadrunner-api-dto), that contains all the generated PHP DTO classes proto files, making it easy to work with these files in your PHP application. -- [API](https://github.com/roadrunner-server/api/blob/master/proto/kv/v1/kv.proto) +- [API](https://github.com/roadrunner-server/api/blob/master/roadrunner/api/kv/v1/kv.proto) ### RPC API @@ -364,6 +369,8 @@ RoadRunner provides an RPC API, which allows you to manage key-value in your app The RPC API provides a set of methods that map to the available methods of the `Spiral\RoadRunner\KeyValue\Cache` class in PHP. +The plugin uses Goridge RPC with `kv/v1` Protobuf messages. Each request must name a configured storage. The `memory`, `redis`, and `memcached` drivers reject empty item lists for `Has`, `MGet`, `Set`, and `Delete`. The `memory` and `redis` drivers also reject empty `TTL` requests. Skip empty batches in direct RPC clients. `Clear` needs a storage name but no items. + #### Has Checks for the presence of one or more keys in the specified storage. @@ -404,6 +411,8 @@ func (r *rpc) MGet(in *kvv1.Request, out *kvv1.Response) error {} Sets the expiration time for one or more keys in the specified storage. +The `timeout` field is an absolute RFC 3339 timestamp, not a duration in seconds. Read the [memory expiration warning](./memory.md#expiration) before sending deadlines near the current time. + {% code %} ```go diff --git a/kv/redis.md b/kv/redis.md index 8e0fa61..ab1d26f 100644 --- a/kv/redis.md +++ b/kv/redis.md @@ -48,7 +48,7 @@ kv: dial_timeout: 0 # Optional section. - # Default: 0 (equivalent to the default value of 3 retries) + # Default: 0 (3 command retries after the initial attempt). max_retries: 0 # Optional section. @@ -95,7 +95,7 @@ kv: # Default: false read_only: false - # Optional section. + # Optional. Remove this section for a non-TLS connection. tls: # Optional section. # Default: "" @@ -175,8 +175,7 @@ disables idle timeout check. ### Retries -`max_retries`: Maximum number of retries before giving up. Specifying `0` is equivalent to the default (`3` attempts). -If you need to specify an infinite number of connection attempts, specify the value `-1`. +`max_retries`: Maximum number of command retries after the initial attempt. A value of `0` selects the default of three retries. A value of `-1` disables command retries. It does not enable unlimited connection attempts. `min_retry_backoff`: Minimum backoff between each retry. Must be in the format of a "numeric value" + "time format suffix". A value of `0` is equivalent to a timeout of 8 milliseconds (`8ms`). A value of `-1` disables backoff. @@ -184,6 +183,8 @@ suffix". A value of `0` is equivalent to a timeout of 8 milliseconds (`8ms`). A `max_retry_backoff`: Maximum backoff between each retry. Must be in the format of a "numeric value" + "time format suffix". A value of `0` is equivalent to a timeout of 512 milliseconds (`512ms`). A value of `-1` disables backoff. +`min_retry_backoff` controls the minimum retry delay independently of `max_retry_backoff`. + ### Pool Size `pool_size`: Maximum number of RoadRunner socket connections. A value of `0` is equivalent to a `10` connections per @@ -192,18 +193,15 @@ cores in your system, then setting the option to 2 you will get 16 connections. ### TLS Configuration -The `tls` section allows you to configure Transport Layer Security (TLS) for secure communication with the Redis server. -If no options are defined in this section, the connection will default to non-TLS. +The `tls` section enables TLS for the Redis connection. Omit the section for a non-TLS connection. A TLS configuration without `root_ca` uses the system trust store. + +`cert`: Path to the client certificate file. Set this with `key` when the server requires client authentication. -`cert`: Path to a file containing the client certificate. This certificate is used to authenticate the client -when communicating with the server. +`key`: Path to the client private key file. -`key`: Path to a file containing the client private key. This key is used in conjunction with the client -certificate for mutual authentication. +`root_ca`: Optional path to PEM-encoded CA certificates. The driver adds these certificates to the system trust store to verify the server certificate. This option does not require `cert` or `key`. -`root_ca`: Path to a file containing the Certificate Authority (CA) certificates used to verify the server's certificate. -**Note**: This option can be used independently of the `cert` and `key` options. In cases where the server does not -require client certificate verification, you only need to provide the `root_ca` option. +An unreadable CA file or a CA file without valid PEM certificates prevents startup. ### Other @@ -273,7 +271,7 @@ Where new options means: Redis Sentinel provides high availability for Redis. You can find more information about [Sentinel on the documentation page](https://redis.io/topics/sentinel). -There are two additional options available for the Sentinel configuration: `master_name` and `sentinel_password`. +Sentinel configuration adds three options: `master_name`, `sentinel_username`, and `sentinel_password`. Set `addrs` to the Sentinel endpoints. {% code title=".rr.yaml" %} @@ -285,19 +283,33 @@ kv: driver: redis config: - # Required section. - master_name: "" + addrs: + - "${REDIS_SENTINEL_ADDRESS}" - # Optional section. - # Default: "" (no password) - sentinel_password: "" + # Required: master name configured in Sentinel. + master_name: "${REDIS_SENTINEL_MASTER_NAME}" + + # Optional: Redis master credentials. + username: "${REDIS_USERNAME}" + password: "${REDIS_PASSWORD}" + + # Optional: Sentinel credentials, independent of the master credentials. + sentinel_username: "${REDIS_SENTINEL_USERNAME}" + sentinel_password: "${REDIS_SENTINEL_PASSWORD}" ``` {% endcode %} -Where Sentinel's options means: +Sentinel options: + +- `master_name`: The name of the Redis master monitored by Sentinel. + +- `sentinel_username`: Optional Sentinel ACL username. Defaults to an empty string. + +- `sentinel_password`: Optional password for the Sentinel ACL user, or the password configured with `requirepass` for the default user. Defaults to an empty string. + +`username` and `password` authenticate against the Redis master. `sentinel_username` and `sentinel_password` authenticate against Sentinel. For password-only Sentinel authentication, omit `sentinel_username` or set it to `""`. -- `master_name`: The name of the Sentinel's master in string format. +## Expiration Errors -- `sentinel_password`: Sentinel password from "requirepass `password`" - (if enabled) in Sentinel configuration. +`kv.MExpire` returns Redis command errors through RPC. Handle these errors before assuming that an expiration was set. A batch can update some keys before a later key fails. diff --git a/lab/accesslogs.md b/lab/accesslogs.md index 696844d..e98635c 100644 --- a/lab/accesslogs.md +++ b/lab/accesslogs.md @@ -1,6 +1,6 @@ # Access logs (HTTP) -RoadRunner supports HTTP access logs, which provide detailed information about incoming HTTP requests and responses. +RoadRunner writes HTTP access logs through the `http` logger channel at `info` level. {% hint style="info" %} This feature is disabled by default, but it can be enabled by configuring the HTTP server. @@ -21,21 +21,52 @@ http: address: 127.0.0.1:8000 access_logs: true # ... + +logs: + channels: + http: + mode: production + level: info +``` + +{% endcode %} + +The message is `http access log`. Use production mode or a [custom format](logger.md#custom-format) with `%attrs%` to retain its fields: + +| Field | Value | +| --- | --- | +| `method` | HTTP request method. | +| `remote_address` | Client address. | +| `read_bytes` | Request body bytes read. | +| `write_bytes` | Response body bytes written. | +| `content_len` | Request content length. | +| `host` | Request host. | +| `URI` | Request URI. | +| `URL` | Request URL. | +| `query` | Query string. | +| `status` | Final HTTP response status. | +| `user_agent` | User-Agent header. | +| `referer` | Referer header. | +| `time_local` | Request start time in the `02/Jan/06:15:04:05 -0700` layout. | +| `request_time`, `start` | Request start timestamp. | +| `elapsed` | Request duration in whole milliseconds. | + +Use `elapsed` to measure request duration. The `request_time` field is a timestamp. See [Logger](logger.md) for JSON record fields and asynchronous output. + +## Client Addresses Behind a Proxy + +Enable `proxy_ip_parser` to use forwarded client addresses in `remote_address`. Set `trusted_subnets` to the networks that contain your proxies. Set `trusted_headers` to select the headers and their priority: + +{% code title=".rr.yaml fragment" %} + +```yaml +http: + access_logs: true + middleware: [proxy_ip_parser] + trusted_subnets: ["10.0.0.0/8"] + trusted_headers: ["X-Forwarded-For", "X-Real-IP"] ``` {% endcode %} -Once enabled, RoadRunner will log the following information for each incoming HTTP request: - -- `method` - HTTP method of the request -- `remote_addr` - Remote address of the client -- `bytes_sent` - Content length of the response -- `http_host` - Host name of the server -- `request` - Query string of the request -- `time_local` - Local time of the server in Common Log Format -- `request_length` - Request body size including headers in bytes (content length + size of all headers). The maximum allowed header - size for RR is 1 MB. -- `request_time` - Request processing time in seconds with millisecond resolution -- `status` - HTTP response status -- `http_user_agent` - HTTP user [agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent) string of the client -- `http_referer` - HTTP [referer](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer) string of the client +See [Trusted Subnets](../http/proxy.md) for address selection and configuration. diff --git a/lab/applogger.md b/lab/applogger.md index 7e8a25d..7272d17 100644 --- a/lab/applogger.md +++ b/lab/applogger.md @@ -1,17 +1,10 @@ # Application logger -The RoadRunner server has a useful `app-logger` plugin that allows users to send logs from their applications to the -RoadRunner server using an RPC interface. This plugin is enabled by default and does not require any additional -configurations. It can be used to observe all application and server logs in one place. This is especially useful when -debugging and monitoring applications. - -{% hint style="info" %} -It will send raw messages to the RoadRunner `STDERR` -{% endhint %} +The app-logger plugin accepts log messages over RPC. Level-based methods use the `app` logger channel. The `log()` method writes directly to RoadRunner's standard error. ## Configuration -The `logs` section in the RoadRunner configuration file allows you to configure logging behavior for their application. +Configure the `app` channel in the `logs` section to select the output format and minimum level: {% code title=".rr.yaml" %} @@ -22,21 +15,20 @@ rpc: logs: channels: app: + mode: production level: info ``` {% endcode %} {% hint style="warning" %} -To interact with the RoadRunner app-logger plugin, you will need to have the RPC defined in the rpc configuration -section. You can refer to the documentation page [here](../php/rpc.md) to learn more about the configuration. +Configure [RPC](../php/rpc.md#configuration) to use app-logger. {% endhint %} -The `level` key is used to specify the logging level for this channel. This means that only log messages with a severity -level of info or higher will be sent to this channel. +This example emits JSON records at `info` level or higher and filters out `debug`. These settings do not control raw log calls. {% hint style="info" %} -Read more about logging in the [Logging β€” Logger](./logger.md) section. +See [Logger](./logger.md) for formats, levels, and output destinations. Level-based methods use the logger's [asynchronous output queue](logger.md#asynchronous-output). {% endhint %} ## PHP client @@ -81,7 +73,7 @@ $logger->error('Houston, we have a problem!'); {% endcode %} {% hint style="info" %} -You can refer to the documentation page [here](../php/rpc.md) to learn more about creating the RPC connection. +See [RPC connections](../php/rpc.md) for connection setup. {% endhint %} ### Available methods @@ -92,75 +84,28 @@ You can refer to the documentation page [here](../php/rpc.md) to learn more abou - `warning(string): void`: Sends a warning log message to the server - `log(string): void`: Sends a log message directly to the `STDERR` of the server -## API - -### RPC API - -RoadRunner provides an RPC API, which allows you to manage app-logger in your applications using remote -procedure calls. The RPC API provides a set of methods that map to the available methods of -the `RoadRunner\Logger\Logger`class in PHP. - -{% hint style="info" %} -All methods accept a `string` (which will be log message) as a first argument and a `bool` placeholder for the second -arg. -{% endhint %} - -#### Error +### Raw Output -Method sends an `error` log message with the specified message to the RoadRunner server. +The raw RPC methods `app.Log` and `app.LogWithContext` write directly to standard error. They bypass logger levels, formats, output settings, and the asynchronous queue. These calls wait for the write to complete. The logger's `raw` mode uses its configured level, destinations, and asynchronous queue. -{% code %} - -```go -func (r *RPC) Error(in string, _ *bool) error {} -``` - -{% endcode %} - -#### Info +With app-logger v6, `app.Log` appends LF (`\n`) only when the message does not already end with LF. `app.LogWithContext` uses the same rule when there are no attributes. V5 wrote these messages without adding a line ending. Update consumers that depended on concatenated messages without line endings. -Method sends an `info` log message with the specified message to the RoadRunner server. +With attributes, `app.LogWithContext` writes the message, a space, comma-separated `key:value` pairs, and LF. It preserves newlines inside the message. Send the raw RPC message without a trailing newline if you need the attributes on the same line. -{% code %} - -```go -func (r *RPC) Info(in string, _ *bool) error {} -``` - -{% endcode %} - -#### Warning - -Method sends a `warning` log message with the specified message to the RoadRunner server. - -{% code %} - -```go -func (r *RPC) Warning(in string, _ *bool) error {} -``` - -{% endcode %} - -#### Debug - -Method sends a `debug` log message with the specified message to the RoadRunner server. - -{% code %} - -```go -func (r *RPC) Debug(in string, _ *bool) error {} -``` - -{% endcode %} +## API -#### Log +### RPC API -Method sends a log message with the specified message directly to the `STDERR` of the RoadRunner server. +The string methods accept a message and a boolean reply placeholder. Call them by their registered RPC names: -{% code %} +| Method | Output | +| --- | --- | +| `app.Error` | Error-level record through the `app` logger. | +| `app.Info` | Info-level record through the `app` logger. | +| `app.Warning` | Warning-level record through the `app` logger. | +| `app.Debug` | Debug-level record through the `app` logger. | +| `app.Log` | Raw standard error output. | -```go -func (r *RPC) Log(in string, _ *bool) error {} -``` +Each method also has a `WithContext` variant, such as `app.InfoWithContext`. These methods accept `LogEntry` and a `Response` placeholder from `github.com/roadrunner-server/api-go/v6/applogger/v1`. The entry carries the message and `LogAttrs` key/value pairs. -{% endcode %} +The plugin's Go RPC receiver is no longer exported as `app.RPC`. Custom containers obtain it through `Plugin.RPC()`; PHP RPC method names are unchanged. diff --git a/lab/health.md b/lab/health.md index 4d15994..0e14885 100644 --- a/lab/health.md +++ b/lab/health.md @@ -27,11 +27,12 @@ can change the address to any IP address and port number of your choice. To access the health check, use the following URL: `http://127.0.0.1:2114/health`. This URL will return the health status of all plugins that are enabled and support health probes. To specify a particular plugin, you need to use the `plugin` query parameter: `http://127.0.0.1:2114/health?plugin=http`. In that case, the health status of the `http` plugin will be returned. {% hint style="info" %} -You can specify multiple plugins by separating them with a comma. For example, to check the health status of both the -http and grpc plugins, you can use the following URL: http://127.0.0.1:2114/health?plugin=http&plugin=grpc. +Repeat the `plugin` query parameter to check multiple plugins: `http://127.0.0.1:2114/health?plugin=http&plugin=grpc`. Names that do not support the check are skipped. {% endhint %} -The health check endpoint will return `HTTP 200` if there is at least one worker ready to serve requests. If there are no workers ready to service requests, the endpoint will return `HTTP 503` (or your unavailable status code, which can be set via configuration of the plugin). If there are any other errors, the endpoint will also return `HTTP 503` (or your unavailable status code). The `status` plugin also returns a payload with a list of checked plugins and errors, if any, in the following format: +The `/health` endpoint calls each selected plugin's health check. For worker-pool checks, an active worker can be busy with a request. Use `/ready` to check for idle workers. + +If a checked plugin reports a status of `500` or higher, the HTTP response uses `unavailable_status_code` (`503` by default). The JSON response lists the checked plugins and their reported status or errors: ```json [ @@ -52,9 +53,7 @@ The health check endpoint will return `HTTP 200` if there is at least one worker To access the readiness check, use the following URL: `http://127.0.0.1:2114/ready`. -The readiness check endpoint will return `HTTP 200` if there is at least one worker ready to take the request (i.e., not -currently busy with another request). If there is no worker ready or all workers are busy, the endpoint will return -`HTTP 503` status code (you can override this with the `unavailable_status_code` option). +For worker-pool checks, `/ready` returns `HTTP 200` when each checked plugin has at least one idle worker. If a checked pool has no ready workers, including when all workers are busy, the endpoint returns `unavailable_status_code` (`503` by default). Like the health check, you can target a specific plugin using the `plugin` query parameter: @@ -62,12 +61,22 @@ Like the health check, you can target a specific plugin using the `plugin` query - `http://127.0.0.1:2114/ready?plugin=grpc` {% hint style="info" %} -You can specify multiple plugins by separating them with a comma. For example: -`http://127.0.0.1:2114/ready?plugin=http&plugin=grpc`. +Repeat the `plugin` query parameter to check multiple plugins: `http://127.0.0.1:2114/ready?plugin=http&plugin=grpc`. {% endhint %} The response format is the same JSON structure as the `/health` endpoint. +## Kubernetes Endpoint Aliases + +The status server provides these endpoint aliases: + +| Endpoint | Alias | Check | +| --- | --- | --- | +| `/health` | `/livez` | Liveness | +| `/ready` | `/readyz` | Readiness | + +Each alias uses the same handler as its original endpoint. Both names accept the same `plugin` query parameters and return the same status codes and response bodies. For example, use `/livez?plugin=http` or `/readyz?plugin=http&plugin=grpc`. + ## Customizing the Not-Ready Status Code By default, the Status Plugin uses a `503` status code. However, you can replace this status code with a custom one. @@ -86,10 +95,15 @@ status: {% endcode %} +## Graceful Shutdown + +During graceful shutdown, `/health` and `/livez` return `200`. The `/ready`, `/readyz`, and `/jobs` endpoints return `unavailable_status_code` (`503` by default). These responses contain the text `service is shutting down`, not the usual JSON report. + +Use `/health` for liveness and `/ready` for readiness. This lets the process finish its current work after readiness checks stop new traffic. + ## Check Timeout -The status plugin uses a timeout when checking the status of plugins. By default, this timeout is **60 seconds**. You -can customize it using the `check_timeout` option: +Set `check_timeout` to an integer number of seconds. The default is `60`. This value sets the status server's HTTP request and header read timeouts. It does not set a deadline for `Status()`, `Ready()`, or `JobsState()` execution. {% code title=".rr.yaml" %} @@ -98,7 +112,7 @@ version: "3" status: address: 127.0.0.1:2114 - check_timeout: 30s + check_timeout: 30 ``` {% endcode %} @@ -108,21 +122,59 @@ status: In addition to checking the health status of the workers, you can also examine the pipelines in the Jobs plugin using the following URL: http://127.0.0.1:2114/jobs -This URL will return the status of the pipelines in the Jobs plugin. The output will be in the following format: +This endpoint returns a JSON array of pipeline states: -```log -plugin: jobs: pipeline: test-1 | priority: 13 | ready: true | queue: test-1 | active: 0 | delayed: 0 | reserved: 0 | driver: memory | error: +```json +[ + { + "pipeline": "test-1", + "priority": 13, + "ready": true, + "queue": "test-1", + "active": 0, + "delayed": 0, + "reserved": 0, + "driver": "amqp", + "error_message": "" + } +] ``` +If the Jobs plugin is absent, `/jobs` returns `unavailable_status_code`. The handler passes the HTTP request context to `JobsState()` so the check can respond to request cancellation. + ## Use cases The health check endpoint serves the following purposes: ### Kubernetes Readiness and Liveness Probes -In Kubernetes, you can use readiness and liveness probes to check the health of your application. It can be used as a -readiness or liveness probe to ensure that your application is ready to serve requests. You can configure Kubernetes to -check the health check endpoint and take appropriate action if the endpoint returns an error. +Configure the liveness probe to use `/livez` and the readiness probe to use `/readyz`. Busy workers can fail readiness without failing liveness. During shutdown, readiness fails while liveness remains successful. + +Bind the status server to an address that Kubernetes can reach, such as `0.0.0.0:2114`: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +status: + address: 0.0.0.0:2114 +``` + +{% endcode %} + +Add these probes to the RoadRunner container specification: + +```yaml +livenessProbe: + httpGet: + path: /livez + port: 2114 +readinessProbe: + httpGet: + path: /readyz + port: 2114 +``` **Read more [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/)** diff --git a/lab/logger.md b/lab/logger.md index 5c810b2..afc3df4 100644 --- a/lab/logger.md +++ b/lab/logger.md @@ -1,11 +1,9 @@ # Logger -Logger Plugin is responsible for collecting logs from server plugins and PHP application workers' `STDERR` and -displaying them in the RoadRunner `STDERR`/`STDOUT`. It comes with a variety of options that allow you to customize the -way your application logs are collected and displayed. +The logger plugin writes logs from RoadRunner plugins and PHP workers to standard error, standard output, or files. The `logs` section controls the format, minimum level, and output destinations. {% hint style="info" %} -PHP workers are internally, mapped to a logger with an INFO log level severity. Use `channels` to map the `server` plugin (responsible for PHP workers) log level to at least `info`: +PHP worker standard error is logged at `info` level through the `server` channel. Set that channel to `info` or `debug` to retain worker output when the root level is higher: {% code title=".rr.yaml" %} @@ -13,11 +11,10 @@ PHP workers are internally, mapped to a logger with an INFO log level severity. version: "3" logs: - encoding: console # default value - level: error # mapped to all plugins - mode: "production" # mapped to all plugins + mode: production + level: error channels: - server: # mapped to only server plugin + server: mode: production level: info ``` @@ -39,39 +36,58 @@ logs: {% endcode %} -There are three available modes: +| Mode | Output | +| --- | --- | +| `production` | JSON records with `time`, `level`, `msg`, and structured attributes. | +| `development` | Key/value text without console colors. This is the default mode. | +| `raw` | The message only. Attributes and groups are discarded. | +| `off`, `none` | No output from this logger. Channel overrides can still enable their own output. | -1. `production` - This mode uses logger settings that are optimized for production usage. -2. `development` - This mode is enabled by default and is designed for use during application development. In - development mode, DPanicLevel logs panic, console colors are used, and logs are written to standard error. Sampling - is disabled, and stack traces are automatically included on logs of WarnLevel and above. -3. `raw` - This mode displays messages as raw output without any formatting. This mode is useful in production - environments where you need to parse logs programmatically. +Unknown modes use the same text format as `development`. -{% hint style="info" %} -Use `production` mode in production environments. It is optimized for production usage. -{% endhint %} +Logger plugin v6 production records use a `time` string in RFC3339 format with up to nanosecond precision. Level names use uppercase, such as `INFO`. In v5, production records used a numeric `ts` value in epoch nanoseconds and lowercase levels. Update log parsers for these changes. Development output also changes from the v5 colored console format to key/value text. + +Example v6 production record: + +```json +{"time":"2026-08-17T12:00:00.123Z","level":"INFO","msg":"worker output","logger":"server"} +``` ### Encoding -Logger supports two types of encoding, `console` and `json`. By default, `console` encoding is used, which outputs logs -in a friendly format. JSON encoding, on the other hand, returns messages in a JSON Structured logging format. This -format presents log messages as JSON objects with key-value pairs representing each log message field, making them more -machine-readable and easier to process programmatically. JSON encoding is also better suited for production usage. +V6 ignores the `encoding` setting. Remove it from the root and channel configurations. Use `mode: production` for JSON or `mode: development` for text. + +### Custom Format + +Use `format` to select a custom text format. It takes precedence over `mode`, except that `off` and `none` still disable the logger. The `time_format` setting uses a [Go time layout](https://pkg.go.dev/time#Layout). Its default is RFC3339, and it applies only to `%time%` in a custom format. {% code title=".rr.yaml" %} ```yaml logs: - encoding: console + level: info + format: "%time% [%level%] %logger% %message% %attrs%" + time_format: "2006-01-02 15:04:05" ``` {% endcode %} +| Placeholder | Value | +| --- | --- | +| `%time%` | Record time, using `time_format`. | +| `%level%` | Level name, such as `INFO`. | +| `%message%` | Log message. | +| `%logger%` | Logger name. | +| `%attrs%` | Attributes as space-separated `key=value` pairs. | +| `%source_file%` | Go source file path. | +| `%source_line%` | Go source line number. | +| `%source_func%` | Go function name. | + +Unknown placeholders remain unchanged. When `%logger%` is present, `%attrs%` omits the `logger` attribute. Custom formats do not escape message or attribute text. Use production mode when you need JSON encoding. + ### Level -The level is used to specify the logging level. This means that only log messages with a severity level will be sent to -this channel. Available levels include `panic`, `error`, `warn`, `info`, and `debug`. +The `level` setting is the minimum severity to emit. Supported values are `debug`, `info`, `warn`, and `error`. The value `warning` also selects `warn`. Values are case-insensitive. {% code title=".rr.yaml" %} @@ -83,13 +99,12 @@ logs: {% endcode %} {% hint style="info" %} -The default level is `debug`. +In v6, an empty or unknown level selects `debug`, including in production and raw channels. Values such as `panic`, `dpanic`, and `fatal` are not supported. Replace these v5 values with a supported level; they do not disable logging or fail configuration validation. {% endhint %} ### Output -By default, RoadRunner sends logs to `STDERR`. However, you can configure RoadRunner to send logs to `STDOUT` by using -the output key. +The logger writes to standard error by default. The `output` list accepts `stderr`, `stdout`, and file paths. The logger writes each enabled record to every destination in the list. Paths are file paths, not URLs. {% code title=".rr.yaml" %} @@ -100,37 +115,35 @@ logs: {% endcode %} -### Error Output +### Asynchronous Output -You can configure a separate output destination for error-level logs. By default, error logs are sent to `STDERR`. +Log output is asynchronous for production, development, raw, and custom formats. Each configured logger has a queue for up to 1,024 records. A slow output destination does not block the caller. When the queue is full, the logger discards new records. The queue size is fixed. -{% code title=".rr.yaml" %} +During shutdown, the plugin drains the root and channel queues for up to five seconds, within the shutdown context deadline. If the deadline expires, it discards queued records. Output errors are returned during shutdown. -```yaml -logs: - error_output: [ stderr ] -``` +### Error Output -{% endcode %} +All enabled levels of a logger use the same output destinations. V6 ignores `err_output`. In v5, this setting controlled internal logger errors, not error-level records. Remove it from your configuration. The `error_output` key is not supported. ### Line Endings -It allows configuring custom line endings for the logger. By default, the plugin uses `\n` as the line ending. Note that the `\n` is a forced default. This means that if the value is empty, RoadRunner will still use `\n`. So no empty line endings are allowed. +Custom formats append `\n` by default. Set `line_ending` to change it. To append nothing, set `skip_line_ending: true`. This takes precedence over `line_ending`. An empty `line_ending` without `skip_line_ending` still selects `\n`. {% code title=".rr.yaml" %} ```yaml logs: + format: "%message%" line_ending: "\r\n" ``` {% endcode %} +These settings require a nonempty `format`. Standard production and development handlers append `\n`. Standard raw mode appends `\n` only when the message does not already end with it. + ### Channels -In addition, you can configure each plugin log messages individually using the `channels` section. It allows you to -customize the logger settings for each plugin independently. You can disable logging for a particular plugin or change -its log mode and output destination. +Use `channels` to configure a plugin logger separately. A channel configuration replaces the root settings for that plugin. Omitted channel settings use their own defaults, not the root values. Plugins without a channel override use the root logger. {% code title=".rr.yaml" %} @@ -138,81 +151,45 @@ its log mode and output destination. version: "3" logs: - encoding: console # default value - level: info - mode: none # disable server logging. Also, `off` can be used. + mode: none channels: http: mode: production + level: info output: [ http.log ] ``` {% endcode %} -## File Logger +If a channel output cannot be opened, the logger reports the error through the root logger and uses the root settings for that channel. A disabled root logger hides this error. Check the root configuration when a channel file is missing. A root-output open error fails initialization. -It is possible to redirect channels or the entire log output to a file. To use the file logger, you need to set -the `file_logger_options.log_output` option to the filename where you want to write the logs. +## File Output -### Entire log +Use a file path in `output` to write logs to a file. The plugin creates the file if needed and appends to an existing file. Create its parent directory before starting RoadRunner. {% code title=".rr.yaml" %} ```yaml logs: - mode: development - file_logger_options: - log_output: "test.log" - max_size: 10 - max_age: 24 - max_backups: 10 - compress: true + mode: production + level: info + output: ["rr.log"] ``` {% endcode %} -### Channel +V6 removes `file_logger_options`. Replace the v5 `file_logger_options.log_output` setting with an entry in `output`. For a channel, use `logs.channels..output` as shown above. -You can also redirect a specific channel to a file. To do this, you need to specify the channel name in the `channels` +There is no built-in log rotation, backup retention, or compression in v6. The v5 `max_size`, `max_age`, `max_backups`, and `compress` settings have no effect. The plugin does not reopen files after external rotation. Send logs to stdout or stderr and let a service manager, container runtime, or log collector manage rotation. -{% code title=".rr.yaml" %} +## Startup Logs -```yaml -logs: - mode: development - level: debug - channels: - http: - file_logger_options: - log_output: "test.log" - max_size: 10 - max_age: 24 - max_backups: 10 - compress: true -``` - -{% endcode %} - -### Available options - -1. `log_output`: Filename is the file to write logs to in the same directory. It uses `processname-lumberjack.log` in - `os.TempDir()` if empty. -2. `max_size`: is the maximum size in megabytes of the log file before it gets rotated. It defaults to 100 megabytes. -3. `max_age`: is the maximum number of days to retain old log files based on the timestamp encoded in their filename. - Note that a day is defined as 24 hours and may not exactly correspond to calendar days due to daylight savings, leap - seconds, etc. The default is not to remove old log files based on age. -4. `max_backups`: is the maximum number of old log files to retain. The default is to retain all old log files (though - MaxAge may still cause them to get deleted.) -5. `compress`: determines if the rotated log files should be compressed using gzip. The default is not to perform - compression. -6. `log_ending`: line ending to use in the logger. Default is new line - `\n`. - -### Startup logs +The build time below is a placeholder. {% code %} ```log -[INFO] RoadRunner server started; version: 2024.3.0, buildtime: 2024-12-05T18:39:32+0000 +[INFO] RoadRunner server started; version: 3.0.0, buildtime: [INFO] sdnotify: not notified ``` @@ -220,6 +197,8 @@ logs: These logs are not controlled by the logs configuration section. They are emitted directly by the RoadRunner core and can be turned off using the `-s` or `--silent` CLI option. -## ZapLogger +## Go Loggers + +V6 uses Go's [log/slog](https://pkg.go.dev/log/slog) instead of Zap. Named loggers return `*slog.Logger`. Use a `slog.Handler` for custom output. -Feel free to register your own [ZapLogger](https://github.com/uber-go/zap) extensions. +The plugin drains its queues and closes file outputs during `Stop`. If you call `Config.BuildLogger()` directly, close each resource in the returned `BuildResult.Closers` to drain its queue. If you construct a `Log` with `NewLogger`, call `Log.Close()` to drain its channel outputs. diff --git a/lab/metrics.md b/lab/metrics.md index d626153..591f542 100644 --- a/lab/metrics.md +++ b/lab/metrics.md @@ -43,6 +43,8 @@ The general metrics provided by the metrics plugin include: be `rr_jobs_total_workers`. {% endhint %} +Per-worker memory metrics have a `pid` label. Worker-state metrics have `pid` and `state` labels. Pool totals have no plugin-defined labels. + ### HTTP Metrics {% hint style="info" %} @@ -66,18 +68,24 @@ The HTTP metrics provided by the metrics plugin include: - `rr_http_request_total` - Total number of handled HTTP requests after server restart. - `rr_http_request_duration_seconds` - HTTP request duration. -- `rr_http_requests_queue` - Total number of queued requests waiting for a worker. +- `rr_http_requests_queue` - Requests currently inside the metrics middleware, including requests waiting for a worker and requests being processed. - `rr_http_uptime_seconds` - Plugin uptime in seconds. - `rr_http_no_free_workers_total` - Total number of NoFreeWorkers errors. +The request counter and duration histogram have a `status` label. A response body written without an explicit status records `status="200"`. The duration histogram has finite buckets from `0.005` through `60` seconds, including `20`, `30`, and `60` for slow requests. + +The duration histogram includes downstream request execution. The `http_metrics` and `http_metrics:post` spans described in [OpenTelemetry](otel.md) measure work before and after the downstream handler. Their duration does not change histogram duration. + ### gRPC Metrics The gRPC metrics provided by the metrics plugin include: -- `rr_grpc_requests_queue` - Total number of queued requests waiting for a worker. -- `rr_gprc_request_total` - Total number of handled gRPC requests after server restart. +- `rr_grpc_requests_queue` - Requests currently inside the gRPC interceptor, including requests waiting for a worker and requests being processed. +- `rr_grpc_request_total` - Total number of handled gRPC requests after server restart. - `rr_grpc_request_duration_seconds` - gRPC request duration. +The request counter has `grpc_method` and `status_code` labels. The duration histogram has a `grpc_method` label. The status value is the gRPC code name, such as `OK` or `Internal`. + ### Redis Metrics The Redis metrics provided by the redis plugin include: @@ -95,13 +103,27 @@ The JOBS metrics provided by the metrics plugin include: - `rr_jobs_jobs_err` - Number of jobs that failed while processing in the worker. - `rr_jobs_jobs_ok` - Number of successfully processed jobs. +- `rr_jobs_jobs_requeue` - Number of jobs requeued by worker responses. +- `rr_jobs_push_ok` - Number of successful job pushes. - `rr_jobs_push_err` - Number of jobs that failed to push. -- `rr_jobs_push_latency` - Histogram that represents the latency for pushed operation. Available filters: driver, job ( - pipeline), source. -- `rr_jobs_push_latency_sum` - Histogram that represents the latency for pushed operation. Available filters: driver, - job (pipeline), source. -- `rr_jobs_push_latency_count` - Histogram that represents the latency for pushed operation and the number of processed - jobs for the metric. Available filters: driver, job (pipeline), source. +- `rr_jobs_requests_total` - Job push attempts for an existing pipeline. +- `rr_jobs_push_latency_bucket` - Histogram buckets for successful job push duration, in seconds. +- `rr_jobs_push_latency_sum` - Total duration of successful job pushes, in seconds. +- `rr_jobs_push_latency_count` - Number of successful job pushes observed by the histogram. + +The push request counter and latency histogram have `driver`, `job`, and `source` labels. `job` is the pipeline name. `source` is `single` or `batch`. Each job in a batch is counted separately. The outcome counters have no plugin-defined labels. + +The v6 Jobs plugin exports `rr_jobs_jobs_ok`, `rr_jobs_jobs_err`, `rr_jobs_push_ok`, and `rr_jobs_push_err` as counters instead of gauges. Their names do not change. Use `rate()` or `increase()` for dashboards that measure changes over time. + +Failed and requeued worker responses no longer increment `rr_jobs_jobs_ok`. They increment `rr_jobs_jobs_err` or the new `rr_jobs_jobs_requeue` counter instead. Update success-rate queries to keep these outcomes separate. The job outcome counters measure processing attempts, not unique jobs. + +For example, compare processing rates with these PromQL queries: + +```promql +rate(rr_jobs_jobs_ok[5m]) +rate(rr_jobs_jobs_err[5m]) +rate(rr_jobs_jobs_requeue[5m]) +``` ### Temporal Metrics @@ -150,6 +172,9 @@ Prometheus. To do this, you need to register collectors in your configuration fi ```yaml version: "3" +rpc: + listen: tcp://127.0.0.1:6001 + metrics: address: 127.0.0.1:2112 collect: @@ -173,18 +198,21 @@ You can also use tagged (labels) metrics to group values: ```yaml version: "3" +rpc: + listen: tcp://127.0.0.1:6001 + metrics: address: 127.0.0.1:2112 collect: registered_users: - type: histogram + type: counter help: "Total registered users." labels: [ "type", "is_admin" ] ``` {% endcode %} -In the example below we will show you how to send metrics into `registered_users` collector. +Choose either the basic configuration or this labeled configuration for `registered_users`. Every update to the labeled collector must include two label values, in the order `type`, `is_admin`. ### PHP client @@ -208,7 +236,7 @@ composer require spiral/roadrunner-metrics After the installation, you can create an instance of the `Spiral\RoadRunner\Metrics\Metrics` class, which will allow you to use the available class methods. -**Here is an example:** +Use the [basic configuration](#application-metrics) for this example. Its collector has no labels: {% code title="metrics.php" %} @@ -237,7 +265,7 @@ grouping, and aggregating the data. - **Simplified querying:** You can use labels to filter and aggregate your metric data, making it easier to extract meaningful insights from the data. -You can also specify labels for your metrics by passing an array of labels to the `add` method: +Use the [tagged configuration](#tagged-metrics) for this call. Pass one value for `type` and one for `is_admin`, in that order: {% code title="metrics.php" %} @@ -273,7 +301,7 @@ $metrics->add('earned_money', 100_000_000); {% endcode %} -You can also declare labeled metrics: +You can also declare the labeled collector in PHP instead of YAML. For this example, remove `registered_users` from `metrics.collect` before starting RoadRunner. A declaration does not change the labels of an existing collector. Use the two-label `add()` call above after this declaration: {% code title="metrics.php" %} @@ -297,7 +325,7 @@ the `Spiral\RoadRunner\Metrics\Metrics` class in PHP. #### Add -Method is used to add a new metric to the declared collector. +Add a value to a declared gauge or counter. A negative counter increment returns an RPC error. Use a gauge for values that must decrease. {% code %} diff --git a/lab/otel.md b/lab/otel.md index eb54b90..1a331f5 100644 --- a/lab/otel.md +++ b/lab/otel.md @@ -1,10 +1,10 @@ # OpenTelemetry -RoadRunner offers OTEL (OpenTelemetry) plugin, which provides a unified standard for tracing, logging, and metrics -information. This plugin allows you to send tracing data from RoadRunner to tracing collectors -like [New Relic](https://newrelic.com/), [Zipkin](https://zipkin.io), [Jaeger](https://www.jaegertracing.io/), -[Datadog](https://www.datadoghq.com/), and more. -Starting with `v2023.3`, the `Jaeger` exporter is deprecated. Please use `OTLP` instead: [docs](https://www.jaegertracing.io/docs/1.49/architecture/). +The RoadRunner OpenTelemetry (OTEL) plugin exports tracing data to an OTLP receiver, standard output, or standard error. + +{% hint style="warning" %} +The v6 plugin rejects `exporter: zipkin`. The native Jaeger exporter is also unavailable. Use `exporter: otlp` and an OTLP receiver instead. A Zipkin `/api/v2/spans` endpoint cannot receive OTLP data. +{% endhint %} ![OpenTelemetry](https://user-images.githubusercontent.com/773481/213914208-cd944ca8-f218-4baf-8a54-5a4e42a1ed40.jpg) @@ -12,12 +12,7 @@ Starting with `v2023.3`, the `Jaeger` exporter is deprecated. Please use `OTLP` Read more about OpenTelemetry on the [official site](https://opentelemetry.io/). {% endhint %} -The OpenTelemetry plugin is designed to integrate with various tracing collectors to provide end-to-end tracing of -requests across multiple services. The plugin is built to support the OpenTelemetry standard, which provides a unified -way of collecting telemetry data across various languages and frameworks. - -The plugin supports tracing, logging, and metrics data, but currently only tracing information is stable and safe to use -in production. +This page describes trace export. For Prometheus metrics, see [Metrics](metrics.md). ## Configuration @@ -29,7 +24,6 @@ Here is an example configuration file: version: "3" otel: - # https://github.com/open-telemetry/opentelemetry-specification/blob/v1.25.0/specification/resource/semantic_conventions/README.md resource: service_name: "rr_test" service_version: "1.0.0" @@ -38,33 +32,35 @@ otel: insecure: true compress: false exporter: otlp + client: grpc endpoint: 127.0.0.1:4317 ``` + {% endcode %} {% hint style="info" %} -Note, that you may also use OTEL envs in the `OTEL` plugin configuration using [Shell-Parameter-Expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html) syntax. For example: +You can use environment variables in the configuration with [shell parameter expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html). This example leaves `client` and `endpoint` unset so the OTLP exporter can use its environment configuration. {% code title=".rr.yaml" %} + ```yaml version: "3" otel: - # https://github.com/open-telemetry/opentelemetry-specification/blob/v1.25.0/specification/resource/semantic_conventions/README.md resource: service_name: "${OTEL_SERVICE_NAME:-rr_test}" service_version: "${OTEL_SERVICE_VERSION:-1.0.0}" insecure: "${OTEL_EXPORTER_OTLP_INSECURE:-true}" exporter: "${OTEL_TRACES_EXPORTER:-otlp}" - endpoint: "${OTEL_EXPORTER_OTLP_ENDPOINT:-127.0.0.1:4317}" - ``` + {% endcode %} +When `client` is unset, `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL` takes precedence over `OTEL_EXPORTER_OTLP_PROTOCOL`. Supported values are `grpc` and `http/protobuf`. When `endpoint` is unset, the SDK reads its OTLP endpoint environment variables. For example, use `OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf` with `OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318`. + {% endhint %} -Once the plugin is activated, the `grpc` and `jobs` plugins will use the configuration to send tracing data to the -collector. The `http` plugin requires the `otel` middleware to be added to the middleware list. +The `grpc` and `jobs` plugins use the configured trace provider. Temporal collects the `otel` worker interceptor. The `http` plugin requires the `otel` middleware in its middleware list. **Here is an example:** @@ -73,28 +69,40 @@ collector. The `http` plugin requires the `otel` middleware to be added to the m ```yaml .rr.yaml http: address: 127.0.0.1:15389 - middleware: [ gzip, otel ] + middleware: [ otel, gzip ] ``` {% endcode %} +Requests pass through the middleware list from left to right. Put `otel` before the middleware you want to trace. In this example, the HTTP server span starts before `gzip` runs. + +The `headers`, `gzip`, `zstd`, `proxy_ip_parser`, `rate_limiter`, and `http_metrics` spans end before the next handler starts. They measure work before that call. Compression spans do not measure the full response compression time. + +The `static` span includes file serving when it handles the request. When it forwards the request, its span ends before the next handler starts. The HTTP plugin's `http` span has kind `Internal` and includes worker and response processing. The outer HTTP `Server` span covers the handlers that follow `otel`. + +The `sendfile:post` and `http_metrics:post` spans measure work after the downstream handler returns. The Prometheus duration histogram includes downstream request execution. See [HTTP tracing](../http/http.md#tracing) for these span boundaries. + +For Jobs, pass the producer's `traceparent` and optional `tracestate` values in the task headers to continue its trace. Without valid trace context, the push RPC starts a new trace. See [Jobs trace context](../queues/overview-queues.md#trace-context) for batch behavior and driver storage requirements. + **The `otel` section of the configuration file contains the following options:** -| Option | Description | -|---------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **insecure** | a boolean that determines whether to use insecure endpoints (HTTP/HTTPS) or insecure gRPC. The default value is `false`. | | -| **compress** | a boolean that determines whether to use gzip to compress the spans. The default value is `false`. | -| **exporter** | a string that provides functionality to emit telemetry to consumers. Possible values are `otlp` (used for New Relic, Datadog, Jaeger), `zipkin`, `stdout` or `stderr`. The default value is `otlp`. | -| **custom_url** | a string that is used for the http client to override the default URL. The default value is `empty`. | -| **client** | a string that determines the client to send the spans. Possible values are http and grpc. The default value is `http`. | -| **endpoint** | a string that specifies the consumer's endpoint. The default value is `127.0.0.1:4318`. | -| **service_name** | a string that specifies the user's service name. The default value is `RoadRunner`. | -| **service_version** | a string that specifies the user's service version. The default value is `1.0.0`. | -| **headers** | a key-value map that contains user-defined headers. The `api-key` for New Relic should be here. | -| **resource** | a key-value map that contains OTEL resource (https://github.com/open-telemetry/opentelemetry-specification/blob/v1.25.0/specification/resource/semantic_conventions/README.md) | +| Option | Description | +| -------- | ------------- | +| **insecure** | Use an OTLP connection without TLS. The configuration default is `false`. | +| **compress** | Compress exported spans with gzip. The configuration default is `false`. | +| **exporter** | Trace exporter: `otlp`, `stdout`, or `stderr`. The default is `otlp`. | +| **custom_url** | Override the HTTP request path, for example `/v1/traces`. This option does not set the receiver address. | +| **client** | OTLP transport: `http` or `grpc`. If unset, the plugin checks the protocol environment variables and defaults to `http`. | +| **endpoint** | OTLP receiver address as `host:port`, without a scheme or path. If unset, the SDK uses its environment configuration or transport default. | +| **service_name** | Deprecated. Use `resource.service_name`. The default resource value is `RoadRunner`. | +| **service_version** | Deprecated. Use `resource.service_version`. The default resource value is `1.0.0`. | +| **headers** | Headers sent to the OTLP receiver, such as an `api-key` header. | +| **resource** | Service attributes: `service_name`, `service_version`, `service_namespace`, and `service_instance_id`. | + +Values in `resource` take precedence over environment resource attributes. For service name and version, the deprecated top-level settings take precedence over the environment when their `resource` values are empty. The plugin also reads `OTEL_SERVICE_NAME` and `OTEL_RESOURCE_ATTRIBUTES`. {% hint style="warning" %} -The OpenTelementy OTLP endpoint can be used with 2 different ports: `4318` and `4317`, [docs](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/). In general, port `4317` is used for the `gRPC` traces (with the `gRPC` OTLP client). While port `4318` is used for the `http` OTLP client. Keep in mind that having a `gRPC` collector with an `http` endpoint will cause a send error. +Match `client` to the receiver protocol. OTLP normally uses port `4317` for `grpc` and port `4318` for `http`. For example, `client: http` requires an HTTP receiver such as `endpoint: 127.0.0.1:4318`. See the [OTLP exporter configuration](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/). {% endhint %} ## Collector @@ -136,7 +144,7 @@ Read more about the OpenTelemetry Collector on the [official site](https://opent The collector is started with the `otel-collector-config.yml` configuration file, which specifies how the collector should receive, process, and export the tracing data. -Here is an example configuration file that sends data to Zipkin and Datadog: +RoadRunner sends OTLP data to the Collector. This Collector configuration prints received traces with the `debug` exporter. Replace it with your backend exporter to store traces. {% code title="otel-collector-config.yml" %} @@ -145,45 +153,34 @@ receivers: otlp: protocols: grpc: + endpoint: 0.0.0.0:4317 http: + endpoint: 0.0.0.0:4318 processors: batch: timeout: 1s exporters: - logging: - loglevel: debug - - zipkin: - endpoint: "http://zipkin:9411/api/v2/spans" - - datadog: - api: - site: datadoghq.eu - key: ... - - otlp: - endpoint: https://otlp.eu01.nr-data.net:443 - headers: - api-key: ... + debug: + verbosity: detailed service: pipelines: traces: receivers: [ otlp ] processors: [ batch ] - exporters: [ zipkin, datadog, otlp, logging ] + exporters: [ debug ] ``` {% endcode %} -| Option | Description | -|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **receivers** | a key-value map that specifies the protocols the collector should use to receive the tracing data. | -| **processors** | a key-value map that specifies the processors to apply to the tracing data before exporting it. In this example, it uses the batch processor, which batches tracing data before exporting it. | -| **exporters** | a key-value map that specifies the destination collectors to which the tracing data should be exported. In this example, it exports the tracing data to `zipkin`, `datadog`, `otlp`, and `logging`. | -| **service** | a key-value map that specifies the service name and the pipelines through which the tracing data flows. In this example, the traces pipeline includes the otlp receiver, batch processor, and zipkin, datadog, otlp, and logging exporters. | +| Option | Description | +| --- | --- | +| **receivers** | Protocols that receive trace data. | +| **processors** | Processing steps for trace data. This example uses `batch` to group traces before export. | +| **exporters** | Output destinations. This example uses `debug` to print traces. | +| **service** | Enabled pipelines. This example connects the OTLP receiver, batch processor, and debug exporter. | {% hint style="info" %} Read more about the OpenTelemetry Collector configuration on @@ -249,6 +246,7 @@ Here is the list of currently supported plugin | **Kafka** | Kafka driver. | | **NATS** | NATS driver. | | **Beanstalk** | Beanstalk driver. | +| **Temporal** | Workflow and activity tracing through the `otel` worker interceptor. | {% hint style="info" %} Thanks to [Brett McBride](https://github.com/brettmc), he created a rr-otel [PHP demo](https://github.com/brettmc/rr-otel-demo). diff --git a/php/auto-scaling.md b/php/auto-scaling.md index 56d2b9b..d6d5760 100644 --- a/php/auto-scaling.md +++ b/php/auto-scaling.md @@ -1,12 +1,10 @@ -# Auto worker scaling [BETA] +# Automatic worker scaling -## Beta notice -This feature is still in beta, errors are expected. Do not use it in production environments. +This page describes automatic scaling in RoadRunner v3, using [pool/v2](https://github.com/roadrunner-server/pool). ## Introduction -This feature became available starting with the RoadRunner `2024.3` release. -Users can now scale their RoadRunner workers automatically, up to an additional 100 workers. +Automatic scaling has been available since RoadRunner `2024.3`. It adds workers when the pool cannot supply a free worker before the allocation timeout. It removes extra workers when allocation pressure stops. ### Supported plugins @@ -16,13 +14,27 @@ Users can now scale their RoadRunner workers automatically, up to an additional - This feature is unavailable when running RoadRunner in debug mode (`*.pool.debug=true`). - This feature does not scale Temporal workflow workers; only activity workers are scaled. +- The initial `num_workers` value cannot exceed 500. The combined base and additional worker count cannot exceed 2048. ### How it works -RoadRunner uses the `pool.allocate_timeout` option to determine when to start spawning additional workers. If no workers are available to handle the request before the timeout expires, RoadRunner begins dynamically allocating additional workers according to the `spawn_rate`. + +If no worker becomes free within `pool.allocate_timeout`, RoadRunner attempts to add up to `spawn_rate` workers. It does not exceed `max_workers` additional workers or the combined pool limit. + +Only one allocation batch can run at a time. Concurrent triggers do not each start a batch. The next batch can start after a one-second cooldown once the previous batch finishes. + +After a batch adds workers, the request retries worker acquisition. A concurrent request that cannot start a batch because of the cooldown also retries. Each retry waits up to `allocate_timeout`, subject to the caller's context deadline. The total wait can include the initial timeout, worker startup, and the retry timeout. + +If a batch adds no workers because the pool is at its limit or worker startup fails, the request returns `NoFreeWorkers`. A batch that adds at least one worker still permits the retry, even if a later worker fails to start. Choose `allocate_timeout` to allow both worker startup and waiting for a free worker. + +Idle checks start with the first automatic allocation attempt. After an `idle_timeout` interval without allocation triggers, the allocator removes up to `spawn_rate` extra workers per tick. Recent allocation triggers postpone removal, including triggers rejected by the cooldown. This is not a separate idle timer for each worker. + +A removal batch is limited to the number of workers that are free at the start of the check. Each removal attempt waits up to 500 ms for a free worker. If none becomes free, the allocator stops that batch. It does not interrupt a busy worker to scale down. Stopping a selected worker can take longer than the 500 ms wait. + +The allocator counts all workers above the base `num_workers` value. This includes workers added through [manual scaling](manual-scaling.md). Worker resets preserve the current pool size; extra workers remain subject to idle removal. ### Usage -Below is a configuration example demonstrating how to use this new feature: +Configure the allocator in the plugin's `pool` section: {% code title=".rr.yaml" %} @@ -55,7 +67,8 @@ logs: ### Configuration -The new `dynamic_allocator` section has been added to the `*.pool` configuration. It contains the following parameters: -- `max_workers` - the maximum number of workers that can be additionally spawned. -- `spawn_rate` - the number of workers that can be spawned per NoFreeWorkers error (but up to `max_workers`). -- `idle_timeout` - the time after which dynamically allocated workers are considered not needed and will be deallocated. +| Option | Meaning | Default and limit | +| --- | --- | --- | +| `max_workers` | Maximum additional worker count, not the total pool size. | Default: 10. Reduced if necessary so `num_workers + max_workers` does not exceed 2048. | +| `spawn_rate` | Maximum number of workers added per allocation batch or removed per idle tick. | Default: 5. Maximum: 100. | +| `idle_timeout` | Interval for idle checks and minimum time without allocation triggers before removal. | Default: `1m`. Values below `1s` use the default. | diff --git a/php/debugging.md b/php/debugging.md index 1b2f3ed..cf556d9 100644 --- a/php/debugging.md +++ b/php/debugging.md @@ -18,13 +18,15 @@ To enable Xdebug in your application, set the environment variable `XDEBUG_SESSI {% code title=".rr.yaml" %} ```yaml +version: "3" + rpc: listen: tcp://127.0.0.1:6001 server: command: "php worker.php" env: - - XDEBUG_SESSION: 1 + XDEBUG_SESSION: 1 http: address: "0.0.0.0:8080" @@ -54,6 +56,8 @@ php -dvariables_order=EGPCS artisan octane:start --max-requests=250 --server=roa ## Xdebug for RoadRunner in Docker +The application image below uses the [official RoadRunner v3 image](../app-server/docker.md). + First, create a `docker-compose.yml` file in your project root, or copy the `environment` and `extra_hosts` sections into your existing `docker-compose.yml`: {% code title="docker-compose.yml" %} @@ -81,11 +85,13 @@ services: {% endcode %} -Next, create a `.rr.yaml` file in your project root or copy the following content: +Create `api/public/` for the static middleware. Put the PHP worker and its Composer dependencies in `api/`, which is mounted at `/app`. Create `api/.rr.yaml` with this configuration: {% code title=".rr.yaml" %} ```yaml +version: "3" + rpc: listen: 'tcp://127.0.0.1:6001' http: @@ -107,7 +113,7 @@ server: command: 'php app.php' relay: pipes env: - - XDEBUG_SESSION: 1 + XDEBUG_SESSION: 1 metrics: address: '127.0.0.1:2112' ``` @@ -119,12 +125,11 @@ Next, create a `Dockerfile` in your project root or copy the following content: {% code title="Dockerfile" %} ```dockerfile -FROM ghcr.io/roadrunner-server/roadrunner:2025.1.1 AS roadrunner +FROM ghcr.io/roadrunner-server/roadrunner:3.0.0 AS roadrunner -FROM php:8.4-cli-alpine3.21 +FROM php:8.5-cli-alpine -# change version to latest if you want to use the latest version of xDebug, see https://xdebug.org -ENV XDEBUG_VERSION=3.4.4 +ENV XDEBUG_VERSION=3.5.3 RUN apk add --no-cache autoconf g++ make postgresql-dev coreutils --update linux-headers \ && pecl install xdebug-$XDEBUG_VERSION \ @@ -154,7 +159,7 @@ USER app {% endcode %} -Important: Make sure to change `XDEBUG_VERSION` to the version you want to use (see xdebug.org). Note that the example container runs as the `app` user. +The example uses Xdebug `3.5.3` and runs as the `app` user. Next, create an `xdebug.ini` file in your project root or copy the following content: diff --git a/php/environment.md b/php/environment.md index 6215794..315c12f 100644 --- a/php/environment.md +++ b/php/environment.md @@ -57,32 +57,55 @@ configuration file. These variables will be applied to all workers when they are server: command: "php worker.php" env: - APP_RUNTIME: prod + APP_RUNTIME: "${APP_RUNTIME:-prod}" ``` {% endcode %} -In this example, when RoadRunner starts a PHP worker, it will set the `APP_RUNTIME` environment variable to `prod`. +This example uses the RoadRunner process value of `APP_RUNTIME`. If it is unset or empty, the worker receives `prod`. + +Use a mapping for `server.env`, as shown above. Environment defaults are expanded in map values. A sequence of one-key maps does not receive the same expansion. + +Values in `server.env` override inherited environment values in the worker processes. They do not change the RoadRunner process environment. Use `server.on_init.env` separately for the [initialization command](../plugins/server.md#server-initialization). {% hint style="warning" %} -All environment variable keys will be automatically converted to uppercase. +Keys in `server.env` are automatically converted to uppercase. {% endhint %} ## Dotenv -RoadRunner supports reading environment variables from `.env` files, which are typically used to store sensitive or -environment-specific variables outside your codebase. +Use the root `envfile` setting to load a file before environment variable expansion in the main configuration and included files: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +envfile: env/.env -To read environment variables from an `.env` file, you can use the `--dotenv` CLI option when starting RoadRunner. +logs: + level: ${RR_LOG_LEVEL:-info} +``` + +{% endcode %} + +With config plugin v6, `envfile` no longer requires experimental mode. Its path is relative to the main configuration file's directory, not the process working directory. In this example, a main configuration at `/var/www/.rr.yaml` loads `/var/www/env/.env`. + +The file supplies variables that are not already present in the RoadRunner process environment. It does not replace existing values. For example, an exported `RR_LOG_LEVEL=error` takes precedence over `RR_LOG_LEVEL=info` in the file. A missing or unreadable file stops startup. + +The `reset`, `workers`, and `jobs` CLI commands use their own configuration reader. It does not load the root `envfile` setting. Use `--dotenv` for those commands when their RPC address depends on variables from that file: {% code %} ```bash ./rr serve --dotenv /var/www/config/.env +./rr workers --dotenv /var/www/config/.env ``` {% endcode %} +The `DOTENV_PATH` environment variable also selects a dotenv file and takes precedence over `--dotenv`. Relative CLI dotenv paths use the effective working directory. + ## Default environment variables in PHP workers RoadRunner comes with a set of default environment (ENV) values that facilitate proper communication between the PHP @@ -96,7 +119,7 @@ various aspects of the worker's operation. | **RR_MODE** | Identifies the mode the worker should run in (`http`, `temporal`, `grpc`, `jobs`, `tcp`, `centrifuge`, etc.) | | **RR_RPC** | Contains RPC connection address when enabled. | | **RR_RELAY** | `pipes` or `tcp://...`, depends on server relay configuration. | -| **RR_VERSION** | RoadRunner version that started the PHP worker (minimum `2023.1.0`) | +| **RR_VERSION** | RoadRunner version that started the PHP worker. Available since RoadRunner `v2023.1.0`. | These default environment values can be used within your PHP worker to configure various settings and adapt the worker's behavior according to the specific requirements of your application. diff --git a/php/manual-scaling.md b/php/manual-scaling.md index 1e44d13..dda30c7 100644 --- a/php/manual-scaling.md +++ b/php/manual-scaling.md @@ -2,18 +2,18 @@ ## Introduction -This feature became available starting from the RoadRunner `2023.3` release. -Users can now scale their RoadRunner workers dynamically via RPC. -A new class, `Spiral\RoadRunner\WorkerPool`, has been introduced to provide an easy interface to **add** or **remove** -workers from the RoadRunner workers pool. +Use `Spiral\RoadRunner\WorkerPool` to add or remove workers through [Goridge RPC](rpc.md). RoadRunner v3 retains these PHP calls. ### Limitations - This feature is not available when running RoadRunner in debug mode (`pool.debug=true`). +- A pool can hold at most 2048 workers. The initial `num_workers` value cannot exceed 500. +- A removal selects a free worker. It does not interrupt an active request or remove the last worker. +- When an automatic allocator is active, workers above the base `num_workers` count are subject to its idle removal rules, including manually added workers. ### Usage -Below is a brief example demonstrating how to use this new feature: +Add or remove a worker from the HTTP pool: {% code title="worker.php" %} @@ -37,5 +37,4 @@ $pool->removeWorker('http'); - `http`, `grpc`, `temporal`, `centrifuge`, `tcp`, `jobs`. -This provides developers with more control and flexibility over their RoadRunner setup, -allowing for better resource allocation based on the needs of their application. +For automatic allocation, see [automatic worker scaling](auto-scaling.md). diff --git a/php/pool.md b/php/pool.md index 64d15a3..9ba0651 100644 --- a/php/pool.md +++ b/php/pool.md @@ -1,19 +1,22 @@ # Worker pool -RoadRunner uses a worker pool to manage the PHP workers (PHP CLI processes). Internally, the worker pool consists of a worker watcher used to control the workersβ€”releasing, allocating, preventing zombie processes, resetting, destroyingβ€”and an internal stack responsible for manipulating (popping and pushing) already allocated workers. +RoadRunner uses a worker pool to start PHP CLI processes, assign work, and replace stopped workers. The pool also handles worker reset and shutdown. The worker pool is not used in every RoadRunner plugin but only in the `http`, `gRPC`, `tcp`, `roadrunner-temporal`, `jobs`, and `centrifuge` plugins. Additionally, the worker pool contains an internal `supervisor` to control the execution TTL of the workers, overall TTL, and execution time limits. -## Workers pool configuration: +## Worker pool configuration + +RoadRunner v3 uses [pool/v2](https://github.com/roadrunner-server/pool). {% code title=".rr.yaml" %} ```yaml # Workers pool settings. pool: - # Debug mode for the pool. In this mode, the pool will not pre-allocate the worker. A worker (only 1; num_workers ignored) will be allocated right after a request arrives. + # Start a fresh worker for each request. Do not pre-allocate workers. + # In debug mode, num_workers is ignored. # # Default: false debug: false @@ -23,7 +26,7 @@ Additionally, the worker pool contains an internal `supervisor` to control the e # Default: empty command: "php my-super-app.php" - # How many worker processes will be started. Zero (or nothing) means the number of logical CPUs. + # Initial worker count, at most 500. Zero means the number of logical CPUs. # # Default: 0 num_workers: 0 @@ -34,7 +37,7 @@ Additionally, the worker pool contains an internal `supervisor` to control the e max_jobs: 0 # [2023.3.10] - # Maximum size of the internal requests queue. After reaching the limit, all additional requests would be rejected with error. + # Request admission limit. Concurrent requests can exceed this value. # # Default: 0 (no limit) max_queue_size: 0 @@ -44,7 +47,7 @@ Additionally, the worker pool contains an internal `supervisor` to control the e # Default: 60s allocate_timeout: 60s - # Timeout for the reset operation. Zero means 60s. + # Wait for active work before stopping workers during reset. Zero means 60s. # # Default: 60s reset_timeout: 60s @@ -54,12 +57,12 @@ Additionally, the worker pool contains an internal `supervisor` to control the e # Default: 60s stream_timeout: 60s - # Timeout for worker destroying before process killing. Zero means 60s. + # Wait for active work before stopping workers during shutdown. Zero means 60s. # # Default: 60s destroy_timeout: 60s - # Dynamic allocator settings. + # Dynamic allocator settings. Base and additional workers share a limit of 2048. # # Default: empty dynamic_allocator: @@ -99,14 +102,22 @@ Additionally, the worker pool contains an internal `supervisor` to control the e {% endcode %} -Tips and tricks: +## Timeouts and admission -{% hint style="info" %} -The worker pool has an internal queue that holds requests waiting for execution, which can be limited with the `pool.max_queue_size` option. -{% endhint %} +`allocate_timeout` limits worker allocation and each wait for a free worker. With [automatic scaling](auto-scaling.md), a request can wait again after a scale-up attempt. This setting does not limit PHP request execution. Set `supervisor.exec_ttl` to limit execution time. Without `exec_ttl`, canceling the caller's context does not stop normal pool execution in PHP. + +For a stream, `exec_ttl` applies separately to each read, not to the entire stream. `stream_timeout` applies to stream cancellation. + +When a stream consumer cancels its context, the pool stops waiting to send response frames to that consumer. It cancels an incomplete worker stream before releasing the worker. A worker blocked in a stream read still needs `exec_ttl` to bound that read. + +`reset_timeout` and `destroy_timeout` limit the wait for active work before the pool starts stopping workers. They are not strict limits on the entire operation. A worker stop has its own ten-second grace period. + +`max_queue_size` checks the number of active pool `Exec` calls before another call is registered. This includes calls waiting for a worker and calls executing a request. Concurrent calls can pass the check together, so the value is not a strict queue-capacity guarantee. Zero disables the check. + +Tips and tricks: {% hint style="info" %} -Workers can be dynamically scaled: [link](auto-scaling.md) +See [automatic worker scaling](auto-scaling.md) for allocation batches, capacity limits, and timeout behavior. {% endhint %} {% hint style="info" %} diff --git a/php/rpc.md b/php/rpc.md index f3e01e6..6491b30 100644 --- a/php/rpc.md +++ b/php/rpc.md @@ -22,6 +22,16 @@ composer require spiral/goridge {% endcode %} +## v6 compatibility + +RoadRunner v3 uses Goridge with Go `net/rpc`. Keep the existing TCP or Unix listener address and `plugin.Method` calls. No Connect client migration is required. + +The Go transport, `goridge/v4`, no longer supports MessagePack. PHP clients that select a MessagePack codec must switch to a supported codec, even if their PHP Goridge version still offers MessagePack. The server supports JSON, protobuf, Gob, and raw bytes. Use JSON for JSON RPC arguments and protobuf for methods that accept protobuf DTOs. + +The Go module version does not change the frame protocol version. The base header, payload-length encoding, CRC calculation, and frame version remain compatible. This does not add a new frame-size limit or make MessagePack calls compatible. + +RoadRunner protobuf source, generated Go bindings, and Go plugin contracts now have separate repositories. This relocation alone does not require a PHP worker-loop rewrite. See [plugin migration](../customization/plugin.md#v6-migration) for the Go imports and DTO exceptions. + ## Configuration You can change the RPC port from the default (`127.0.0.1:6001`) using the following configuration: @@ -37,6 +47,23 @@ rpc: {% endcode %} +### Unix Socket + +The RPC plugin can set [Unix socket attributes](../intro/config.md#unix-socket-attributes) independently of worker credentials: + +{% code title=".rr.yaml fragment" %} + +```yaml +rpc: + listen: "unix:///run/roadrunner/rpc.sock" + unix_socket: + mode: "0600" +``` + +{% endcode %} + +Direct clients must use the same listener address. PHP workers can read it through the environment-based client shown below. Keep RPC permissions separate from sockets used by a web server. + ## Connecting to RoadRunner Once you have installed Goridge, you can connect to the RoadRunner server. To do so, create an instance of @@ -126,20 +153,20 @@ the RPC Go definitions for these plugins in the following repositories: - [Metrics](https://github.com/roadrunner-server/metrics/blob/master/rpc.go) - [Lock](../plugins/locks.md) - Provides a way to obtain and release locks on resources. [GitHub](https://github.com/roadrunner-server/lock/blob/master/rpc.go) -- [Service](../plugins/service.md) - Provides a simple API to monitor and control - processes [GitHub](https://github.com/roadrunner-server/service/blob/master/rpc.go) +- [Service](../plugins/service.md) - Monitors and controls processes through Goridge RPC. [`service.Update`](../plugins/service.md#update-service-configuration) updates stored service settings. [GitHub](https://github.com/roadrunner-server/service/blob/master/rpc.go) - [RPC](https://github.com/roadrunner-server/rpc/blob/master/rpc.go) ### Async PHP RPC interface You can use `Spiral\Goridge\RPC\AsyncRPCInterface` and an implementation with multiple relays to offer non-blocking I/O for RoadRunner communication. -The interface provides the following new methods: - - `callIgnoreResponse(string $method, mixed $payload): void` - Invoke the remote RoadRunner service method using the given payload (free form) non-blocking and ignore the response. - - `callAsync(string $method, mixed $payload): int` - Invokes the specified method with the specified payload and returns an integer identifier that can be used to retrieve the response when it's ready. - - `hasResponse(int $seq): bool, getResponse(int $seq, mixed $options = null): mixed` - - `hasResponses(array $seqs): array` - - `getResponses(array $seqs, mixed $options = null): iterable` - methods to check for and retrieve one or more results of executed requests. +The interface provides these methods: + +- `callIgnoreResponse(string $method, mixed $payload): void` - Invoke the method without waiting for its response. +- `callAsync(string $method, mixed $payload): int` - Invoke the method and return an identifier for its response. +- `hasResponse(int $seq): bool, getResponse(int $seq, mixed $options = null): mixed` +- `hasResponses(array $seqs): array` +- `getResponses(array $seqs, mixed $options = null): iterable` - Retrieve responses for the supplied identifiers. The `callIgnoreResponse` method can be used to invoke RPC methods without waiting for a response. If you don't need a response, this can greatly improve performance. For example, consider sending metric data. @@ -251,9 +278,10 @@ final class AsyncCache public function commitAsync(): bool { try { - $this->rpc->getResponses($this->responses, Response::class); - } catch (ServiceException $e) { - // ... + foreach ($this->rpc->getResponses($this->responses, Response::class) as $response) { + // Read each response to detect RPC errors. + } + } catch (ServiceException) { return false; } finally { $this->responses = []; @@ -282,6 +310,8 @@ final class AsyncCache {% endcode %} +`getResponses()` returns a lazy iterator. `commitAsync()` returns `true` only after it reads all responses. It returns `false` on the first server-side RPC error; other exceptions propagate. It does not undo completed operations or read the remaining responses after an error. + Usage: {% code %} diff --git a/php/worker.md b/php/worker.md index 632c768..d4eaff0 100644 --- a/php/worker.md +++ b/php/worker.md @@ -11,10 +11,10 @@ RoadRunner server. ### Worker types -RoadRunner provides several plugins that use workers to receive requests, -including [HTTP](https://github.com/roadrunner-php/http), [Jobs](https://github.com/roadrunner-php/jobs), -[Centrifuge](https://github.com/roadrunner-php/centrifugo), [gRPC](https://github.com/roadrunner-php/grpc), -[TCP](https://github.com/roadrunner-php/tcp), and [Temporal](https://legacy-documentation-sdks.temporal.io/php/workers). +RoadRunner provides worker plugins for [HTTP](https://github.com/roadrunner-php/http), [Jobs](https://github.com/roadrunner-php/jobs), [Centrifuge](https://github.com/roadrunner-php/centrifugo), [gRPC](https://github.com/roadrunner-php/grpc), and [Temporal](../workflow/worker.md). + +RoadRunner v3 does not support [TCP workers](../plugins/tcp.md). They need RoadRunner v2025. + You should choose the appropriate plugin based on the requirements of your application. In the examples below, we will explore the creation of an HTTP worker and a simple implementation of an entry point that can handle several types of requests. @@ -136,8 +136,7 @@ composer require spiral/roadrunner-http spiral/roadrunner-jobs nyholm/psr7 {% endcode %} -Let's start by creating an enum to enumerate the possible operating modes of RoadRunner. In this example, we will only -require the values **http** and **jobs**, but we will list all available modes: +The following enum lists worker modes. The `Tcp` mode applies only to RoadRunner v2025, because RoadRunner v3 does not support the TCP plugin. This example uses only `Http` and `Jobs`. {% code title="RoadRunnerMode.php" %} @@ -488,7 +487,7 @@ http: {% endcode %} {% hint style="warning" %} -Please pay attention, that the RRv1 section had the name **limit**, `>=v2.0` - **supervisor** +RoadRunner v3 uses `supervisor`. This section was named `limit` in RoadRunner v1 and was renamed in v2.0. {% endhint %} ## Troubleshooting diff --git a/plugins/centrifuge.md b/plugins/centrifuge.md index 1b7f980..3aeb630 100644 --- a/plugins/centrifuge.md +++ b/plugins/centrifuge.md @@ -44,17 +44,18 @@ composer require roadrunner-php/centrifugo ### Configuration -First you need to add centrifuge section to your RoadRunner configuration. For example, such a configuration would be -quite feasible to run: +Add a `centrifuge` section to your RoadRunner configuration. The plugin starts an incoming proxy listener and a PHP worker pool. If `proxy_address` is omitted or empty, it uses `tcp://127.0.0.1:30000`. These components also start when the application only sends outgoing API calls. {% code title=".rr.yaml" %} ```yaml +version: "3" + rpc: listen: tcp://127.0.0.1:6001 server: - command: "php app.php" + command: "php centrifuge-worker.php" relay: pipes centrifuge: @@ -71,7 +72,7 @@ centrifuge: use_compressor: true # Your application version - # Optional, default: v1.0.0 + # Optional, default: 1.0.0 version: "v1.0.0" # Your application name @@ -107,7 +108,6 @@ For example: "allowed_origins": [ "*" ], - "token_hmac_secret_key": "test", "proxy_publish": true, "proxy_subscribe": true, "allow_subscribe_for_client": true, @@ -119,8 +119,6 @@ For example: "proxy_subscribe_timeout": "10s", "proxy_refresh_endpoint": "grpc://127.0.0.1:30000", "proxy_refresh_timeout": "10s", - "proxy_sub_refresh_endpoint": "grpc://127.0.0.1:30000", - "proxy_sub_refresh_timeout": "1s", "proxy_rpc_endpoint": "grpc://127.0.0.1:30000", "proxy_rpc_timeout": "10s" } @@ -129,13 +127,40 @@ For example: {% endcode %} {% hint style="info" %} -`proxy_connect_endpoint`, `proxy_publish_endpoint`, `proxy_subscribe_endpoint`, `proxy_refresh_endpoint`, `proxy_sub_refresh_endpoint`, `proxy_rpc_endpoint` - +`proxy_connect_endpoint`, `proxy_publish_endpoint`, `proxy_subscribe_endpoint`, `proxy_refresh_endpoint`, `proxy_rpc_endpoint` - endpoint address of roadrunner server with activated centrifuge plugin. {% endhint %} +### Unix Socket + +Configure [Unix socket attributes](../intro/config.md#unix-socket-attributes) for the incoming proxy listener: + +{% code title=".rr.yaml fragment" %} + +```yaml +centrifuge: + proxy_address: "unix:///run/roadrunner/centrifuge.sock" + proxy_socket: + mode: "0660" +``` + +{% endcode %} + +Configure Centrifugo to connect to the same Unix socket. `proxy_socket` does not configure `grpc_api_address` or the TLS client used for outgoing API calls. + ### PHP worker example -Here is an example of a PHP worker: +This worker authenticates one configured service account with a bearer token. Set `APP_CENTRIFUGO_USER` to that account's ID in the RoadRunner process environment. Generate a token with the following command: + +```bash +php -r 'echo bin2hex(random_bytes(32)), PHP_EOL;' +``` + +Set `APP_CENTRIFUGO_TOKEN` to the command output in the RoadRunner process environment. Give the token only to that account. Send it in the Centrifugo client's connection `data` as `{"token": ""}`. Use WSS outside local tests. Do not put the token in public JavaScript or logs. The worker obtains the user ID from server configuration, not from client data. It rejects missing or incorrect credentials. + +Connections expire after five minutes. The refresh handler marks them as expired, so clients must reconnect and authenticate again. To revoke the token, replace it. Then restart RoadRunner. Existing connections remain valid until they expire or you disconnect them through the Centrifugo API. + +For multiple users, validate a separate credential for each user against your application's session or token store. The subscribe, publish, and RPC handlers below are examples, not per-channel authorization rules. Replace the sample admin and API credentials in the Centrifugo configuration before exposing the server. {% code title="centrifuge-worker.php" %} @@ -150,6 +175,12 @@ use RoadRunner\Centrifugo\Request; use RoadRunner\Centrifugo\Request\RequestFactory; use Spiral\RoadRunner\Worker; +$authToken = (string) getenv('APP_CENTRIFUGO_TOKEN'); +$authUser = (string) getenv('APP_CENTRIFUGO_USER'); +if (strlen($authToken) < 64 || $authUser === '') { + throw new \RuntimeException('Configure APP_CENTRIFUGO_TOKEN and APP_CENTRIFUGO_USER.'); +} + $worker = Worker::create(); $requestFactory = new RequestFactory($worker); @@ -171,11 +202,24 @@ while ($request = $centrifugoWorker->waitRequest()) { continue; } + if ($request instanceof Request\Connect) { + $token = $request->getData()['token'] ?? null; + if (!is_string($token) || !hash_equals($authToken, $token)) { + $request->error(1000, 'Invalid credentials.'); + continue; + } + + $request->respond(new Payload\ConnectResponse( + user: $authUser, + expireAt: time() + 300, + )); + continue; + } + if ($request instanceof Request\Refresh) { try { - // Do something $request->respond(new Payload\RefreshResponse( - // ... + expired: true, )); } catch (\Throwable $e) { $request->error($e->getCode(), $e->getMessage()); @@ -191,8 +235,7 @@ while ($request = $centrifugoWorker->waitRequest()) { // ... )); - // You can also disconnect connection - $request->disconnect('500', 'Connection is not allowed.'); + // Use disconnect() instead of respond() to reject a connection. } catch (\Throwable $e) { $request->error($e->getCode(), $e->getMessage()); } @@ -207,8 +250,7 @@ while ($request = $centrifugoWorker->waitRequest()) { // ... )); - // You can also disconnect connection - $request->disconnect('500', 'Connection is not allowed.'); + // Use disconnect() instead of respond() to reject a connection. } catch (\Throwable $e) { $request->error($e->getCode(), $e->getMessage()); } @@ -218,12 +260,8 @@ while ($request = $centrifugoWorker->waitRequest()) { if ($request instanceof Request\RPC) { try { - $response = $router->handle( - new Request(uri: $request->method, data: $request->data), - ); // ['user' => ['id' => 1, 'username' => 'john_smith']] - $request->respond(new Payload\RPCResponse( - data: $response + data: $request->getData(), )); } catch (\Throwable $e) { $request->error($e->getCode(), $e->getMessage()); @@ -252,12 +290,18 @@ specifications and proxies these events to the PHP worker. To determine what proxy method was called inside the PHP, RR adds a `type` : `endpoint` metadata. For example, if the `Subscribe` method was called, RR will add `type`:`subscribe` metadata to the worker's context. +The proxy supports the unary events listed below. Unidirectional and bidirectional subscription streams are not implemented. + ### RPC You may also use RPC methods to communicate with Centrifugo server. RR follows the official [Centrifugo proto API](https://github.com/centrifugal/centrifugo/blob/master/internal/apiproto/api.proto). Official documentation available [here](https://centrifugal.dev/docs/server/server_api#grpc-api) +{% hint style="warning" %} +The v6 plugin no longer exposes `centrifuge.RateLimit`. Remove calls to this RPC before upgrading. The plugin does not provide a replacement method. +{% endhint %} + ### Proxy events With the incoming payload, RoadRunner also adds the type of the proxied request to the headers before sending it to the PHP worker. The key in the headers is called `type`. Here is the complete list of types supported by RoadRunner: @@ -268,8 +312,11 @@ With the incoming payload, RoadRunner also adds the type of the proxied request - `publish`: Publish proxy request. - `rpc`: RPC proxy request. - `subrefresh`: Subscription refresh proxy request. +- `notifycacheempty`: Notify cache empty proxy request (`NotifyCacheEmpty`). - `notifychannelstate`: Notify channel state proxy request. +To handle `NotifyCacheEmpty`, use a PHP DTO package and worker request handler that support this method. RoadRunner forwards the event with `type: notifycacheempty`. + ## Metrics RoadRunner has a [metrics plugin](../lab/metrics.md) that provides metrics for the Centrifuge plugin, which can be used diff --git a/plugins/config.md b/plugins/config.md index 9748a9a..c4c3d90 100644 --- a/plugins/config.md +++ b/plugins/config.md @@ -1,16 +1,13 @@ # Config -The config plugin is an essential component in RoadRunner, responsible for parsing configuration files and environment -variables. It serves as a central hub for managing configurations for the plugins and RoadRunner server itself. +The config plugin reads configuration files and expands environment variables. Other plugins use it to read their settings. ## Configuration File Structure -The RoadRunner configuration file should be formatted using **YAML** or **JSON**. Each configuration file must include -a `version` at the top, indicating the format's version. The currently supported configuration version is **version 3**. +Use YAML or JSON for the RoadRunner configuration. Set `version: "3"` in each file. This value identifies the configuration format. {% hint style="warning" %} -The configuration version is not synonymous with the RoadRunner (RR) version. The configuration version and RR version -have separate versioning systems. +The configuration format version is separate from the RoadRunner release number. {% endhint %} **Example of a YAML configuration file:** @@ -29,6 +26,20 @@ version: '3' Version numbers are strings, not numbers. For example, `version: "3"` is correct, but `version: 3` is not. {% endhint %} +The file-based configuration loader rejects a missing or non-string version. The legacy value `"2.7"` still loads with an upgrade warning. Each included file must have the same version as the main file. + +## Includes and overrides + +Use [`include`](../intro/config.md#include-configuration-files) to load additional files. The loader applies CLI `-o` values before includes, so an included section can replace an override. List values are replaced. Includes have no delete operation and do not load nested include lists. + +Each include is required. The config plugin does not support optional include objects. Use environment variables with defaults for values that differ between systems. + +## Environment Files + +The root `envfile` setting loads a file before the config plugin expands environment variables. With config plugin v6, it no longer requires experimental mode. A setting that was ignored without experimental mode in v5 is now active. + +Remove an unused `envfile` setting. If you need it, supply the file it names. A missing or unreadable file stops initialization. See [Dotenv](../php/environment.md#dotenv) for configuration examples, relative paths, and environment precedence. + ## Tips -1. By default, `.rr.yaml` used as the configuration, located in the same directory with RR binary. +1. RoadRunner looks for `.rr.yaml` in the current working directory. Use `-c` to select a file and `-w` to select a working directory. diff --git a/plugins/intro.md b/plugins/intro.md index 0304923..880b6c9 100644 --- a/plugins/intro.md +++ b/plugins/intro.md @@ -14,17 +14,22 @@ development process. Some of the most notable plugins include: - [**Logger**](../lab/logger.md): Robust logging capabilities for various output formats and destinations. - [**App-Logger**](../lab/applogger.md): Send application logs to the RoadRunner logger from a PHP application. - [**gRPC**](../grpc/grpc.md): Efficient and scalable gRPC server implementation. +- [**Protoreg**](../grpc/protoreg.md): Protobuf descriptors for gRPC reflection and custom plugins. - [**Temporal**](../workflow/temporal.md): Workflow and task orchestration with distributed computing capabilities. - [**Server**](./server.md): Core server functionality and lifecycle management. - [**Service**](./service.md): Start and monitor services like a supervisor. - [**Locks**](./locks.md): Distributed locking mechanisms for concurrency control. -- [**TCP**](./tcp.md): High-performance TCP server for custom networking solutions. - [**Metrics**](../lab/metrics.md): Application-level metrics and monitoring. - [**KV**](../kv/overview-kv.md): Key-value store interface for storage and retrieval of data. - [**Jobs**](../queues/overview-queues.md): Background job processing and management. +- [**NSQ**](../queues/nsq.md): Jobs driver with topics, channels, and broker discovery. +- [**Rate limiter**](../http/rate-limiter.md): HTTP request limits by global, IP, or header key. +- [**Zstd**](../http/zstd.md): HTTP response compression with Zstandard. - [**HealthChecks**](../lab/health.md): Health monitoring and reporting for system components. - [**OpenTelemetry (OTEL)**](../lab/otel.md): Distributed tracing and observability with OpenTelemetry integration. +RoadRunner v3 does not support the [TCP plugin](tcp.md). Its reference applies to RoadRunner v2025. + ## Custom plugins In addition, RoadRunner encourages developers to create their own custom plugins, tailored to meet specific requirements diff --git a/plugins/locks.md b/plugins/locks.md index 201b995..2677a46 100644 --- a/plugins/locks.md +++ b/plugins/locks.md @@ -1,17 +1,74 @@ # Lock -The RoadRunner lock plugin is a powerful tool that enables you to manage resource locks in their applications using the -RPC protocol. By leveraging the benefits of using GO with PHP, it provides a lightweight, fast, and reliable way to -acquire, release, and manage locks. With this plugin, you can easily manage critical sections of your application and -prevent race conditions, data corruption, and other synchronization issues that can occur in multiprocess environments. +The Lock plugin provides exclusive and shared resource locks through RPC. Use it to coordinate access to a resource from multiple PHP workers. -{% hint style="warning" %} -RoadRunner lock plugin uses an in-memory storage to store information about locks at this moment. When multiple -instances of RoadRunner are used, each instance will have its own in-memory storage for locks. As a result, if a -process -acquires a lock on one instance of RoadRunner, it will not be aware of the lock state on the other instances. +{% hint style="info" %} +The plugin has two lock backends. The in-memory backend is the default, and it keeps the lock state in one RoadRunner instance. The Redis backend shares locks between RoadRunner instances. {% endhint %} +## Configuration + +Omit the `lock` section or set `driver: memory` to use the in-memory backend. Each RoadRunner instance then keeps its own lock state. + +Set `driver: redis` to share locks between RoadRunner instances: + +{% code title=".rr.yaml" %} + +```yaml +lock: + driver: redis + config: + addrs: ["127.0.0.1:6379"] + username: "" + password: "" + db: 0 + master_name: "" + sentinel_password: "" + pool_size: 0 + dial_timeout: 5s + read_timeout: 5s + write_timeout: 5s + tls: + root_ca: "" + cert: "" + key: "" +``` + +{% endcode %} + +The Redis backend requires Redis 7 or later. It uses the `go-redis` v9 client. + +A set `master_name` selects a failover client for any number of addresses. Without `master_name`, one address selects a standalone client, and two or more addresses select a cluster client. The `db` setting applies to a standalone client and to a failover client. A negative `db` is rejected at startup. A cluster client uses database 0 only. Without `master_name`, a non-zero `db` with more than one address is rejected at startup. + +Set `master_name` to use Redis Sentinel. The `addrs` list then holds the Sentinel addresses. Set `sentinel_password` when the Sentinel nodes need their own password. + +The lock backend accepts password-only Sentinel authentication. Its configuration does not include the `sentinel_username` option of the Redis KV driver. `username` and `password` authenticate against the Redis data nodes. + +Set `pool_size` to size the connection pool for one Redis node. The client opens more connections when the pool is busy. The value `0` selects the `go-redis` default. + +The `dial_timeout`, `read_timeout`, and `write_timeout` settings accept Go durations, such as `5s`. Negative values are rejected at startup. Omitted timeouts use the Redis client defaults. + +Add the `tls` block to connect with TLS. Remove the block for a plaintext connection. An empty `root_ca` selects the system root certificates. Set `root_ca` to the PEM file of a private certificate authority. Set `cert` and `key` together to send a client certificate. The backend reads the pair for each handshake, so a renewed certificate needs no restart. The minimum protocol version is TLS 1.2. + +Write at least one key in the `tls` block. The configuration reader drops a block that has no keys, and the connection then stays plaintext. Write `root_ca: ""` for a server with a public certificate authority. + +The backend does not accept some keys of the [RoadRunner Redis plugin](../kv/redis.md). The `max_retries` key is absent, because a retry after a lost reply reports contention for a lock that the caller now holds. The `route_by_latency`, `route_randomly`, and `read_only` keys are absent, because the lock script writes and must run on the master. The `min_retry_backoff`, `max_retry_backoff`, `min_idle_conns`, `max_conn_age`, `pool_timeout`, `idle_timeout`, and `idle_check_freq` keys are absent as well. + +The `lock` section requires `driver: memory` or `driver: redis`. Invalid configuration and connection failures stop plugin initialization. + +The backend stores lock state under the fixed `rr:lock:` key prefix. The resource name is the namespace. Give resources unique names when different applications share one Redis server. Redis server time controls expiration. Each read lock has its own TTL. A TTL of `0` creates a lock that remains until release. + +### Redis lock behavior + +- `ForceRelease` removes all locks on the resource. It returns `Ok: true` only if it removed at least one lock. +- Both backends refuse a second read lock with the same ID on the same resource. Use `UpdateTTL` to extend a held lock. +- Contention, or an expired wait during an acquisition, returns `Ok: false`. +- A failed Redis command, or a deadline that occurs during a Redis command, returns an RPC error. The lock state is then unknown. Call `Exists` or `Release` to find the state of the lock. +- A `wait` of `0` on Redis makes one acquisition attempt, bounded by `read_timeout`, which is 5 seconds by default. The memory backend waits 1 millisecond instead. +- Stored locks stay in Redis after a RoadRunner restart until release or expiry. + +For `Release`, `ForceRelease`, `Exists`, and `UpdateTTL`, Redis uses `wait` as the deadline for the command. These calls do not wait for lock contention. A deadline during the command returns an RPC error with an unknown result. All waiting acquisition calls in one RoadRunner instance share one Redis Pub/Sub connection. + ## PHP client The RoadRunner lock plugin comes with a convenient PHP package that simplifies the process of integrating the @@ -59,20 +116,38 @@ The `RoadRunner\Lock\Lock` class provides four methods that allow you to manage #### Acquire lock -Locks a resource so that it can be accessed by one process at a time. When a resource is locked, other processes that -attempt to lock the same resource will be blocked until the lock is released. +Attempts to acquire an exclusive lock on a resource. Set a positive `waitTTL` to wait for a conflicting lock to be released. The method returns a lock ID on success or `false` on failure. Check the result before accessing the protected resource. + +The PHP SDK uses seconds for numeric `ttl` and `waitTTL` values. It converts these values to microseconds for RPC. In PHP SDK 1.0.x, `waitTTL` defaults to `0`. The memory backend then uses a one-millisecond acquisition window. The Redis backend makes one acquisition attempt, bounded by `read_timeout`. Set `waitTTL` explicitly when you need a longer wait. {% code title="app.php" %} ```php -$id = $lock->lock('pdf:create'); +$id = $lock->lock('pdf:create', ttl: 10, waitTTL: 5); +if ($id === false) { + throw new \RuntimeException('Could not acquire pdf:create.'); +} + +try { + // Access the protected resource here. +} finally { + $lock->release('pdf:create', $id); +} +``` -// Acquire lock with ttl - 10 microseconds +{% endcode %} + +These calls show alternative arguments. Use the same success check and release handling for each call: + +{% code title="app.php" %} + +```php +// Set a ten-second TTL. $id = $lock->lock('pdf:create', ttl: 10); // or $id = $lock->lock('pdf:create', ttl: new \DateInterval('PT10S')); -// Acquire lock and wait 5 microseconds until lock will be released +// Wait for at most five seconds to acquire the lock. $id = $lock->lock('pdf:create', waitTTL: 5); // or $id = $lock->lock('pdf:create', waitTTL: new \DateInterval('PT5S')); @@ -85,18 +160,36 @@ $id = $lock->lock('pdf:create', id: '14e1b600-9e97-11d8-9f32-f2801f1b9fd1'); #### Acquire read lock -Locks a resource for shared access, allowing multiple processes to access the resource simultaneously. When a resource -is locked for shared access, other processes that attempt to lock the resource for exclusive access will be blocked -until all shared locks are released. +Attempts to acquire a shared lock on a resource. Multiple readers can hold shared locks. An exclusive acquisition must wait until all readers release their locks or its wait expires. Set a positive `waitTTL` to wait for an existing exclusive lock. `lockRead()` returns a lock ID on success or `false` on failure. The PHP time units and default wait are the same as for `lock()`. + +{% code title="app.php" %} + +```php +$id = $lock->lockRead('pdf:create', ttl: 10, waitTTL: 5); +if ($id === false) { + throw new \RuntimeException('Could not acquire a read lock on pdf:create.'); +} + +try { + // Read the protected resource here. +} finally { + $lock->release('pdf:create', $id); +} +``` + +{% endcode %} + +These calls show alternative arguments. Check each result before reading the resource: {% code title="app.php" %} ```php -$id = $lock->lockRead('pdf:create', ttl: 100000); +// Set a ten-second TTL. +$id = $lock->lockRead('pdf:create', ttl: 10); // or $id = $lock->lockRead('pdf:create', ttl: new \DateInterval('PT10S')); -// Acquire lock and wait 5 microseconds until lock will be released +// Wait for at most five seconds to acquire the read lock. $id = $lock->lockRead('pdf:create', waitTTL: 5); // or $id = $lock->lockRead('pdf:create', waitTTL: new \DateInterval('PT5S')); @@ -143,15 +236,20 @@ if($status) { #### Update TTL -Updates the time-to-live (TTL) for the locked resource. +Replaces the remaining TTL with a new duration from the time the server applies the update. It does not add time to the previous expiry. Numeric PHP values use seconds. A shorter value can release the lock before the protected operation finishes. Check the update result. Stop accessing the resource if the update fails. Finish the operation or renew the lock before its TTL expires. {% code title="app.php" %} ```php -// Add 10 microseconds to lock ttl -$lock->updateTTL('pdf:create', $id, 10); -// or -$lock->updateTTL('pdf:create', $id, new \DateInterval('PT10S')); +// Set the remaining TTL to ten seconds. +if (!$lock->updateTTL('pdf:create', $id, 10)) { + throw new \RuntimeException('Could not renew pdf:create.'); +} + +// Alternative: use a DateInterval for the same duration. +if (!$lock->updateTTL('pdf:create', $id, new \DateInterval('PT10S'))) { + throw new \RuntimeException('Could not renew pdf:create.'); +} ``` {% endcode %} @@ -200,17 +298,22 @@ To make it easy to use the Lock proto API in PHP, we provide a [GitHub repository](https://github.com/roadrunner-php/roadrunner-api-dto), that contains all the generated PHP DTO classes proto files, making it easy to work with these files in your PHP application. -- [API](https://buf.build/roadrunner-server/api/file/main:lock/v1beta1/lock.proto) +- [Lock protobuf API](https://github.com/roadrunner-server/api/blob/25217e9/roadrunner/api/lock/v1/lock.proto) ### RPC API RoadRunner provides an RPC API, which allows you to manage locks in your applications using remote procedure calls. The RPC API provides a set of methods that map to the available methods of the `RoadRunner\Lock\Lock` class in PHP. +Raw RPC requests use microseconds for `Request.ttl` and `Request.wait`. For example, `wait: 5000000` allows an acquisition wait of five seconds. An omitted or zero `wait` is not an unlimited wait. The memory backend then uses a one-millisecond acquisition window, and the Redis backend makes one acquisition attempt, bounded by `read_timeout`. PHP SDK arguments use seconds and are converted before the RPC call. + +`Lock`, `LockRead`, and `UpdateTTL` accept `ttl` values from `0` to `9223372036854775` microseconds. All methods accept `wait` values in the same range. A value outside the range returns an RPC error and changes no lock state. + +For `Lock` and `LockRead`, require `Response.Ok == true` before accessing the resource. A completed RPC call with no error can still return `Ok == false`, including when the acquisition wait expires. + ### Lock -Acquires an exclusive lock on a resource so that it can be accessed by one process at a time. When a resource is locked, -other processes that attempt to lock the same resource will be blocked until the lock is released. +Attempts to acquire an exclusive lock. If the resource has a conflicting lock, the call waits until that lock is released or the acquisition deadline expires. Check `Response.Ok` before entering the protected section. {% code %} @@ -222,9 +325,7 @@ func (r *rpc) Lock(req *lockApi.Request, resp *lockApi.Response) error {} ### LockRead -Acquires a read lock on a resource, allowing multiple processes to access the resource simultaneously. When a resource -is locked for shared access, other processes that attempt to lock the resource for exclusive access will be blocked -until all shared locks are released. +Attempts to acquire a shared lock. Readers can share the resource, but an existing exclusive lock prevents acquisition. The call waits until the conflict ends or the acquisition deadline expires. Check `Response.Ok` before reading the protected resource. An exclusive acquisition also has a finite wait when readers hold the resource. {% code %} @@ -272,7 +373,7 @@ func (r *rpc) Exists(req *lockApi.Request, resp *lockApi.Response) error {} #### UpdateTTL -Updates the time-to-live (TTL) for the locked resource. +Replaces the remaining TTL with `Request.ttl` microseconds from the time the server applies the update. It does not add to the previous expiry. Check `Response.Ok` to detect a failed update. {% code %} diff --git a/plugins/server.md b/plugins/server.md index 115b24d..cc26f15 100644 --- a/plugins/server.md +++ b/plugins/server.md @@ -25,17 +25,15 @@ server: # Default: "" user: "" - # Script execute timeout - # - # Default: 60s [60m, 60h], if used w/o units its means - NANOSECONDS. + # Timeout after the command starts. Default: 60s. Include a unit. exec_timeout: 20s - # Environment variables for the worker processes. + # Environment variables for the initialization command. # # Default: env: - - SOME_KEY: "SOME_VALUE" - - SOME_KEY2: "SOME_VALUE2" + SOME_KEY: "SOME_VALUE" + SOME_KEY2: "SOME_VALUE2" # Exit RR if the `on_init` command fails or exceeds the `exec_timeout`. exit_on_error: false @@ -50,17 +48,12 @@ server: # Default: "" user: "" - # Group name (not GID) for the worker processes. An empty value means to use the RR process user. - # - # Default: "" - group: "" - # Environment variables for the worker processes. # # Default: env: - - SOME_KEY: "SOME_VALUE" - - SOME_KEY2: "SOME_VALUE2" + SOME_KEY: "SOME_VALUE" + SOME_KEY2: "SOME_VALUE2" relay: pipes ``` @@ -78,16 +71,36 @@ processes. It does not require any network connections or external libraries, ma Use `on_init.user` option to execute the on_init command under a different user. {% endhint %} +### Unix Relay + +The server plugin supports [Unix socket attributes](../intro/config.md#unix-socket-attributes) for the shared worker relay: + +{% code title=".rr.yaml fragment" %} + +```yaml +server: + command: "php worker.php" + relay: "unix:///run/roadrunner/relay.sock" + relay_socket: + mode: "0600" +``` + +{% endcode %} + +`relay_socket` controls only the socket file. It does not select the [worker user](#worker-user) or `on_init.user`. Its permissions must allow the configured workers to connect. Omit this object when using the default `pipes` relay or TCP. + ### Server initialization The `on_init` section is used for application initialization or warming up before starting workers. It allows you to set a command script that will be executed before starting the workers. You can also set environment variables to pass to this script. +With server plugin v6, `on_init.env` values override inherited process environment values. In v5, inherited values took precedence for this command. Check for conflicting variable names when upgrading. `server.env` does not configure the initialization command. + +The `on_init.exec_timeout` interval starts after the command starts successfully. Use a duration with a unit, such as `20s`. + {% hint style="info" %} -If the `on_init` command fails (i.e., returns a non-zero exit code), RoadRunner will log the error but continue -execution. This ensures that a failure during initialization does not interrupt the application's operation. -Use `on_init.exit_on_error: true` to stop RoadRunner if `on_init` command fails or exceeds the `exec_timeout`. +By default, RoadRunner logs an `on_init` command error and continues startup. Set `on_init.exit_on_error: true` to stop RoadRunner if the command fails or exceeds `exec_timeout`. {% endhint %} ### Worker starting command @@ -98,21 +111,30 @@ The `server.command` option is required and is used to start the worker pool for This option can be overridden by plugins with a pool section, such as the `http.pool.command`, or in general `.pool.command`. {% endhint %} -The `user` and `group` options allow you to set the user and group that will start and own the worker process. This -feature provides an additional layer of security and control over the application's execution environment. +In v6, a scalar command or a one-element sequence is split at whitespace. Repeated spaces and tabs do not create empty arguments. This is not shell parsing: quotes inside a scalar do not keep words in one argument. Use a sequence with one element for the executable and one for each argument when an argument contains spaces: -{% hint style="info" %} -An empty value means to use the RoadRunner process user. -{% endhint %} +{% code title=".rr.yaml" %} + +```yaml +server: + command: ["php", "worker.php", "--label", "my worker"] +``` + +{% endcode %} + +The same argument rules apply to `server.on_init.command` and pool command overrides. + +### Worker User + +Set `server.user` to an account name, not a numeric UID. An empty value keeps the RoadRunner process user. The server plugin resolves the selected account's UID and GID during initialization. `server.group` does not override that GID. + +In v6, a failed account lookup or an invalid numeric UID or GID stops initialization. Correct the account name before restarting RoadRunner. A nonempty `server.user` is not supported on Windows and also stops initialization; remove that setting on Windows. {% hint style="warning" %} -RoadRunner must be started from the root user. Root access is needed only to fork the process under a -different user. Once the worker process is started, it will run with the specified user and group permissions, -providing a secure and controlled execution environment for the application. All temporary files (`http` for example) -would be created with the provided user/group +On Unix, RoadRunner needs permission to change process credentials when `server.user` selects a different account. {% endhint %} -The `env` option allows you to set environment variables to pass to the worker script. +Use `server.env` to set [worker environment variables](../php/environment.md#setting-environment-variables). ## PHP Client diff --git a/plugins/service.md b/plugins/service.md index 176095f..90722cc 100644 --- a/plugins/service.md +++ b/plugins/service.md @@ -37,7 +37,7 @@ service: remain_after_exit: true service_name_in_log: false env: - - foo: "BAR" + foo: "BAR" restart_sec: 1 ``` @@ -53,15 +53,18 @@ The following are the available configuration settings for each service: |-----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **command** | The command to execute. There are no restrictions on commands. It could be a binary, a PHP file, a script, etc. | | **process_num** | The number of processes for the command to fire. The default value is `1`. | -| **exec_timeout** | The maximum allowed time to run for the process. The default value is 0s, which means unlimited time. The timeout can be set in the form of `1h`, `1m`, or `1s` (h,m,s). | -| **timeout_stop_sec** | The maximum allowed time to wait for the process to stop. | -| **remain_after_exit** | If set to `true`, the process will remain after exit. For example, if you need to restart the process every 10 seconds, exec_timeout should be set to `10s`, and `remain_after_exit` should be set to `true`. Note that if you kill the process from outside and `remain_after_exit` is `true`, the process will be restarted. | -| restart_sec | The delay between process stop and restart. The default value is 30 seconds. | -| service_name_in_log | If set to `true`, the service name will be shown in the log in the form `%plugin%.%service_name%`. The default value is `false`. | +| **exec_timeout** | The maximum execution time as a YAML duration, such as `1h`, `1m`, or `1s`. The default is `0s` (unlimited). RPC and PHP use integer seconds. | +| **timeout_stop_sec** | The stop timeout in integer seconds. The default is `5`; `0` selects the default. | +| **remain_after_exit** | If `true`, automatically restarts the process after any exit code. The time between starts includes execution time and `restart_sec`. | +| restart_sec | The delay after process exit, in integer seconds. The default is `30`; `0` selects the default. | +| service_name_in_log | If `true`, adds a `service` log attribute with the service name. The logger name remains `service`. The default is `false`. | | env | Environment variables to pass to the underlying process from the config. | | user | Username (not UID) for the Service process. An empty value means to use the RR process user. | + Services will be started when RoadRunner starts and will be stopped when RoadRunner stops. +Service plugin v5 used `service_name_in_log` to change the logger name to `service.NAME`. In v6, update log filters to use the separate `service` attribute. Use production mode or a custom [logger format](../lab/logger.md#custom-format) with `%attrs%` to retain that attribute; raw mode discards it. + ## PHP client The RoadRunner Service Plugin PHP Client Library allows you to manage processes in PHP application using the Service @@ -126,6 +129,67 @@ try { {% endcode %} +#### Update service configuration + +{% hint style="info" %} +The PHP examples use `Manager::update()` and the `Update` and `Environment` DTOs. The server exposes this operation as `service.Update` over Goridge RPC. +{% endhint %} + +Use `update()` to change the desired configuration of an existing service. A `true` result confirms acceptance. A name-only call succeeds. Omitted arguments and `null` keep stored values. Explicit `false`, `0`, and `[]` are sent. + +{% code title="app.php" %} + +```php +$result = $manager->update( + name: 'listen-jobs', + processNum: 2, + execTimeout: 60, + remainAfterExit: true, + env: ['APP_ENV' => 'production', 'QUEUE' => 'priority'], + restartSec: 5, + serviceNameInLogs: true, + stopTimeout: 10 +); +``` + +{% endcode %} + +| RPC field | PHP argument | Behavior | +| --- | --- | --- | +| `process_num` | `processNum` | Target process count, at least `1`. | +| `exec_timeout` | `execTimeout` | Integer seconds; `0` means unlimited. A running execution keeps its deadline. | +| `remain_after_exit` | `remainAfterExit` | Automatic restart after any exit code. The latest value controls exit handling and queued starts. `false` cancels queued automatic starts. | +| `env` | `env` | Replaces the complete service override map for new executions. `[]` clears overrides. | +| `restart_sec` | `restartSec` | Integer seconds; `0` selects `30`. A queued timer keeps its delay. The next scheduled restart uses the new delay. | +| `service_name_in_logs` | `serviceNameInLogs` | Adds the separate `service` log attribute for new executions. Current output keeps its logger. YAML uses `service_name_in_log`. | +| `timeout_stop_sec` | `stopTimeout` | Integer seconds; `0` selects `5`. New executions use this stop timeout. Current executions keep theirs. | + +Updates keep current PIDs. When automatic restart is enabled, the next exit or queued restart adjusts the process count. Excess processes finish without replacement. An increase starts the missing processes at the next restart opportunity. To change the command or user, terminate the service and create it with the new settings. + +Processes inherit the RoadRunner environment. Override keys become uppercase. Values expand from the RoadRunner environment. Empty strings and the string `'0'` are preserved. + +For example, send explicit values to clear settings and disable automatic restart: + +{% code title="app.php" %} + +```php +$result = $manager->update( + name: 'listen-jobs', + execTimeout: 0, + remainAfterExit: false, + env: [], + serviceNameInLogs: false +); +``` + +{% endcode %} + +A later explicit `Restart` or `Reset` uses unlimited execution time, cleared environment overrides, and no separate service log attribute. Automatic restart stays disabled. + +Server validation rejects the whole invalid request. An unknown service returns an RPC error. `Manager` converts these RPC errors to `Spiral\RoadRunner\Services\Exception\ServiceException`. + +Updates stay in memory. RoadRunner process startup reloads file configuration. Explicit `Restart` and `Reset` use the latest settings and count. An inactive service with no queued starts stays inactive until `Restart` or `Reset`. + #### Checking Service Status To check the status of a service, use the `statuses` method: @@ -150,6 +214,8 @@ try { To restart a service, use the `restart` method: +Restart cancels queued automatic starts and waits for all current processes to stop before it starts replacements. A process receives `SIGINT`, then a forced stop if its stop timeout expires. The service has an interval with no running process. + {% code title="app.php" %} ```php @@ -168,6 +234,8 @@ try { {% endcode %} +If a replacement fails to start, RoadRunner stops the replacements that already started and removes the service from its registry. Fix the startup error. Then call `create` with the required service settings. + #### Terminating a Service To terminate a service, use the `terminate` method: @@ -223,4 +291,32 @@ To make it easy to use the Service proto API in PHP, we provide a [GitHub repository](https://github.com/roadrunner-php/roadrunner-api-dto), that contains all the generated PHP DTO classes proto files, making it easy to work with these files in your PHP application. -- [API](https://github.com/roadrunner-server/api/blob/master/proto/service/v1/service.proto) +- [Service protobuf API](https://github.com/roadrunner-server/api/blob/master/roadrunner/api/service/v1/service.proto) + +### Update with Goridge + +Use Goridge RPC with `ProtobufCodec` for `service.Update`. It takes `Update` and returns `Response`. Use a PHP DTO package that provides the types listed [above](#update-service-configuration). + +{% code title="app.php" %} + +```php +use RoadRunner\Service\DTO\V1\Environment; +use RoadRunner\Service\DTO\V1\Response; +use RoadRunner\Service\DTO\V1\Update; +use Spiral\Goridge\RPC\Codec\ProtobufCodec; +use Spiral\Goridge\RPC\RPC; + +require __DIR__ . '/vendor/autoload.php'; + +$rpc = RPC::create('tcp://127.0.0.1:6001')->withCodec(new ProtobufCodec()); +$update = (new Update()) + ->setName('listen-jobs') + ->setEnv(new Environment()); + +$response = $rpc->call('service.Update', $update, Response::class); +$result = $response->getOk(); +``` + +{% endcode %} + +Call setters only for fields to change. Omitting `setEnv()` keeps the override map. A present empty `Environment` clears it. To replace the map, use `setEnv((new Environment())->setValues(['APP_ENV' => 'production']))`. diff --git a/plugins/tcp.md b/plugins/tcp.md index 73d2b60..9513525 100644 --- a/plugins/tcp.md +++ b/plugins/tcp.md @@ -1,5 +1,9 @@ # TCP +{% hint style="warning" %} +RoadRunner v3 does not support the TCP plugin. This page describes the plugin for RoadRunner v2025. Applications that use the `tcp:` plugin stay on RoadRunner v2025. This does not affect `tcp://` transport for RPC or worker relays. +{% endhint %} + The RoadRunner TCP plugin helps you handle TCP requests. You can use this plugin to make your own servers like an SMTP server, and send TCP requests directly to PHP workers for handling. @@ -78,7 +82,7 @@ registered plugins, such as `grpc`, `http`, `jobs`, etc. {% tab title="Worker command" %} -You can also define command to start server in the `grpc.pool.command` section to separate server and grpc workers. +Set `tcp.pool.command` to use a separate command for TCP workers. {% code title=".rr.yaml" %} @@ -214,4 +218,4 @@ while ($request = $tcpWorker->waitRequest()) { ## What's Next? -1. [Plugins β€” KV](../kv/overview-kv.md) - Learn how to use the Key Value plugin to store data between requests. +1. [Plugins - KV](../kv/overview-kv.md) - Learn how to use the Key Value plugin to store data between requests. diff --git a/queues/amqp.md b/queues/amqp.md index a70138e..bff5faa 100644 --- a/queues/amqp.md +++ b/queues/amqp.md @@ -14,293 +14,339 @@ To install and configure the RabbitMQ, use the corresponding [documentation page](https://www.rabbitmq.com/download.html). {% hint style="info" %} -Every message pushed to the RabbitMQ server uses publisher confirms. A message is considered sent only after the server confirms it. This is a reliable way to ensure delivery to the server. +Immediate publishes wait for a publisher confirm from RabbitMQ. Delayed publishes do not wait for that confirmation before returning. A publisher confirm does not prove that a message reached a queue. {% endhint %} +## Named Connections -After that, you should configure the connection to the server in the `amqp` section. This configuration section -contains exactly one `addr` key with a [connection DSN](https://www.rabbitmq.com/uri-spec.html). The `TLS` configuration sits in the `amqp.tls` section and consists of the following options: +Define each connection under `amqp.`. Each entry requires an explicit `addr` with a [connection DSN](https://www.rabbitmq.com/uri-spec.html). Each entry can also have an optional `tls` section. Every YAML AMQP pipeline must set `config.connection` to a configured name. + +RR rejects pipeline creation if the connection selector is missing or empty. An unknown connection name also causes an error. RR also rejects a selected connection with a missing or empty `addr`. + +There is no implicit default connection or localhost fallback. Top-level `amqp.addr` and `amqp.tls` are not supported. Connection names select configuration. Each pipeline keeps its own broker sockets. + +Connection names are separate from [named worker pools](overview-queues.md#named-worker-pools). Connection selection does not add a message header. + +TLS options belong under `amqp..tls`: - `key`: path to a key file. - `cert`: path to a certificate file. - `root_ca`: path to Root CAs used by the AMQP client to trust and verify the broker/server certificate during TLS dial. -- `client_auth_type`: also known as `mTLS`. Possible values are: `no_client_cert`, `request_client_cert`, `require_any_client_cert`, `verify_client_cert_if_given`, `require_and_verify_client_cert`. +- `client_auth_type`: possible values are `no_client_cert`, `request_client_cert`, `require_any_client_cert`, `verify_client_cert_if_given`, `require_and_verify_client_cert`. This setting does not control broker certificate verification. + +Configure [RabbitMQ TLS support](https://www.rabbitmq.com/ssl.html) on the broker. -You should also configure `rabbitMQ` with `TLS` support: [link](https://www.rabbitmq.com/ssl.html). +`root_ca` adds trust roots for broker certificate verification. Reconnects reuse the configured TLS settings. Use an `amqps://` address with valid `key` and `cert` files. A `root_ca`-only TLS block is not supported. Omit `tls` when using a plain `amqp://` connection. {% code title=".rr.yaml" %} ```yaml +version: "3" + amqp: - addr: amqp://guest:guest@127.0.0.1:5672 + brokerA: + addr: amqps://guest:guest@rabbitmq.example.com:5671/ - # AMQPS TLS configuration - # - # This section is optional - tls: - # Path to the key file + # AMQPS TLS configuration # - # This option is required - key: "" + # This section is optional + tls: + # Path to the key file + # + # This option is required + key: /etc/rr/tls/client.key - # Path to the certificate - # - # This option is required - cert: "" + # Path to the certificate + # + # This option is required + cert: /etc/rr/tls/client.crt - # Path to Root CAs used by the AMQP client to trust and verify the broker/server certificate during TLS dial. - # - # This option is optional - root_ca: "" + # Path to Root CAs used by the AMQP client to trust and verify the broker/server certificate during TLS dial. + # + # This option is optional + root_ca: /etc/rr/tls/ca.crt - # Client auth type (mTLS, peer verification). - # - # This option is optional. Default value: no_client_cert. Possible values: no_client_cert, request_client_cert, require_any_client_cert, verify_client_cert_if_given, require_and_verify_client_cert - client_auth_type: no_client_cert + # Legacy client_auth_type setting. + # + # This option is optional. Default value: no_client_cert. Possible values: no_client_cert, request_client_cert, require_any_client_cert, verify_client_cert_if_given, require_and_verify_client_cert + client_auth_type: no_client_cert ``` {% endcode %} -Upon establishing a connection to the server, you can create a new queue that utilizes this connection and encompasses -the queue settings, including those specific to AMQP. +Configure each pipeline's exchange and queue as shown below. + +## Pipeline Configuration + +Static AMQP configuration uses only nested `exchange` and `queue` sections under `jobs.pipelines..config`. At least one section is required. Missing sections receive default values. + +AMQP `config.version` is removed. Keep the root `version: "3"`. Use the [migration table](#migration) to move old flat entity settings. + +### Options in `config` + +- `connection`: required connection name from `amqp`. +- `priority`: pipeline priority. If a job has priority `0`, it inherits the pipeline priority. Default: `10`. +- `prefetch`: RabbitMQ QoS prefetch. Default: `10`. +- `redial_timeout`: reconnect timeout in seconds. Default: `60`. + +Zero or negative values for `priority`, `prefetch`, and `redial_timeout` use their defaults. -## Configuration +### Exchange settings + +- `name`: exchange name. Default: `amqp.default`. +- `type`: exchange type. Supported: `direct`, `fanout`, `topic`, `headers`. Default: `direct`. +- `durable`: durable exchange flag. Default: `false`. +- `auto_delete`: auto-delete exchange when last queue is unbound. Default: `false`. +- `declare`: declare exchange during pipeline creation. Default: `true`. + +### Queue settings + +- `name`: queue name. Optional for producer-only pipelines; required for `run`, `resume`, and `pause`. +- `routing_key`: routing key. Required when `exchange.type != fanout`. +- `durable`: durable queue flag. Default: `false`. +- `auto_delete`: auto-delete queue after the last consumer unsubscribes. Default: `false`. +- `exclusive`: exclusive queue flag. Default: `false`. +- `consumer_id`: consumer identifier. Default: `roadrunner-`. +- `delete_on_stop`: delete queue when pipeline stops. Default: `false`. +- `multiple_ack`: ACK this and prior unacked deliveries on the same channel. Default: `false`. +- `requeue_on_fail`: use RabbitMQ requeue on failure (Nack). Default: `false`. +- `headers`: queue declaration arguments (for example, `x-queue-mode: lazy`). +- `declare`: declare and bind queue when `Run` or `Resume` starts consumption. Default: `true`. + +For both entities, an omitted `declare` has the same effect as `true`. Explicit `false` disables declaration. Pipeline creation does not declare or bind the queue. + +{% hint style="info" %} +See also [AMQP model](https://www.rabbitmq.com/tutorials/amqp-concepts.html#amqp-model) documentation section. +{% endhint %} + +{% hint style="info" %} +Producer-only pipeline: `queue.name` can be empty, `push` works, but `run`, `resume`, and `pause` will fail without a queue name. +{% endhint %} + +{% hint style="info" %} +If `exchange.type` is not `fanout`, `queue.routing_key` must be set when RR initializes or declares the pipeline. This also applies to consume-only pipelines and pipelines with declarations disabled. +{% endhint %} + +{% hint style="info" %} +Read more about Nack in RabbitMQ official docs: https://www.rabbitmq.com/confirms.html#consumer-nacks-requeue +{% endhint %} + +This example consumes from `brokerA` through `consume-a`. It publishes to `brokerB` through `publish-b`. {% code title=".rr.yaml" %} ```yaml version: "3" -amqp: - addr: amqp://guest:guest@127.0.0.1:5672 - - # AMQPS TLS configuration - # - # This section is optional - tls: - # Path to the key file - # - # This option is required - key: "" +rpc: + listen: tcp://127.0.0.1:6001 - # Path to the certificate - # - # This option is required - cert: "" - - # Path to Root CAs used by the AMQP client to trust and verify the broker/server certificate during TLS dial. - # - # This option is optional - root_ca: "" +server: + command: php worker.php + relay: pipes - # Client auth type (mTLS, peer verification). - # - # This option is optional. Default value: no_client_cert. Possible values: no_client_cert, request_client_cert, require_any_client_cert, verify_client_cert_if_given, require_and_verify_client_cert - client_auth_type: no_client_cert +amqp: + brokerA: + addr: amqp://guest:guest@broker-a:5672/ + brokerB: + addr: amqp://guest:guest@broker-b:5672/ jobs: + consume: ["consume-a"] pipelines: - # User defined name of the queue. - example: - # Driver name - # - # This option is required. + consume-a: driver: amqp - - # Driver's configuration - # - # Should not be empty config: - - # QoS - prefetch. - # - # Default: 10 + connection: brokerA + priority: 10 prefetch: 10 - - # Pipeline priority - # - # If the job has priority set to 0, it will inherit the pipeline's priority. Default: 10. - priority: 1 - - # Redial timeout (in seconds). How long to try to reconnect to the AMQP server. - # - # Default: 60 redial_timeout: 60 - - # Durable queue - # - # Default: false - durable: false - - # Durable exchange (rabbitmq option: https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchanges) - # - # Default: false - exchange_durable: false - - # Auto-delete (exchange is deleted when last queue is unbound from it): https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchanges - # - # Default: false - exchange_auto_delete: false - - # Auto-delete (queue that has had at least one consumer is deleted when last consumer unsubscribes) (rabbitmq option: https://www.rabbitmq.com/queues.html#properties) - # - # Default: false - queue_auto_delete: false - - # Delete queue when stopping the pipeline - # - # Default: false - delete_queue_on_stop: false - - # Queue name - # - # Optional for producer-only pipelines. Required for run/resume/pause. - # Can be omitted for push-only mode. - queue: test-1-queue - - # Exchange name - # - # Optional. Default: amqp.default - exchange: amqp.default - - # Exchange type - # - # Default: direct. Possible values: direct, fanout, topic, headers. - exchange_type: direct - - # Routing key for the queue - # - # Default: empty. Required for push when exchange_type != fanout. - routing_key: test - - # Declare a queue exclusive at the exchange - # - # Default: false - exclusive: false - - # When multiple is true, this delivery and all prior unacknowledged deliveries - # on the same channel will be acknowledged. This is useful for batch processing - # of deliveries - # - # Default: false - multiple_ack: false - - # The consumer_id is identified by a string that is unique and scoped for all consumers on this channel. - # - # Default: "roadrunner" + uuid. - consumer_id: "roadrunner-uuid" - - # Use rabbitmq mechanism to requeue the job on fail - # - # Default: false - requeue_on_fail: false - - # Queue headers (new in 2.12.2) - # - # Default: null - queue_headers: - x-queue-mode: lazy + exchange: + name: amqp.default + type: direct + durable: false + auto_delete: false + declare: true + queue: + name: team-a-queue + routing_key: team-a + durable: false + auto_delete: false + exclusive: false + consumer_id: "" + delete_on_stop: false + multiple_ack: false + requeue_on_fail: false + headers: + x-queue-mode: lazy + declare: true + publish-b: + driver: amqp + config: + connection: brokerB + exchange: + name: team-b-exchange + type: direct + queue: + routing_key: team-b ``` {% endcode %} -## Configuration options +Only `consume-a` is in `jobs.consume`. The producer-only `publish-b` pipeline has no queue name. Create a destination queue on broker B before publishing. Bind it to `team-b-exchange` with routing key `team-b`. -**Here is a detailed description of each of the amqp-specific options:** +{% code title="worker.php" %} -### Priority +```php +connect('publish-b'); +$consumer = new Consumer(); -### Queue +while ($task = $consumer->waitTask()) { + try { + $destination->dispatch($destination->create($task->getName(), $task->getPayload())); + $task->ack(); + } catch (\Throwable $e) { + $task->nack($e, redelivery: true); + } +} +``` -`queue` - AMQP internal (inside the driver) queue name. Optional for producer-only pipelines, required for consumer lifecycle operations (`run`, `resume`, `pause`). +{% endcode %} -### Exchange +RR does not provide a cross-broker transaction. A failure after publishing to B but before acknowledging on A can cause duplicate messages on B. Make the destination handler safe to repeat. -`exchange` - rabbitMQ exchange name. Optional, default: `amqp.default`. +## Per-job Routing -{% hint style="info" %} -See also [AMQP model](https://www.rabbitmq.com/tutorials/amqp-concepts.html#amqp-model) documentation section. -{% endhint %} +Set the `x-routing-key` job header to override `queue.routing_key` for one publish. Supply exactly one nonempty string value. RR removes this header before it sends the message. Missing, empty, or multiple values use the pipeline's routing key. -{% hint style="info" %} -Producer-only pipeline: `queue` can be empty, `push` works, but `run`, `resume`, and `pause` will fail without a queue. -{% endhint %} +The `pool` job header selects a [PHP worker pool](overview-queues.md#named-worker-pools). It does not select an AMQP connection, exchange, or routing key. The `rr_connection` key selects a connection only during [RPC pipeline declaration](#runtime-declaration). + +## Read-only RabbitMQ Permissions + +Disable declarations when the RabbitMQ user has no `configure` permission: -### Exchange type +- `jobs.pipelines..config.exchange.declare: false` +- `jobs.pipelines..config.queue.declare: false` -`exchange_type` - rabbitMQ exchange type. May be one of `direct`, `fanout`, `topic`, `headers`. +Create the exchange, queue and binding before starting RR. The broker user still needs `read` permission to consume and `write` permission to publish. Keep `queue.delete_on_stop: false` to avoid a queue deletion request. -### Routing key +With `queue.declare: false`, RR skips both queue binding and passive queue inspection. `jobs.Stat` reports `active: 0` without querying queue depth. This does not mean the queue is empty. -`routing_key` - queue's routing key. Required for `push` when `exchange_type != fanout`. +Delayed publishing and delayed requeue still declare and bind temporary queues. These operations still require `configure` permission. -### Exclusive +{% code title=".rr.yaml" %} -`exclusive` - applied to the queue, exclusive queues cannot be redeclared. If set to true, and you attempt to declare -the same pipeline twice, it will result in an error. +```yaml +version: "3" -### Multiple ack +amqp: + brokerA: + addr: amqp://readonly:readonly@127.0.0.1:5675/TEST -`multiple_ack` - this delivery, along with all prior unacknowledged deliveries on the same channel, will be -acknowledged. This feature is beneficial for batch processing of deliveries and is applicable only for `Ack`, not for -`Nack`. +jobs: + pipelines: + readonly: + driver: amqp + config: + connection: brokerA + exchange: + name: test-1-exchange + type: fanout + durable: true + auto_delete: false + declare: false + queue: + name: test-1-queue + routing_key: test-1 + durable: true + auto_delete: false + exclusive: false + declare: false +``` -### Requeue on fail +{% endcode %} -`requeue_on_fail` - requeue on Nack (by RabbitMQ). +## Runtime Declaration -{% hint style="info" %} -Read more about Nack in RabbitMQ official docs: https://www.rabbitmq.com/confirms.html#consumer-nacks-requeue -{% endhint %} +Dynamic pipeline declaration over RPC remains a flat string map. It is separate from static YAML configuration. It does not use nested `config.exchange` or `config.queue` sections. -### Queue headers +The `jobs.Declare` payload accepts these declaration controls as strings: -`queue_headers` - used to pass arguments to the `Queue` create method, such as `x-queue-mode: lazy` +- `exchange_declare`: `"true"` (default) or `"false"`. +- `queue_declare`: `"true"` (default) or `"false"`. -### Durable +Set the flat `connection` field to a configured name such as `"brokerB"`. Do not send a DSN or TLS settings in the RPC payload. -`durable` - create a durable queue. +The existing PHP 4.x [AMQPCreateInfo API](https://github.com/roadrunner-php/jobs/blob/4.x/src/Queue/AMQPCreateInfo.php) accepts `queueHeaders`. `Jobs` serializes this map as JSON in the flat `queue_headers` field. Set the reserved `rr_connection` key in that map to select a connection. No PHP package change is required. -Default: `false` +RR reads `rr_connection` only when the flat `connection` field is absent. The flat field takes precedence whenever it is present, even when it is empty. An empty value causes an error. RR removes the reserved key from queue arguments before broker declaration and configuration storage. Other queue arguments stay unchanged. This key is RPC configuration metadata, not a message header. YAML pipelines must still set `config.connection`. -### Delete queue on stop +This runtime example creates the `runtime-b` pipeline on `brokerB` and declares its exchange. It does not declare or bind the queue. PHP 4.x `Jobs` requires an RPC instance: -`delete_queue_on_stop` - delete the queue when the pipeline is stopped. +{% code title="create.php" %} -Default: `false` +```php +use Spiral\Goridge\RPC\RPC; +use Spiral\RoadRunner\Jobs\Jobs; +use Spiral\RoadRunner\Jobs\Queue\AMQPCreateInfo; -### Redial timeout +$jobs = new Jobs(RPC::create('tcp://127.0.0.1:6001')); +$queue = $jobs->create(new AMQPCreateInfo( + name: 'runtime-b', + queue: 'team-b-queue', + exchange: 'team-b-exchange', + routingKey: 'team-b', + queueHeaders: ['rr_connection' => 'brokerB'], +)); +``` -`redial_timeout` - Redial timeout (in seconds). How long to try to reconnect to the AMQP server. +{% endcode %} -### Exchange durable +Call `$jobs->resume('runtime-b')` after creation to declare and bind the queue and start consumption. For a producer-only pipeline, omit this call. Its destination queue and binding must exist before publishing. -`exchange_durable` - Durable -exchange ([rabbitmq option](https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchanges)). +## Migration -Default: `false` +To update an existing configuration: -### Exchange auto delete +1. Move `amqp.addr` to `amqp..addr`. Move optional `amqp.tls` to `amqp..tls`. Set an explicit address for each connection. +2. Set `config.connection` on every YAML AMQP pipeline. There is no implicit default connection or localhost fallback. +3. Remove `config.version` from every YAML AMQP pipeline. Keep the root `version: "3"`. +4. For runtime declarations, send flat `connection` or use the existing PHP `queueHeaders` map with `rr_connection`. The flat field takes precedence even when empty. RR removes the reserved key before broker declaration and configuration storage. -`exchange_auto_delete` - Auto-delete (exchange is deleted when last queue is unbound from it): [link](https://www.rabbitmq.com/tutorials/amqp-concepts.html#exchanges). +Move old flat entity settings to the nested fields below. All paths are relative to `jobs.pipelines..config`. -Default: `false` +| Old Field | New Field | +| --- | --- | +| `exchange` | `exchange.name` | +| `exchange_type` | `exchange.type` | +| `exchange_durable` | `exchange.durable` | +| `exchange_auto_delete` | `exchange.auto_delete` | +| `queue` | `queue.name` | +| `durable` | `queue.durable` | +| `queue_auto_delete` | `queue.auto_delete` | +| `delete_queue_on_stop` | `queue.delete_on_stop` | +| `queue_headers` | `queue.headers` | +| `routing_key`, `exclusive`, `consumer_id`, `multiple_ack`, `requeue_on_fail` | Same key under `queue` | -### Queue auto delete +Keep `priority`, `prefetch` and `redial_timeout` directly in `config`. -`queue_auto_delete` - Auto-delete (queue that has had at least one consumer is deleted when last consumer -unsubscribes): [link](https://www.rabbitmq.com/queues.html#properties). +The configuration schema rejects `config.version` and removed flat keys. The normal config provider can ignore unknown keys, but old scalar `exchange` or `queue` values fail to decode. Flat flags do not set nested values. For example, a flat `durable: true` does not set `queue.durable`. Migrate all entity settings before starting RR. -Default: `false` +For restricted RabbitMQ permissions, set `exchange.declare: false` and `queue.declare: false`. Review the [declaration limitations](#read-only-rabbitmq-permissions) before using delayed jobs. -### Consumer id +## What's Next? -`consumer_id` - string that is unique and scoped for all consumers on this channel. +1. [Queues and Jobs overview](overview-queues.md) - Review the full jobs pipeline model before configuring AMQP in production. +2. [Read-only RabbitMQ permissions](https://www.rabbitmq.com/docs/access-control) - See declaration flags for restricted users. Use [Runtime Declaration](#runtime-declaration) for flat `jobs.Declare` keys. +3. [Pipeline configuration](#pipeline-configuration) - Use nested AMQP entity settings. See [RoadRunner configuration](../intro/config.md) for the general configuration structure. +4. [Exchange settings](#exchange-settings) and [Queue settings](#queue-settings) - Review routing and consumption settings. +5. [Allocate Timeout](../known-issues/allocate-timeout.md) and [CRC validation failed](../known-issues/stdout-crc.md) - Use these troubleshooting references when workers fail to process queue jobs as expected. diff --git a/queues/beanstalk.md b/queues/beanstalk.md index a83daf9..c667b29 100644 --- a/queues/beanstalk.md +++ b/queues/beanstalk.md @@ -78,3 +78,11 @@ value should not exceed `int32` size. ### Tube `tube` - The name of the inner "tube" specific to the Beanstalk driver. + +## Headers + +RR stores headers with the job body and preserves them on delivery and requeue. This includes retry counts, trace context, and the `pool` header for named worker pools. Existing v5 jobs need no conversion, but headers that v5 did not store cannot be recovered. Process those jobs before switching to named pools. + +## Statistics + +Queue counters describe the configured tube. An unused tube reports zero jobs. Pipelines that share a tube report the same tube counters. diff --git a/queues/boltdb.md b/queues/boltdb.md index 03b5834..553e075 100644 --- a/queues/boltdb.md +++ b/queues/boltdb.md @@ -7,6 +7,10 @@ Data in this driver is stored in the boltdb database file. You can't use the sam pipelines or for the for KV plugin and Jobs plugin. This is a boltdb limitation on simultaneous access of 2 processes to the same file. +{% hint style="warning" %} +Stored jobs do not retain application headers, including the `pool` routing header. Use a single `jobs.pool` when consuming BoltDB pipelines. [Named worker pools](overview-queues.md#named-worker-pools) cannot route these jobs. This restriction applies to new jobs as well as existing data. +{% endhint %} + ## Configuration {% code title=".rr.yaml" %} @@ -71,3 +75,11 @@ Default: `rr.db`. ### Permissions `permissions` - Permissions for the boltdb database file. Default: `0755`. + +## Recovery + +At startup, RR moves unacknowledged jobs from the in-flight bucket back to the pending bucket in one database transaction. Workers must tolerate repeated deliveries after a restart. + +Delayed jobs enter the in-memory priority queue only after their database transaction commits. A failed transaction leaves those jobs in the database for a later attempt. + +Keep the same database file when upgrading if you need to retain queued jobs. Stop RoadRunner before backing up the file. The driver reads the existing v5 job format; no conversion is required. diff --git a/queues/google-pub-sub.md b/queues/google-pub-sub.md index f795f1f..b748d06 100644 --- a/queues/google-pub-sub.md +++ b/queues/google-pub-sub.md @@ -1,18 +1,6 @@ -# Google Pub/Sub driver [Beta] +# Google Pub/Sub Driver -{% hint style="warning" %} -Driver is still in beta. Please report any issues you encounter. Configuration might be updated due to the received feedback. -{% endhint %} - -Pub/Sub is an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages. - -Pub/Sub allows services to communicate asynchronously, with latencies on the order of 100 milliseconds. - -Pub/Sub is used for streaming analytics and data integration pipelines to load and distribute data. It's equally effective as a messaging-oriented middleware for service integration or as a queue to parallelize tasks. - -Pub/Sub lets you create systems of event producers and consumers, called publishers and subscribers. Publishers communicate with subscribers asynchronously by broadcasting events, rather than by synchronous remote procedure calls (RPCs). - -Publishers send events to the Pub/Sub service, without regard to how or when these events are to be processed. Pub/Sub then delivers events to all the services that react to them. In systems communicating through RPCs, publishers must wait for subscribers to receive the data. However, the asynchronous integration in Pub/Sub increases the flexibility and robustness of the overall system. +RoadRunner publishes jobs to Google Pub/Sub topics and receives them through subscriptions. {% hint style="info" %} Read more about Google Pub/Sub [here](https://cloud.google.com/pubsub/docs/overview). @@ -20,6 +8,8 @@ Read more about Google Pub/Sub [here](https://cloud.google.com/pubsub/docs/overv ## Configuration +The global `google_pub_sub` section is required. This example connects to a local Pub/Sub emulator: + {% code title=".rr.yaml" %} ```yaml @@ -30,6 +20,7 @@ google_pub_sub: endpoint: 127.0.0.1:8085 jobs: + consume: ["test-1"] pool: num_workers: 10 @@ -41,11 +32,27 @@ jobs: project_id: test topic: rrTopic1 dead_letter_topic: "dead-letter-topic" - max_delivery_attempts: 3 + max_delivery_attempts: 10 ``` {% endcode %} +### Google Cloud Connection + +For Google Cloud, set `insecure: false` and an explicit service endpoint. The client uses TLS and [Application Default Credentials](https://docs.cloud.google.com/docs/authentication/provide-credentials-adc). + +{% code title=".rr.yaml" %} + +```yaml +google_pub_sub: + insecure: false + endpoint: pubsub.googleapis.com:443 +``` + +{% endcode %} + +Set each pipeline's `project_id` to your Google Cloud project ID. Without an explicit `endpoint`, RR uses `127.0.0.1:8085`. Use `insecure: true` for an emulator. This setting disables TLS, authentication, and Google client telemetry. + ## Configuration options **Here is a detailed description of each of the Google Pub/Sub-specific options:** @@ -66,14 +73,22 @@ from `pipe1` have been processed. ### Dead letter topic -`dead_letter_topic` - optional, string. Should be used with `max_delivery_attempts`. If the Pub/Sub service attempts to deliver a message but the subscriber can't acknowledge it, Pub/Sub can forward the undeliverable message to a dead-letter topic. For more information, see: [link](https://cloud.google.com/pubsub/docs/handling-failures#dead_letter_topic) +`dead_letter_topic`: Optional topic ID. Use it with `max_delivery_attempts` to configure forwarding of messages that cannot be acknowledged. See [Pub/Sub dead-letter topics](https://cloud.google.com/pubsub/docs/handling-failures#dead_letter_topic). ### Max delivery attempts -`max_delivery_attempts` - optional, int. Should be used with `dead_letter_topic`. The maximum number of delivery attempts for a message. For more information, see: [link](https://cloud.google.com/pubsub/docs/handling-failures#dead_letter_topic) +`max_delivery_attempts`: Optional delivery-attempt setting for dead-letter forwarding. RoadRunner defaults to `10` when `dead_letter_topic` is set. See [Pub/Sub dead-letter configuration](https://cloud.google.com/pubsub/docs/handling-failures#dead_letter_topic). + +## Subscriptions + +RR reuses existing topics and subscriptions. The pipeline name is the subscription ID. Keep the project, topic, and pipeline names unchanged during an upgrade to use the same subscription. + +RR applies `dead_letter_topic` and `max_delivery_attempts` when it creates subscriptions for both YAML pipelines and dynamically declared pipelines. These settings do not update an existing subscription. Use Pub/Sub administration to change an existing subscription's dead-letter policy. Do not delete a subscription with pending messages just to apply new settings. + +## Pause and Resume + +Pausing a pipeline cancels its receive operation and stops new message pulls. Publishing remains available while paused. Jobs already received by RoadRunner can still complete. Resume starts receiving from the same subscription again. -## Limitations +## Statistics -1. TLS is not supported at the moment. -2. Metrics are not supported at the moment. -3. Telemetry and Authentication are not supported at the moment. Use `insecure: true` to test this driver. +Use Google Pub/Sub monitoring for queue counts. RR statistics return pipeline identity only. Zero counts and `ready: false` do not report the broker's actual state. diff --git a/queues/kafka.md b/queues/kafka.md index 1fc96c3..fa7288e 100644 --- a/queues/kafka.md +++ b/queues/kafka.md @@ -14,6 +14,8 @@ Version `2023.2.0` update: ## Configuration +For direct partition consumption, use the [direct partitions example](#direct-partitions). + {% code title=".rr.yaml" %} ```yaml @@ -130,7 +132,6 @@ kafka: user_agent: baz jobs: - num_pollers: 10 pipeline_size: 100000 pool: num_workers: 10 @@ -231,7 +232,8 @@ jobs: # topics: adds topics to consume # - # Default: empty (produces an error); possible to use regexp if `consume_regexp` is set to true. + # Default: empty. Required unless consume_partitions is set. + # Regex is supported when consume_regexp is true. topics: [ "foo", "bar", "^[a-zA-Z0-9._-]+$" ] # consume_regexp sets the client to parse all topics passed to `topics` as regular expressions. @@ -257,35 +259,6 @@ jobs: # Optional, default: 1. min_fetch_message_size: 1 - # consume_partitions sets partitions to consume from directly and the offsets to start consuming those partitions from. - # This option is basically a way to explicitly consume from subsets of partitions in topics, or to consume at exact offsets. - # - # NOTE: This option is not compatible with group consuming and regex consuming. - # - # Optional, default: null - consume_partitions: - - # Topic for consume_partitions - # - # At least one topic is required. - foo: - - # Partition for the topic. - # - # At least one partition is required. - 0: - - # Partition offset. - # - # Required if all options are used. No default; error on empty. - # Possible values: AtEnd, At, AfterMilli, AtStart, Relative, WithEpoch - type: AtStart - - # Value for the At, AfterMilli, Relative, and WithEpoch offsets. - # - # Optional, default: 0. - value: 1 - # consumer_offset sets the offset to start consuming from, or, if OffsetOutOfRange is seen while fetching, # to restart consuming from. # @@ -326,3 +299,57 @@ jobs: ``` {% endcode %} + +## Acknowledgments + +Pipelines with `group_options.group_id` set mark acknowledged records for automatic offset commits. A commit advances the consumer group's position for a partition. RR does not wait for all earlier records in that partition to complete. + +{% hint style="warning" %} +Workers can complete jobs out of order. If offset `101` is acknowledged while offset `100` is unfinished in the same partition, a commit can advance the group to `102`. After a crash, the group then skips offset `100`, although that job was not acknowledged. Do not assume that every unacknowledged job will be delivered again. +{% endhint %} + +Direct partition consumption without `group_options` does not use these consumer-group commits. + +## Direct Partitions + +Use `consumer_options.consume_partitions` to select topic partitions and starting offsets without a consumer group. + +A nonempty `consumer_options.topics` list takes precedence: RR ignores `consume_partitions` in that case. For direct partition consumption, omit `topics` and `group_options`. Leave `consume_regexp` unset or `false`. + +This example consumes partition `0` of the existing `orders` topic, starting at offset `100`: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +server: + command: php consumer.php + relay: pipes + +kafka: + brokers: ["127.0.0.1:9092"] + +jobs: + pool: + num_workers: 2 + consume: ["orders"] + pipelines: + orders: + driver: kafka + config: + consumer_options: + consume_partitions: + orders: + 0: + type: At + value: 100 +``` + +{% endcode %} + +Each topic maps partition numbers to offsets. The offset `type` is required: `At`, `AfterMilli`, `AtEnd`, `AtStart`, `Relative` or `WithEpoch`. The `value` defaults to `0` and is used by `At`, `AfterMilli`, `Relative` and `WithEpoch`. + +## Consumer Recovery + +When a Kafka client closes unexpectedly or reports a non-retriable broker error, the driver asks the Jobs plugin to recreate the pipeline. The driver releases blocked group rebalances when its listener exits. Check the logs if pipeline recreation fails. A broker error that remains, such as denied topic access, still requires operator action. diff --git a/queues/nats.md b/queues/nats.md index 9560397..643b0fc 100644 --- a/queues/nats.md +++ b/queues/nats.md @@ -13,7 +13,6 @@ nats: addr: "demo.nats.io" jobs: - num_pollers: 10 pipeline_size: 100000 pool: num_workers: 10 @@ -43,10 +42,12 @@ jobs: # The consumer will only start receiving messages that were created after the consumer was created # Default: false (deliver all messages from the stream beginning) + # RR does not apply this option to the consumer. deliver_new: true # Consumer rate-limiter in bytes https://docs.nats.io/jetstream/concepts/consumers#ratelimit # Default: 1000 + # RR does not apply this option to the consumer. rate_limit: 100 # Delete the stream after the pipeline is stopped @@ -56,11 +57,12 @@ jobs: # Delete message from the stream after successful acknowledge # Default: false delete_after_ack: false - - # Time in seconds after which NATS will redeliver the message if no ACK received - # Default: 30 - ack_wait: 30 + + # Time before redelivery if no acknowledgment is received. + # Default: 30s + ack_wait: 30s ``` + {% endcode %} ## Configuration options @@ -81,15 +83,19 @@ To prevent duplicate message consumption, ensure that each pipeline is configure ### Deliver new -`deliver_new` - the consumer will only start receiving messages that were created after the consumer was created. +RR parses `deliver_new` but does not apply it to the consumer. It does not prevent replay after a consumer restart. ### Rate limit -`rate_limit` - NATS rate [limiter](https://docs.nats.io/jetstream/concepts/consumers#ratelimit). +RR parses `rate_limit` but does not apply it to the consumer. ### Delete stream on stop -`delete_stream_on_stop` - delete the whole stream when pipeline stopped. +`delete_stream_on_stop` deletes the whole stream when the pipeline stops. Default: `false`. + +Stopping a pipeline with `delete_stream_on_stop: false` preserves the stream. Messages remain subject to the stream's retention policy. Setting this option to `true` deletes the stream and its messages. + +RR creates a new consumer on each run or resume. Retained messages can be delivered again, including messages acknowledged by an earlier consumer when `delete_after_ack` is `false`. Review stream retention before upgrading. Make job handlers safe to process the same job more than once. ### Delete after ack @@ -97,4 +103,12 @@ To prevent duplicate message consumption, ensure that each pipeline is configure ### Ack wait -`ack_wait` - time in seconds after which NATS will redeliver the message if no ACK received. +`ack_wait` sets the time before JetStream redelivers an unacknowledged message. + +YAML uses a Go duration string, such as `ack_wait: 30s` or `ack_wait: 2m`. The default is `30s`. Use a duration longer than the expected queueing and processing time. + +The `jobs.Declare` RPC payload instead uses integer seconds encoded as a string: `"ack_wait": "30"`. Do not send a duration string such as `"30s"` through RPC. + +## Headers + +RR preserves job headers when it republishes a job for retry. This includes application headers, trace context, and the `pool` header used by [named worker pools](overview-queues.md#named-worker-pools). diff --git a/queues/nsq.md b/queues/nsq.md new file mode 100644 index 0000000..a6a71f5 --- /dev/null +++ b/queues/nsq.md @@ -0,0 +1,76 @@ +# NSQ Driver + +RoadRunner publishes jobs to an NSQ topic and consumes them through an NSQ channel. + +## Configuration + +Start `nsqd` before RoadRunner. The example uses a [Jobs consumer](./overview-queues.md) in `consumer.php`. + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +rpc: + listen: tcp://127.0.0.1:6001 + +server: + command: php consumer.php + relay: pipes + +nsq: + addr: tcp://127.0.0.1:4150 + lookupd_poll_interval: 5s + +jobs: + consume: [tasks] + pipelines: + tasks: + driver: nsq + config: + topic: tasks + channel: workers + prefetch: 10 + priority: 10 + max_attempts: 5 +``` + +{% endcode %} + +The global `nsq` section is required. Put connection settings in this section. Put pipeline settings under `jobs.pipelines..config`. Global values override matching fields in YAML pipeline configurations. + +- `addr`: The nsqd TCP address. Both `host:port` and `tcp://host:port` are accepted. The default is `127.0.0.1:4150`. The producer always uses this address, including when consumer discovery is enabled. Driver initialization fails if the producer cannot connect. +- `lookupd`: An optional list of nsqlookupd HTTP addresses, such as `["127.0.0.1:4161"]`. With this option, consumers discover nsqd servers through nsqlookupd. Without it, consumers connect to `addr`. +- `lookupd_poll_interval`: The duration between discovery polls. It also controls the delay before a direct consumer connection reconnects. The default is `60s`. +- `dial_timeout`: The connection timeout. The default is `1s`. Duration settings use a unit suffix, such as `5s`. +- `read_timeout` and `write_timeout`: Network read and write timeouts. Positive values override the NSQ client defaults of `60s` and `1s`. + +## Pipeline Options + +- `topic`: The topic to publish to and consume from. It defaults to the pipeline name. +- `channel`: The consumer channel. It defaults to `default`. Consumers on the same topic and channel share the work. Separate channels each receive a copy of the topic's messages. +- `prefetch`: The maximum number of messages in flight for the consumer. It defaults to `10`. Zero or negative values also select `10`. +- `priority`: The default job priority in RoadRunner. It defaults to `10`. Lower numbers have higher priority. This affects RR's priority queue, not NSQ delivery order. +- `max_attempts`: The delivery-attempt limit. See [Retry Limit](#retry-limit) before changing it. + +## Delivery Behavior + +- Acknowledgement completes the NSQ message. Auto-ack completes it before PHP processes it. Do not enable auto-ack when failed jobs must be retried. +- Job delays and explicit retry delays use seconds. A negative acknowledgement can requeue the original message. Disabling requeue completes the message without another attempt. +- A retry that changes headers publishes a new message to the topic before acknowledging the original. The two operations are not atomic. Workers must tolerate repeated deliveries. +- Pause stops requesting new messages without closing the consumer connection. Jobs already received can still run. Publishing remains available while paused. Resume restores the configured prefetch limit. +- The driver can consume raw messages from other producers. It passes the raw body to PHP with the job name `deduced_by_rr`. + +## Retry Limit + +{% hint style="warning" %} +Omitting `max_attempts` or setting it to `0` keeps the client default of five attempts. Zero does not enable unlimited retries. On a delivery beyond the limit, the client acknowledges the message without sending it to PHP. The driver does not forward it to a dead-letter queue. +{% endhint %} + +Use a positive `max_attempts` value for a different limit. A retry that publishes a new message starts a new broker attempt count. This setting is not a total retry limit across those new messages. Applications that must retain failed jobs need their own failure storage. + +## Limitations + +RR statistics report pipeline identity, listener readiness, and a locally tracked delayed-job count. They do not report the broker backlog. Use [nsqadmin](https://nsq.io/components/nsqadmin.html) for topic and channel counts. + +The driver does not expose TLS or authentication settings. diff --git a/queues/overview-queues.md b/queues/overview-queues.md index f036c79..5cfa06c 100644 --- a/queues/overview-queues.md +++ b/queues/overview-queues.md @@ -1,13 +1,8 @@ # Jobs (Queue) plugin -Starting with RoadRunner >= 2.4, a queuing system (aka "jobs") is available. -This plugin allows you to move arbitrary "heavy" code into separate tasks to -execute them asynchronously in an external worker, which will be referred to -as "consumer" in this documentation. +The Jobs plugin sends tasks to queues and runs them in PHP workers. A producer sends tasks. A consumer receives and processes them. -The RoadRunner PHP library provides both API implementations: The client one, -which allows you to dispatch tasks, and the server one, which provides the -consumer who processes the tasks. +The RoadRunner PHP library provides APIs for both producers and consumers. ![RR queue](https://github.com/roadrunner-server/roadrunner-docs/assets/773481/31f7aed2-efae-4088-8f46-b67b8c04a3dd) @@ -64,7 +59,6 @@ Let's now focus on the common settings of the queue server. In full, it may look version: "3" jobs: - num_pollers: 64 timeout: 60 pipeline_size: 100000 options: @@ -85,12 +79,9 @@ jobs: {% endcode %} -Above is a complete list of all possible common Jobs settings. Let's now figure out what they are responsible for. +Common Jobs settings: -- `num_pollers`: The number of threads that are simultaneously reading from the priority - queue and send payloads to the workers. There is no optimal number, it depends - heavily on the performance of the PHP worker. For example, echo workers - can process over 300k jobs per second within 64 pollers (on a 32 core CPU). `num_pollers` should not be less than the number of workers to properly load all of them with the jobs. +- `num_pollers`: RR derives the number of queue pollers from the worker count and ignores this setting. A single pool uses `num_workers + 2` pollers. If `num_workers` is omitted or zero, RR uses the default worker count of the pool. Named pools use the total worker count plus two. - `timeout`: The internal Golang context timeouts (in seconds). For example, if the connection was disconnected or your push was in the middle of a @@ -98,32 +89,88 @@ Above is a complete list of all possible common Jobs settings. Let's now figure or the queue is full. If the timeout is exceeded, your call will be rejected with an error. Default: 60 (seconds). -- `options.parallelism`: The number of goroutines that process jobs from the binary heap priority queue simultaneously. Default: 10. +- `options.parallelism`: Limits concurrent pipeline initialization and destruction. Default: `10` when `options` is omitted; `5` when `options.parallelism` is zero. -- `pipeline_size`: The binary heaps priority queue (PQ) settings. The priority - queue stores jobs in order of priority. The priority can be set - for the job or inherited by the pipeline. When worker performance is poor, PQ - will accumulate jobs until `pipeline_size` is reached. After that, PQ - is then blocked until the workers have processed all the jobs in it. **Lower number means higher priority.** +- `pipeline_size`: The size of the priority queue (PQ). All pipelines share one PQ. The PQ stores jobs in order of priority, and jobs with the same priority leave the PQ in the order they arrived. The priority can be set for the job or inherited by the pipeline. When the workers are slower than the drivers, the PQ fills up to `pipeline_size`. After that, each driver waits for a free slot before it adds the next job. Default: `100000`. **Lower number means higher priority.** {% hint style="info" %} -Blocked PQ means that you can push the job into the driver, but RoadRunner -will not read that job until PQ is empty. If RoadRunner is running -with jobs in the PQ, they won't be lost because jobs are not removed from the driver's -driver queue until after Ack. +A full PQ blocks further inserts. Publishing can continue if the driver has capacity. + +A job can wait in the PQ for about `pipeline_size` multiplied by the average job time, divided by the number of workers. Keep this time shorter than the acknowledgment deadline of the broker, for example the beanstalk TTR, the SQS visibility timeout or the NATS `ack_wait`. If the wait is longer, the broker can deliver the job again while it is in the PQ. + +The PQ does not guarantee recovery after a crash. Recovery depends on the driver's persistence, broker retention, and acknowledgment settings. The memory driver loses jobs when RoadRunner stops. Auto-ack can acknowledge a job before PHP processes it. + +Kafka pipelines with a configured consumer group use partition offset commits. A later acknowledgment can cause unfinished earlier records in the same partition to be skipped after a crash. See [Kafka acknowledgments](./kafka.md#acknowledgments). {% endhint %} -- `pool`: All settings in this section are similar to the worker pool settings - described on the [configuration page](https://roadrunner.dev/docs/intro-config). +- `pool`: A single [worker pool](../php/pool.md) for all consumed pipelines. + +- `pools`: Named worker pools. Configure either `pool` or `pools`. See [Named Worker Pools](#named-worker-pools). -- `consume`: Contains an array of the names of all queues specified in the - `"pipelines"` section, which should be processed by the concierge specified in - the global `"server"` section (see the [PHP worker's settings](../php/worker.md)). +- `consume`: Pipeline names to start consuming when RoadRunner starts. Pipelines outside this list are available for publishing. Use `jobs.Resume` to start consumption later. - `pipelines`: This section contains a list of all queues created in the RoadRunner. The key is a unique *queue identifier*, and the value is an object of the driver-specific configuration (we will talk about this later). +### Named Worker Pools + +Named pools let you assign worker capacity and worker settings to different pipelines. Configure them under `jobs.pools`. Setting both `jobs.pool` and a nonempty `jobs.pools` is an error. + +Set each pipeline's `pool` to a configured pool name: + +{% code title=".rr.yaml" %} + +```yaml +version: "3" + +rpc: + listen: tcp://127.0.0.1:6001 + +server: + command: php consumer.php + relay: pipes + +jobs: + pools: + default: + num_workers: 4 + reports: + num_workers: 1 + consume: ["emails", "reports"] + pipelines: + emails: + driver: memory + pool: default + config: + prefetch: 10 + reports: + driver: memory + pool: reports + config: + prefetch: 10 +``` + +{% endcode %} + +Each pool accepts the [worker pool settings](../php/pool.md), including `command` to select a different PHP worker command. Jobs worker metrics include workers from all pools. A jobs reset resets all pools. Worker add and remove operations apply to each pool. + +On `jobs.Push` and `jobs.PushBatch`, RR copies a nonempty pipeline `pool` setting into the lowercase `pool` job header. This replaces any existing value. The consumer selects the named pool from the first header value. + +Treat `pool` as a reserved header. An unknown pool name causes a negative acknowledgment, even in single-pool mode. In multi-pool mode, a missing or empty `pool` header also causes a negative acknowledgment. A pool named `default` is not an automatic fallback. + +Producers that publish directly to a broker must supply the `pool` header in the driver's job format. The pipeline setting alone does not route incoming jobs. Process existing jobs without this header before switching to named pools. Producer and consumer instances must agree on pool names. + +{% hint style="warning" %} +BoltDB does not preserve the `pool` header in stored jobs. Use a single `jobs.pool` when consuming BoltDB pipelines. This also affects newly published jobs. +{% endhint %} + +### Producer-only Pipelines + +Omit a pipeline from `jobs.consume` to use it only for publishing on that RoadRunner instance. Set `consume: []` to start no consumers. Driver connections and pipeline setup still run at startup. + +The Jobs plugin still creates PHP worker pools with this configuration. A `num_workers` value of `0` selects the default worker count. See the [AMQP example](amqp.md#pipeline-configuration) for a pipeline that publishes without a queue name. + ## PHP Client (Producer) ### Installation @@ -354,6 +401,8 @@ IP address, the user's token or session id, etc. Headers can only contain string values and are not serialized in any way during transmission, so be careful when specifying them. +The lowercase `pool` header is [reserved for worker-pool routing](#named-worker-pools). + In the case to add a new header to the task, you can use methods [similar to PSR-7](https://www.php-fig.org/psr/psr-7/). **That is:** @@ -375,6 +424,14 @@ $queue->dispatch($task); {% endcode %} +### Trace Context + +Enable the [OpenTelemetry plugin](../lab/otel.md) to export job traces. Put the producer's `traceparent` and optional `tracestate` values in the task headers to continue its trace. The `jobs.Push` RPC handler extracts this context before it starts the push span. Without valid trace context, RR starts a new trace. + +`jobs.PushBatch` uses the first valid trace context in the batch as the parent of its push span. Submit jobs from separate traces in separate calls when their trace boundaries must remain separate. + +Trace propagation through storage depends on the driver. BoltDB does not store job headers. Beanstalk jobs written by v5 also lack stored headers. + ### Task Delayed Dispatching If you want to specify that a job should not be immediately available for processing by a jobs worker, you can use the @@ -599,9 +656,7 @@ $handler->process($task); ### Task auto acknowledge -RoadRunner version `v2.10.0+` supports an auto acknowledge task option. You might use this option to acknowledge a task -right after RR receive it from the queue. You can use this option for the non-important tasks which can fail or break -the worker. +Automatic task acknowledgment has been supported since RoadRunner `v2.10.0`. This option acknowledges a task as soon as RR receives it from the queue. Use it for tasks that can be lost if processing fails. To use this option you may update the Options: @@ -826,10 +881,25 @@ $jobs->resume('emails', 'billing'); {% endcode %} +## Metrics + +Enable the [Metrics plugin](../lab/metrics.md) to export these Prometheus counters: + +| Metric | Meaning | +| --- | --- | +| `rr_jobs_jobs_ok` | Successfully acknowledged processing attempts. | +| `rr_jobs_jobs_err` | Failed processing attempts, including worker, routing, and response errors. | +| `rr_jobs_jobs_requeue` | Attempts that the worker response handler successfully requeues. | +| `rr_jobs_push_ok` | Successful job pushes to drivers. | +| `rr_jobs_push_err` | Failed driver push calls. | + +A successful requeue increments `rr_jobs_jobs_requeue` without incrementing `rr_jobs_jobs_ok`. A failed requeue increments `rr_jobs_jobs_err`. A later successful attempt increments `rr_jobs_jobs_ok`. Broker redelivery without a worker requeue response does not increment the requeue counter. + +These metrics count attempts across all pipelines and pools. They are counters and reset when RoadRunner restarts. Use Prometheus `rate()` or `increase()` to measure activity over time. + ## RPC Interface -All communication between PHP and GO made by the RPC calls with protobuf payloads. You can find versioned proto-payloads -here: [Proto](https://github.com/roadrunner-server/roadrunner/blob/e9713a1d08a93e2be70c889c600ed89f54822b54/proto/jobs/v1beta). +PHP communicates with the Jobs plugin through Goridge RPC with [jobs/v1 Protobuf messages](https://github.com/roadrunner-server/api/blob/master/roadrunner/api/jobs/v1/jobs.proto). The statistics method is `jobs.Stat`. - `Push(in *jobsv1.PushRequest, out *jobsv1.Empty) error` - The arguments: the first argument is a `PushRequest`, which contains one field of the `Job` being sent to the queue; the second argument is `Empty`, which means that the function @@ -856,7 +926,7 @@ here: [Proto](https://github.com/roadrunner-server/roadrunner/blob/e9713a1d08a93 an `DeclareRequest`, which contains one `map` pipeline field of queue configuration; the second argument is `Empty`, which means that the function does not return a result. The error returned if the request fails. -- `Stat(in *jobsv1beta.Empty, out *jobsv1beta.Stats) error` - The arguments: the first argument is an `Empty`, meaning +- `Stat(in *jobsv1.Empty, out *jobsv1.Stats) error` - The arguments: the first argument is an `Empty`, meaning that the function does not accept anything (from the point of view of the PHP API, an empty string should be passed); the second argument is `Stats`, which contains one repeated (list) field named `Stats` of type `Stat`. The error returned if the request fails. diff --git a/queues/sqs.md b/queues/sqs.md index 62446c5..c254801 100644 --- a/queues/sqs.md +++ b/queues/sqs.md @@ -131,7 +131,7 @@ jobs: config: # Optional section. - # Default: 10 + # Default: 1 prefetch: 10 # Get queue URL only @@ -185,6 +185,12 @@ process the jobs in correct order. 3. You should make sure to either use an explicit non-zero `visibility_timeout` **or** make sure that the default visibility configuration attribute on your queue (`VisibilityTimeout`) is non-zero. +### Retries + +A republished FIFO retry uses a new SQS deduplication ID. The application job ID stays the same. Initial sends use the job ID for deduplication within SQS's five-minute deduplication window. + +`retain_failed_jobs: true` applies to native Nack handling. An explicit requeue still republishes the job, even with this option enabled, and can change FIFO processing order. The new deduplication behavior does not add exactly-once job processing. Job handlers must remain safe to retry. + ## Configuration Options ### Prefetch diff --git a/releases/v3-0-0.md b/releases/v3-0-0.md new file mode 100644 index 0000000..3ea4fec --- /dev/null +++ b/releases/v3-0-0.md @@ -0,0 +1,119 @@ +# πŸš€ v3.0.0 πŸš€ + +RoadRunner v3 uses v6 Go plugins. See [Upgrade from v2025 to v3](../intro/v3-migration.md) for the required configuration and API changes. + +## Changelog + +### 🎯 Core + +- ✨ **Go module**: Import RoadRunner through `github.com/roadrunner-server/roadrunner/v3`. See [Embedding a server](../customization/embedding.md). +- ✨ **Plugin API**: Protobuf schemas, generated Go messages, and Go plugin contracts now have separate repositories: `api`, `api-go/v6`, and `api-plugins/v6`. Custom plugins use `log/slog`, `pool/v2`, and `goridge/v4`. See [Writing a Plugin](../customization/plugin.md), [PR](https://github.com/roadrunner-server/roadrunner/pull/2383). +- ✨ **Unix sockets**: Configure socket mode, owner, and group for supported listeners and worker relays. See [Configuration](../intro/config.md#unix-socket-attributes), [FR](https://github.com/roadrunner-server/roadrunner/issues/1789). +- ✨ **Environment files**: Load the root `envfile` without experimental mode. A configured file is required when the server starts. See [Environment](../php/environment.md). +- ✨ **RPC codecs**: Goridge v4 removes MessagePack support. JSON, protobuf, Gob, and raw payloads remain available through the existing Goridge RPC transport. See [RPC](../php/rpc.md), [PR](https://github.com/roadrunner-server/goridge/pull/194). +- ✨ **Worker scaling**: Retry worker acquisition after automatic scale-up and revise idle-worker removal. `dynamic_allocator.max_workers` counts workers added above the base pool size. See [Auto workers scaling](../php/auto-scaling.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2086). +- ✨ **Initialization environment**: Values in `server.on_init.env` override inherited values. See [Server](../plugins/server.md), [PR](https://github.com/roadrunner-server/server/pull/137). +- ✨ **Default plugins**: Add NSQ, HTTP rate limiting, Zstd compression, and the protobuf registry to the default container. RoadRunner v3 does not support the [TCP plugin](../plugins/tcp.md). See [NSQ](https://github.com/roadrunner-server/roadrunner/pull/2383), [rate limiting](https://github.com/roadrunner-server/roadrunner/pull/2389), and [Zstd and Protoreg](https://github.com/roadrunner-server/roadrunner/pull/2410). + +### πŸ“¦ `http` plugin + +- ✨ **Middleware order**: Requests enter `http.middleware` from left to right. Reverse a v5 list to keep its previous order. See [Middleware order](../http/http.md#middleware-order), [PR](https://github.com/roadrunner-server/http/pull/226). +- ✨ **Cleartext HTTP/2**: Use HTTP/2 prior knowledge for H2C connections. HTTP/1.1 `Upgrade: h2c` is no longer supported. See [HTTP/2](../http/http.md#http2). +- ✨ **PROXY protocol**: Accept v1 and v2 headers on HTTP and HTTPS listeners with explicit trusted proxy addresses. See [HTTP](../http/http.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/937). + +### πŸ“¦ `proxy_ip_parser` middleware + +- ✨ **Trusted headers**: Select forwarding headers and their priority through `http.trusted_headers`. See [Proxy IP parser](../http/proxy.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1515). + +### πŸ“¦ `static` middleware + +- ✨ **File serving controls**: Configure URL prefixes, metadata cache TTL, missing-file cache TTL, and cache limits. Reset both caches with `rr reset static`. Weak ETags now use file size and modification time. See [Static files](../http/static.md), [PR](https://github.com/roadrunner-server/static/pull/147). + +### πŸ“¦ `rate_limiter` middleware + +- ✨ **Request limits**: Limit requests by global, IP, or header key. Rejected requests return `429` with `Retry-After`. See [Rate limiter](../http/rate-limiter.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/934). + +### πŸ“¦ `zstd` middleware + +- ✨ **Response compression**: Compress HTTP responses with Zstandard when the client accepts `zstd`. See [Zstd](../http/zstd.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2080). + +### πŸ“¦ `grpc` plugin + +- ✨ **Server reflection**: Discover services through reflection and retrieve PHP service descriptors through the bundled Protoreg plugin. See [Protoreg](../grpc/protoreg.md#server-reflection), [FR](https://github.com/roadrunner-server/roadrunner/issues/1208). +- ✨ **Error details**: Include supported `google.rpc.Status` details in failed unary-call logs. See [gRPC](../grpc/grpc.md#error-details-in-logs), [FR](https://github.com/roadrunner-server/roadrunner/issues/1897). + +### πŸ“¦ `amqp` driver + +- ✨ **Named connections**: Connect pipelines to different brokers. Static pipelines require `config.connection` and nested `exchange` and `queue` settings; AMQP `config.version` is removed. See [RabbitMQ](../queues/amqp.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1962). +- ✨ **Declaration controls**: Configure queue and exchange declarations independently. Use existing broker resources with accounts that cannot declare them. See [RabbitMQ](../queues/amqp.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2275). + +### πŸ“¦ `jobs` plugin + +- ✨ **Named worker pools**: Assign pipelines to separate PHP worker pools with their own commands and capacity. The `pool` header selects the destination pool. See [Jobs](../queues/overview-queues.md#named-worker-pools), [PR](https://github.com/roadrunner-server/jobs/pull/139). +- ✨ **Trace continuity**: Preserve valid producer trace context when jobs are submitted through RPC. See [Jobs](../queues/overview-queues.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2211). +- ✨ **Outcome metrics**: Count successful, failed, and requeued attempts separately with `rr_jobs_jobs_requeue`. Outcome and push totals use counters. See [Metrics](../lab/metrics.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1566). +- ✨ **Queue size**: The default `pipeline_size` is `100000`. See [Jobs](../queues/overview-queues.md), [PR](https://github.com/roadrunner-server/jobs/pull/157). +- πŸ› **Full queue**: A driver waits for a free slot in a full priority queue without CPU use. Jobs with the same priority leave the queue in arrival order. See [Jobs](../queues/overview-queues.md), [BUG](https://github.com/roadrunner-server/roadrunner/issues/2411). +- πŸ› **Pollers**: A `jobs.pool` without `num_workers` uses `num_workers + 2` pollers with the default worker count of the pool. See [Jobs](../queues/overview-queues.md), [PR](https://github.com/roadrunner-server/jobs/pull/157). + +### πŸ“¦ `nsq` driver + +- ✨ **NSQ jobs**: Publish and consume jobs through NSQ topics and channels. Support broker discovery, delayed delivery, acknowledgements, and configurable reconnection intervals. See [NSQ](../queues/nsq.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1948). + +### πŸ“¦ `nats` driver + +- ✨ **Stream retention**: Preserve stream contents when a pipeline stops unless stream deletion is enabled. Review retention and repeated-delivery handling when upgrading. See [NATS](../queues/nats.md), [PR](https://github.com/roadrunner-server/nats/pull/226). +- ✨ **Acknowledgment wait**: Configure `ack_wait` for JetStream redelivery timing. See [NATS](../queues/nats.md), [PR](https://github.com/roadrunner-server/nats/pull/190). + +### πŸ“¦ `kafka` driver + +- ✨ **Direct partition consumption**: Apply `consume_partitions` to select topic partitions and starting offsets. See [Kafka](../queues/kafka.md), [PR](https://github.com/roadrunner-server/kafka/pull/580). + +### πŸ“¦ `sqs` driver + +- πŸ› **FIFO retry delivery**: Use a fresh broker deduplication ID for each republished retry while retaining the application job ID. This prevents SQS from discarding the retry as a duplicate of the deleted original. See [SQS](../queues/sqs.md), [PR](https://github.com/roadrunner-server/sqs/pull/864). + +### πŸ“¦ `beanstalk` driver + +- ✨ **Job headers**: Preserve headers through storage and retries, including trace context and named-pool routing. Existing messages cannot recover headers that were not stored. See [Beanstalk](../queues/beanstalk.md). + +### πŸ“¦ `logger` plugin + +- ✨ **Logging configuration**: Use Go `log/slog` with JSON, text, and raw formats. Configure file destinations through `output`; built-in rotation and `file_logger_options` are removed. See [Logger](../lab/logger.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1750). +- ✨ **Custom formats**: Set `format` placeholders and `time_format` to control log records and timestamps. See [Logger](../lab/logger.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/1376). +- ✨ **Non-blocking output**: Queue log records before writing them. A full output queue drops new records to keep application calls responsive. See [Logger](../lab/logger.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2195). + +### πŸ“¦ `lock` plugin + +- ✨ **Redis backend**: Share locks between RoadRunner instances through Redis with the existing lock RPC API. The in-memory backend remains the default. See [Locks](../plugins/locks.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2070). + +### πŸ“¦ `redis` driver + +- ✨ **Sentinel authentication**: Configure `sentinel_username` and `sentinel_password` independently from Redis master credentials. Password-only Sentinel authentication remains supported. See [Redis](../kv/redis.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2407). + +### πŸ“¦ `service` plugin + +- ✨ **Deferred updates**: Change stored service configuration through `service.Update`. Existing processes keep their current settings; new processes use the updated values. PHP applications need client and DTO support for this method. See [Service](../plugins/service.md), [FR](https://github.com/roadrunner-server/roadrunner/issues/2119). + +### πŸ“¦ `status` plugin + +- ✨ **Kubernetes probes**: Use `/livez` as an alias for `/health` and `/readyz` as an alias for `/ready`. See [HealthChecks](../lab/health.md), [PR](https://github.com/roadrunner-server/status/pull/154). + +### πŸ“¦ `otel` plugin + +- ✨ **Span timing**: End named middleware spans before the next handler starts. Use the outer HTTP server span for request latency. See [OpenTelemetry](../lab/otel.md) and [HTTP tracing](../http/http.md#tracing), [FR](https://github.com/roadrunner-server/roadrunner/issues/2276). +- ✨ **Exporters**: Remove the native Zipkin exporter. Send traces through OTLP to a compatible receiver. See [OpenTelemetry](../lab/otel.md). + +### πŸ“¦ `centrifuge` plugin + +- ✨ **Protocol update**: Add `NotifyCacheEmpty` event support and remove `centrifuge.RateLimit`. The new event needs matching PHP DTO and handler support. See [Centrifuge](../plugins/centrifuge.md). + +### πŸ“¦ `temporal` plugin + +- ✨ **Worker heartbeats**: Configure the worker heartbeat interval and report PHP SDK identity with host CPU and memory metrics. See [Temporal](../workflow/temporal.md), [PR](https://github.com/temporalio/roadrunner-temporal/pull/790). +- ✨ **Dynamic workflows**: Register a catch-all workflow for workflow types without a named registration. See [Worker](../workflow/worker.md), [PR](https://github.com/temporalio/roadrunner-temporal/pull/784). +- πŸ› **Activity-worker recovery**: Replace a failed activity worker without restarting healthy workers in the same activity pool. See [Worker](../workflow/worker.md), [BUG](https://github.com/roadrunner-server/roadrunner/issues/2335). + +### πŸ“¦ `velox` builder + +- ✨ **Module configuration**: Select plugins through `[plugins.]`, with `module_name` and `tag`. Support module replacements, exclusions, and repeatable build timestamps. The remote build server and Windows targets are removed. See [Building a Server](../customization/build.md), [PR](https://github.com/roadrunner-server/velox/pull/350). diff --git a/workflow/temporal.md b/workflow/temporal.md index 5c085da..5d4a0af 100644 --- a/workflow/temporal.md +++ b/workflow/temporal.md @@ -1,10 +1,9 @@ # About Temporal.IO -Temporal is a distributed, scalable, durable, and highly available orchestration engine used to execute asynchronous -long-running business logic in a scalable and resilient way. +Temporal runs durable workflows and activities for long-running business operations. Read more at [official website](https://docs.temporal.io/). -RoadRunner >= 2.0 includes a plugin to execute Temporal workflows and activities. Make sure to write [temporal worker](/workflow/worker.md). +RoadRunner includes a plugin to execute Temporal workflows and activities. Configure a [Temporal worker](worker.md). Activate plugin via config: @@ -17,15 +16,44 @@ server: temporal: address: "127.0.0.1:7233" + worker_heartbeat_interval: 10s activities: num_workers: 10 logs: level: debug channels: - temporal.level: error + temporal: + level: error ``` +## Worker Heartbeats + +The `temporal.worker_heartbeat_interval` option sets how often SDK workers report their state to the Temporal server. Use a duration from `1s` through `60s`. If the option is omitted or set to `0s`, the SDK uses its default of `60s`. + +Positive intervals below `1s` are clamped to `1s`. Intervals above `60s` are clamped to `60s`. The plugin logs a warning when it clamps a value. + +The plugin supplies host CPU and memory usage and the PHP SDK identity for these heartbeats. This option does not control activity heartbeats or the activity heartbeat timeout. + +## Custom Payload Converters + +The plugin collects Go implementations of Temporal's `converter.PayloadConverter` interface from a [custom RoadRunner build](../customization/build.md). Use `data_converters` to select converters by their `Encoding()` value and set their order: + +{% code title=".rr.yaml fragment" %} + +```yaml +temporal: + address: "127.0.0.1:7233" + data_converters: + - "json/custom" +``` + +{% endcode %} + +This example requires a registered converter with the encoding `json/custom`. An unknown encoding fails initialization. If the list is empty or omitted, the plugin uses all registered converters. Set the list when converter order matters. + +The composite converter tries the standard nil, byte, and Protobuf converters first. It then tries the selected custom converters, followed by the standard JSON converter. See the [payload converter example](https://github.com/roadrunner-server/data-converter-example). + ## Example Integrated workflow server provides the ability to create very complex, long-running activities. diff --git a/workflow/worker.md b/workflow/worker.md index 3008c80..b03ff8f 100644 --- a/workflow/worker.md +++ b/workflow/worker.md @@ -39,6 +39,30 @@ $factory->run(); Read more about Temporal configuration and usage on the [official website](https://docs.temporal.io/develop/php/core-application#run-a-dev-worker). +## Dynamic Workflows + +A dynamic workflow handles workflow types that have no named registration in that SDK worker. Register at most one dynamic workflow per worker. Multiple dynamic registrations cause worker initialization to fail. + +Use a PHP SDK version with [dynamic workflow support](https://github.com/temporalio/sdk-php/pull/774). The SDK must set the `dynamic` field to `true` in the workflow registration metadata. This is a worker registration option. + +## Worker Recovery + +After an activity worker exits, the pool replaces that worker without an explicit activity-pool reset. A workflow-worker exit still triggers a full reset and clears the sticky workflow cache so Temporal can replay workflow history. + +## Graceful Shutdown + +RoadRunner stops Temporal task polling before it destroys the PHP worker pools. Set the PHP SDK worker stop timeout to give running task handlers time to finish: + +```php +$worker = $factory->newWorker( + 'taskQueue', + \Temporal\Worker\WorkerOptions::new() + ->withWorkerStopTimeout(new \DateInterval('PT20S')) +); +``` + +The plugin passes this timeout to the Go SDK. Set the RoadRunner [shutdown grace period](../app-server/cli.md#stopping-the-server) to allow time for SDK workers and PHP pools to stop. Activity heartbeats can continue while the plugin waits for running work. + ## Multi-worker environment To serve both HTTP and Temporal from the same worker, use the `getMode()` option of `Environment`: