diff --git a/.editorconfig b/.editorconfig index 7e5f93a2f5..e769ddc280 100644 --- a/.editorconfig +++ b/.editorconfig @@ -25,17 +25,16 @@ max_line_length = 120 ij_wrap_on_typing = true ij_visual_guides = 120 - [*.{java,xml,py}] indent_style = space indent_size = 4 [*.{java,xml}] -# Ignore the IDEA unsupported warning & it works well (indeed) continuation_indent_size = 8 [*.md] -max_line_length = off +max_line_length = unset +ij_visual_guides = 160 ij_wrap_on_typing = false [*.java] diff --git a/.serena/README.md b/.serena/README.md new file mode 100644 index 0000000000..85a533d253 --- /dev/null +++ b/.serena/README.md @@ -0,0 +1,27 @@ +# Serena project context + +This directory is a compact navigation cache for coding agents. It is not an independent documentation set and must not duplicate product guides, option tables or release history. + +## Read order + +1. [`../AGENTS.md`](../AGENTS.md) for repository-wide constraints and validation expectations. +2. [`../docs/README.md`](../docs/README.md) for the authoritative documentation map. +3. The nearest module `AGENTS.md` and README for the code being changed. +4. A memory below only when a quick architecture or file-location map is useful. + +## Maintained memories + +| Memory | Purpose | +|---|---| +| [project_overview.md](memories/project_overview.md) | Stable product scope and authoritative entry points | +| [architecture_and_modules.md](memories/architecture_and_modules.md) | High-level module ownership and runtime relationships | +| [key_file_locations.md](memories/key_file_locations.md) | Search anchors for common work areas | + +Build commands, style rules, test selection and completion checks belong in `AGENTS.md`, module files and executable CI configuration. Keeping them there prevents this cache from drifting. + +## Maintenance policy + +- Store stable relationships and search anchors, not timestamps, onboarding status, pull-request links, commit hashes or copied release numbers. +- Link to the owning file instead of reproducing long procedures. +- Update or remove a memory in the same change when module ownership or a referenced path changes. +- Keep each memory small enough to load on demand; do not turn the cache into a second README tree. diff --git a/.serena/memories/architecture_and_modules.md b/.serena/memories/architecture_and_modules.md index 687ae2251f..92b422e81f 100644 --- a/.serena/memories/architecture_and_modules.md +++ b/.serena/memories/architecture_and_modules.md @@ -1,36 +1,26 @@ -# Architecture and Module Structure - -## Three-Tier Architecture - -### 1. Client Layer -- Gremlin/Cypher queries, REST APIs, Swagger UI - -### 2. Server Layer (hugegraph-server, 8 submodules) -- **REST API** (hugegraph-api): GraphAPI, SchemaAPI, GremlinAPI, CypherAPI, AuthAPI, GraphSpaceAPI (distributed only), ManagerAPI (distributed only) -- **Graph Engine** (hugegraph-core): Schema (with TTL update), traversal, task scheduling, GraphSpace multi-tenancy -- **Backend Interface**: Pluggable via `BackendStore` - -### 3. Storage Layer -- RocksDB (default/embedded), HStore (distributed/production) -- HBase (deprecated; planned for removal in 2.0) - -## Module Structure (7 top-level modules) - -### hugegraph-server (8 submodules) -`hugegraph-core`, `hugegraph-api` (includes `opencypher/`, `space/`), `hugegraph-dist`, `hugegraph-test`, `hugegraph-example`, plus backends: `hugegraph-rocksdb`, `hugegraph-hstore`, `hugegraph-hbase` - -### hugegraph-pd (8 submodules) -Placement Driver: `hg-pd-core`, `hg-pd-service`, `hg-pd-client`, `hg-pd-common`, `hg-pd-grpc`, `hg-pd-cli`, `hg-pd-dist`, `hg-pd-test` - -### hugegraph-store (9 submodules) -Distributed storage + Raft: `hg-store-core`, `hg-store-node`, `hg-store-client`, `hg-store-common`, `hg-store-grpc`, `hg-store-rocksdb`, `hg-store-cli`, `hg-store-dist`, `hg-store-test` - -### Others -- **hugegraph-commons**: Shared utilities, RPC framework -- **hugegraph-struct**: Data structures (must build before PD/Store) -- **install-dist**: Distribution packaging, license files -- **hugegraph-cluster-test**: Cluster integration tests - -## Distributed Deployment (BETA) -PD manages placement and metadata; Store owns partition data; Server serves graph queries. -Development and HA topologies differ; see `docker/README.md` for node counts and configuration. +# Architecture and module ownership + +```text +Clients and ecosystem tools + │ + ▼ +HugeGraph Server ─────────────────► RocksDB (standalone) + │ + ├─ metadata / placement lookup ─► PD ── coordinates ─► HStore nodes + └─ graph data requests ──────────────────────────────► HStore nodes (distributed) +``` + +PD supplies metadata, placement and membership information; Server sends graph data requests directly to the selected HStore nodes. + +## Top-level ownership + +- `hugegraph-server/`: graph engine, schema, tasks, traversers, REST/Gremlin/Cypher APIs and storage adapters. +- `hugegraph-pd/`: placement, metadata, cluster scheduling and PD clients. +- `hugegraph-store/`: partition storage, Store clients and Raft replication. +- `hugegraph-struct/`: shared graph structures, identifiers and serialization used across distributed modules. +- `hugegraph-commons/`: shared utilities and RPC infrastructure. +- `hugegraph-cluster-test/`: cross-component cluster validation. +- `install-dist/`: distribution assembly, dependency inventory and release metadata. +- `helm/hugegraph/` and `docker/`: Kubernetes and container deployment assets. + +Read the nearest module `AGENTS.md` before changing an implementation. For current submodule membership and dependency edges, inspect the root and module POMs rather than maintaining a copied count here. diff --git a/.serena/memories/code_style_and_conventions.md b/.serena/memories/code_style_and_conventions.md deleted file mode 100644 index ce3809cdf4..0000000000 --- a/.serena/memories/code_style_and_conventions.md +++ /dev/null @@ -1,28 +0,0 @@ -# Code Style and Conventions - -## Style Tools -- `.editorconfig` — Primary style definition -- `style/checkstyle.xml` — Enforcement (note: `hugegraph-style.xml` was removed) -- `.licenserc.yaml` + apache-rat-plugin + skywalking-eyes — License header validation - -## Core Rules -- **Line length**: 120 chars -- **Indent**: 4 spaces, continuation 8 spaces -- **Charset**: UTF-8, LF line endings, final newline -- **Imports**: Sorted `$*` → `java` → `javax` → `org` → `com` → `*`, no star imports (threshold 100) -- **Braces**: `if`/`while`/`for` forced if multiline, `do-while` always -- **Blank lines**: Max 1 in code/declarations -- **JavaDoc**: `

` on empty lines, no wrap if one line - -## Naming -- Packages: `org.apache.hugegraph.*` (lowercase dot-separated) -- Classes: PascalCase, Methods/Variables: camelCase, Constants: UPPER_SNAKE_CASE - -## License -- All source files require Apache 2.0 header -- gRPC generated code excluded from checks -- Validate: `mvn apache-rat:check -ntp` + `mvn editorconfig:check` - -## Build -- Java 17 release target, `-Xlint:unchecked`, Lombok 1.18.30 (provided/optional) -- Swagger: `io.swagger.core.v3:swagger-jaxrs2-jakarta` for REST API docs diff --git a/.serena/memories/ecosystem_and_related_projects.md b/.serena/memories/ecosystem_and_related_projects.md deleted file mode 100644 index b5bd9c1eeb..0000000000 --- a/.serena/memories/ecosystem_and_related_projects.md +++ /dev/null @@ -1,24 +0,0 @@ -# HugeGraph Ecosystem and Related Projects - -## This Repo: apache/hugegraph (server, OLTP) - -## Ecosystem -| Repo | Purpose | -|------|---------| -| hugegraph-toolchain | Loader, Hubble (visualization), Tools CLI, Java Client | -| hugegraph-computer | OLAP: PageRank, Connected Components, Shortest Path | -| apache/hugegraph-ai | Graph RAG, KG construction, NL→Gremlin/Cypher | -| hugegraph-doc | Docs & website (hugegraph.apache.org) | - -## Data Flow -``` -Sources → hugegraph-loader → hugegraph-server → Hubble / Computer / AI -``` - -## Integrations -- Big Data: Flink, Spark, HDFS -- Queries: Gremlin (TinkerPop 3.8.1), OpenCypher, REST API + Swagger UI -- Storage: RocksDB (default), HStore (distributed) - -## Versions -Read Server revision from root `pom.xml` and TinkerPop from `hugegraph-server/pom.xml`. diff --git a/.serena/memories/implementation_patterns_and_guidelines.md b/.serena/memories/implementation_patterns_and_guidelines.md deleted file mode 100644 index ce83957f48..0000000000 --- a/.serena/memories/implementation_patterns_and_guidelines.md +++ /dev/null @@ -1,49 +0,0 @@ -# Implementation Patterns and Guidelines - -## Backend Architecture -- Backends implement `BackendStore` interface from `hugegraph-core` -- Each backend = separate Maven module under `hugegraph-server/` -- Configured via `hugegraph.properties` → `backend` property -- **Supported backends**: RocksDB (default/embedded), HStore (distributed), HBase (deprecated; planned for removal in 2.0) - -## GraphSpace Multi-Tenancy -- Core: `hugegraph-core/.../space/` (GraphSpace, SchemaTemplate, Service, register/) -- API: `hugegraph-api/.../api/space/GraphSpaceAPI.java` (includes GS profile endpoints) -- **Standalone mode**: GraphSpaceAPI and ManagerAPI are disabled - -## Auth System -- Disabled by default, enable via `bin/enable-auth.sh` -- ConfigAuthenticator was removed, use standard auth -- Multi-level: Users, Groups, Projects, Targets, Access control -- Location: `hugegraph-api/.../api/auth/` - -## gRPC Protocol -- PD protos: `hugegraph-pd/hg-pd-grpc/src/main/proto/` -- Store protos: `hugegraph-store/hg-store-grpc/src/main/proto/` -- After `.proto` changes: `mvn clean compile`; PD/Store gRPC POMs output to each module's `src/main/java/`. - -## Query Languages -- **Gremlin**: Native TinkerPop 3.8.1 -- **OpenCypher**: `hugegraph-api/opencypher/` -- TinkerPop exceptions are passed through in Gremlin responses - -## Schema -- Labels support TTL with runtime update -- Edge label conflicting conditions are handled safely - -## Testing -- **Profiles**: `unit-test`, `core-test`, `api-test`, `tinkerpop-structure-test`, `tinkerpop-process-test` -- **Backends in CI**: memory, rocksdb, hbase (matrix) -- **Single test class**: `mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,memory -Dtest=ClassName` -- TinkerPop tests: `release-*`/`test-*` and upgrade branches selected in `server-ci.yml` -- Server Raft API tests are branch-gated in `server-ci.yml`; Store raft-core tests run in normal `pd-store-ci.yml` CI. - -## Docker -- Standalone: `docker/docker-compose.yml` (bridge network, Server + Hubble) -- Cluster: `docker/docker-compose-3pd-3store-3server.yml` -- Container logs: stdout-based - -## CI Pipelines -- `server-ci.yml`: compile + unit/core/API tests (memory/rocksdb/hbase × Java 17) -- `rerun-ci.yml`: auto-rerun flaky failures (max 2 reruns, 180s delay) -- `auto-pr-review.yml`: auto-comment on new PRs diff --git a/.serena/memories/key_file_locations.md b/.serena/memories/key_file_locations.md index bde5926a3c..32dee92d5c 100644 --- a/.serena/memories/key_file_locations.md +++ b/.serena/memories/key_file_locations.md @@ -1,44 +1,21 @@ -# Key File and Directory Locations +# Key file locations -## Build & Config -- `pom.xml` — Root multi-module POM -- `.editorconfig` — Code style rules -- `style/checkstyle.xml` — Checkstyle enforcement (hugegraph-style.xml was removed) -- `.licenserc.yaml` — License checker config +Use these paths as search anchors; follow the nearest README or `AGENTS.md` for local rules. -## Server (hugegraph-server) -- Core engine: `hugegraph-core/src/main/java/org/apache/hugegraph/` → `backend/`, `schema/`, `traversal/`, `task/` -- GraphSpace: `hugegraph-core/.../space/` → `GraphSpace.java`, `SchemaTemplate.java`, `Service.java`, `register/` -- REST APIs: `hugegraph-api/src/main/java/org/apache/hugegraph/api/` → `graph/`, `schema/`, `gremlin/`, `cypher/`, `auth/`, `space/` (GraphSpaceAPI), `metrics/`, `arthas/` -- OpenCypher: `hugegraph-api/.../opencypher/` -- Backend interface: `hugegraph-core/.../backend/store/BackendStore.java` -- Distribution: `hugegraph-dist/src/assembly/static/` → `bin/`, `conf/`, `lib/`, `logs/` -- Tests: `hugegraph-test/src/main/java/.../` → `unit/`, `core/`, `api/`, `tinkerpop/` +| Work area | Start here | +|---|---| +| Repository modules and versions | `pom.xml` | +| Build, contribution and documentation map | `docs/README.md`, `docs/BUILDING.md`, `docs/CONTRIBUTING.md` | +| Graph engine, schema, traversal and tasks | `hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/` | +| REST, Gremlin, Cypher and authentication APIs | `hugegraph-server/hugegraph-api/src/main/java/org/apache/hugegraph/api/` | +| Server distribution and runtime configuration | `hugegraph-server/hugegraph-dist/` | +| Server tests | `hugegraph-server/hugegraph-test/` | +| PD implementation and protocol | `hugegraph-pd/`, especially `hg-pd-core/`, `hg-pd-service/` and `hg-pd-grpc/` | +| Store implementation and protocol | `hugegraph-store/`, especially `hg-store-core/`, `hg-store-node/` and `hg-store-grpc/` | +| Shared graph structures | `hugegraph-struct/` | +| CI workflows and policy | `.github/workflows/`, `.github/scripts/`, `docs/ci.md` | +| Dependency and release metadata | `install-dist/release-docs/`, `install-dist/scripts/dependency/` | +| Docker and Compose | `docker/README.md`, `docker/` | +| Kubernetes and Helm | `helm/hugegraph/README.md`, `helm/hugegraph/docs/README.md`, `helm/hugegraph/values.yaml` | -## Docker -- `docker/docker-compose.yml` — Standalone Server + Hubble (bridge network) -- `docker/docker-compose-3pd-3store-3server.yml` — 3-node cluster -- `docker/docker-compose.dev.yml` — Dev mode - -## PD Module -- Proto: `hugegraph-pd/hg-pd-grpc/src/main/proto/` -- Dist: `hugegraph-pd/hg-pd-dist/src/assembly/static/` - -## Store Module -- Proto: `hugegraph-store/hg-store-grpc/src/main/proto/` -- Dist: `hugegraph-store/hg-store-dist/src/assembly/static/` - -## CI Workflows (.github/workflows/) -- `server-ci.yml` — Server tests (matrix: memory/rocksdb/hbase × Java 17) -- `pd-store-ci.yml` — PD, Store & HStore tests -- `commons-ci.yml` — Commons tests -- `cluster-test-ci.yml` — Cluster integration -- `licence-checker.yml` — License headers -- `rerun-ci.yml` — Auto-rerun for flaky workflows -- `auto-pr-review.yml` — Auto-comment on new PRs -- `check-dependencies.yml` — Dependency checks -- `codeql-analysis.yml` — CodeQL security scanning -- `stale.yml` — Stale issue/PR cleanup - -## Docs -- `README.md`, `docs/BUILDING.md`, `docs/CONTRIBUTING.md`, `AGENTS.md`, `CLAUDE.md` +Search the repository before assuming a path or configuration name is still current. This memory intentionally omits workflow filenames and submodule counts that change frequently. diff --git a/.serena/memories/project_overview.md b/.serena/memories/project_overview.md index deac1e6821..8a60b3cc93 100644 --- a/.serena/memories/project_overview.md +++ b/.serena/memories/project_overview.md @@ -1,26 +1,19 @@ -# Apache HugeGraph Project Overview +# Project overview -## Project Purpose -Apache HugeGraph is a fast-speed, highly-scalable graph database supporting 10+ billion vertices/edges for OLTP workloads. Graduated from Apache Incubator (incubating branding removed). +Apache HugeGraph is a graph database with standalone and distributed deployment modes. It provides a property-graph data model, schema and indexes, graph traversals, REST/Gremlin/Cypher access, authentication and operational tooling. -## Key Capabilities -- Apache TinkerPop 3 compliant graph database -- Gremlin + OpenCypher query languages -- Schema metadata management (VertexLabel, EdgeLabel, PropertyKey, IndexLabel) with TTL update support -- Multi-type indexes (exact, range, complex conditions) -- Pluggable backend storage (RocksDB default, HStore distributed) -- GraphSpace multi-tenancy (standalone mode disables GraphSpaceAPI/ManagerAPI) -- Swagger UI for REST API documentation -- Integration with Flink/Spark/HDFS +## Deployment modes -## Technology Stack -- **Language**: Java 17 (required; currently supported release) -- **Build**: Maven 3.6.3+ -- **Graph Framework**: Apache TinkerPop 3.8.1 -- **RPC**: gRPC + Protocol Buffers -- **API Docs**: Swagger (io.swagger.core.v3) -- **Storage**: RocksDB (default/embedded), HStore (distributed/production), HBase (deprecated; planned for removal in 2.0) +- **Standalone:** HugeGraph Server uses embedded RocksDB by default. The HBase backend remains supported but is deprecated. RocksDB is the shortest path for development and smaller deployments. +- **Distributed:** Server handles graph APIs, PD owns placement and metadata coordination, and HStore nodes own partition data and replication. -## Version -- Read `${revision}` from the root `pom.xml`; Maven flatten resolves it for published artifacts. -- License: Apache License 2.0 +## Authoritative entry points + +- Product and deployment overview: [`../../README.md`](../../README.md) +- Repository documentation map: [`../../docs/README.md`](../../docs/README.md) +- Build and contribution rules: [`../../docs/BUILDING.md`](../../docs/BUILDING.md), [`../../docs/CONTRIBUTING.md`](../../docs/CONTRIBUTING.md) +- Kubernetes deployment: [`../../helm/hugegraph/README.md`](../../helm/hugegraph/README.md) +- Repository agent guidance: [`../../AGENTS.md`](../../AGENTS.md) +- Current versions and module membership: [`../../pom.xml`](../../pom.xml) + +Do not copy release versions or detailed feature lists into this memory. Read the owning source when exact current behavior matters. diff --git a/.serena/memories/suggested_commands.md b/.serena/memories/suggested_commands.md deleted file mode 100644 index c1aa4d8e5b..0000000000 --- a/.serena/memories/suggested_commands.md +++ /dev/null @@ -1,59 +0,0 @@ -# Suggested Development Commands - -## Build -```bash -mvn clean install -DskipTests # Full build -mvn clean install -pl hugegraph-server -am -DskipTests # Server only -mvn clean compile -U -Dmaven.javadoc.skip=true -ntp # Compile only -mvn clean package -DskipTests # Aggregate distribution → root target/ -``` - -## Test -```bash -# Server tests (memory/rocksdb/hbase backends) -mvn test -pl hugegraph-server/hugegraph-test -am -P unit-test -mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,memory -mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,rocksdb -mvn test -pl hugegraph-server/hugegraph-test -am -P api-test,rocksdb - -# Single test class -mvn test -pl hugegraph-server/hugegraph-test -am -P core-test,memory -Dtest=YourTestClass - -# TinkerPop compliance (can be run locally on any branch) -mvn test -pl hugegraph-server/hugegraph-test -am -P tinkerpop-structure-test,memory - -# PD/Store (build struct first) -mvn install -pl hugegraph-struct -am -DskipTests -mvn test -pl hugegraph-pd/hg-pd-test -am -mvn test -pl hugegraph-store/hg-store-test -am -``` - -## Validation -```bash -mvn apache-rat:check -ntp # License headers -mvn editorconfig:check # Code style (.editorconfig) -mvn checkstyle:check # Code style (style/checkstyle.xml) -mvn clean compile -Dmaven.javadoc.skip=true # Compile warnings -``` - -## Server Ops -Scripts in `hugegraph-server/hugegraph-dist/src/assembly/static/bin/` (or extracted distribution `bin/`): -```bash -bin/init-store.sh && bin/start-hugegraph.sh # Init + start -bin/stop-hugegraph.sh # Stop -bin/enable-auth.sh # Enable auth -``` - -## Docker -Create the required authentication environment first; follow `docker/README.md`. -```bash -cd docker && docker compose up -d # Standalone Server + Hubble (bridge network) -cd docker && docker compose -f docker-compose-3pd-3store-3server.yml up -d # Cluster -``` - -## Distributed Build (BETA) -```bash -mvn install -pl hugegraph-struct -am -DskipTests # 1. Struct first -mvn clean package -pl hugegraph-pd -am -DskipTests # 2. PD -mvn clean package -pl hugegraph-store -am -DskipTests # 3. Store -``` diff --git a/.serena/memories/task_completion_checklist.md b/.serena/memories/task_completion_checklist.md deleted file mode 100644 index 58a32fb0e4..0000000000 --- a/.serena/memories/task_completion_checklist.md +++ /dev/null @@ -1,37 +0,0 @@ -# Task Completion Checklist - -## 1. Validation scope -Documentation-only changes: check links, paths and `git diff --check`. -For code changes, select relevant checks below and follow root `AGENTS.md` before pushing. -```bash -mvn apache-rat:check -ntp # License headers -mvn editorconfig:check # Style (.editorconfig) -mvn checkstyle:check # Style (style/checkstyle.xml) -mvn clean compile -Dmaven.javadoc.skip=true # Compile warnings -``` - -## 2. Testing -- Choose backend: `memory` (fast), `rocksdb` (realistic), `hbase` (deprecated compatibility) -- Single-test selection: check the module test POM and confirm the requested test actually ran. -- Bug fix → existing tests; New feature → write tests; Refactor → affected module tests - -## 3. Dependencies (if adding new) -1. License file → `install-dist/release-docs/licenses/` -2. Declare in `install-dist/release-docs/LICENSE` -3. Append NOTICE → `install-dist/release-docs/NOTICE` -4. Run `./install-dist/scripts/dependency/regenerate_known_dependencies.sh` - -## 4. CI Awareness -- `server-ci.yml`: memory/rocksdb/hbase × Java 17 -- `rerun-ci.yml`: auto-retries flaky failures -- `licence-checker.yml`: header validation -- Server Raft API tests are branch-gated; Store raft-core and core tests run in normal PD/Store CI. -- TinkerPop tests: only `release-*`/`test-*` branches - -## 5. Documentation -- Follow root `AGENTS.md`: user-visible feature/configuration/deployment changes ship with matching docs. - -## 6. Commit -- NEVER commit unless explicitly asked -- Format: `feat|fix|refactor(module): msg` -- Include issue ID if available diff --git a/AGENTS.md b/AGENTS.md index e7768ad5eb..3bf6deba08 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,62 +1,58 @@ # AGENTS.md -Repository-specific guidance; module files add local details. +Repository-specific guidance for coding agents. Module files add local ownership, commands and failure modes. -## Key relationships +## System map ```text -Server (hugegraph-server): graph engine + REST/Gremlin API - ├─ RocksDB: embedded backend - └─ HStore adapter → Store client → Store: partition data + Raft +Server: graph engine and user APIs + ├─ RocksDB: embedded standalone backend + └─ HStore adapter → Store client → Store partition data and Raft -PD: placement + metadata; consulted by Server and Store -Struct: shared types/codecs; separate implementations also exist in Server -Commons: shared utilities; independent compiler and test settings +PD: placement and metadata coordination +Struct: shared graph types and codecs +Commons: shared utilities and RPC infrastructure ``` ## Load on demand -| Work area | Read when working there | +| Work area | Read first | |---|---| -| Graph engine, API, backends | [Server](hugegraph-server/AGENTS.md) | -| Placement, metadata, PD client | [PD](hugegraph-pd/AGENTS.md) | -| Distributed storage and client | [Store](hugegraph-store/AGENTS.md) | -| Shared data types and serialization | [Struct](hugegraph-struct/AGENTS.md) | -| Utilities and RPC | [Commons](hugegraph-commons/AGENTS.md) | -| Container deployment | [Docker guide](docker/README.md) | -| PR requirements | [Contribution guide](docs/CONTRIBUTING.md) | +| Documentation ownership and durable links | [Repository docs index](docs/README.md) | +| Graph engine, API and backends | [Server guidance](hugegraph-server/AGENTS.md) | +| Placement, metadata and PD clients | [PD guidance](hugegraph-pd/AGENTS.md) | +| Distributed storage and Store clients | [Store guidance](hugegraph-store/AGENTS.md) | +| Shared graph types and serialization | [Struct guidance](hugegraph-struct/AGENTS.md) | +| Utilities and RPC | [Commons guidance](hugegraph-commons/AGENTS.md) | +| Docker deployment | [Docker guide](docker/README.md) | +| Kubernetes deployment | [Helm guide](helm/hugegraph/README.md) | +| Contribution requirements | [Contribution guide](docs/CONTRIBUTING.md) | ## Repository constraints -- Use existing module boundaries; keep unrelated refactors out of a fix. -- User-visible feature, configuration or deployment behavior changes must ship with matching docs. - Update in-repository docs in the same PR; link a paired `apache/hugegraph-doc` PR when website - docs are affected and coordinate both merges. A follow-up issue alone does not satisfy this rule. - Internal-only changes can use `Doc - No Need`. -- Keep README as an entry point; link detailed deployment instructions instead of duplicating them. -- Keep AGENTS.md under 100 lines where practical. Shared rules belong here, local exceptions - in module files. Keep key relationships, common commands and non-obvious pitfalls inline; - reference existing sources for versions, configuration details and lengthy procedures. +- Keep changes inside the module that owns the behavior; exclude unrelated cleanup from a functional fix. +- A user-visible feature, configuration or deployment change requires matching documentation in the same change. Link a coordinated `apache/hugegraph-doc` change when the website is affected; a follow-up issue alone is not sufficient. +- Keep root and module READMEs as entry points. Put detailed procedures in a focused document and link it from the relevant index. +- Treat `docs/README.md` as the repository documentation map. New durable Markdown files must be linked from it or from a module-local index. +- Treat `.serena/` as a small navigation cache. Do not store duplicated procedures, fixed release numbers, timestamps, review history, pull-request links or commit hashes there. +- Write documentation for the current contract and operational boundary, not as a development diary. Keep prose paragraphs as logical lines; wrap near 160 characters only when necessary for review. ## Build and validation -Commands run from the repository root. Java 17 and Maven 3.6.3+; -versions come from [pom.xml](pom.xml), including `${revision}`. +Run commands from the repository root. Java and Maven requirements are defined by [pom.xml](pom.xml) and summarized in [BUILDING.md](docs/BUILDING.md). ```bash mvn clean install -DskipTests mvn clean install -pl hugegraph-server -am -DskipTests ``` -For separate distributed-module builds, install `hugegraph-struct` first, -then build PD, Store and Server. Module guidance covers tests and CI prerequisites. -Commons tests require `-DskipCommonsTests=false`; a successful build does not imply its tests ran. - -- Java style: 120 columns, 4 spaces, no star imports; see [.editorconfig](.editorconfig). -- Before pushing code: `mvn editorconfig:format`, - `mvn clean compile -Dmaven.javadoc.skip=true`, and relevant module tests. - Documentation-only changes need link/path checks and `git diff --check`. -- PD/Store `.proto` builds generate Java into each `hg-*-grpc/src/main/java/` directory; - see the corresponding gRPC module POM before editing or cleaning generated sources. -- New dependencies require updating [release LICENSE/NOTICE/licenses](install-dist/release-docs/) - and [dependency inventory](install-dist/scripts/dependency/known-dependencies.txt). +For separate distributed-module builds, install `hugegraph-struct` first, then build PD, Store and Server. Commons tests require `-DskipCommonsTests=false`; a successful aggregate build does not imply those tests ran. + +Before pushing code, run `mvn editorconfig:format`, `mvn clean compile -Dmaven.javadoc.skip=true` and the tests selected by the affected modules. Documentation-only changes require relative-link review, rendered-heading review and `git diff --check`. + +## Non-obvious boundaries + +- Server changes commonly require both Server and PD/Store CI because the HStore path crosses module boundaries. +- PD and Store protocol builds generate Java into their gRPC modules; inspect the module POM before editing or deleting generated sources. +- New dependencies require updates to `install-dist/release-docs/` and `install-dist/scripts/dependency/known-dependencies.txt`. +- Read versions, plugin settings and module membership from Maven configuration instead of copying them into agent notes. diff --git a/README.md b/README.md index 2237a5e2da..dc49bc4823 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ --- -**Quick Navigation:** [Architecture](#architecture) • [Quick Start](#quick-start) • [Module Map](#module-map) • [Ecosystem](#ecosystem) • [For Contributors](#contributing) • [Community](#community) +**Quick Navigation:** [Architecture](#architecture) • [Quick Start](#quick-start) • [Module Map](#module-map) • [Documentation](#documentation) • [Ecosystem](#ecosystem) • [For Contributors](#contributing) • [Community](#community) --- @@ -33,10 +33,56 @@ achieved through the powerful [Gremlin](https://tinkerpop.apache.org/gremlin.htm - **Schema Metadata Management**: VertexLabel, EdgeLabel, PropertyKey, and IndexLabel - **Multi-type Indexes**: Exact query, range query, and complex conditions combination query - **Plug-in Backend Store Framework**: RocksDB powers standalone deployments and HStore powers distributed clusters. See the [backend evolution guide](hugegraph-server/README.md#backend-evolution-and-compatibility) for compatibility details. +- **Backend Compatibility**: HBase is deprecated but remains supported; see the [backend evolution guide](hugegraph-server/README.md#backend-evolution-and-compatibility). - **Big Data Integration**: Seamless integration with `Flink`/`Spark`/`HDFS` - **Complete Graph Ecosystem**: In/out-memory Graph Computing + Graph Visualization & Tools + Graph Learning & AI - **Dual Query Language Support**: [Gremlin](https://tinkerpop.apache.org/gremlin.html) (via [Apache TinkerPop 3](https://tinkerpop.apache.org/)) and [Cypher](https://en.wikipedia.org/wiki/Cypher_(query_language)) (OpenCypher) +## Quick Start + +### 5 Minutes Quick Start + +Start a standalone server for local evaluation, bind it to the loopback interface and wait until a Gremlin query succeeds before continuing: + +```bash +( + docker run -d --name hugegraph \ + -p 127.0.0.1:8080:8080 hugegraph/hugegraph || exit 1 + + for attempt in $(seq 1 60); do + if curl -fsS --max-time 2 -o /dev/null \ + -X POST http://127.0.0.1:8080/gremlin \ + -H 'Content-Type: application/json' \ + -d '{"gremlin":"g.V().limit(1)"}'; then + exit 0 + fi + if [ "${attempt}" -eq 60 ]; then + printf 'Server query did not become ready; inspect the logs before continuing.\n' >&2 + docker logs hugegraph + exit 1 + fi + sleep 2 + done +) +``` + +After the readiness query succeeds, run a small graph query: + +```bash +curl -X POST http://127.0.0.1:8080/gremlin \ + -H 'Content-Type: application/json' \ + -d '{"gremlin":"g.V().limit(5)"}' +``` + +A timeout deliberately leaves the named container available for diagnosis. Inspect `docker logs hugegraph` and preserve any required data or attached volumes first. When it is safe to discard the failed container, stop and remove it before retrying: + +```bash +docker stop hugegraph 2>/dev/null || true +docker rm hugegraph +``` + +The loopback bind keeps this evaluation endpoint off ordinary remote interfaces while authentication is not enabled. Users of older Docker or unusual host-network setups should also restrict host firewall access explicitly. Pin a released image for repeatable deployments and enable authentication before exposing the service outside a trusted network. See the [server deployment guide](https://hugegraph.apache.org/docs/quickstart/hugegraph-server/) for packages, configuration and production preparation. + ## Ecosystem Complete **HugeGraph** ecosystem components: @@ -51,23 +97,64 @@ Complete **HugeGraph** ecosystem components: 3. **[hugegraph-ai](https://github.com/apache/hugegraph-ai)** - **Graph AI/LLM/Knowledge Graph** integration -4. **[hugegraph-website](https://github.com/apache/hugegraph-doc)** - **Documentation & website** repository +4. **[hugegraph-doc](https://github.com/apache/hugegraph-doc)** - **Documentation & website** repository ## Architecture HugeGraph supports both **standalone** and **distributed** deployments: +```mermaid +flowchart TB + CLIENTS["Client Layer
Gremlin Console · REST Client · Cypher Client · SDK/Tools"] + + subgraph Server["HugeGraph Server :8080"] + API[REST API
Jersey 3] + GS[Gremlin Server
TinkerPop] + CS[Cypher Engine
OpenCypher] + CORE[Graph Engine
hugegraph-core] + + API --> CORE + GS --> CORE + CS --> CORE + end + + subgraph Storage["Storage Layer"] + subgraph Standalone["Standalone Mode"] + ROCKS[(RocksDB
Embedded)] + end + + subgraph Distributed["Distributed Mode"] + PD[HugeGraph PD
Metadata and placement] + HSTORE[(HStore
Partitioned storage)] + end + end + + CLIENTS --> Server + CORE --> ROCKS + CORE -.->|metadata and placement| PD + CORE -->|graph data requests| HSTORE + PD -.->|coordinates| HSTORE + + style Server fill:#e1f5ff + style Distributed fill:#fff4e1 + style Standalone fill:#f0f0f0 +``` + + +

+Text architecture diagram + ``` ┌─────────────────────────────────────────────────────┐ │ Client Layer │ - │ Gremlin Console │ REST API │ Cypher │ SDK/Tools │ + │ Gremlin Console │ REST Client │ Cypher │ SDK/Tools │ └─────────────────────────┬───────────────────────────┘ │ ┌─────────────────────────▼───────────────────────────┐ │ HugeGraph Server (:8080) │ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │ │ REST API │ │ Gremlin │ │ Cypher Engine │ │ - │ │(Jersey 3)│ │(TP 3.8.1)│ │ (OpenCypher) │ │ + │ │(Jersey 3)│ │TinkerPop │ │ (OpenCypher) │ │ │ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │ │ └─────────────┼─────────────────┘ │ │ ┌────────▼────────┐ │ @@ -85,30 +172,36 @@ HugeGraph supports both **standalone** and **distributed** deployments: │ │ (embedded) │ │ │ │ (Raft, 3-5 nodes) │ │ │ └───────────────────────┘ │ │ │ :8620/:8686 │ │ │ │ │ └───────────┬───────────┘ │ - │ Use Case: │ │ │ │ + │ Use Case: │ │ coordinates │ │ │ Development/Testing │ │ ┌───────────▼───────────┐ │ │ Single Node │ │ │ HStore │ │ │ │ │ │ (Raft, 3+ nodes) │ │ - │ Data Scale: < 1TB │ │ │ :8520 │ │ + │ Single-node storage │ │ │ :8520 │ │ └─────────────────────────────┘ │ └───────────────────────┘ │ │ │ │ Use Case: │ │ Production/HA/Cluster │ │ │ - │ Data Scale: < 1000 TB │ + │ Horizontal scaling │ └─────────────────────────────┘ ``` +
+ +In distributed mode, Server asks PD for metadata and placement, then sends graph data requests directly to HStore; PD coordinates the storage nodes and does not forward graph data. + See the [backend evolution guide](hugegraph-server/README.md#backend-evolution-and-compatibility) for lifecycle and historical compatibility guidance. To use ToplingDB with the RocksDB backend, follow the [three-step switching guide](docs/storage-lifecycle.md#storage-provider-selection-and-toplingdb). ### Deployment Mode Comparison -| Mode | Components | Use Case | Data Scale | High Availability | -|------|------------|----------|------------|-------------------| -| **Standalone** | Server + RocksDB | Development, Testing, Single Node | < 1TB | Basic | -| **Distributed** | Server + PD + HStore | Production, HA, Horizontal Scaling | < 1000 TB | Yes | +| Mode | Components | Use Case | Scaling | High Availability | +|------|------------|----------|---------|-------------------| +| **Standalone** | Server + RocksDB | Development, Testing, Single Node | Single node | Basic | +| **Distributed** | Server + PD + HStore | Production, HA, Horizontal Scaling | Horizontal scaling | Yes | + +Actual capacity depends on hardware, data and workload. ### Module Overview @@ -122,66 +215,12 @@ To use ToplingDB with the RocksDB backend, follow the [three-step switching guid See the [1.8.0 shared-foundation migration guide](docs/shared-foundation-migration.md) for Java API changes, module ownership and coordinated upgrade requirements. -
-📊 Click to view detailed architecture diagram (Mermaid) - -```mermaid -flowchart TB - CLIENTS["Client Layer
Gremlin Console · REST Client · Cypher Client · SDK/Tools"] - - subgraph Server["HugeGraph Server :8080"] - API[REST API
Jersey 3] - GS[Gremlin Server
TinkerPop 3.8.1] - CS[Cypher Engine
OpenCypher] - CORE[Graph Engine
hugegraph-core] - - API --> CORE - GS --> CORE - CS --> CORE - end - - subgraph Storage["Storage Layer"] - subgraph Standalone["Standalone Mode"] - ROCKS[(RocksDB
Embedded)] - end - - subgraph Distributed["Distributed Mode"] - PD_HSTORE[PD + HStore
Raft Cluster] - end - end - - CLIENTS --> Server - CORE --> ROCKS - CORE --> PD_HSTORE - - style Server fill:#e1f5ff - style Distributed fill:#fff4e1 - style Standalone fill:#f0f0f0 -``` -
-## Quick Start +## Installation -### 5 Minutes Quick Start - -```bash -# Start HugeGraph (standalone mode) -docker run -itd --name=hugegraph -p 8080:8080 hugegraph/hugegraph:1.7.0 - -# Verify server is running -curl http://localhost:8080/versions - -# Try a Gremlin query -curl -X POST http://localhost:8080/gremlin \ - -H "Content-Type: application/json" \ - -d '{"gremlin":"g.V().limit(5)"}' -``` - -> **Production Note**: For production environments or public network exposure, you **must** enable the [AuthSystem](https://hugegraph.apache.org/docs/config/config-authentication/) for security. - ### Prerequisites - **Java 17** (required and currently the only supported Java release) @@ -196,17 +235,17 @@ for client configuration and query compatibility checks. ### Option 1: Docker (Fastest) -Docker is the quickest way to get started for **testing or development**: +Docker is the quickest way to get started for **testing or development**. Each command below is an alternative; remove or rename an existing `hugegraph` container before trying another. The loopback port binding is for local use: ```bash # Basic usage -docker run -itd --name=hugegraph -p 8080:8080 hugegraph/hugegraph:1.7.0 +docker run -itd --name=hugegraph -p 127.0.0.1:8080:8080 hugegraph/hugegraph:1.7.0 # With sample graph preloaded -docker run -itd --name=hugegraph -e PRELOAD=true -p 8080:8080 hugegraph/hugegraph:1.7.0 +docker run -itd --name=hugegraph -e PRELOAD=true -p 127.0.0.1:8080:8080 hugegraph/hugegraph:1.7.0 # With authentication enabled -docker run -itd --name=hugegraph -e PASSWORD=your_password -p 8080:8080 hugegraph/hugegraph:1.7.0 +docker run -itd --name=hugegraph -e PASSWORD=your_password -p 127.0.0.1:8080:8080 hugegraph/hugegraph:1.7.0 ``` For advanced Docker configurations, see: @@ -233,7 +272,7 @@ upgrade guidance.
Option 3: Download Binary Package -Download pre-built packages from the [Download Page](https://hugegraph.apache.org/docs/download/download/): +Download pre-built packages from the [Download Page](https://hugegraph.apache.org/docs/download/download/). Replace `{version}` below with the version of the package you choose: ```bash # Download and extract @@ -276,9 +315,10 @@ cd hugegraph # Build all modules (skip tests for faster build) mvn clean package -DskipTests -# Extract built package -tar -xzf target/apache-hugegraph-{version}.tar.gz -cd apache-hugegraph-{version}/apache-hugegraph-server-{version} +# Replace VERSION with the project version from pom.xml before extracting +VERSION="" +tar -xzf "hugegraph-server/apache-hugegraph-server-${VERSION}.tar.gz" +cd "apache-hugegraph-server-${VERSION}" # Initialize and start bin/init-store.sh @@ -298,20 +338,15 @@ Once the server is running, verify the installation: # Check server version curl http://localhost:8080/versions -# Expected output: -# { -# "versions": { -# "version": "v1", -# "core": "1.7.0", -# "gremlin": "3.8.1", -# "api": "1.7.0" -# } -# } +# The response reports the installed server and Gremlin versions. # Try Gremlin console (if installed locally) bin/gremlin-console.sh +``` + +In the Gremlin console: -# In Gremlin console: +```text gremlin> :remote connect tinkerpop.server conf/remote.yaml gremlin> :> g.V().limit(5) ``` @@ -324,6 +359,9 @@ Read the [storage lifecycle and recovery guide](docs/storage-lifecycle.md) befor ## Module Map +
+Developer navigation + **Developer Navigation**: Find the right module for your task | I want to... | Module | Key Path | @@ -341,6 +379,8 @@ Read the [storage lifecycle and recovery guide](docs/storage-lifecycle.md) befor For repository constraints and module guidance, see [AGENTS.md](AGENTS.md). +
+
For Contributors @@ -378,6 +418,12 @@ For repository constraints and module guidance, see [AGENTS.md](AGENTS.md).
+## Documentation + +- [Repository documentation index](docs/README.md): build, API, runtime, storage and internal design guides. +- [Official HugeGraph documentation](https://hugegraph.apache.org/docs/): user guides and released-product instructions. +- [Helm documentation index](helm/hugegraph/docs/README.md): Kubernetes installation, operations and optional components. + ## Contributing Welcome to contribute to HugeGraph! @@ -411,4 +457,4 @@ HugeGraph is licensed under [Apache 2.0 License](LICENSE). HugeGraph relies on the [Apache TinkerPop](http://tinkerpop.apache.org) framework. We are grateful to the TinkerPop community, Titan, and DataStax for their foundational work. Thanks to all contributors and organizations who have helped make HugeGraph possible. -You are welcome to contribute to HugeGraph, and we look forward to working with you to build an excellent open-source community. +You are welcome to contribute to HugeGraph, and we look forward to working with you to build an excellent open-source community. \ No newline at end of file diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000000..a00c93e9d4 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,55 @@ +# Repository documentation + +This directory contains source-specific documentation for Apache HugeGraph. It complements the [official website documentation](https://hugegraph.apache.org/docs/) and should not duplicate complete user guides that already live there. + +## Find the right document + +| Task | Document | +|---|---| +| Understand the repository and deployment choices | [Root README](../README.md) | +| Build the project from source | [BUILDING.md](BUILDING.md) | +| Prepare and validate a contribution | [CONTRIBUTING.md](CONTRIBUTING.md) | +| Understand CI selection and required checks | [ci.md](ci.md) | +| Call the REST API | [rest-api-usage.md](rest-api-usage.md) | +| Use the Cypher endpoint | [cypher-api.md](cypher-api.md) | +| Use the same-neighbors batch API | [same-neighbors-batch.md](same-neighbors-batch.md) | +| Configure clients or diagnose Server runtime behavior | [server-runtime.md](server-runtime.md) | +| Select, upgrade or recover a storage backend | [storage-lifecycle.md](storage-lifecycle.md) | +| Understand shared module ownership and migration boundaries | [shared-foundation-migration.md](shared-foundation-migration.md) | +| Deploy or operate Kubernetes resources | [Helm README](../helm/hugegraph/README.md) and [Helm docs index](../helm/hugegraph/docs/README.md) | +| Run Docker or Compose environments | [Docker README](../docker/README.md) | +| Navigate the repository with Serena | [Serena project context](../.serena/README.md) | + +Module-specific implementation details belong beside the module they describe. Contributor and agent navigation belongs in the root or module `AGENTS.md`; short tool-specific caches under `.serena/` must link back to these authoritative files. + +## Source-of-truth hierarchy + +Use this order when information overlaps: + +1. Executable configuration and source code define current behavior. +2. The closest module README documents module usage and local constraints. +3. This directory documents cross-module build, API, runtime and design topics. +4. The website repository owns long-form user tutorials and release-oriented guides. +5. `AGENTS.md` and `.serena/` summarize navigation for development tools; they must not become independent product documentation. + +A secondary document should link to the canonical source instead of copying version numbers, option tables or procedures that will drift. + +## Writing durable documentation + +- Describe the current contract, supported workflow and operational boundary. Do not record implementation chronology or review discussions. +- Do not embed pull-request links, commit hashes or temporary branch names as durable explanations. Link to a stable file, configuration key, public issue or website section only when it remains useful after the change is merged. +- Give every durable document a clear owner and add it to this index or a module-local index. Avoid isolated Markdown files that are discoverable only through repository search. +- Use task-oriented headings and explicit terms such as component names, configuration keys, file paths and expected results. This improves both human navigation and LLM retrieval. +- Keep prose paragraphs as logical lines. Do not hard-wrap every sentence; wrap near 160 characters only when a long source line becomes difficult to review. Preserve intentional line breaks in lists, tables, code and quoted output. +- Prefer one concise example that users can run over multiple nearly identical examples. Move uncommon recovery procedures or compatibility details into a focused document and link to it. +- State prerequisites, safety implications and verification steps near the operation they qualify. + +## Adding or changing a document + +Before merging a documentation change: + +1. Confirm that an existing document is not already the canonical owner of the topic. +2. Link the new or moved page from this index, a module README or another obvious entry point. +3. Remove duplicated or obsolete text rather than leaving two competing versions. +4. Check relative links, headings, code blocks and commands from the directory where users will run them. +5. Update matching website documentation when public installation, configuration or behavior changes. diff --git a/helm/hugegraph/README.md b/helm/hugegraph/README.md index ec1e2420d6..4e66812fd1 100644 --- a/helm/hugegraph/README.md +++ b/helm/hugegraph/README.md @@ -9,9 +9,13 @@ are not supported. Provide a default StorageClass or prepare matching PVs. The default installation needs **7 volumes**: 3 × 10 GiB for PD, 3 × 50 GiB for Store, and 1 × 1 GiB for Hubble. The chart creates PVCs; it does not create a storage provisioner. -SeaTunnel and Vermeer are **disabled by default**, including in the single-node +SeaTunnel, Vermeer and RAG are **disabled by default**, including in the single-node and cluster presets. Optional components add resources only when enabled. +For a map of the chart guides, see the [documentation index](docs/README.md). +For complete configuration options, see [values.yaml](values.yaml) and the +[deployment examples](examples/README.md). + ## Install a published Chart For a quick trial, install the latest available Chart from GHCR: @@ -56,10 +60,12 @@ kubectl -n hugegraph get secret hugegraph-admin \ -o jsonpath='{.data.password}' | base64 --decode ``` -Open **`http://:`** and log in as `admin` with that -password. Server API uses `http://:`. -HTTP works directly; a hostname, certificate or extra acknowledgement is not -required. PD remains internal. +On a trusted network, open **`http://:`** and log in +as `admin` with that password. Server API uses +`http://:`. These HTTP connections are unencrypted; +use local port-forwarding or a trusted network for evaluation. Before allowing +access from an untrusted network, use an HTTPS reverse proxy or Ingress and +restrict access appropriately. PD remains internal. If node access is unavailable, use port-forward: @@ -84,11 +90,14 @@ Hubble is optional. Current Hubble login requires Server authentication. helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ --set hubble.enabled=false -# Also disable API authentication, if wanted +# For local or trusted-network testing only, also disable API authentication # Add: --set server.auth.enabled=false ``` -Inactive credential values may remain in the values file when auth is disabled. +The default easy mode exposes HTTP services through NodePort. Disable Server +authentication only for local or trusted-network testing; retain it in other +environments. Inactive credential values may remain in the values file when +authentication is disabled. ### Optional components @@ -97,6 +106,7 @@ Inactive credential values may remain in the values file when auth is disabled. | Hubble | Enabled | `hubble.enabled=false` selects API-only deployment | | SeaTunnel | Disabled | `seatunnel.enabled=true` adds data import/export and its native UI | | Vermeer | Disabled | `vermeer.enabled=true` adds graph computation, Hubble integration and its native UI | +| RAG | Disabled | `rag.enabled=true` adds the RAG service and its resource storage |
SeaTunnel: optional import, export and migration @@ -144,6 +154,17 @@ See the [Vermeer deployment guide](docs/vermeer.md) for administration and recov
+
+RAG: optional GraphRAG service + +Enable RAG with `rag.enabled=true` or the [RAG values preset](values-rag.yaml). +RAG is a single process with a separate resource volume; model endpoints must be +reachable from its Pod. Use a compatible HugeGraph AI image, and protect external +access with an authenticated reverse proxy. The [RAG guide](docs/rag.md) covers +model credentials, persisted configuration, network exposure and compatibility. + +
+ ### Internal image registry Mirror the four component images (`hugegraph/pd`, `hugegraph/store`, @@ -154,8 +175,12 @@ helm install hugegraph ./helm/hugegraph -n hugegraph --create-namespace \ --set global.imageRegistry=registry.internal.example.com ``` -A component repository with an explicit registry takes priority over the global -prefix. Set `imagePullSecrets` if the registry needs credentials. +The global prefix applies to unqualified image references such as +`hugegraph/server`. A component repository with an explicit registry, such as +`ghcr.io/team/server` or `localhost:5000/team/server`, takes priority and is +not rewritten. Set the component `image.repository` explicitly to move such an +image; update qualified `server.waitImage` or `store.waitImage` overrides too. +Set `imagePullSecrets` if the registry needs credentials. Store's wait init container and Helm test reuse the Store and Server images, which already contain `curl`; no separate curl image is needed by default. See the [complete registry example](examples/values-internal-registry.yaml). @@ -192,6 +217,20 @@ inherit `persistence.enabled`; see [Vermeer storage](docs/vermeer.md#storage-and Hubble persists its Text2GQL master key alongside H2. Existing releases may need a one-time API-key re-entry; see [Text2GQL credentials](docs/operations.md#hubble-text2gql-credentials). +### Partition sharding + +For a new installation, `pd.partition.defaultShardCount` sets the initial +replica count for each partition. When unset, the chart seeds RF=1 with fewer +than three Stores and RF=3 with three or more Stores. An explicit value must be +odd and no greater than `store.replicas`. `pd.partition.storeMaxShardCount` +controls the initial shard capacity per Store; see [values.yaml](values.yaml) +for the derivation and defaults. + +PD persists these values during first bootstrap. Scaling Store or changing Helm +values later does not change the persisted replica policy or move existing data. +Use the [bootstrap topology guide](docs/operations.md#stable-bootstrap-topology) +for adoption and coordinated changes. + ### PD metadata namespace `metadataCluster` defaults to `hg-test`, preserving the namespace used by existing @@ -247,14 +286,20 @@ still need verification with a running Linux cluster. See the The independent `vermeer.mode` and `seatunnel.mode` select optional engine topologies; none of these presets enables an optional engine automatically. -`mode: easy` defaults HTTP entries to NodePort. `mode: strict` defaults them to ClusterIP -and enables full NetworkPolicy isolation; authentication, Hubble and persistence stay enabled. -Use `--set mode=strict` or `-f helm/hugegraph/values-strict.yaml`. - -**Explicit component value > global value > mode default.** An explicit `false` -is preserved; null component values inherit. The global settings are -`persistence.enabled` and `global.imageRegistry`. SeaTunnel/Vermeer HTTP stays -authenticated and peer channels remain internal. No mode requires HTTPS. +`mode: easy` defaults HTTP entries to NodePort. `mode: strict` defaults them to +ClusterIP and enables NetworkPolicy resources; authentication, Hubble and +persistence stay enabled. Network isolation requires a CNI plugin that supports +and actually enforces Kubernetes NetworkPolicy. Use `--set mode=strict` or +`-f helm/hugegraph/values-strict.yaml`. + +An explicit component value takes precedence. An unset component Service type +inherits the mode default; an unset component storage flag inherits +`persistence.enabled`. Explicit `false` is preserved. The separate +`global.imageRegistry` rule is described in [Internal image registry](#internal-image-registry). +SeaTunnel/Vermeer HTTP stays authenticated and peer channels remain internal. +The chart does not automatically configure TLS. For access over untrusted +networks, provide TLS through an HTTPS proxy or Ingress and restrict access +appropriately. Server/Hubble entry choices: @@ -263,7 +308,7 @@ Server/Hubble entry choices: | NodePort | Reachable node IP + assigned port | Direct access without another controller; easy default | | ClusterIP | Internal Service IP/DNS | In-cluster clients, an existing gateway or port-forward | | LoadBalancer | Infrastructure-assigned address | A load balancer controller is available | -| Ingress | HTTP routes through a controller | Shared HTTP routing; hostname and TLS are optional | +| Ingress | Routes through a controller | Shared routing; configure HTTPS for untrusted networks | ClusterIP is an internal virtual service address, not a node IP. An ordinary laptop usually cannot reach it directly. With default `externalTrafficPolicy: @@ -299,6 +344,7 @@ with `-f`; see [values.yaml](values.yaml) for all fields and their comments. | `pd.replicas` / `store.replicas` / `server.replicas` | Component replicas | `3` / `3` / `3` | | `hubble.enabled` | Install the UI | `true` | | `seatunnel.enabled` / `vermeer.enabled` | Install optional data/computation engines | `false` / `false` | +| `rag.enabled` | Install the optional RAG service | `false` | | `vermeer.mode` | One Pod with two containers, or separate Master/Worker Pods | `standalone` | | `vermeer.worker.replicas` | Worker Pods in Vermeer cluster mode | `3` | | `server.auth.enabled` | Server authentication | `true` | @@ -338,10 +384,28 @@ with `-f`; see [values.yaml](values.yaml) for all fields and their comments. ## Upgrade and verify +Inspect the saved values, then use the same chart source as the installation. +For the published OCI chart, specify the published chart version: + +```bash +helm get values hugegraph -n hugegraph --all +CHART_VERSION="REPLACE_WITH_PUBLISHED_VERSION" +helm upgrade hugegraph oci://ghcr.io/hugegraph/hugegraph-helm \ + --version "${CHART_VERSION}" -n hugegraph --reuse-values \ + --wait --timeout 15m +``` + +For a source checkout, run from the repository root: + ```bash helm get values hugegraph -n hugegraph --all helm upgrade hugegraph ./helm/hugegraph -n hugegraph --reuse-values \ --wait --timeout 15m +``` + +After either upgrade, verify the release: + +```bash helm test hugegraph -n hugegraph --logs --timeout 5m ``` @@ -351,6 +415,15 @@ NodePort-default notice in NOTES; explicitly configured entry types are preserve To adopt new defaults, use `--reset-values` with a reviewed values file retaining all settings your release needs. +A release installed with an older chart may have a NodePort stored in its release values even when the operator never set the Service type explicitly. `--reuse-values` can retain that value, so changing only `mode=strict` does not guarantee that existing Server or Hubble Services become `ClusterIP`. Inspect the effective values and Services after the upgrade: + +```bash +helm get values hugegraph -n hugegraph --all +kubectl -n hugegraph get svc hugegraph-server hugegraph-hubble +``` + +To make existing entries private, explicitly set `server.service.type=ClusterIP` and `hubble.service.type=ClusterIP`—plus the Service type of each enabled optional component—or use `--reset-values` with a complete reviewed values file. Verify the resulting Service types rather than assuming a mode change replaced values retained from an older release. + PD, Store and Server images built from this version run as UID/GID `10001`. The PD and Store pods get `fsGroup: 10001` (with `fsGroupChangePolicy: OnRootMismatch`) unless their `podSecurityContext` sets `fsGroup`, also when `--reuse-values` keeps @@ -372,12 +445,18 @@ If a release already ran root images with `fsGroup` set, roll out the first in the same command, so the recursive change runs for the `10001` image rather than for a pinned root image. The example clears any reused digest so the tag applies; a release pinned by digest sets `pd.image.digest` and -`store.image.digest` to the digests of this version's images instead: +`store.image.digest` to the digests of this version's images instead. +The migration commands below use a source checkout and run from the repository +root. For an OCI installation, replace `./helm/hugegraph` with +`oci://ghcr.io/hugegraph/hugegraph-helm` and add the same explicit +`--version "${CHART_VERSION}"` used in the published-chart upgrade above. ```bash +# Replace with the tag of the matching UID/GID 10001 images. +IMAGE_TAG="REPLACE_WITH_IMAGE_TAG" helm upgrade hugegraph ./helm/hugegraph -n hugegraph --reuse-values \ - --set pd.image.tag= \ - --set store.image.tag= \ + --set-string pd.image.tag="${IMAGE_TAG}" \ + --set-string store.image.tag="${IMAGE_TAG}" \ --set-string pd.image.digest= \ --set-string store.image.digest= \ --set pd.podSecurityContext.fsGroupChangePolicy=Always \ @@ -424,4 +503,4 @@ login and graph query. Templates and linting alone do not prove live volume, network or application behavior. Website guides: [Helm deployment](https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-helm/) -and [Kubernetes operations](https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-helm-operations/). +and [Kubernetes operations](https://hugegraph.apache.org/docs/quickstart/hugegraph/hugegraph-helm-operations/). \ No newline at end of file diff --git a/helm/hugegraph/docs/README.md b/helm/hugegraph/docs/README.md new file mode 100644 index 0000000000..f3b38c0a6e --- /dev/null +++ b/helm/hugegraph/docs/README.md @@ -0,0 +1,49 @@ +# Helm documentation index + +The chart [README](../README.md) is the installation and configuration entry point. This directory contains focused operational and optional-component guides that would make the entry point difficult to scan if kept inline. + +## Documents + +| Document | Read it when you need to... | +|---|---| +| [distribution.md](distribution.md) | Install from OCI or HTTP repositories, mirror images, work offline, or choose a single-node distribution path | +| [operations.md](operations.md) | Manage credentials, persistence, bootstrap, membership, scaling, upgrades, scheduling, policies, probes and troubleshooting | +| [rag.md](rag.md) | Deploy and secure the HugeGraph AI RAG service through the chart | +| [seatunnel.md](seatunnel.md) | Configure SeaTunnel import, export and migration in standalone or cluster mode | +| [vermeer.md](vermeer.md) | Configure Vermeer computation, Hubble integration, native administration, storage and recovery | + +Reviewed value combinations live under [`../examples/`](../examples/). The complete value schema and comments live in [`../values.yaml`](../values.yaml). Keep those as the source of truth instead of copying every value into each guide. + +## Recommended reading paths + +### First installation + +1. [Chart README](../README.md) +2. [Distribution](distribution.md) when the published OCI chart or default images are not directly reachable +3. [Examples](../examples/README.md) for a complete registry, storage, Ingress or GitOps configuration + +### Production preparation or upgrade + +1. [Operations](operations.md) +2. The active release values from `helm get values ... --all` +3. Relevant optional-component guide when SeaTunnel, Vermeer or RAG is enabled + +### Incident or recovery + +Start with the matching section in [operations.md](operations.md). Follow links into component-specific guides only when the failure belongs to that component. Record durable recovery rules in the canonical guide rather than appending an incident timeline. + +## Documentation ownership + +- `../README.md` owns chart purpose, first install, topology selection, common settings and high-level boundaries. +- `operations.md` owns procedures that can affect availability, identity, credentials or persistent data. +- Optional-component guides own their component-specific setup and failure modes. +- `../values.yaml` owns option names, defaults and field-level comments. +- The website owns long-form user tutorials and release-facing documentation. + +## Authoring rules + +- Link every durable page from this index or an obvious parent section. Do not leave operational instructions as isolated Markdown files. +- Explain the supported current procedure, prerequisites, safety boundary and verification result. Avoid review history, temporary workarounds and change-by-change narratives. +- Do not use pull-request links, commit hashes or branch names as permanent evidence. Prefer stable source paths, value names and public user documentation. +- Keep prose paragraphs as logical lines. Wrap near 160 characters only when necessary for review; preserve structure in tables, lists and code blocks. +- When a value or default changes, update `values.yaml`, the smallest relevant guide and any corresponding website page in the same change.