Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .changes/browser-collection-runtime.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
修复远端开发镜像缺少 SDK 源码与 LICENSE 的构建输入,固定 PDM 安装器的兼容依赖;构建失败时保留两个输出流中的原因并遮盖凭据。
5 changes: 3 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ ENV PDM_CHECK_UPDATE=false \
PDM_VENV_IN_PROJECT=1
WORKDIR /app

# PDM 2.28 requires the httpx client replaced in unearth 0.19.
RUN pip install --no-cache-dir "pdm==${PDM_VERSION}" "unearth==0.18.3"
# Keep the installer on the httpx API used by PDM 2.28.
RUN pip install --no-cache-dir "pdm==${PDM_VERSION}" "unearth==0.18.3" "hishel==1.3.0" \
&& python -c "from pdm.models.session import PDMPyPIClient"

COPY pyproject.toml pdm.lock README.md LICENSE ./
COPY inkcre_core/ inkcre_core/
Expand Down
1 change: 1 addition & 0 deletions docs/30-unit-tdd/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This directory owns expensive internal design truth for logical units delivered
| [memos-extension.md](memos-extension.md) | Memos adapter identity, graph grammar, persistence, and failure boundaries |
| [capture-extension.md](capture-extension.md) | Capture submission identity, original materials, shared graph, and atomic recovery |
| [local-files-extension.md](local-files-extension.md) | 本地文件资料、原件状态与正文的读取分发 |
| [bilibili-extension.md](bilibili-extension.md) | B 站共享收藏图的 Python 读取与词法投影边界 |
| [rss-extension.md](rss-extension.md) | RSS adapter identity, collection lifecycle, reconciliation, and materialization |
| [twitter-extension.md](twitter-extension.md) | Twitter 共同身份、Bookmark 过渡、事务合并和按需媒体读取 |
| [security-model.md](security-model.md) | Core-py realization of the shared security boundary model |
Expand Down
17 changes: 17 additions & 0 deletions docs/30-unit-tdd/bilibili-extension.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# B 站资料读取

当 Python Peer 需要读取或索引 Web Peer 已采集的 B 站资料时,启用 `inkcre/bilibili` 的 Python Distribution。它与 Web Distribution 解释同一份共享图,不拥有采集任务或浏览器会话。

## 读取与索引边界

`extensions/bilibili/schema.py` 验证四类 v1 原生记录,拒绝未知字段、显式 null、非字符串身份和错误类型。User、Folder、Video 的原生身份保存在 Block 内容中;Membership 由所属 Folder 和指向 Video 的关系定位,不复制关联 ID。Python 解释这些已保存的资料。

Resolver 只从已持久化内容产生 Block-local 标签与文本。视频与收藏夹投影包括已保存的标题、简介,未知标题回退到 aid、BVID 或列表 ID。词法维护使用既有 `LexicalRetrievalManager` 和普通维护 Job;插件不直接写派生检索表,也不递归抓取相关视频以补全文本。

直接关系可通过 Core 的 Resolver 图接口读取。收藏条目的可用性与收藏时间独立于视频,避免不同收藏夹的观察互相覆盖。失效占位文字不得在采集阶段抹去旧标题;读取阶段如实解释已保存的较完整记录,不尝试恢复源站内容。

## Runtime 与交付

标准 Python Extension entry point 只装配 Resolver,没有 Source、HTTP 路由、Peer inbound 或私有后台任务。安装与启停沿用原有 Host;Python 类注册随进程保留,停用不模拟卸载已 import 的模块,本插件没有额外的 active effect。Host SDK 范围以 `app.version.CORE_VERSION` 的 `core-py` 边界为准,不把 `inkcre-core` 项目版本当成 Host 版本。

内容契约和用户使用范围见 [插件说明](../../extensions/bilibili/README.md)。发布、安装和精确版本选择继续遵守 [First-party Extension Distribution](../40-deployment/first-party-extension-distribution.md)。
1 change: 1 addition & 0 deletions extensions/bilibili/.changes/reader.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
新增 B 站收藏夹的 Python 读取 Distribution,解释同一插件 Web 采集的账号、收藏夹、视频和收藏条目 v1 图,为现有词法索引提供标题与简介投影;不采集源站、不保存视频二进制。
5 changes: 5 additions & 0 deletions extensions/bilibili/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Changelog

此文件由本 Distribution 的 Towncrier 发布片段生成。

<!-- towncrier release notes start -->
21 changes: 21 additions & 0 deletions extensions/bilibili/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 InKCre contributors

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
33 changes: 33 additions & 0 deletions extensions/bilibili/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# B 站收藏夹读取

`inkcre/bilibili` 的 Python Distribution 解释共享信息库中的 B 站 v1 资料,为 Python Peer 提供读取与现有词法检索所需的文本投影。采集由同一插件的 Web Distribution 借用用户选择的浏览器完成;Python 读取不需要该浏览器在线,也不接触 B 站账号或 cookie。

## 使用

在需要解释或索引 B 站资料的 Python Peer 上安装并启用本 Distribution。Web 采集得到的账号、收藏夹、视频和收藏条目随后可由普通 Resolver 读取。现有 `core.feature_retrieval.lexical.maintain.v1` Job 负责维护索引;本插件只提供标签和文本解释,不另外写一套索引。

视频文本包含已保存的标题与简介,收藏夹文本包含名称与简介。未知标题使用原生身份作为可读回退;缺省字段表示未知。记录的直接关系继续通过普通图接口读取,包含收藏夹归属、成员与 UP 主。

## 持久契约

本版解释四个精确 Resolver ID:

- `extensions.bilibili.user.v1`:B 站账号,含 MID 和实际取得的名称。
- `extensions.bilibili.folder.v1`:收藏夹,含完整列表 ID、名称、可见性及来源报告数量。
- `extensions.bilibili.video.v1`:视频,以 aid 为身份,BVID 为已知别名。
- `extensions.bilibili.membership.v1`:收藏条目的可用性、收藏时间和本次页面观察位置。

图关系为 `账号 —owns→ 收藏夹 —contains→ 收藏条目 —item→ 视频 —author→ UP 主`。同一视频可由多个收藏条目引用。成员记录中的页码和页内位置是采集观察事实,不承诺原站的实时绝对顺序。

本 Distribution 不注册 Source,不下载视频二进制、字幕、封面或其他远程资料,不恢复失效视频,也不修改源站收藏。封面 URL 仅作为已保存事实被解释。采集分发按 B 站原生身份保存普通 Block 与 Relation,Python 读取分发解释已保存的资料。

## 兼容与验证

Host SDK 为 `core-py >=0.3.4 <0.4.0`;该版本来自 Python App 的 Host SDK 版本,区别于独立 `inkcre-core` SDK 项目的版本号。

```sh
pdm run python -m pytest -q tests/extensions/bilibili
pdm run python -m build --wheel extensions/bilibili
```

单元测试使用与 Web v1 形状一致的虚构资料,并调用现有词法投影入口。真实浏览器采集、数据库持久化和跨 Peer 检索由主任务验收记录报告。
20 changes: 20 additions & 0 deletions extensions/bilibili/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""B 站插件的 Python 读取 Distribution。"""

import pydantic

from app.business.extension import ExtensionBase


class BilibiliConfig(pydantic.BaseModel):
model_config = pydantic.ConfigDict(extra="forbid")


class Extension(
ExtensionBase[BilibiliConfig], ext_id="bilibili", config_cls=BilibiliConfig
):
@classmethod
def _init_resolvers(cls) -> None:
from . import resolver as resolver_module # noqa: F401


__all__ = ["BilibiliConfig", "Extension"]
30 changes: 30 additions & 0 deletions extensions/bilibili/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
[build-system]
requires = ["setuptools>=80,<81"]
build-backend = "setuptools.build_meta"

[project]
name = "inkcre-ext-bilibili"
version = "0.1.0"
description = "Read shared Bilibili favorites in Python peers"
requires-python = ">=3.12,<3.13"
readme = "README.md"
license = "MIT"
license-files = ["LICENSE"]
dependencies = [
"pydantic>=2.11.7,<3.0.0",
]

[project.entry-points."inkcre.core.extensions"]
bilibili = "extensions.bilibili:Extension"

[tool.inkcre-extension]
name = "inkcre/bilibili"
nickname = "B 站收藏夹"
host-sdk = "core-py"
host-sdk-version = ">=0.3.4 <0.4.0"

[tool.setuptools]
packages = ["extensions.bilibili"]

[tool.setuptools.package-dir]
"extensions.bilibili" = "."
76 changes: 76 additions & 0 deletions extensions/bilibili/resolver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
"""读取共享收藏资料;词法索引归现有 LexicalRetrievalManager 维护。"""

from __future__ import annotations

import typing

from inkcre_core.business.info_base.resolver import Resolver, TextProjectionContext

from .schema import (
FOLDER_RESOLVER,
MEMBERSHIP_RESOLVER,
USER_RESOLVER,
VIDEO_RESOLVER,
BilibiliRecord,
Folder,
Membership,
User,
Video,
record_title,
)


RecordT = typing.TypeVar("RecordT", bound=BilibiliRecord)


class _Reader(typing.Generic[RecordT]):
content_model: type[RecordT]

async def _get_solved_content(
self, *, refresh: bool = False, materialize_missing: bool = True
) -> RecordT:
del materialize_missing
resolver = typing.cast(Resolver[RecordT, str], self)
return self.content_model.model_validate_json(
await resolver.get_raw_content(refresh=refresh)
)

async def get_text(
self,
*,
context: TextProjectionContext = "default",
refresh: bool = False,
materialize_missing: bool = True,
) -> str:
del context, materialize_missing
resolver = typing.cast(Resolver[RecordT, str], self)
value = await resolver.get_solved_content(refresh=refresh, materialize_missing=False)
description = value.description if isinstance(value, (Folder, Video)) else None
return "\n".join(part for part in (record_title(value), description) if part)

async def get_label(self, *, refresh: bool = False) -> str:
resolver = typing.cast(Resolver[RecordT, str], self)
return record_title(
await resolver.get_solved_content(refresh=refresh, materialize_missing=False)
)


class UserResolver(_Reader[User], Resolver[User, str], rso_type=USER_RESOLVER):
content_model = User


class FolderResolver(_Reader[Folder], Resolver[Folder, str], rso_type=FOLDER_RESOLVER):
content_model = Folder


class VideoResolver(_Reader[Video], Resolver[Video, str], rso_type=VIDEO_RESOLVER):
content_model = Video


class MembershipResolver(
_Reader[Membership], Resolver[Membership, str], rso_type=MEMBERSHIP_RESOLVER
):
content_model = Membership


__all__ = ["FolderResolver", "MembershipResolver", "UserResolver", "VideoResolver"]
113 changes: 113 additions & 0 deletions extensions/bilibili/schema.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
"""B 站 v1 持久图记录,与 MF Distribution 共用语义。"""

from __future__ import annotations

from typing import Annotated, Literal

from pydantic import (
AnyUrl,
AwareDatetime,
BaseModel,
ConfigDict,
Field,
StrictBool,
StrictInt,
field_validator,
model_validator,
)


USER_RESOLVER = "extensions.bilibili.user.v1"
FOLDER_RESOLVER = "extensions.bilibili.folder.v1"
VIDEO_RESOLVER = "extensions.bilibili.video.v1"
MEMBERSHIP_RESOLVER = "extensions.bilibili.membership.v1"

NativeID = Annotated[str, Field(strict=True, pattern=r"^[1-9][0-9]*$")]
NonnegativeInt = Annotated[StrictInt, Field(ge=0)]
PositiveInt = Annotated[StrictInt, Field(ge=1)]
UTCInstant = Annotated[AwareDatetime, Field(strict=False)]


class Record(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True, strict=True)

@model_validator(mode="before")
@classmethod
def sparse_not_null(cls, value: object) -> object:
# The MF contract uses omitted fields for unknown facts, never authored nulls.
if isinstance(value, dict) and any(item is None for item in value.values()):
raise ValueError("未知字段应省略,不能保存 null")
return value


class User(Record):
mid: NativeID
name: str | None = None
avatar_url: AnyUrl | None = None


class Folder(Record):
id: NativeID
title: str | None = None
description: str | None = None
is_private: StrictBool | None = None
reported_count: NonnegativeInt | None = None


class Video(Record):
aid: NativeID
bvid: Annotated[str, Field(pattern=r"^BV[0-9A-Za-z]{10}$")] | None = None
title: str | None = None
description: str | None = None
duration_seconds: NonnegativeInt | None = None
published_at: UTCInstant | None = None
cover_url: AnyUrl | None = None

@field_validator("published_at", mode="before")
@classmethod
def utc_string(cls, value: object) -> object:
if not isinstance(value, str) or not value.endswith("Z"):
raise ValueError("来源时间必须是 UTC ISO 字符串")
return value


class Membership(Record):
availability: Literal["available", "unavailable", "unknown"]
page_number: PositiveInt | None = None
position_in_page: NonnegativeInt | None = None
favored_at: UTCInstant | None = None
observed_at: UTCInstant

@field_validator("favored_at", "observed_at", mode="before")
@classmethod
def utc_string(cls, value: object) -> object:
if not isinstance(value, str) or not value.endswith("Z"):
raise ValueError("来源时间必须是 UTC ISO 字符串")
return value


BilibiliRecord = User | Folder | Video | Membership


def record_title(value: BilibiliRecord) -> str:
if isinstance(value, User):
return value.name or f"UID {value.mid}"
if isinstance(value, Folder):
return value.title or f"收藏夹 {value.id}"
if isinstance(value, Video):
return value.title or value.bvid or f"av{value.aid}"
return "已失效的收藏条目" if value.availability == "unavailable" else "收藏条目"


__all__ = [
"FOLDER_RESOLVER",
"MEMBERSHIP_RESOLVER",
"USER_RESOLVER",
"VIDEO_RESOLVER",
"BilibiliRecord",
"Folder",
"Membership",
"User",
"Video",
"record_title",
]
4 changes: 3 additions & 1 deletion scripts/dev_database_provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,10 @@
BUILD_CONTEXT_PATHS = (
Path("Dockerfile"),
Path("README.md"),
Path("LICENSE"),
Path("alembic.ini"),
Path("app"),
Path("inkcre_core"),
Path("extensions"),
Path("libs"),
Path("migrations"),
Expand Down Expand Up @@ -201,7 +203,7 @@ def _compose_failure(
compose_environment: Path,
provider: DatabaseProvider,
) -> RuntimeError:
raw_diagnostics = error.stderr or error.stdout or b""
raw_diagnostics = b"\n".join(value for value in (error.stderr, error.stdout) if value)
diagnostics = raw_diagnostics.decode(errors="replace")
try:
for line in compose_environment.read_text().splitlines():
Expand Down
Loading
Loading