Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,19 @@ Structured Markdown notes with timestamps

NoteForge downloads subtitles only. It does not download the video or audio.

## Media service and credentials

`noteforge.media.MediaService` is the sole media boundary for YouTube and
Comment on lines 51 to +55
Bilibili. It exposes metadata, formats, playlists, subtitles, audio, and video
without leaking yt-dlp options. yt-dlp runs in isolated worker processes, while
downloaded assets live in expiring leases and are removed when `MediaAsset` is
closed. Persistence requires an explicit `export_to()` call.

Browser credentials are handled by `CookieService`. A task receives a private,
short-lived Cookie lease that is deleted by default. Opt-in retained credentials
are AEAD-encrypted and their master key is stored in the operating-system
keyring; plaintext Cookie files are never retained.

---

## Capabilities
Expand Down Expand Up @@ -393,12 +406,10 @@ Project structure:
noteforge/
├── src/noteforge/
│ ├── cli/ # Typer commands
│ ├── collector/ # Source inspection and Bilibili collection
│ ├── subtitle/ # Subtitle selection, parsing, and normalization
│ ├── media/ # Unified media, Cookie, platform, and worker service
│ ├── knowledge/ # Chunking, semantic analysis, and extraction
│ ├── llm/ # OpenAI-compatible, Anthropic Messages, and Ollama adapters
│ ├── document/ # Learning-document construction
│ ├── renderer/ # Markdown rendering and writing
│ ├── document/ # Learning-document construction and Markdown rendering
│ └── core/ # End-to-end pipeline
├── tests/
├── .github/workflows/publish.yml
Expand Down
20 changes: 11 additions & 9 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,15 @@

## 视频采集配置

Media Extractor 统一支持 Bilibili 与 YouTube。复制
`config.example.yaml` 为 `config.yaml` 后,可分别配置代理、缓存、下载目录和
Cookie。推荐同时设置 `cookie_file` 与 `cookies_from_browser`:首次运行从已登录
浏览器导入 Cookie,之后只复用持久文件,不会反复读取浏览器 Cookie 或提示系统
密码。`config.yaml`、`.noteforge/` 与 `.cache/` 已被 Git 忽略,请勿提交 Cookie。
`noteforge.media.MediaService` 是 Bilibili 与 YouTube 的唯一媒体入口,统一提供
元数据、格式、播放列表、字幕、音频和视频 API。yt-dlp 在独立 Worker 进程中执行,
调用方不会接触其参数或下载路径。媒体默认保存在有 TTL 的临时租约中,退出
`MediaAsset` 上下文后立即删除;只有显式调用 `export_to()` 才会持久化。

浏览器身份由独立 `CookieService` 管理。每个任务只获得权限为 `0600` 的短期
Cookie 租约,任务结束默认销毁。用户显式选择 `CookiePersistence.RETAIN` 时,
目标平台 Cookie 使用 AEAD 加密保存,主密钥进入系统 Keyring;不会持久化明文
`cookies.txt`。`config.yaml`、`.noteforge/` 与 `.cache/` 已被 Git 忽略。

字幕 fallback 顺序为人工字幕、自动字幕、可选的音频转录器;媒体层目前可解析
VTT、SRT、ASS 和 JSON3,并为 Whisper 实现保留了 `AudioTranscriber` 接口。
Expand Down Expand Up @@ -411,12 +415,10 @@ GitHub 中不需要保存长期 PyPI Token。
noteforge/
├── src/noteforge/
│ ├── cli/ # Typer 命令
│ ├── collector/ # 来源检查与 B 站采集
│ ├── subtitle/ # 字幕选择、解析与规范化
│ ├── media/ # 统一媒体、Cookie、平台适配与 Worker 服务
│ ├── knowledge/ # 分块、语义分析与知识提取
│ ├── llm/ # OpenAI-compatible、Anthropic Messages 和 Ollama 适配器
│ ├── document/ # 学习文档构建
│ ├── renderer/ # Markdown 渲染与写入
│ ├── document/ # 学习文档构建与 Markdown 渲染
│ └── core/ # 端到端流水线
├── tests/
├── .github/workflows/publish.yml
Expand Down
14 changes: 6 additions & 8 deletions config.example.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# 复制为 config.yaml。Cookie 文件属于敏感信息,请勿提交到版本库
# 复制为 config.yaml。媒体与 Cookie 均由租约管理,不配置明文 Cookie 文件
extractor:
cache_path: .cache/noteforge/media
download_path: .cache/noteforge/downloads
runtime_path: .cache/noteforge/runtime
credential_vault_path: .noteforge/credentials
worker_count: 2
proxy:
youtube:
# 首次运行时,从已登录的浏览器导入 Cookie 到此文件。
# 后续运行只使用持久化文件,不会再次提示授权。
cookie_file: .noteforge/cookies/youtube.txt
cookies_from_browser: chrome
proxy:
bilibili:
cookie_file: .noteforge/cookies/bilibili.txt
cookies_from_browser: chrome
proxy:
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ classifiers = [
"Topic :: Text Processing :: Markup :: Markdown",
]
dependencies = [
"cryptography>=45,<48",
"httpx>=0.28,<1",
"keyring>=25,<27",
"rich>=13,<15",
"typer>=0.12,<1",
"yt-dlp[curl-cffi]>=2026.7.4",
Expand Down
4 changes: 2 additions & 2 deletions src/noteforge/cli/commands/doctor.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
import typer

from noteforge.cli.ui import StatusUI
from noteforge.collector import discover_video
from noteforge.collector import source as inspection
from noteforge.config import LLMSettings, llm_api_format_label
from noteforge.exceptions import CollectionError, LLMConfigurationError
from noteforge.llm import LLMMessage, create_llm_client
from noteforge.media import discover_video
from noteforge.media import source as inspection
from noteforge.media.models import Subtitle, VideoResource


Expand Down
4 changes: 2 additions & 2 deletions src/noteforge/cli/commands/generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@
from noteforge.cli.commands.configure import load_configured_llm_settings
from noteforge.cli.renderer import PipelineRenderer
from noteforge.cli.ui import StatusUI
from noteforge.collector import collect_video
from noteforge.collector import source as inspection
from noteforge.config import LLMSettings, llm_api_format_label
from noteforge.core import NoteGenerationPipeline
from noteforge.core.events import compose_event_handlers
from noteforge.exceptions import NoteForgeError, PipelineExecutionError
from noteforge.llm import create_llm_client
from noteforge.media import collect_video
from noteforge.media import source as inspection
from noteforge.media.models import VideoResource
from noteforge.run import RunRecorder

Expand Down
4 changes: 2 additions & 2 deletions src/noteforge/cli/commands/inspect.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
import typer

from noteforge.cli.serialization import subtitle_debug_output
from noteforge.collector import collect_video
from noteforge.collector import source as inspection
from noteforge.exceptions import CollectionError, SubtitleError
from noteforge.media import collect_video
from noteforge.media import source as inspection


def inspect(
Expand Down
24 changes: 0 additions & 24 deletions src/noteforge/collector/__init__.py

This file was deleted.

83 changes: 0 additions & 83 deletions src/noteforge/collector/factory.py

This file was deleted.

7 changes: 0 additions & 7 deletions src/noteforge/collector/platforms/__init__.py

This file was deleted.

Loading