Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/catalog-listing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Publish LLM catalog listing

# The catalog listing (catalog/listing.yaml + catalog/screenshots) is
# presentation only: a change never implies a release. On main it replaces the
# listing of the entry the catalog already records and requests a publication.

on:
push:
branches: [main]
paths:
- catalog/listing.yaml
- catalog/screenshots/**
- hack/catalog-listing.ts
- .github/workflows/catalog-listing.yml
workflow_dispatch:

permissions:
contents: read

# Shares the writer with the release publisher: one read/merge/PUT at a time.
concurrency:
group: llm-catalog-publish
cancel-in-progress: false

jobs:
listing:
if: github.repository == 'appthrust/llm'
runs-on: ubuntu-24.04
timeout-minutes: 20
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
with:
persist-credentials: false
- uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
with:
bun-version: 1.4.1
- name: Validate the listing
run: bun hack/publish-entry.ts validate-listing
- name: Submit the listing
id: catalog
env:
CLOUD_API_URL: ${{ vars.CLOUD_API_URL || 'https://api.appthrust.com' }}
CLOUD_CATALOG_ADMIN_TOKEN: ${{ secrets.CLOUD_CATALOG_ADMIN_TOKEN }}
run: bun hack/publish-entry.ts listing
- name: Mint the Cloud workflow dispatch token
if: steps.catalog.outputs.state == 'snapshot'
id: cloud-token
uses: actions/create-github-app-token@fee1f7d63c2ff003460e3d139729b119787bc349 # v2
with:
app-id: ${{ secrets.APPTHRUST_GITHUB_ACTIONS_APP_ID }}
private-key: ${{ secrets.APPTHRUST_GITHUB_ACTIONS_APP_PRIVATE_KEY }}
owner: appthrust
repositories: cloud
- name: Dispatch Cloud catalog publication
if: steps.catalog.outputs.state == 'snapshot'
env:
GH_TOKEN: ${{ steps.cloud-token.outputs.token }}
SNAPSHOT_ID: ${{ steps.catalog.outputs.snapshot_id }}
run: gh api -X POST repos/appthrust/cloud/dispatches -f event_type=catalog-publish -f "client_payload[snapshot_id]=$SNAPSHOT_ID"
64 changes: 64 additions & 0 deletions catalog/listing.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
locales:
en:
title: LLM
tagline: Use generative AI in your Project's apps without prompts or documents leaving the cluster.
overview: |
LLM runs a chat model on CPU inside your Project, so apps can summarise, answer questions and rewrite text without sending prompts or documents to an outside AI service. It is meant for teams that want AI features but keep their data inside their own environment.

Apps do not call LLM directly. They send requests to the Project's AI Gateway, which lists LLM as an in-cluster provider next to any external providers and applies the Project's AI policy: which providers and models are allowed and what is recorded. Knowledge, for example, uses the gateway for Ask answers, reranking and query expansion.

The model (gemma3:4b) is built into each release; nothing is downloaded at startup, and a new model arrives as a new version through the installation's update setting. LLM speaks the OpenAI-compatible chat completions API, including streamed replies, and its model list states the context window (16,384 tokens) and the recommended prompt budget (3,000 tokens) so apps can size their requests. It handles two requests at a time; further requests get an immediate busy response instead of waiting in a queue.

LLM needs no GPU node, but it reserves several gigabytes of memory in the Project. It reports ready only after the model has answered a short test prompt, and it restarts and warms up again on its own if the model process stops. Prompts, documents and answers stay inside the cluster, and the model runtime's cloud features are turned off.
ja:
title: LLM
tagline: プロンプトや文書をクラスタの外に出さずに、Project 内のアプリで生成 AI を使えます。
overview: |
LLM は、Project の中でチャットモデルを CPU で動かすアプリです。要約や質問への回答、文章の書き換えを、プロンプトや文書を外部の AI サービスへ送らずに行えます。AI の機能は使いたいが、データは自分たちの環境から出したくないチームに向いています。

アプリは LLM を直接呼び出しません。Project の AI ゲートウェイにリクエストを送り、ゲートウェイが LLM をクラスタ内のプロバイダーとして外部のプロバイダーと並べて扱います。どのプロバイダーとモデルを使ってよいか、何を記録するかは、Project の AI ポリシーで決まります。たとえば Knowledge は、質問への回答、検索結果の並べ替え、クエリの補完にゲートウェイを使います。

モデル(gemma3:4b)はリリースごとに組み込まれていて、起動時のダウンロードはありません。モデルの更新は新しいバージョンとして届き、インストールの更新設定に沿って反映されます。API は OpenAI 互換のチャット補完で、ストリーミング応答にも対応しています。モデル一覧ではコンテキスト長(16,384 トークン)と推奨のプロンプト上限(3,000 トークン)を返すので、アプリ側でリクエストの大きさを調整できます。同時に処理するのは 2 件までで、それを超えたリクエストは待たせずにすぐ「混雑中」と返します。

GPU ノードは不要ですが、Project 内で数 GB のメモリを確保します。短いテスト用の問いかけにモデルが答えてはじめて準備完了になり、モデルのプロセスが止まった場合は自動で再起動して準備し直します。プロンプト、文書、回答はクラスタの外に出ず、モデル実行環境のクラウド機能も無効にしています。
highlights:
- screenshot: installation
locales:
en:
title: Runs inside your Project
body: The chat model runs as an app in your Project, on CPU. You need no GPU node and no account with an outside AI service, and the installation page shows whether it is ready and healthy.
ja:
title: Project の中で動く
body: チャットモデルは Project 内のアプリとして CPU で動きます。GPU ノードも外部 AI サービスのアカウントも要りません。準備完了かどうか、正常かどうかはインストール画面で確認できます。
- screenshot: ai-policy
locales:
en:
title: A provider for the AI Gateway
body: The Project's AI policy lists LLM and its model gemma3:4b next to external providers. Apps call the AI Gateway, and the policy decides which providers and models may answer and what is recorded.
ja:
title: AI ゲートウェイのプロバイダーとして
body: Project の AI ポリシーには、LLM とモデル gemma3:4b が外部のプロバイダーと並んで表示されます。アプリは AI ゲートウェイを呼び出し、どのプロバイダーとモデルが答えてよいか、何を記録するかはポリシーで決まります。
- screenshot: capability
locales:
en:
title: Healthy means it can answer
body: LLM offers OpenAI-compatible chat completions and reports ready only after the model has answered a test prompt. If the model process stops, it restarts and warms up again on its own.
ja:
title: 答えられてはじめて「正常」
body: LLM は OpenAI 互換のチャット補完を提供し、テスト用の問いかけにモデルが答えてから準備完了になります。モデルのプロセスが止まった場合も、自動で再起動して準備し直します。
screenshots:
- name: installation
file: installation.webp
alt:
en: "AppThrust Dashboard installation page for LLM version 0.2.0, with Ready and Healthy badges, the app's address, and branch and update settings."
ja: "AppThrust ダッシュボードの LLM(バージョン 0.2.0)のインストール画面。「準備完了」「正常」のバッジ、アプリのアドレス、ブランチと更新の設定が表示されています。"
- name: ai-policy
file: ai-policy.webp
alt:
en: "The Project's effective AI policy in the Dashboard, listing the provider llm (llm.internal) with models gemma3* and gemma3:4b above the openai provider, plus approval modes, logging and 90-day retention."
ja: "ダッシュボードに表示された Project の有効な AI ポリシー。プロバイダー llm(llm.internal)とモデル gemma3*、gemma3:4b が openai の上に並び、承認モード、記録の設定、保持期間 90 日が表示されています。"
- name: capability
file: capability.webp
alt:
en: "Details of the LLM installation: catalog entry llm, application builtin-app-llm, capability chat-completions.openai/v1, and a conditions table where Ready and Healthy are True."
ja: "LLM のインストール詳細。カタログのエントリ llm、アプリケーション builtin-app-llm、機能 chat-completions.openai/v1 と、Ready と Healthy が True になっている条件の一覧が表示されています。"
Binary file added catalog/screenshots/ai-policy.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added catalog/screenshots/capability.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added catalog/screenshots/installation.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/appthrust/llm
go 1.26.0

require (
github.com/appthrust/appserver-go v0.1.0
github.com/appthrust/appserver-go v0.3.0
github.com/stretchr/testify v1.11.1
)

Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
github.com/appthrust/appserver-go v0.1.0 h1:MYpkwwm4wqa+YcKe1AwywcRe9xuUtTooK4+o7Z5Jbqo=
github.com/appthrust/appserver-go v0.1.0/go.mod h1:0KSf9RQZ6OxBcE0lxYzzjKF37nfwbKps3wXUGlE0oFo=
github.com/appthrust/appserver-go v0.3.0 h1:zxOz0hkspZSXjLq3KWGDLkZIz42Jag6wqRujZ8lTru0=
github.com/appthrust/appserver-go v0.3.0/go.mod h1:0KSf9RQZ6OxBcE0lxYzzjKF37nfwbKps3wXUGlE0oFo=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
Expand Down
130 changes: 130 additions & 0 deletions hack/catalog-listing.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
import { describe, expect, test } from 'bun:test'
import { createHash } from 'node:crypto'
import { mkdtemp, mkdir, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { pathToFileURL } from 'node:url'
import { imageDimensions, loadListing, uploadMedia, validateListing, type Listing } from './catalog-listing'
import { renderEntry, renderListingUpdate, type CatalogEntry } from './publish-entry'

/** Minimal valid PNG: 8-byte signature + IHDR chunk; dimensions only, no pixel data. */
function png(width: number, height: number): Uint8Array {
const bytes = new Uint8Array(33)
bytes.set([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a], 0)
const view = new DataView(bytes.buffer)
view.setUint32(8, 13)
bytes.set([0x49, 0x48, 0x44, 0x52], 12)
view.setUint32(16, width)
view.setUint32(20, height)
return bytes
}

function webpVP8X(width: number, height: number): Uint8Array {
const bytes = new Uint8Array(30)
bytes.set([0x52, 0x49, 0x46, 0x46], 0)
bytes.set([0x57, 0x45, 0x42, 0x50], 8)
bytes.set([0x56, 0x50, 0x38, 0x58], 12)
const w = width - 1, h = height - 1
bytes.set([w & 0xff, (w >> 8) & 0xff, (w >> 16) & 0xff], 24)
bytes.set([h & 0xff, (h >> 8) & 0xff, (h >> 16) & 0xff], 27)
return bytes
}

const listing: Listing = {
locales: {
en: { tagline: 'People, groups and profiles.', overview: 'Who is who.\n\nSearch and browse.' },
ja: { tagline: 'この組織のメンバー、グループ、プロフィール。', overview: '誰が誰か。' },
},
highlights: [{ locales: { en: { title: 'Search' }, ja: { title: '検索' } }, screenshot: 'people' }],
screenshots: [{
name: 'people', digest: `sha256:${'a'.repeat(64)}`, mediaType: 'image/png', size: 33, width: 1280, height: 800,
alt: { en: 'People list', ja: 'メンバー一覧' },
}],
}

describe('imageDimensions', () => {
test('reads PNG IHDR and WebP VP8X', () => {
expect(imageDimensions(png(1280, 800))).toEqual({ mediaType: 'image/png', width: 1280, height: 800 })
expect(imageDimensions(webpVP8X(1440, 900))).toEqual({ mediaType: 'image/webp', width: 1440, height: 900 })
expect(() => imageDimensions(new Uint8Array(40))).toThrow('PNG or WebP')
})
})

describe('validateListing', () => {
test('accepts the sample and enforces the contract', () => {
expect(() => validateListing(listing)).not.toThrow()
expect(() => validateListing({ locales: { ja: { tagline: 'x' } } })).toThrow('requires the "en" locale')
expect(() => validateListing({ ...listing, highlights: [{ locales: { en: { title: 'x' } }, screenshot: 'missing' }] }))
.toThrow('unknown screenshot "missing"')
expect(() => validateListing({ ...listing, screenshots: [{ ...listing.screenshots![0], size: 600_000 }] })).toThrow('bytes')
expect(() => validateListing({ ...listing, screenshots: [{ ...listing.screenshots![0], alt: { ja: 'x' } }] })).toThrow('requires the "en" locale')
expect(() => validateListing({ locales: { en: { tagline: 'x'.repeat(201) } } })).toThrow('exceeds 200')
})
})

describe('loadListing', () => {
test('resolves screenshot files to digests, sizes and dimensions', async () => {
const dir = await mkdtemp(join(tmpdir(), 'listing-'))
await mkdir(join(dir, 'screenshots'))
const image = png(1200, 750)
await writeFile(join(dir, 'screenshots', 'people.png'), image)
await writeFile(join(dir, 'listing.yaml'), [
'locales:', ' en: { tagline: People, groups and profiles. }', ' ja: { tagline: メンバー一覧 }',
'highlights:', ' - screenshot: people', ' locales: { en: { title: Search }, ja: { title: 検索 } }',
'screenshots:', ' - name: people', ' file: people.png', ' alt: { en: People list, ja: メンバー一覧 }',
].join('\n'))
const { listing: loaded, media } = await loadListing(pathToFileURL(`${dir}/`))
const digest = `sha256:${createHash('sha256').update(image).digest('hex')}`
expect(media).toHaveLength(1)
expect(media[0].bytes).toEqual(image)
expect(loaded.screenshots).toEqual([{ name: 'people', digest, mediaType: 'image/png', size: 33, width: 1200, height: 750, alt: { en: 'People list', ja: 'メンバー一覧' } }])
expect(loaded.highlights?.[0].screenshot).toBe('people')
})

test('rejects paths escaping catalog/screenshots', async () => {
const dir = await mkdtemp(join(tmpdir(), 'listing-'))
await writeFile(join(dir, 'listing.yaml'), 'locales: { en: { tagline: x } }\nscreenshots: [{ name: a, file: ../x.png, alt: { en: a } }]\n')
await expect(loadListing(pathToFileURL(`${dir}/`))).rejects.toThrow('relative path')
})
})

describe('uploadMedia', () => {
test('PUTs each blob by hex digest and checks the stored descriptor', async () => {
const image = png(10, 10)
const digest = `sha256:${createHash('sha256').update(image).digest('hex')}`
const calls: Array<{ url: string; type: string }> = []
const fetcher = async (url: string, init: RequestInit): Promise<Response> => {
calls.push({ url, type: (init.headers as Record<string, string>)['content-type'] })
return Response.json({ digest, mediaType: 'image/png', size: 33 })
}
await uploadMedia([{ ...listing.screenshots![0], digest, bytes: image }], 'https://api.example', 'token', fetcher)
expect(calls).toEqual([{ url: `https://api.example/api/v1/catalog/media/${digest.slice(7)}`, type: 'image/png' }])
const mismatch = async (): Promise<Response> => Response.json({ digest, mediaType: 'image/png', size: 1 })
await expect(uploadMedia([{ ...listing.screenshots![0], digest, bytes: image }], 'https://api.example', 'token', mismatch))
.rejects.toThrow('does not match')
})
})

describe('listing in entries', () => {
const template = [
'apiVersion: appthrust.io/v1alpha1', 'kind: ApplicationStackDefinition', 'metadata: { name: llm }',
'spec:', ' display: { title: Directory, description: People., category: collaborate, iconRef: { name: llm } }',
' versions: []', ' updatePolicy: { default: pinned, allowed: [pinned, follow] }',
].join('\n')
const digest = `sha256:${'b'.repeat(64)}`

test('renderEntry carries the authored listing and otherwise preserves the stored one', () => {
const first = renderEntry(template, '0.2.0', digest, undefined, listing)
expect(first.listing).toEqual(listing)
const next = renderEntry(template, '0.2.1', `sha256:${'c'.repeat(64)}`, first)
expect(next.listing).toEqual(listing)
expect(renderEntry(template, '0.2.0', digest).listing).toBeUndefined()
})

test('renderListingUpdate replaces only the listing', () => {
const current: CatalogEntry = { ...renderEntry(template, '0.2.0', digest), publishedAt: '2026-11-01T00:00:00Z' }
const updated = renderListingUpdate(current, listing)
expect(updated).toEqual({ ...current, listing })
expect(() => renderListingUpdate({ ...current, slug: 'other' }, listing)).toThrow('slug llm')
})
})
Loading
Loading