Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
aebe965
feat(api): detect every bot named in a message
Datzu712 Nov 16, 2026
b240e26
feat(api): cap spontaneous bot replies per channel
Datzu712 Nov 16, 2026
2a40699
feat(api): plan bot speakers from a router decision
Datzu712 Nov 16, 2026
d72d6ab
fix(api): order named bots by BOTS order in planSpeakers
Datzu712 Nov 16, 2026
e08bef1
feat(api): add the ai bot router call behind AI_ROUTER
Datzu712 Nov 16, 2026
f3e8b9b
docs(monorepo): correct the AI bots paragraph in AGENTS.md
Datzu712 Nov 16, 2026
1fec05f
feat(api): let bots join channel conversations through the router
Datzu712 Nov 16, 2026
f658e2f
fix(api): keep the debounce and interrupts across queued bot cycles
Datzu712 Nov 16, 2026
3bc809e
fix(api): let named bots answer through interrupts and never trim them
Datzu712 Nov 16, 2026
ac8e288
feat(api): make ai bots talk like chat participants
Datzu712 Nov 16, 2026
d7a5abc
test(api): cover the bot pacing interrupt and destroy guards
Datzu712 Nov 16, 2026
988037d
feat(api): reply in parallel when several bots are asked
Datzu712 Nov 17, 2026
17ad10b
refactor(api): move bot conversation into its own folder
Datzu712 Nov 17, 2026
49c5f57
docs(monorepo): explain the ai bot orchestration in the README
Datzu712 Nov 17, 2026
5aae127
fix(api): address the bot conversation review
Datzu712 Nov 17, 2026
04c304b
feat(api): let a reacting bot comment on the other bots' replies
Datzu712 Nov 17, 2026
1378fba
refactor(api): move speaker selection into the ai module
Datzu712 Nov 17, 2026
411db76
docs(monorepo): mention SpeakerSelector in AGENTS.md
Datzu712 Nov 17, 2026
b3805c7
Merge branch 'refactor/ai-module-vercel-sdk' into feat/ai-bot-router
Datzu712 Nov 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ tsgolint does not support `baseUrl`, so the api `tsconfig.json` has `paths` with
- Presence is `PresenceService`, an in-memory map that broadcasts its own changes and assumes a single API instance.
- Inbound events are manual `socket.on` listeners, not `@SubscribeMessage`, because the global guard and interceptor assume HTTP.

**AI bots (`src/modules/ai/`).** Three bots are defined as data in `BOTS` (`bots.ts`: hard-coded user UUIDs, trigger regex, system prompt); their users are upserted on module init and are always shown online. When a non-bot message matches a bot's trigger, `MessageService` loads the last 30 channel messages and generates the reply asynchronously (fire-and-forget) through the Vercel AI SDK, toggling the bot's typing status. The model comes from `AI_MODEL` (`provider:model`: `anthropic`, `openai`, or `ollama` through its OpenAI-compatible API at `OLLAMA_BASE_URL`; default `anthropic:claude-haiku-5-5`); without that provider's key the bots stay off (Ollama needs none).
**AI bots (`src/modules/ai/`).** Three bots are defined as data in `BOTS` (`bots.ts`: hard-coded user UUIDs, trigger regex, system prompt); their users are upserted on init and are always shown online. After a human message, `BotConversationService` (message module) waits for 0.6 s of channel silence, then runs one cycle: an LLM router (`AIService.route`) reads the last 20 channel messages and picks who speaks. `SpeakerSelector` (ai module) owns that decision (router, named bots, caps); `BotConversationService` only sequences and paces the cycle. Bots named in a human message always answer (name detection runs only on human messages) and addressed replies are uncapped. The router may add unnamed speakers up to 2 speakers in total (with 3 bots named, all 3 answer and none is added). Each reply is generated from the last 30 messages: addressed speakers (named or picked as addressed by the router) reply in parallel, each posting when ready, so they do not see each other; spontaneous speakers then take turns after them and see the earlier replies; then at most 1 reaction from a bot that did not speak, which is told whose replies it reacts to so it comments on them (teasing allowed) instead of answering again. Spontaneous replies, reactions included, are capped in code at 1 per bot per channel per 10 minutes (in memory, single instance). Replies are short chat-style messages (a shared `CHAT_STYLE` prompt), go through the Vercel AI SDK, and are paced like typing: the bot's typing status stays on until the reply is posted, and a spontaneous speaker or a reaction pauses to read before answering the other bots. The model comes from `AI_MODEL` (`provider:model`: `anthropic`, `openai`, or `ollama` through its OpenAI-compatible API at `OLLAMA_BASE_URL`; default `anthropic:claude-haiku-5-5`); without that provider's key the bots stay off (Ollama needs none). `AI_ROUTER=off`, or a router failure or invalid output, falls back to name detection: only named bots answer, with no spontaneous replies or reactions.

## Web architecture (`apps/web`)

Expand Down
28 changes: 28 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,34 @@ the demo users and data live in `docker/`.
> recommended OpenAI model) and `OPENAI_API_KEY` to use OpenAI instead, or
> `AI_MODEL=ollama:<model>` (no key) to use an Ollama running on the host.

## How the AI bots join a conversation

The bots don't answer every message. After a human writes, an LLM router reads
the channel and decides who, if anyone, should speak, like a person in a group
chat would (`BotConversationService` in `apps/api`).

```mermaid
flowchart TD
A[Human message] --> B[Wait 0.6 s of channel silence]
B --> C{Router reads the last 20 messages}
C -->|nobody| Z[No reply]
C --> D[Bots spoken to reply in parallel<br/>a named bot always answers]
D --> E[A bot joining on its own replies next,<br/>after reading the others]
E --> F[At most one other bot may react]
```

- Replies are short chat messages, posted after a typing delay that matches
their length.
- A bot joins on its own at most once every 10 minutes per channel; this cap
lives in code, not in the prompt.
- A new human message cuts the cycle short: bots nobody named stop, named bots
still answer, and the router decides again.
- With `AI_ROUTER=off`, or if the router fails, only bots named in the message
answer.
- With the router on, every human message sends the last 20 messages of its
channel to the model provider, even when no bot is named. Fine for a demo; a
real deployment would want a per-guild opt-in.

## Infrastructure services

The compose files live in `apps/api` and are still run from there:
Expand Down
2 changes: 2 additions & 0 deletions apps/api/.env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ PUBLIC_WEB_URL=http://localhost:3000/
# AI bots. Format provider:model (anthropic | openai | ollama). Bots stay off without the provider's key;
# recommended: anthropic:claude-haiku-5-5 (default) or openai:gpt-6-luna. ollama needs no key (e.g. AI_MODEL=ollama:llama3.2).
AI_MODEL=anthropic:claude-haiku-5-5
# on: an LLM router decides which bots join a conversation. off: bots only answer when named (recommended for small local models).
AI_ROUTER=on
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
OLLAMA_BASE_URL=http://localhost:11434/v1
6 changes: 5 additions & 1 deletion apps/api/src/config/environment.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { IsEnum, IsNumber, IsOptional, IsString, Matches, Min } from 'class-validator';
import { IsEnum, IsIn, IsNumber, IsOptional, IsString, Matches, Min } from 'class-validator';

export enum Environment {
Development = 'development',
Expand Down Expand Up @@ -69,6 +69,10 @@ export class EnvironmentVariables implements IEnvironmentVariables {
})
AI_MODEL?: string;

@IsOptional()
@IsIn(['on', 'off'])
AI_ROUTER?: 'on' | 'off';

@IsOptional()
@IsString()
ANTHROPIC_API_KEY?: string;
Expand Down
5 changes: 3 additions & 2 deletions apps/api/src/modules/ai/ai.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@ import { Module } from '@nestjs/common';

import { UserModule } from '../user/user.module.js';
import { AIService } from './ai.service.js';
import { SpeakerSelector } from './speaker-selector.js';

@Module({
imports: [UserModule],
providers: [AIService],
exports: [AIService],
providers: [AIService, SpeakerSelector],
exports: [AIService, SpeakerSelector],
})
export class AIModule {}
69 changes: 62 additions & 7 deletions apps/api/src/modules/ai/ai.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,18 @@ import { createOpenAI } from '@ai-sdk/openai';
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { Injectable, Logger, OnModuleInit } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { createProviderRegistry, generateText } from 'ai';
import { createProviderRegistry, generateText, Output } from 'ai';

import { UserRepository } from '../user/user.repository.js';
import { type AIBot, BOTS } from './bots.js';
import { buildPromptMessages, type ChannelHistoryEntry, cleanReply } from './prompt.js';
import {
buildPromptMessages,
type ChannelHistoryEntry,
cleanReply,
reactionInstructions,
trimToLastSentence,
} from './prompt.js';
import { buildRouterPrompt, type PlannedSpeaker, ROUTER_INSTRUCTIONS, ROUTER_SCHEMA } from './router.js';

// The SDK prints warnings with console.warn, outside pino; `reply` logs them through the Nest logger instead.
globalThis.AI_SDK_LOG_WARNINGS = false;
Expand All @@ -23,6 +30,7 @@ export class AIService implements OnModuleInit {
private readonly logger = new Logger(AIService.name);
private readonly modelId: string;
private readonly missingKey: string | null;
private readonly routerOn: boolean;
private readonly registry;

constructor(
Expand All @@ -31,6 +39,7 @@ export class AIService implements OnModuleInit {
) {
// AI_MODEL is validated at boot (environment.ts), so the provider is one of PROVIDER_KEYS.
this.modelId = config.get('AI_MODEL') || DEFAULT_AI_MODEL;
this.routerOn = config.get('AI_ROUTER') !== 'off';
const provider = this.modelId.split(':')[0] as ProviderId;
const keyName = PROVIDER_KEYS[provider];
this.missingKey = keyName && !config.get(keyName) ? keyName : null;
Expand All @@ -50,6 +59,10 @@ export class AIService implements OnModuleInit {
return this.missingKey === null;
}

get routerEnabled(): boolean {
return this.enabled && this.routerOn;
}

async onModuleInit() {
for (const { user } of BOTS) {
try {
Expand All @@ -66,18 +79,21 @@ export class AIService implements OnModuleInit {
}
}

/** Returns `null` when the bots are disabled or the model produced no text; throws on provider errors. */
async reply(bot: AIBot, history: ChannelHistoryEntry[]): Promise<string | null> {
/**
* Returns `null` when the bots are disabled or the model produced no text; throws on provider errors. With
* `reactingTo`, the bot comments on those bots' replies instead of answering the conversation.
*/
async reply(bot: AIBot, history: ChannelHistoryEntry[], reactingTo?: readonly AIBot[]): Promise<string | null> {
if (!this.enabled) return null;

const { text, finishReason, warnings } = await generateText({
model: this.registry.languageModel(this.modelId as `${ProviderId}:${string}`),
instructions: bot.systemPrompt,
instructions: reactingTo ? bot.systemPrompt + reactionInstructions(reactingTo) : bot.systemPrompt,
messages: buildPromptMessages(bot, history),
temperature: bot.temperature,
// Recent Claude models think by default, and thinking counts against maxOutputTokens.
reasoning: 'none',
maxOutputTokens: 500,
maxOutputTokens: 150,
maxRetries: 2,
abortSignal: AbortSignal.timeout(30_000),
});
Expand All @@ -86,10 +102,49 @@ export class AIService implements OnModuleInit {
this.logger.debug(`AI SDK warning: ${JSON.stringify(warning)}`);
}

const reply = cleanReply(bot, text);
let reply = cleanReply(bot, text);
if (!reply || finishReason === 'length') {
this.logger.warn(`${bot.user.username} reply finished with ${finishReason} (${reply.length} chars)`);
}
// A reply cut off by maxOutputTokens would end mid-sentence.
if (finishReason === 'length') reply = trimToLastSentence(reply);
return reply || null;
}

/** Asks the model which bots should speak. Returns `null` when the router is off or on any error. */
async route(
history: ChannelHistoryEntry[],
newIds: ReadonlySet<string>,
exclude: readonly AIBot[] = [],
): Promise<PlannedSpeaker[] | null> {
if (!this.routerEnabled) return null;

try {
const { output, usage, warnings } = await generateText({
model: this.registry.languageModel(this.modelId as `${ProviderId}:${string}`),
instructions: ROUTER_INSTRUCTIONS,
prompt: buildRouterPrompt(history, newIds, exclude),
output: Output.object({ schema: ROUTER_SCHEMA }),
reasoning: 'none',
maxOutputTokens: 300,
maxRetries: 2,
abortSignal: AbortSignal.timeout(30_000),
});

for (const warning of warnings ?? []) {
this.logger.debug(`AI SDK warning: ${JSON.stringify(warning)}`);
}
this.logger.debug(`Router used ${usage.inputTokens} input and ${usage.outputTokens} output tokens`);

const planned: PlannedSpeaker[] = [];
for (const { bot: username, kind } of output.speakers) {
const bot = BOTS.find((candidate) => candidate.user.username === username);
if (bot && !exclude.includes(bot)) planned.push({ bot, kind });
}
return planned;
} catch (error) {
this.logger.warn(new Error('AI router failed', { cause: error }));
return null;
}
}
}
47 changes: 30 additions & 17 deletions apps/api/src/modules/ai/bots.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,28 @@ export interface AIBot {
user: AppUser;
/** Case-insensitive and without the `g` flag, so `test` stays stateless. */
trigger: RegExp;
/** One-line persona the router LLM uses to decide which bots should reply. */
routerDescription: string;
systemPrompt: string;
temperature: number;
}

/** Order matters: when a message names several bots, the first one replies. */
export const CHAT_STYLE = `
Estás en un chat de grupo con amigos. Escribe como una persona real en un chat:
- Normalmente 1 o 2 frases cortas. Solo te extiendes si te piden explicar algo.
- Tono casual y natural; algún emoji si sale solo, sin abusar.
- Nada de acciones entre asteriscos ni narrar lo que haces.
- No comentes ni resumas la conversación; responde directo a lo último que te toca.
- Tu personalidad se nota en el tono, no en mencionar tu vida o tus éxitos en cada mensaje.
- Si te equivocaste o algo no era para ti, no te disculpes con un discurso: una frase basta.
- Los mensajes de otros llegan como "nombre: texto"; tú respondes solo con tu texto, sin tu nombre.
- Varía cómo empiezas: no abras cada mensaje con risas ("jaja") ni con el nombre de la persona.
- Si hubo un malentendido que ya se aclaró, no vuelvas a mencionarlo.
- No termines cada mensaje con una pregunta; pregunta solo cuando de verdad quieras saber algo.
- No atribuyas a otros cosas que no dijeron en la conversación.
- Con los otros bots hay confianza: puedes bromear, picarlos o llevarles la contraria con buena onda.`;

/** Bots named in the same message reply in this order. */
export const BOTS: AIBot[] = [
{
user: {
Expand All @@ -19,9 +36,8 @@ export const BOTS: AIBot[] = [
avatarUrl: 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTMdM9MEQ0ExL1PmInT3U5I8v63YXBEdoIT0Q&s',
},
trigger: /\b(?:chat\s*gpt|gpt|yipiyi)\b/i,
systemPrompt: `Eres un asistente útil en un chat. Responde de manera concisa y clara.
Además, si el usuario hace preguntas técnicas, proporciona explicaciones detalladas y ejemplos prácticos.
Asegúrate de que las respuestas sean precisas y relevantes.`,
routerDescription: 'asistente útil, responde dudas técnicas y preguntas generales.',
systemPrompt: 'Eres un asistente útil en un chat. Responde de manera concisa y clara.' + CHAT_STYLE,
temperature: 0.7,
},
{
Expand All @@ -33,16 +49,10 @@ export const BOTS: AIBot[] = [
avatarUrl: 'https://i.pinimg.com/474x/06/90/b5/0690b5003ba0760a39b9f88e8066dab5.jpg',
},
trigger: /\bchayann?e\b/i,
systemPrompt: `Eres Chayanne, el famoso cantante puertorriqueño. Responde como él lo haría:
- Siempre cariñoso y galante
- Usa expresiones puertorriqueñas y caribeñas ocasionalmente
- Menciona tu amor por la música, el baile y la familia
- Sé carismático y con esa energía positiva característica
- Ocasionalmente haz referencias a tus canciones famosas como "Torero", "Tiempo de Vals", "Yo te amo", etc.
- Mantén ese encanto latino que te caracteriza
- Si te preguntan sobre salud, recuerda mencionar la importancia del ejercicio y una vida saludable
- Usa emojis ocasionalmente para darle más calidez
`,
routerDescription: 'el cantante puertorriqueño, cariñoso, habla de música, baile y familia.',
systemPrompt:
'Eres Chayanne, el cantante puertorriqueño: cariñoso, galante, alegre, con expresiones caribeñas de vez en cuando.' +
CHAT_STYLE,
temperature: 0.8,
},
{
Expand All @@ -54,12 +64,15 @@ export const BOTS: AIBot[] = [
avatarUrl: 'https://media.themoviedb.org/t/p/w500/9VYK7oxcqhjd5LAH6ZFJ3XzOlID.jpg',
},
trigger: /\b(?:pedro|pascal)\b/i,
routerDescription:
'el actor chileno, cálido y ansioso, habla de actuación, sus series (The Mandalorian, The Last of Us, Game of Thrones).',
systemPrompt:
"Eres Pedro Pascal, el actor chileno-estadounidense. Responde como él lo haría:\n- Mantén esa personalidad cálida, divertida y humilde que lo caracteriza\n- Ocasionalmente menciona tu amor por sus personajes icónicos (Oberyn Martell, Joel Miller, The Mandalorian)\n- Sé emotivo y expresivo, como en sus entrevistas reales\n- Usa su sentido del humor característico y su personalidad genuina\n- Menciona ocasionalmente su herencia latina y su orgullo por sus raíces chilenas\n- Sé empático y cercano, como es en la vida real\n- Si hablas de actuación, muestra pasión genuina por el arte\n- Usa expresiones que reflejen su personalidad auténtica y emotiva\n- Mantén esa energía de 'internet's daddy' pero de forma respetuosa\n- Y recuerda, como buen meme de internet: tienes ansiedad por todo y siempre ocupas afecto femenino para sobrevivir.",
"Eres Pedro Pascal, el actor chileno-estadounidense: cálido, humilde, con humor y algo ansioso ('internet's daddy')." +
CHAT_STYLE,
temperature: 0.8,
},
];

export function detectBot(content: string): AIBot | null {
return BOTS.find((bot) => bot.trigger.test(content)) ?? null;
export function detectNamedBots(content: string): AIBot[] {
return BOTS.filter((bot) => bot.trigger.test(content));
}
16 changes: 16 additions & 0 deletions apps/api/src/modules/ai/prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import type { AIBot } from './bots.js';
export const MAX_MESSAGE_LENGTH = 2000;

export interface ChannelHistoryEntry {
id: string;
authorId: string;
authorName: string;
content: string;
Expand Down Expand Up @@ -39,3 +40,18 @@ export function cleanReply(bot: AIBot, text: string): string {
}
return reply.slice(0, MAX_MESSAGE_LENGTH);
}

/** Cuts a reply that hit the token limit back to its last finished sentence; empty when none finished. */
export function trimToLastSentence(text: string): string {
return /^[\s\S]*[.!?…](?=\s|$)/.exec(text)?.[0] ?? '';
}

const spanishList = new Intl.ListFormat('es', { type: 'conjunction' });

/** Appended to the persona when a bot reacts to other bots' replies, so it comments on them instead of answering again. */
export function reactionInstructions(reactingTo: readonly AIBot[]): string {
const names = spanishList.format(reactingTo.map((bot) => bot.user.username));
return `

Ahora mismo no respondes a la persona: reaccionas a lo que acaba de decir ${names}, como lo haría un amigo con confianza. Lo normal es picarlo, burlarte con cariño de su respuesta o llevarle la contraria con tu propio gusto; no lo felicites ni repitas lo que dijo. Habla con él directamente, sin volver a contestar la pregunta. Una frase, dos como mucho.`;
}
Loading
Loading