add update_inner_state tool, life loop with tools, timeout protection
- update_inner_state: LLM can update its own persistent inner state - inner_state injected into chat loop system prompt (read-only) - Life Loop now uses run_openai_with_tools (full tool access) - Life Loop LLM calls wrapped in 120s tokio::time::timeout - All reqwest clients: 120s timeout (whisper: 60s) - doc/life.md: life loop architecture design doc - todo.md: removed completed items
This commit is contained in:
69
doc/life.md
Normal file
69
doc/life.md
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
# Life Loop 设计
|
||||||
|
|
||||||
|
## 核心理念
|
||||||
|
|
||||||
|
小乖不只是一个对话机器人。对话是她跟用户交流的窗口,但 Life Loop 才是她"活着"的地方。
|
||||||
|
|
||||||
|
## 双循环架构
|
||||||
|
|
||||||
|
```
|
||||||
|
Chat Loop (被动) Life Loop (主动)
|
||||||
|
收到消息 → 处理 → 回复 每 30 秒醒来 → 检查 timers
|
||||||
|
context: context:
|
||||||
|
persona persona
|
||||||
|
inner_state (只读) inner_state (读写)
|
||||||
|
对话历史 + scratch timer payload
|
||||||
|
memory_slots 无对话历史
|
||||||
|
tools (全量)
|
||||||
|
决策:
|
||||||
|
- 发消息给某个 chat
|
||||||
|
- 更新 inner_state
|
||||||
|
- 什么都不做
|
||||||
|
|
||||||
|
┌─── SQLite (共享状态层) ───┐
|
||||||
|
│ inner_state │
|
||||||
|
│ timers │
|
||||||
|
│ conversations/messages │
|
||||||
|
│ memory_slots / scratch │
|
||||||
|
│ config │
|
||||||
|
└───────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
## 状态层级
|
||||||
|
|
||||||
|
| 层级 | 名称 | 生命周期 | 用途 |
|
||||||
|
|------|------|---------|------|
|
||||||
|
| persona | 人格 | 永久 | 定义小乖是谁 |
|
||||||
|
| inner_state | 内在状态 | 永久,LLM 自更新 | 小乖对当前情况的感知 |
|
||||||
|
| memory_slots | 记忆槽 | 永久,LLM 管理 | 跨会话的关键事实/偏好 |
|
||||||
|
| summary | 对话摘要 | 按 session | 长对话的压缩记忆 |
|
||||||
|
| scratch | 草稿 | session 内 | 当前任务的工作笔记 |
|
||||||
|
|
||||||
|
## Timer 系统
|
||||||
|
|
||||||
|
### 调度格式
|
||||||
|
|
||||||
|
- 相对时间:`5min`, `2h`, `30s`, `1d`
|
||||||
|
- 绝对时间:`once:2026-04-10 09:00`
|
||||||
|
- 周期性:`cron:0 8 * * *`(标准 cron 表达式)
|
||||||
|
|
||||||
|
### 触发流程
|
||||||
|
|
||||||
|
```
|
||||||
|
Life Loop tick
|
||||||
|
→ 扫描 timers 表,找到 next_fire <= now 的
|
||||||
|
→ 构建 LLM 请求:
|
||||||
|
system: persona + inner_state + 当前时间
|
||||||
|
user: [timer] {label}
|
||||||
|
→ 调用 LLM(无工具,轻量)
|
||||||
|
→ 发送回复到 chat
|
||||||
|
→ cron 类型: 计算下次触发时间,更新 next_fire
|
||||||
|
→ 一次性: 删除
|
||||||
|
```
|
||||||
|
|
||||||
|
### 演进方向
|
||||||
|
|
||||||
|
- 给 Life Loop 的 LLM 调用也加工具(查待办、执行命令)
|
||||||
|
- inner_state 自动更新(对话结束后 LLM 反思)
|
||||||
|
- 预设 cron(晨间/晚间报告)
|
||||||
|
- 事件驱动(不只是时间驱动)
|
||||||
57
doc/todo.md
57
doc/todo.md
@@ -1,50 +1,37 @@
|
|||||||
# noc roadmap
|
# noc todo
|
||||||
|
|
||||||
## "会呼吸的助手" — 让 noc 活着
|
|
||||||
|
|
||||||
核心理念:noc 不应该只在收到消息时才被唤醒,而是一个持续运行、有自己节奏的存在。
|
|
||||||
|
|
||||||
### 主动行为
|
### 主动行为
|
||||||
- [ ] 定时任务 (cron):LLM 可以自己设置提醒、定期检查
|
- [ ] 预设 cron:晨间待办汇总、晚间日记、定期记忆整理
|
||||||
- [ ] 事件驱动:监控文件变化、git push、CI 状态等,主动通知
|
- [ ] 事件驱动:监控文件变化、git push、CI 状态等,主动通知
|
||||||
- [ ] 晨间/晚间报告:每天自动汇总待办、提醒重要事项
|
|
||||||
- [ ] 情境感知:根据时间、地点、日历自动调整行为
|
- [ ] 情境感知:根据时间、地点、日历自动调整行为
|
||||||
|
|
||||||
### 记忆与成长
|
### 记忆与成长
|
||||||
- [x] 持久记忆槽 (memory_slots):100 个跨 session 的记忆槽位,注入 system prompt
|
- [ ] AutoMem:后台定时自动分析对话,LLM 决定 SKIP/UPDATE/INSERT 记忆
|
||||||
- [ ] AutoMem:后台定时(如每 10 条消息)自动分析对话,由 LLM 决定 SKIP/UPDATE/INSERT 记忆,无需用户手动触发(参考 luke)
|
- [ ] 分层记忆:核心记忆(始终注入)+ 长期记忆(RAG 检索)+ scratch(当前任务)
|
||||||
- [ ] 分层记忆:核心记忆(身份/原则,始终注入)+ 长期记忆(偏好/事实,RAG 检索)+ scratch(当前任务)(参考 xg 三层 + luke 四层架构)
|
- [ ] 语义搜索:基于 embedding 的记忆检索
|
||||||
- [ ] 语义搜索:基于 embedding 的记忆检索(BGE-M3/Gemini embedding + Qdrant 向量库)
|
- [ ] 记忆合并:相似记忆 cosine >= 0.7 时 LLM 合并
|
||||||
- [ ] 记忆合并:新记忆与已有记忆 cosine ≥ 0.7 时,用 LLM 合并而非插入(参考 xg)
|
- [ ] 时间衰减:记忆按时间指数衰减加权
|
||||||
- [ ] 二次联想召回:第一轮直接检索 → 用 top-K 结果做第二轮关联检索,去重后合并(参考 xg/luke 2-pass recall)
|
- [ ] 自我反思:定期回顾对话质量,优化行为
|
||||||
- [ ] 时间衰减:记忆按时间指数衰减加权,近期记忆优先(参考 xg 30 天半衰期)
|
|
||||||
- [ ] 自我反思:定期回顾对话质量,优化自己的行为
|
|
||||||
|
|
||||||
### 知识图谱(参考 luke concept graph)
|
|
||||||
- [ ] 概念图:Aho-Corasick 模式匹配用户消息中的关键概念,自动注入相关知识
|
|
||||||
- [ ] update_concept tool:LLM 可动态添加/更新概念节点及关联关系
|
|
||||||
- [ ] LRU 缓存:内存中保持热门概念,微秒级匹配
|
|
||||||
|
|
||||||
### 工具系统
|
### 工具系统
|
||||||
- [x] spawn_agent(Claude Code 子代理)
|
- [ ] run_code:安全沙箱执行 Python/Shell
|
||||||
- [x] update_scratch / update_memory
|
- [ ] gen_image:图像生成
|
||||||
- [x] send_file / agent_status / kill_agent
|
- [ ] web_search:网页搜索 + 摘要(简单场景不必 spawn agent)
|
||||||
- [x] 外部脚本工具发现 (tools/ 目录)
|
|
||||||
- [ ] run_code tool:安全沙箱执行 Python/Shell 代码,捕获输出返回(参考 luke run_python)
|
|
||||||
- [ ] gen_image tool:调用图像生成 API(Gemini/FLUX/本地模型)
|
|
||||||
- [ ] gen_voice tool:TTS 语音合成,发送语音消息(参考 luke Elevenlabs / xg Fish-Speech)
|
|
||||||
- [ ] set_timer tool:LLM 可设置延迟/定时任务,到时触发回调(参考 luke timer 系统)
|
|
||||||
- [ ] web_search tool:网页搜索 + 摘要,不必每次都 spawn 完整 agent
|
|
||||||
|
|
||||||
### 感知能力
|
### 感知能力
|
||||||
- [ ] 链接预览/摘要
|
- [ ] 链接预览/摘要
|
||||||
- [ ] 语音转文字 (STT):接收语音消息后自动转写(当前 xg 用 FunASR,luke 用 Whisper)
|
|
||||||
|
|
||||||
### 交互体验
|
### 交互体验
|
||||||
- [ ] 语音回复 (TTS)
|
- [ ] Typing indicator
|
||||||
- [ ] 流式分句发送:长回复按句号/问号断句分批发送,体验更自然
|
- [ ] 语音回复(TTS → Telegram voice message)
|
||||||
- [ ] 多频道支持:同一 bot 核心逻辑支持 Telegram + WebSocket + HTTP(参考 luke MxN 多路复用架构)
|
- [ ] Inline keyboard 交互
|
||||||
|
|
||||||
### 上下文管理
|
### 上下文管理
|
||||||
- [ ] 智能上下文分配:system prompt / 记忆 / 历史消息 / 工具输出各占比可配置,预留 60-70% 给工具输出(参考 luke 保守分配策略)
|
- [ ] 智能上下文分配:token 预算制替代硬上限
|
||||||
- [ ] 对话历史滚动窗口优化:当前 100 条硬上限,可改为 token 预算制
|
- [ ] Context pruning:只裁工具输出,保留对话文本
|
||||||
|
|
||||||
|
### 可靠性
|
||||||
|
- [ ] API 重试策略(指数退避)
|
||||||
|
- [ ] 用量追踪
|
||||||
|
- [ ] Model failover
|
||||||
|
- [ ] Life Loop / API 调用超时保护
|
||||||
|
|||||||
36
src/life.rs
36
src/life.rs
@@ -1,13 +1,15 @@
|
|||||||
use std::sync::Arc;
|
use std::sync::Arc;
|
||||||
|
|
||||||
use teloxide::prelude::*;
|
use teloxide::prelude::*;
|
||||||
use tracing::{error, info};
|
use tracing::{error, info, warn};
|
||||||
|
|
||||||
use crate::config::{BackendConfig, Config};
|
use crate::config::{BackendConfig, Config};
|
||||||
use crate::state::AppState;
|
use crate::state::AppState;
|
||||||
use crate::stream::run_openai_streaming;
|
use crate::stream::run_openai_with_tools;
|
||||||
use crate::tools::compute_next_cron_fire;
|
use crate::tools::compute_next_cron_fire;
|
||||||
|
|
||||||
|
const LIFE_LOOP_TIMEOUT_SECS: u64 = 120;
|
||||||
|
|
||||||
pub async fn life_loop(bot: Bot, state: Arc<AppState>, config: Arc<Config>) {
|
pub async fn life_loop(bot: Bot, state: Arc<AppState>, config: Arc<Config>) {
|
||||||
info!("life loop started");
|
info!("life loop started");
|
||||||
let mut interval = tokio::time::interval(std::time::Duration::from_secs(30));
|
let mut interval = tokio::time::interval(std::time::Duration::from_secs(30));
|
||||||
@@ -39,8 +41,7 @@ pub async fn life_loop(bot: Bot, state: Arc<AppState>, config: Arc<Config>) {
|
|||||||
if inner.is_empty() { "(空)" } else { &inner }
|
if inner.is_empty() { "(空)" } else { &inner }
|
||||||
));
|
));
|
||||||
system_text.push_str(
|
system_text.push_str(
|
||||||
"\n\n你可以使用工具来完成任务。你可以选择发消息给用户,也可以选择什么都不做(直接回复空文本)。\
|
"\n\n你可以使用工具来完成任务。可以用 update_inner_state 更新你的内心状态。\
|
||||||
可以用 update_inner_state 更新你的内心状态。\
|
|
||||||
输出格式:纯文本或基础Markdown,不要LaTeX或特殊Unicode。",
|
输出格式:纯文本或基础Markdown,不要LaTeX或特殊Unicode。",
|
||||||
);
|
);
|
||||||
|
|
||||||
@@ -49,24 +50,36 @@ pub async fn life_loop(bot: Bot, state: Arc<AppState>, config: Arc<Config>) {
|
|||||||
serde_json::json!({"role": "user", "content": format!("[timer] {label}")}),
|
serde_json::json!({"role": "user", "content": format!("[timer] {label}")}),
|
||||||
];
|
];
|
||||||
|
|
||||||
// call LLM (no tools for now — keep life loop simple)
|
|
||||||
if let BackendConfig::OpenAI {
|
if let BackendConfig::OpenAI {
|
||||||
ref endpoint,
|
ref endpoint,
|
||||||
ref model,
|
ref model,
|
||||||
ref api_key,
|
ref api_key,
|
||||||
} = config.backend
|
} = config.backend
|
||||||
{
|
{
|
||||||
match run_openai_streaming(endpoint, model, api_key, &messages, &bot, chat_id)
|
// synthetic session id for life loop (not tied to any real chat session)
|
||||||
.await
|
let sid = format!("life-{chat_id_raw}");
|
||||||
{
|
|
||||||
Ok(response) => {
|
let result = tokio::time::timeout(
|
||||||
|
std::time::Duration::from_secs(LIFE_LOOP_TIMEOUT_SECS),
|
||||||
|
run_openai_with_tools(
|
||||||
|
endpoint, model, api_key, messages, &bot, chat_id, &state, &sid,
|
||||||
|
&config, true,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
|
||||||
|
match result {
|
||||||
|
Ok(Ok(response)) => {
|
||||||
if !response.is_empty() {
|
if !response.is_empty() {
|
||||||
info!(timer_id, "life loop sent response ({} chars)", response.len());
|
info!(timer_id, "life loop response ({} chars)", response.len());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Ok(Err(e)) => {
|
||||||
error!(timer_id, "life loop LLM error: {e:#}");
|
error!(timer_id, "life loop LLM error: {e:#}");
|
||||||
}
|
}
|
||||||
|
Err(_) => {
|
||||||
|
warn!(timer_id, "life loop LLM timeout after {LIFE_LOOP_TIMEOUT_SECS}s");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -79,7 +92,6 @@ pub async fn life_loop(bot: Bot, state: Arc<AppState>, config: Arc<Config>) {
|
|||||||
state.cancel_timer(*timer_id).await;
|
state.cancel_timer(*timer_id).await;
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
// one-shot: delete after firing
|
|
||||||
state.cancel_timer(*timer_id).await;
|
state.cancel_timer(*timer_id).await;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -403,7 +403,8 @@ async fn handle_inner(
|
|||||||
let conv = state.load_conv(&sid).await;
|
let conv = state.load_conv(&sid).await;
|
||||||
let persona = state.get_config("persona").await.unwrap_or_default();
|
let persona = state.get_config("persona").await.unwrap_or_default();
|
||||||
let memory_slots = state.get_memory_slots().await;
|
let memory_slots = state.get_memory_slots().await;
|
||||||
let system_msg = build_system_prompt(&conv.summary, &persona, &memory_slots);
|
let inner = state.get_inner_state().await;
|
||||||
|
let system_msg = build_system_prompt(&conv.summary, &persona, &memory_slots, &inner);
|
||||||
|
|
||||||
let mut api_messages = vec![system_msg];
|
let mut api_messages = vec![system_msg];
|
||||||
api_messages.extend(conv.messages);
|
api_messages.extend(conv.messages);
|
||||||
@@ -514,7 +515,9 @@ fn build_prompt(
|
|||||||
}
|
}
|
||||||
|
|
||||||
async fn transcribe_audio(whisper_url: &str, file_path: &Path) -> Result<String> {
|
async fn transcribe_audio(whisper_url: &str, file_path: &Path) -> Result<String> {
|
||||||
let client = reqwest::Client::new();
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(std::time::Duration::from_secs(60))
|
||||||
|
.build()?;
|
||||||
let url = format!("{}/v1/audio/transcriptions", whisper_url.trim_end_matches('/'));
|
let url = format!("{}/v1/audio/transcriptions", whisper_url.trim_end_matches('/'));
|
||||||
let file_bytes = tokio::fs::read(file_path).await?;
|
let file_bytes = tokio::fs::read(file_path).await?;
|
||||||
let file_name = file_path
|
let file_name = file_path
|
||||||
|
|||||||
@@ -248,7 +248,6 @@ impl AppState {
|
|||||||
.unwrap_or_default()
|
.unwrap_or_default()
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(dead_code)] // used by life loop tools (coming soon)
|
|
||||||
pub async fn set_inner_state(&self, content: &str) {
|
pub async fn set_inner_state(&self, content: &str) {
|
||||||
let db = self.db.lock().await;
|
let db = self.db.lock().await;
|
||||||
let _ = db.execute(
|
let _ = db.execute(
|
||||||
|
|||||||
@@ -120,7 +120,10 @@ pub async fn run_openai_with_tools(
|
|||||||
config: &Arc<Config>,
|
config: &Arc<Config>,
|
||||||
is_private: bool,
|
is_private: bool,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
let client = reqwest::Client::new();
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(std::time::Duration::from_secs(120))
|
||||||
|
.build()
|
||||||
|
.unwrap();
|
||||||
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
||||||
let tools = discover_tools();
|
let tools = discover_tools();
|
||||||
|
|
||||||
@@ -570,7 +573,10 @@ pub async fn run_openai_streaming(
|
|||||||
bot: &Bot,
|
bot: &Bot,
|
||||||
chat_id: ChatId,
|
chat_id: ChatId,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
let client = reqwest::Client::new();
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(std::time::Duration::from_secs(120))
|
||||||
|
.build()
|
||||||
|
.unwrap();
|
||||||
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
||||||
|
|
||||||
let body = serde_json::json!({
|
let body = serde_json::json!({
|
||||||
@@ -685,7 +691,7 @@ pub async fn run_openai_streaming(
|
|||||||
Ok(accumulated)
|
Ok(accumulated)
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn build_system_prompt(summary: &str, persona: &str, memory_slots: &[(i32, String)]) -> serde_json::Value {
|
pub fn build_system_prompt(summary: &str, persona: &str, memory_slots: &[(i32, String)], inner_state: &str) -> serde_json::Value {
|
||||||
let mut text = if persona.is_empty() {
|
let mut text = if persona.is_empty() {
|
||||||
String::from("你是一个AI助手。")
|
String::from("你是一个AI助手。")
|
||||||
} else {
|
} else {
|
||||||
@@ -708,6 +714,11 @@ pub fn build_system_prompt(summary: &str, persona: &str, memory_slots: &[(i32, S
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !inner_state.is_empty() {
|
||||||
|
text.push_str("\n\n## 你的内在状态\n");
|
||||||
|
text.push_str(inner_state);
|
||||||
|
}
|
||||||
|
|
||||||
if !summary.is_empty() {
|
if !summary.is_empty() {
|
||||||
text.push_str("\n\n## 之前的对话总结\n");
|
text.push_str("\n\n## 之前的对话总结\n");
|
||||||
text.push_str(summary);
|
text.push_str(summary);
|
||||||
@@ -747,7 +758,10 @@ pub async fn summarize_messages(
|
|||||||
)
|
)
|
||||||
};
|
};
|
||||||
|
|
||||||
let client = reqwest::Client::new();
|
let client = reqwest::Client::builder()
|
||||||
|
.timeout(std::time::Duration::from_secs(120))
|
||||||
|
.build()
|
||||||
|
.unwrap();
|
||||||
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
let url = format!("{}/chat/completions", endpoint.trim_end_matches('/'));
|
||||||
|
|
||||||
let body = serde_json::json!({
|
let body = serde_json::json!({
|
||||||
|
|||||||
22
src/tools.rs
22
src/tools.rs
@@ -103,6 +103,20 @@ pub fn discover_tools() -> serde_json::Value {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}),
|
}),
|
||||||
|
serde_json::json!({
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "update_inner_state",
|
||||||
|
"description": "更新你的内在状态。这是你自己的持续意识,跨会话保留,Life Loop 和对话都能看到。记录你对当前情况的理解、正在跟踪的事、对 Fam 状态的感知等。",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"content": {"type": "string", "description": "完整的内在状态文本(替换之前的)"}
|
||||||
|
},
|
||||||
|
"required": ["content"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}),
|
||||||
serde_json::json!({
|
serde_json::json!({
|
||||||
"type": "function",
|
"type": "function",
|
||||||
"function": {
|
"function": {
|
||||||
@@ -275,6 +289,11 @@ pub async fn execute_tool(
|
|||||||
Err(e) => format!("Failed to send file: {e:#}"),
|
Err(e) => format!("Failed to send file: {e:#}"),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
"update_inner_state" => {
|
||||||
|
let content = args["content"].as_str().unwrap_or("");
|
||||||
|
state.set_inner_state(content).await;
|
||||||
|
format!("Inner state updated ({} chars)", content.len())
|
||||||
|
}
|
||||||
"update_scratch" => {
|
"update_scratch" => {
|
||||||
let content = args["content"].as_str().unwrap_or("");
|
let content = args["content"].as_str().unwrap_or("");
|
||||||
state.push_scratch(content).await;
|
state.push_scratch(content).await;
|
||||||
@@ -480,7 +499,8 @@ pub async fn agent_wakeup(
|
|||||||
let conv = state.load_conv(sid).await;
|
let conv = state.load_conv(sid).await;
|
||||||
let persona = state.get_config("persona").await.unwrap_or_default();
|
let persona = state.get_config("persona").await.unwrap_or_default();
|
||||||
let memory_slots = state.get_memory_slots().await;
|
let memory_slots = state.get_memory_slots().await;
|
||||||
let system_msg = build_system_prompt(&conv.summary, &persona, &memory_slots);
|
let inner = state.get_inner_state().await;
|
||||||
|
let system_msg = build_system_prompt(&conv.summary, &persona, &memory_slots, &inner);
|
||||||
let mut api_messages = vec![system_msg];
|
let mut api_messages = vec![system_msg];
|
||||||
api_messages.extend(conv.messages);
|
api_messages.extend(conv.messages);
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user