1. 聊天模型可觀測性Chat Model Observability部分ChatModel/StreamingChatModel實現(xiàn)見「Observability」列支持配置ChatModelListener監(jiān)聽三類事件事件回調(diào)方法時機請求onRequest(ChatModelRequestContext)調(diào)用 LLM 提供商 API之前響應(yīng)onResponse(ChatModelResponseContext)成功收到響應(yīng)后立即調(diào)用錯誤onError(ChatModelErrorContext)出錯時僅一次事件屬性遵循 OpenTelemetry Generative AI 語義約定。事件包含的屬性請求Request消息messages模型model溫度TemperatureTop P最大 tokens 數(shù)工具Tools響應(yīng)格式responseFormat等等響應(yīng)Response助手消息AiMessageID模型Token 使用情況TokenUsage結(jié)束原因finishReason等等2. ChatModelListener 完整示例ChatModelListenerlistenernewChatModelListener(){OverridepublicvoidonRequest(ChatModelRequestContextrequestContext){ChatRequestchatRequestrequestContext.chatRequest();ListChatMessagemessageschatRequest.messages();System.out.println(messages);ChatRequestParametersparameterschatRequest.parameters();System.out.println(parameters.modelName());System.out.println(parameters.temperature());System.out.println(parameters.topP());System.out.println(parameters.topK());System.out.println(parameters.frequencyPenalty());System.out.println(parameters.presencePenalty());System.out.println(parameters.maxOutputTokens());System.out.println(parameters.stopSequences());System.out.println(parameters.toolSpecifications());System.out.println(parameters.toolChoice());System.out.println(parameters.responseFormat());// 提供商特有參數(shù)以 OpenAI 為例需 instanceof 判斷if(parametersinstanceofOpenAiChatRequestParametersopenAiParameters){System.out.println(openAiParameters.maxCompletionTokens());System.out.println(openAiParameters.logitBias());System.out.println(openAiParameters.parallelToolCalls());System.out.println(openAiParameters.seed());System.out.println(openAiParameters.user());System.out.println(openAiParameters.store());System.out.println(openAiParameters.metadata());System.out.println(openAiParameters.serviceTier());System.out.println(openAiParameters.reasoningEffort());}System.out.println(requestContext.modelProvider());// 在 onRequest / onResponse / onError 之間傳遞數(shù)據(jù)MapObject,ObjectattributesrequestContext.attributes();attributes.put(my-attribute,my-value);}OverridepublicvoidonResponse(ChatModelResponseContextresponseContext){ChatResponsechatResponseresponseContext.chatResponse();AiMessageaiMessagechatResponse.aiMessage();System.out.println(aiMessage);ChatResponseMetadatametadatachatResponse.metadata();System.out.println(metadata.id());System.out.println(metadata.modelName());System.out.println(metadata.finishReason());if(metadatainstanceofOpenAiChatResponseMetadataopenAiMetadata){System.out.println(openAiMetadata.created());System.out.println(openAiMetadata.serviceTier());System.out.println(openAiMetadata.systemFingerprint());}TokenUsagetokenUsagemetadata.tokenUsage();System.out.println(tokenUsage.inputTokenCount());System.out.println(tokenUsage.outputTokenCount());System.out.println(tokenUsage.totalTokenCount());if(tokenUsageinstanceofOpenAiTokenUsageopenAiTokenUsage){System.out.println(openAiTokenUsage.inputTokensDetails().cachedTokens());System.out.println(openAiTokenUsage.outputTokensDetails().reasoningTokens());}ChatRequestchatRequestresponseContext.chatRequest();System.out.println(chatRequest);System.out.println(responseContext.modelProvider());MapObject,ObjectattributesresponseContext.attributes();System.out.println(attributes.get(my-attribute));}OverridepublicvoidonError(ChatModelErrorContexterrorContext){ThrowableerrorerrorContext.error();error.printStackTrace();ChatRequestchatRequesterrorContext.chatRequest();System.out.println(chatRequest);System.out.println(errorContext.modelProvider());MapObject,ObjectattributeserrorContext.attributes();System.out.println(attributes.get(my-attribute));}};ChatModelmodelOpenAiChatModel.builder().apiKey(System.getenv(OPENAI_API_KEY)).modelName(GPT_4_O_MINI).listeners(List.of(listener)).build();model.chat(Tell me a joke about Java);attributes 映射的作用attributesMapObject, Object用于在同一個ChatModelListener的onRequest→onResponse/onError之間傳遞信息如請求開始時間、traceId。在多個ChatModelListener之間傳遞信息。3. 監(jiān)聽器的工作方式重要規(guī)則調(diào)用順序與線程監(jiān)聽器以ListChatModelListener形式配置按迭代順序依次調(diào)用。同步調(diào)用運行在同一個線程流式場景例外見下。第一個監(jiān)聽器返回之前第二個不會被調(diào)用。調(diào)用次數(shù)onRequest()每次請求只調(diào)用一次LLM API 出錯重試時不會為每次重試重復(fù)調(diào)用。onResponse()只調(diào)用一次成功收到響應(yīng)后立即調(diào)用。onError()只調(diào)用一次重試時不會為每次重試重復(fù)調(diào)用。異常處理監(jiān)聽器方法拋出的異常以WARN級別記錄日志。后續(xù)監(jiān)聽器照常繼續(xù)執(zhí)行不會中斷。請求內(nèi)容是“最終請求”三個 Context 中拿到的ChatRequest是合并后的最終請求ChatModel上配置的默認(rèn)ChatRequestParameters 請求特定的ChatRequestParameters。StreamingChatModel 的特殊點onResponse()/onError()與onRequest()在不同線程中執(zhí)行。線程上下文如 MDC、ThreadLocal不會自動傳播→ 需要用attributes把數(shù)據(jù)從onRequest()傳到onResponse()/onError()。時序onResponse()早于StreamingChatResponseHandler.onCompleteResponse()。onError()早于StreamingChatResponseHandler.onError()。4. 在 Spring Boot 應(yīng)用中的可觀測性原文此處為小節(jié)標(biāo)題展開內(nèi)容待補充。實踐要點結(jié)合上文規(guī)則的落地建議把ChatModelListener注冊為 Spring Bean如Component或Bean在構(gòu)建ChatModel時通過.listeners(...)注入便于統(tǒng)一收集指標(biāo)/日志。在onRequest()中把traceId、開始時間等放入attributes在onResponse()/onError()中取出配合 Micrometer / OpenTelemetry 上報耗時onResponse/onError時間戳 ?onRequest時間戳Token 消耗tokenUsage.inputTokenCount()/outputTokenCount()/totalTokenCount()錯誤率onError計數(shù) errorContext.error()監(jiān)聽器內(nèi)不要拋異常異常只會被 WARN 吞掉應(yīng)自行 try-catch 并落日志避免影響監(jiān)控鏈路完整性。速查小結(jié)三個回調(diào)onRequest請求前一次、onResponse成功響應(yīng)一次、onError出錯一次重試不會重復(fù)觸發(fā)。一個橋梁attributes映射 —— 跨回調(diào)、跨監(jiān)聽器傳值流式場景線程不同必須靠它傳遞上下文。兩類參數(shù)通用ChatRequestParameters/ChatResponseMetadata 提供商特有如OpenAiChatRequestParameters、OpenAiChatResponseMetadata、OpenAiTokenUsage用instanceof取。線程非流式同線程同步流式onResponse/onError在另一線程且早于StreamingChatResponseHandler對應(yīng)回調(diào)。容錯監(jiān)聽器拋異常只記 WARN不影響后續(xù)監(jiān)聽器與主流程。語義約定屬性命名對齊 OpenTelemetry Generative AI 語義約定便于接入現(xiàn)有 APM。