L lama In dex 고급 사용
프 롬 프트 사용자 정의
두 개의 중요한 템 플 릿
응답 모 드가 컴 팩 트 인 경우 text _ q a _ template 템 플 릿 을 사용한 다음 ref ine _ template 템 플 릿 을 사용합니다 .
LL M 창의 크기
여기에 정의 가 있다 .
/ Us ers / xxx / ana con da 3 / en vs / LI 31 1 - h / lib / py thon 3. 11 / site - pack ages / l lama _ index / ll ms / op enai / utili ties . py
AZURE_TURBO_MODELS: Dict[str, int] = {
"gpt-35-turbo-16k": 16384,
"gpt-35-turbo": 4096,
# 0125 (2024) model (JSON mode)
"gpt-35-turbo-0125": 16385,
# 1106 model (JSON mode)
"gpt-35-turbo-1106": 16384,
# 0613 models (function calling):
"gpt-35-turbo-0613": 4096,
"gpt-35-turbo-16k-0613": 16384,
}
Lang C hain 을 사용한 LL M
import os
from langchain_community.chat_models.moonshot import MoonshotChat
from langchain_core.messages import HumanMessage, SystemMessage
from llama_index.llms.langchain import LangChainLLM
os.environ["MOONSHOT_API_KEY"] = "sk-xxx"
llm = LangChainLLM(llm=MoonshotChat(model_name="moonshot-v1-128k"))
response_gen = llm.stream_complete("你是谁？")
for delta in response_gen:
print(delta.delta, end="")
Moon shot Chat 을 사용할 때 오류 가 발생 하기 쉽습니다 .
File "/Users/yanghaibin/anaconda3/envs/LI311-h/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 581, in create
return self._post(
^^^^^^^^^^^
File "/Users/yanghaibin/anaconda3/envs/LI311-h/lib/python3.11/site-packages/openai/_base_client.py", line 1233, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/yanghaibin/anaconda3/envs/LI311-h/lib/python3.11/site-packages/openai/_base_client.py", line 922, in request
return self._request(
^^^^^^^^^^^^^^
File "/Users/yanghaibin/anaconda3/envs/LI311-h/lib/python3.11/site-packages/openai/_base_client.py", line 1013, in _request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': 'Invalid request: Your request exceeded model token limit: 8192', 'type': 'invalid_request_error'}}