Production-gradeLLM tools

programmatically

Due to the powerful understanding and generation capabilities of this wave of LLMs, command-style programming that pays attention to details seems to be no longer needed, and Statement programming that focuses on process or pipeline capabilities of business logic orchestration has become the mainstream "programming" method. Therefore, we have seen that rookies including LCEL, DSPy, and Semantic-kernel have begun to pay attention "upwards" and focus their main energy on business logic orchestration capabilities. Perhaps this indicates a DAG approach similar to Airflow. Will it become a new generation of programming normal form?

totality

Prompt tool

LangSmith or Azure Prompt Flow

Programming language

Python or Javascript

Development framework/orchestration tool

Langchain or Semitic Kernel

API framework/service box frame work

FastAPI or Next.js

Cloud hosting vendor

Render (AWS Lamada) or Vercel

BuildLLM applications

classification	Name	Star Quantity	remark	Learning
frame	LangChain	77	[Required]	*
	LlamaIndex	29	【必学】专注于RAG，Python，JS SDK比较差	*
	Semantic-kernel	17	【不确定？】C#，Python，Java	?
	haystack	13	有了LangChain可能就不需要这个了
	7	Too academic?	太学术了？The framework for programming-not prompting-foundation models
Agent	SuperAgent	4	This feels very professional, and I will Learning again when I have time	?
	Rift	3	专门针对程序员
	AutoGen	22
	Camel	4
GPTs	OpenGPTs	6	Produced by LangChain
low-code	LangFlow		LangChain visualization
	LiteLLM
	Flowise	21
	Langflow	14
	PromptChainer	商业	感觉做的很专业
RAG	vectara	closed source	Very professional RAG commercial soft ware; there are also Open-Source projects; users can upload PDF, Word, PPT, RTF and other file Data to the Vectara platform to build a Data Searching engine.

Archiving and Observation

classification	Name	Star Quantity	remark
observe	Obsidian Copilot		An interesting approach to how to use Semantic Searching and the BM25 implementation of OpenSearch
	Tanuki		一个使用装饰器进行数据验证的 LLM 框架
	griptape	1.5	一个具有稍微更好的内部编码标准的 langchain 替代品
	16	Controlling modern Language Model more Valid Earth than traditional prompts or links	比传统的提示或链接更有效地控制现代语言模型
	intructor	3.6
	0.5	Another alternative to langchain, but has an interesting approach to defining workflows	langchain 的另一个替代品，但在定义工作流程方面有一个有趣的方法
	6.5	There are derivative products for chat, medical/scientific paper workflow, developer semantic Searching, and title and story text semantic Searching	有用于聊天、医学/科学论文工作流程、开发者语义搜索以及标题和故事文本语义搜索的衍生产品
	5.7	A simple, lightweightLLM package	一个简单、轻量级的 LLM 封装器
pigeonhole	Chidori	1.2
	AutoChains		已经3个月没更新
	MiniChain	1.2	已经3个月没更新
	SimpleAIChat	3.3	已经7个月没更新

monitor and analyze

LLM services at the production level require:

Debugging Prompt
Prompt version management
Metric related to the dependence of the test/verification system
Data Set Management
Various Metric monitoring and statistics: visit volume, response time, Token fee, etc.

classification	Name	Star Quantity
SaaS	LangSmith	N/A
Open-Source	LangFuse	2
Open-Source	Prompt Flow	7
	Helicone	1.2

Production-levelLLM App maintenance platform

LangSmith: LangChain's official platform, SaaS service,not Open-Source
LangFuse:Open-Source+ SaaS, LangSmith replacement, Ensemble LangChain integration, docking with OpenAI API(2.1K)
Prompt Flow: MicrosoftCorp Open-Source + Azure AI cloud service, which can Ensemble Semantic Kernel (7.4K)

Prompt Flow

Learning

https://microsoft.github.io/promptflow/how-to-guides/quick-start.html

UI component

LangUI(1.6K)

agentlabs(0.5K)

other

classification	Name	Star Quantity	remark
LCEL
Python template language	Jinja2

reference

classification	Name	Star Quantity	remark
frame	Obsidian Copilot		An interesting approach to how to use Semantic Searching and the BM25 implementation of OpenSearch
	Tanuki		个使用装饰器进行数据验证的 LLM 框架
	LiteLLM	5.7	一个简单、轻量级的 LLM 封装器
	AutoChain	1.7	langchain 的另一个替代品
	griptape	1.5	一个具有稍微更好的内部编码标准的 langchain 替代品
	txtai	6.5	has spinoffs for chat, workflows for medical/scientific papers, semantic search for developers and semantic search for headlines and story text 有用于聊天、医学/科学论文工作流程、开发者语义搜索以及标题和故事文本语义搜索的衍生产品
	Llmflows	0.5	langchain 的另一个替代品，但在定义工作流程方面有一个有趣的方法
	guidance	16	比传统的提示或链接更有效地控制现代语言模型	*
library	intructor	3.6		*

E2B：Frameworks and tools for AI products

Category	Tools/Platforms
Monitoring, Observability, Analytics	Langsmith, Helicone, langfuse, context
Frontend	AgentLabs
Runtime for LLMs	E2B
Building frameworks & platforms	Langchain, Haystack by deepset, Superagent, Poe, GenWorlds, LiteLLM, Rift by Morph, AutoGen, Vercel AI SDK, OpenGPTs, Huggging Face Agents, Fixie
Data integration, memory management	vectara, LlamaIndex, ABACUS.AI, cadea, SID Memory
API and routers for LLMs	Genoss GPT, Martian, OpenRouter
Libraries for building AI products	CAMEL, ChatterBot (hasn't been updated in 5 annum)
Orchestration	AGIXT
Building & deploying LLMs	BANANA

frameworks and tools for AI products