Build Ollama

Install Ollama

official website

More tutorialson Github

Windows Document

Default port: 11434

Shell command

List Model

ollama list

Running Model

ollama ps

Display Model Details

ollama show llama3.1

visit

RESTful API

reference

Complete Document

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt":"Why is the sky blue?",
   "stream": false
}'

OpenAI-compatible interface

Document

LangChain-TS version

library

@langchain/ollama

installation library

pnpm add @langchain/ollama

Document

course

Quick Start

import { StringOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOllama } from "@langchain/ollama";

export async function main() {
  const model = new ChatOllama({
    baseUrl: "http://ai.cdatatec.cn:11434",
    model: "llama3.1",
    temperature: 0,
  });
  const prompt = ChatPromptTemplate.fromTemplate(
    "讲一个关于{topic}的笑话"
  );
  const parser = new StringOutputParser();

  const chain = prompt.pipe(model).pipe(parser);
  const stream = await chain.stream({
    topic: "parrot",
  });
  for await (const chunk of stream) {
    console.log(`${chunk}|`);
  }
}

Configure the Ollama server address through baseUrl

Modality

import { ChatOllama } from "@langchain/ollama";
import { HumanMessage } from "@langchain/core/messages";
import * as fs from "fs/promises";

export async function main() {
  console.log(process.cwd());
  const imageData = await fs.readFile("src/ollama/hot-dog.png");

  const llmForMultiModal = new ChatOllama({
    model: "llava-llama3",
    baseUrl: "http://ai.cdatatec.cn:11434",
  });

  const multiModalRes = await llmForMultiModal.invoke([
    new HumanMessage({
      content: [
        {
          type: "text",
          text: "图片里有什么？",
        },
        {
          type: "image_url",
          image_url: `data:image/jpeg;base64,${imageData.toString("base64")}`,
        },
      ],
    }),
  ]);
  console.log(multiModalRes);
  console.log(multiModalRes.content);
}

Common Model

support visual

Model	Description	Sizes
llava	A novel Grande Modality Model for End-To-End Training that combines Visual coder and Vicuna for general visual and language understanding.	7B, 13B, 34B
llava-llama3	A LLaVA Model Fine-Tuning from the Llama 3 Instruct performed better in multiple Baseline Test.	8B
llava-phi3	A new small LLaVA Model derived from the Phi 3 Mini Fine-Tuning.	3B
moondream	Moondream2 is a small visual Language Model designed to run efficiently on Edge Device.	Not specified
bakllava	A Modality Model enhanced from the Mistral 7B Base Model and the LLaVA Architecture.	7B

Support Function Call

Model	Parameter	Description
mistral	7B	The 7B Model released by Mistral AI has been updated to version 0.3.
llama3.1	8B, 70B	Llama 3.1 is Meta's new state-of-the-art Model
mixtral	8x7B, 8x22B	The Expert Mixed (MoE) Model set with open Weight developed by Mistral AI has two Parameter scales of 8x7B and 8x22B.
mistral-nemo	12B	The most advanced 12B Model developed by Mistral AI in collaboration with NVIDIA has a context length of 128k.
command-r-plus	104B	Command R+ is a powerful, scalable, Grande Language Model built specifically for enterprise use cases that specialize in the real world.
mistral-large	123B	Mistral Large 2 is Mistral's new flagship Model, which has significantly improved capabilities in code generation, mathematics and reasoning. It has a 128k Context Window and and if you do supports dozens of language.
llama3-groq-tool-use	8B, 70B	A series of Model developed by Groq represent a major advancement in Open-Source AI's tool use/Function invocation capabilities.
hermes3	8B	Hermes 3 is the latest version of the flagship Hermes seriesLLM developed by Nous Research.
firefunction-v2	70B	Based on Llama 3 's open Weight Function Call Model, its Function Call Capability is comparable to GPT-4.