Ollama使用

Ollama是一个用于在本地计算机上运行大模型的软件

软件运行后监听11434端口，自己写的程序要调大模型就用这个端口

ollama命令

ollama list：显示模型列表

ollama show：显示模型的信息

ollama pull：拉取模型

ollama push：推送模型

ollama cp：拷贝一个模型

ollama rm：删除一个模型

ollama run：运行一个模型

ollama全是命令行下操作，所以结合web客户端界面使用【安装可选】

主流的web工具

1 Openwebui

2 LobeChat,功能强大，可调用Ollama的模型，也可调用openai,google的等，在设置界面中配置url和key即可

spring Ai框架调用

1 pom.xml，注意添加的依赖和配置了仓库

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>3.2.5</version><relativePath/> <!-- lookup parent from repository --></parent><groupId>com.example</groupId><artifactId>spring-ai-ollama</artifactId><version>0.0.1-SNAPSHOT</version><name>spring-ai-ollama</name><description>spring-ai-ollama</description><properties><java.version>17</java.version><spring-ai.version>0.8.1</spring-ai.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>io.springboot.ai</groupId><artifactId>spring-ai-ollama-spring-boot-starter</artifactId><version>1.0.0</version></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-devtools</artifactId><scope>runtime</scope><optional>true</optional></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency></dependencies><dependencyManagement><dependencies><dependency><groupId>org.springframework.ai</groupId><artifactId>spring-ai-bom</artifactId><version>${spring-ai.version}</version><type>pom</type><scope>import</scope></dependency></dependencies></dependencyManagement><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId><configuration><excludes><exclude><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId></exclude></excludes></configuration></plugin></plugins></build><repositories><repository><id>spring-milestones</id><name>Spring Milestones</name><url>https://repo.spring.io/milestone</url><snapshots><enabled>false</enabled></snapshots></repository></repositories></project>

2 yml配置，写自己的 Ollama 地址，模型用哪个，先用Ollama去下载

spring: application: name: spring-ai-ollama ai: ollama: base-url: http://120.55.99.218:11434 chat: options: model: gemma:7b

3 测试

import org.springframework.ai.chat.ChatResponse;import org.springframework.ai.chat.messages.AssistantMessage;import org.springframework.ai.chat.prompt.Prompt;import org.springframework.ai.chat.prompt.PromptTemplate;import org.springframework.ai.ollama.OllamaChatClient;import org.springframework.ai.ollama.api.OllamaOptions;import org.springframework.beans.factory.annotation.Autowired;import org.springframework.web.bind.annotation.*;@RestControllerpublic class AiController { @Autowired private OllamaChatClient ollamaChatClient; @GetMapping(value = "/chat_1") public String chat_1(@RequestParam(value = "message") String message) { String call = ollamaChatClient.call(message); System.out.println("模型回答 = " + call); return call; } @GetMapping(value = "/chat_2") public Object chat_2(@RequestParam(value = "message") String message) { Prompt prompt = new Prompt( message, OllamaOptions.create() //代码中配置，会覆盖application.yml中的配置 .withModel("gemma:7b") //使用什么大模型 .withTemperature(0.9F) //温度高，更发散,准确性降低，温度低，更保守，准确性高 ); ChatResponse call = ollamaChatClient.call(prompt); AssistantMessage output = call.getResult().getOutput(); System.out.println("模型回答 = " + output.getContent()); return output; } @GetMapping("/chat_3/{size}") public String chatYear(@PathVariable("size") Integer size) { String message = "随便写一句话,{size} 字以内"; PromptTemplate promptTemplate = new PromptTemplate(message); promptTemplate.add("size", size); System.out.println(promptTemplate.render()); return ollamaChatClient.call(promptTemplate.render()); }}