Add How to use
+145
@@ -0,0 +1,145 @@
|
||||
# How to use NeuroDock
|
||||
|
||||
NeuroDock is a multi-modular Ollama wrapper. many of the modules in the repo are so called CTP's so we will ignore them, these are the important once
|
||||
|
||||
- Core
|
||||
- API
|
||||
- Plugin-API
|
||||
- Display
|
||||
|
||||
These can be described as
|
||||
|
||||
## Core
|
||||
The core system/module of NeuroDock, it's the actual wrapper around Ollama's REST API and is what this usage guide will focus on
|
||||
|
||||
## API
|
||||
This provides a RESTful API to connect to a NeuroDock host to add tools via a REST definition and webhook callbacks. This has a sepereat wiki page but is out of date, [API docs](https://git.server.4zellen.se/neurodock/NeuroDock/wiki/API-Docs)
|
||||
|
||||
## Plugin-API
|
||||
This is the provider of RPCP plugin system, or more correctly the end usage as Core contains the actual loader.
|
||||
This can be used to define a plugin that is expected to be more modular, as in it can be loaded at runtime and is not pre-compiled into the application
|
||||
|
||||
## Display
|
||||
This is a example, as well as a sort of Proof of Concept on how to implement NeuroDock
|
||||
|
||||
# How to Implement
|
||||
|
||||
## Add maven dependency
|
||||
|
||||
### Maven
|
||||
```pom.xml
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>gitea</id>
|
||||
<url>https://git.server.4zellen.se/api/packages/neurodock/maven</url>
|
||||
</repository>
|
||||
</repositories>
|
||||
|
||||
<dependency>
|
||||
<groupId>me.neurodock</groupId>
|
||||
<artifactId>Core</artifactId>
|
||||
<version>1.9.1</version><---This might have been updated since time of writing--->
|
||||
</dependency>
|
||||
```
|
||||
|
||||
### Gradle (Groovy)
|
||||
```build.gradle
|
||||
repositories {
|
||||
maven {
|
||||
url = "https://git.server.4zellen.se/api/packages/neurodock/maven"
|
||||
name = "gitea"
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
implementation("me.neurodock:Core:1.9.1") // This might have been updated since time of writing
|
||||
}
|
||||
```
|
||||
|
||||
## Implementation
|
||||
|
||||
Core contains multiple importent parts but you mainly need to care about 3 of them for a simple implementation
|
||||
|
||||
`me.neurodock.core.Core`: This is the actual core component, and will to the Ollama communication
|
||||
`me.neurodock.ollama.OllamaObject`: This is a Ollama object representation and is the equivalent of the main JSON object defined in the Ollama REST API
|
||||
`me.neurodock.core.PrintAdvanceMessageHandler`: The print handler, this can be substituted with `me.neurodock.core.PrintMessageHandler` if you want a simpler implementation
|
||||
|
||||
How to use these?
|
||||
|
||||
`Core` can be the host of multible OllamaObject instences although as of current implementation this is a bit more invovled then it shuld need to be
|
||||
|
||||
How to create your instance of `Core`
|
||||
```java
|
||||
// This should be replaced by a proper implementation ideally of the PrintAdvanceMessageHandler instead
|
||||
PrintMessageHandler printHandler = new PrintMessageHandler(){
|
||||
@Override
|
||||
public void printMessage(String message) {
|
||||
System.out.println(">> "+message);
|
||||
}
|
||||
@Override
|
||||
public boolean color() {
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
Core.setDataDirectory("project-name",false); // Optional, but if not ran things will be stored under the "AI-chat" name
|
||||
|
||||
Core core = new Core(printHandler);
|
||||
```
|
||||
|
||||
Now you have made your `Core` instance and is ready to add your `OllamaObject` instance
|
||||
|
||||
Now this is a bit more involved, it can be as simple as just setting the model, but as involved as defining a full system prompt, please familiarize yourself with the [OllamaObject.Builder](https://git.server.4zellen.se/neurodock/NeuroDock/src/branch/master/Core/src/main/java/me/neurodock/ollama/OllamaObject.java#L371) to know what _your_ implementation needs
|
||||
|
||||
```java
|
||||
// replace with setOllamaObjectNoMemory if you do not what to have memory functions included
|
||||
core.setOllamaObject(
|
||||
OllamaObject.builder()
|
||||
.setModel("llama3.2") // OBS! NeuroDock is as of version 1.9.10 not equipped to fetch models tor you. Instead open up a terminal and run the ollama command or use other means to fetch an ollama model.
|
||||
.keep_alive(5) // This refers to how long Ollama shuld keep the model loaded, this should not be too long as it is still held when you close down the application, but if it's too short in the time you are typing up your next message the model can get un-loaded resulting in slower responses
|
||||
.addMessage(new SystemMessage("You are an helpful AI-Assistentent")) // This is a simple way to add a System prompt, refer to setSystemPrompt and LLMSystemPrompt for a proper implementation of a system prompt!
|
||||
.build());
|
||||
```
|
||||
|
||||
Now you are ready to actually use the wrapper. How you get the user input, or generates your messages is up to you.
|
||||
|
||||
Now depending on your creation of the `OllamaObject` instance you can do this in different ways
|
||||
|
||||
### Streaming
|
||||
```java
|
||||
core.getOllamaObject().addMessage(new OllamaMessage(
|
||||
OllamaMessageRole.USER, // This represents _who_ is "speaking", enum values are USER, SYSTEM, TOOL, ASSISTENCE. for normal messages you shuld ONLY use USER.
|
||||
message // The message that shuld be sent
|
||||
));
|
||||
core.qurryOllama(/* Your implementation of a Consumer<JSONObject> to handle the chunks */).thenApply(json -> {
|
||||
core.handleResponse(json); // THIS SHOULD ALWAYS BE CALLED! This calls the core's implementation of handling responses, which is printing out the messages, and handling tool callings. However if you know better (_you probably don't_) you can implement your own...
|
||||
}).join(); // The qurryOllama returns a CompletableFuture, we use .join() to hold this thread until it has resolved.
|
||||
```
|
||||
|
||||
## Tool Implementation
|
||||
|
||||
This will explain how to register a CTP tool
|
||||
|
||||
First of you need to create your `me.neurodock.ollama.OllamaFunctionTool` implementation
|
||||
|
||||
```java
|
||||
class YourToolName extends OllamaFunctionTool {
|
||||
@Override
|
||||
public @NotNull String name() {
|
||||
return ""; // This is the name of your tool. These are lower snake_case due by convention of NeuroDock
|
||||
}
|
||||
|
||||
@Override
|
||||
public @NotNull OllamaPerameter parameters() {
|
||||
return null; // Make use of the OllamaPerameterBuilder to build your parameters
|
||||
}
|
||||
|
||||
@Override
|
||||
public @NotNull OllamaToolResponse function(OllamaFunctionArgument... args) throws OllamaToolErrorException {
|
||||
// A key note. NEVER throw any exceptions but OllamaToolErrorException, OllamaToolErrorException have a constructor to take in an offending exception but this method should never throw anything else!
|
||||
return null; // Here is where you get an array of the OllamaFunctionArgument's defined in your parameters.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The code is documented and should be a good base for you to look thru. Best of luck!
|
||||
Reference in New Issue
Block a user