Get chat completion from ollama server
get_ollama_chat_completion.Rdget_ollama_chat_completion get a chat completion from the ollama server API
Usage
get_ollama_chat_completion(
ollama_connection,
model,
embedding_model,
prompts_vector,
images_vector = NA,
output_text_only = F,
num_predict = 200,
temperature = 0.8,
role = "user",
repeat_penalty = 1.2,
seed = sample(1:1e+07, 1),
system_prompt = NA,
context_info = NA,
context_usage_mandatory = FALSE,
num_ctx = NA,
tools = NA,
vision = FALSE
)Arguments
- ollama_connection
a connection object that has the information on how to connect to the ollama server
- model
Name of the model to use on the ollama server. Note that you can only use the models that are available.
- embedding_model
Name of the embedding model to use on the ollama server to create embeddings on the fly for prompts.
- prompts_vector
a vector containing one or more Messages acting as prompt for the completion.
- images_vector
an optional vector (defaults to NA) containing images to be send to the model for completion.
- output_text_only
A Boolean value (default False) indicating if you just want the text message as output (TRUE) or the whole response coming from the server.
- num_predict
The number of tokens to generate in the response (maximum amount). Defaults to 200.
- temperature
The temperature value for the answer of the model. A temperature of 0 gives always the same answer. A temperature of 1 has a lot more variation. Default is 0.8.
- role
the role that is taken during the chat prompt. Typically you should be the "user", but it can be also "assistant" or "system". Defaults to "user".
- repeat_penalty
The penalty you give to the model so that you avoid it repeats the same tokens. Default is 1.2.
- seed
The seed used to generate the answer. Note that the seed has no effect if the temperature is zero. By default it is a random number between 1 and 10000000
- system_prompt
The system prompt used for your LLM.
- context_info
This is used for RAG, when you want to provide context to your LLM to ground its answers. Currently incompatible with template_prompt
- context_usage_mandatory
a boolean value (defaults to FALSE) to let the LLM know if it should ONLY use the context to answer, or if it can use the context optionally to answer a question.
- tools
a R list of tools (i.e. functions) that can be passed to the LLM. Note that this is only supported by specific LLMs like Llama3.1 - this may not work at all on LLMs that were not trained on tools calling.
Examples
conn <- get_ollama_connection()
get_ollama_chat_completion(ollama_connection=conn,
model+"llama3:8b-instruct-q4_K_S", prompts_vector="is the sky blue at night?")
#> <simpleError in eval(expr, envir): object 'model' not found>