Ctrl+K

Local Ollama and LiteLLM: Basic Chat Completion API

Local Ollama and LiteLLM: Basic Chat Completion API#

  • Collaborators:

    • Roberto Rodriguez (@Cyb3rWard0g)

Import Modules#

from openai import OpenAI

Initialize OpenAI Client#

client = OpenAI(
    api_key="anything",
    base_url="http://127.0.0.1:8000"
)

Define Completion Function#

def get_completion(prompt, model="ollama/zephyr:7b-beta"):
    messages = [{"role":"user", "content":prompt}]
    response = client.chat.completions.create(
        model=model,
        messages=messages
    )
    return response.choices[0].message.content

Run Prompt#

get_completion("Tell me a Joke!")

"Why did the tomato turn red?\nBecause it saw the salad dressing and wanted to fit in!\n\nWhy don't scientists trust atoms?\nBecause they make up everything!\n\nWhy do seagulls fly over the sea?\nBecause if they flew over the land, they would be bagels!\n\nWhy did the chicken cross the playground?\nTo get to the other slide!\n\nWhy do scarecrows work on farms all summer long but never get a tan?\nBecause they are made of straw!\n\nHow does a penguin build its house?\nIgloos it together!\n\nWhat do you call an alligator in a vest?\nAn investigator!\n\nWhy did the coffee file a police report?\nIt got mugged!\n\nWhy don't ostriches like the circus?\nBecause of all the tightrope walkers!\n\nWhat does a computer wear to bed?\nSoftware!\n\nWhy do elephants paint their toenails red?\nBecause blue looks better!\n\nWhy did the bike fall in love with the car?\nBecause it was two-seater!\n\nWhy don't scientists trust atoms?\nBecause they make up everything!\n\nWhy are ghosts bad liars?\nBecause they are easy to see through!"
Contents