Large language models, or LLMs, are probabilistic systems that predict the next token in a sequence. The fastest way to understand what that means is to ship something with one. In this guide, we will build a command-line research assistant that uses an LLM to explain technical topics in plain English, so you can see the request/response loop, system prompts, and context windows in action.
What you'll need
You will need Python 3.10 or newer, the OpenAI SDK, and an Oxlo.ai API key.
pip install openaiSign up at https://portal.oxlo.ai to grab your key. Oxlo.ai uses flat per-request pricing, so you can experiment with long prompts and multi-turn conversations without a token counter draining your budget. See https://oxlo.ai/pricing for details.
Step 1: Configure the Oxlo.ai client
Oxlo.ai exposes a fully OpenAI-compatible endpoint. We point the SDK at it and pick a general-purpose model. I use Llama 3.3 70B here because it gives clear, structured explanations.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
MODEL = "llama-3.3-70b"Step 2: Write the system prompt
An LLM does not have fixed instructions baked in. You steer its behavior by prepending a system message. Our prompt frames the model as a patient tutor.
SYSTEM_PROMPT = """You are a helpful research assistant.
When given a technical term, explain it in one short paragraph suitable for a beginner.
Always define acronyms on first use.
If the user asks a follow-up, keep the answer concise."""Step 3: Send your first request
This function packages the system prompt and the user question into the messages array, sends it to Oxlo.ai, and returns the generated text.
from openai import OpenAI
client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
def explain_topic(user_message: str) -> str:
response = client.chat.completions.create(
model="llama-3.3-70b",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
)
return response.choices[0].message.content
# Test it
print(explain_topic("What is an LLM?"))Step 4: Add conversation memory
LLMs are stateless. Each request is independent, so the model only remembers what you include in the messages list. To support follow-up questions, we append every exchange to a running history. This demonstrates the context window in action.
from openai import OpenAI
class ResearchAgent:
def __init__(self):
self.client = OpenAI(base_url="https://api.oxlo.ai/v1", api_key="YOUR_OXLO_API_KEY")
self.messages = [
{"role": "system", "content": SYSTEM_PROMPT},
]
def ask(self, user_message: str) -> str:
self.messages.append({"role": "user", "content": user_message})
response = self.client.chat.completions.create(
model="llama-3.3-70b",
messages=self.messages,
)
answer = response.choices[0].message.content
self.messages.append({"role": "assistant", "content": answer})
return answerStep 5: Route hard questions to a reasoning model
Different LLMs excel at different tasks. Oxlo.ai hosts specialized models like DeepSeek V3.2 for reasoning and coding. We add a second method that swaps the model ID so harder questions get a more capable engine.
def ask_deep(self, user_message: str) -> str:
self.messages.append({"role": "user", "content": user_message})
response = self.client.chat.completions.create(
model="deepseek-v3.2",
messages=self.messages,
)
answer = response.choices[0].message.content
self.messages.append({"role": "assistant", "content": answer})
return answerRun it
Instantiate the agent and run a few turns. Notice how the second question relies on context established in the first, even though the underlying model has no persistent memory.
if __name__ == "__main__":
agent = ResearchAgent()
print("=== Turn 1 ===")
print(agent.ask("What is an LLM?"))
print("\n=== Turn 2 ===")
print(agent.ask("How is that different from a search engine?"))
print("\n=== Turn 3 (reasoning) ===")
print(agent.ask_deep("If an LLM has a 128K context window, roughly how many tokens fit in a 300-page novel?"))Example output:
=== Turn 1 ===
An LLM, or Large Language Model, is a type of artificial intelligence trained on vast amounts of text to predict the next word in a sentence. By repeating this prediction over and over, it can generate coherent paragraphs, answer questions, and summarize documents.
=== Turn 2 ===
A search engine retrieves existing web pages that match your keywords. An LLM generates a new response based on patterns it learned during training, so it can synthesize information rather than just link to it.
=== Turn 3 (reasoning) ===
A typical novel page contains roughly 300 to 400 words. At an average of 0.75 tokens per word, 300 pages would be about 67,500 to 90,000 tokens. A 128K context window can therefore hold the full text of a standard novel with room to spare.Next steps
Now that you have a working LLM client, try adding function calling so the agent can query a live API before it answers. Or wrap the class in a FastAPI endpoint and build a small React frontend to turn your research assistant into a shareable web tool.
