Last month I needed to review twenty LLM papers for a literature survey and found myself copy-pasting abstracts into chat tabs. I built a small Python agent that fetches an arXiv paper and returns a structured technical digest in under five seconds. If you need to triage papers quickly, this tool will save you hours.
What you'll need
- Python 3.10 or newer
- The OpenAI SDK:
pip install openai - The requests library:
pip install requests - An Oxlo.ai API key from https://portal.oxlo.ai
- A free arXiv ID to test with, such as
2404.19756
Step 1: Fetch paper metadata from arXiv
We start by pulling the title and abstract from arXiv's public API. I use requests and ElementTree to avoid heavy dependencies.
import sys
import requests
import xml.etree.ElementTree as ET
def fetch_arxiv(arxiv_id: str) -> dict:
url = f"http://export.arxiv.org/api/query?id_list={arxiv_id}"
resp = requests.get(url, timeout=15)
resp.raise_for_status()
root = ET.fromstring(resp.text)
ns = {"atom": "http://www.w3.org/2005/Atom"}
entry = root.find("atom:entry", ns)
title = entry.find("atom:title", ns).text.strip()
summary = entry.find("atom:summary", ns).text.strip()
return {"title": title, "abstract": summary}
paper_id = sys.argv[1] if len(sys.argv) > 1 else "2404.19756"
paper = fetch_arxiv(paper_id)
print(f"Fetched: {paper['title']}")
Step 2: Define the analysis system prompt
The agent needs explicit instructions to output structured JSON. I keep the schema strict so downstream scripts can consume it reliably.
SYSTEM_PROMPT = """You are a senior ML research assistant.
Analyze the provided paper title and abstract.
Respond ONLY with a JSON object containing:
- summary: a 2-sentence technical overview
- key_contributions: a list of 3 bullet strings
- limitations: a list of 2 bullet strings
- relevance_score: an integer from 1 to 10 for an MLOps engineer
Be concise, specific, and avoid marketing language."""
Step 3: Send the paper to Oxlo.ai for structured analysis
I use Oxlo.ai because its request-based pricing means I can stuff the full abstract plus a long system prompt into a single call without watching token meters spin. The endpoint is a drop-in replacement for the OpenAI SDK.
import json
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.oxlo.ai/v1",
api_key=os.environ.get("OXLO_API_KEY", "YOUR_OXLO_API_KEY"),
)
user_message = f"Title: {paper['title']}\n\nAbstract: {paper['abstract']}"
response = client.chat.completions.create(
model="kimi-k2.6",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_message},
],
response_format={"type": "json_object"},
temperature=0.2,
)
raw_json = response.choices[0].message.content
analysis = json.loads(raw_json)
Step 4: Render the digest
Finally, we pretty-print the JSON so the terminal output is readable during a morning stand-up or literature review session.
def render_digest(paper: dict, analysis: dict):
print(f"\n{'='*60}")
print(f"Paper: {paper['title']}")
print(f"{'='*60}")
print(f"\nSummary:\n{analysis['summary']}")
print(f"\nKey Contributions:")
for item in analysis['key_contributions']:
print(f" - {item}")
print(f"\nLimitations:")
for item in analysis['limitations']:
print(f" - {item}")
print(f"\nRelevance Score: {analysis['relevance_score']}/10")
print(f"{'='*60}\n")
render_digest(paper, analysis)
Run it
Export your key and run the script against any arXiv ID.
export OXLO_API_KEY="sk-oxlo.ai-..."
python digest.py 2404.19756
Example output:
============================================================
Paper: The Llama 3 Herd of Models
============================================================
Summary:
Llama 3 is a new family of foundation models scaling up to 405B parameters trained on over 15 trillion tokens. The paper details data filtering, scaling laws, and post-training recipes that significantly improve reasoning and coding benchmarks.
Key Contributions:
- Release of a 405B dense transformer trained with a novel scaling law recipe
- Comprehensive safety training and red-teaming methodology
- Open weights and evaluation pipeline for reproducibility
Limitations:
- Dense 405B model requires substantial inference infrastructure
- Evaluation focuses on English-centric benchmarks
Relevance Score: 9/10
============================================================
Wrap-up and next steps
This agent turns a stack of arXiv IDs into a structured reading list in minutes. Two concrete ways to extend it: first, wire the script to a CSV export from Google Scholar and batch-process fifty papers overnight, since Oxlo.ai's flat per-request pricing keeps costs predictable even when you feed in long abstracts. Second, add a second pass with deepseek-r1-671b to generate implementation notes or reproduce a paper's core algorithm in Python.
