From Curiosity to Creation: Building a Loop Engineering Application with IBM’s Bob
Introduction
For some time now, “Loop Engineering” has been generating significant buzz within computational and AI architecture circles. It is a concept that immediately caught my interest and sparked my curiosity. Rather than settling for theoretical papers, I wanted to see how a production-grade implementation behaves under the hood. So, I enlisted IBM’s Bob to build a sample application from scratch while breaking down the architectural concepts along the way. That is exactly what we achieved, and in this post, we will walk through the code, design patterns, and concrete schemas that make it work.
TL;DR-Understanding Loop Engineering
Loop Engineering represents a paradigm shift from linear, single-pass LLM prompts toward self-correcting cognitive cycles. Instead of expecting a model to output a perfect result in a single “fire-and-forget” inference call, a loop-engineered architecture constructs a controlled environment around the model. The system iteratively executes actions, captures environmental feedback, evaluates progress against quantitative thresholds, and dynamically adjusts its state for the next turn. By orchestrating multi-stage cognitive passes — typically broken down into Perceive, Plan, Act, Observe, and Reflect (PPAOR) — loop engineering enables autonomous agents to recover from tool failures, refine intermediate reasoning, and guarantees deterministic termination through strict state guards.
Application Architecture & Implementation
The reference project built by Bob is an Autonomous Research Summarizer. It uses a two-tier nested loop architecture: an outer orchestrator loop managing high-level milestone phases and an inner engine controlling atomic cognitive steps.
ResearchOrchestrator (outer loop)
├─ PERCEIVE select next research phase
├─ PLAN check prerequisites, skip/execute decision
├─ ACT delegate to LoopEngine inner loop ──────────────────────┐
│ ┌── LoopEngine (inner loop, per subtask) │
│ │ ├─ PERCEIVE load context from AgentMemory │
│ │ ├─ PLAN StubLLM / RealLLM action decision │
│ │ ├─ ACT ToolRegistry dispatch + retry │
│ │ ├─ OBSERVE structure ToolResult │
│ │ └─ REFLECT extract facts, score, loop state │
│ └────────────────────────────────────────────────────-┘
├─ OBSERVE collect LoopState + quality score
└─ REFLECT update global state, check thresholds, early-exit
Core State & Data Schemas
Robust state tracking forms the foundation of any loop-engineered framework. State is explicitly partitioned across three distinct memory lifetimes within memory.py:
from typing import List, Dict, Any, Optional
from pydantic import BaseModel, Field
from enum import Enum
class LoopState(str, Enum):
RUNNING = "RUNNING"
COMPLETE = "COMPLETE"
FAILED = "FAILED"
TIMEOUT = "TIMEOUT"
class WorkingMemory(BaseModel):
"""Volatile per-iteration workspace. Cleared at the start of every PERCEIVE cycle."""
current_task: str
retrieved_context: List[str] = Field(default_factory=list)
planned_action: Optional[str] = None
raw_tool_output: Optional[Any] = None
class IterationRecord(BaseModel):
"""Immutable episodic record capturing a complete PPAOR cycle."""
iteration_id: int
task: str
action_type: str
action_args: Dict[str, Any]
success: bool
observation: str
reflection_score: float
timestamp: float
class SemanticMemory(BaseModel):
"""Persistent state across all inner and outer iterations."""
facts: List[str] = Field(default_factory=list)
summaries: Dict[str, str] = Field(default_factory=dict)
quality_score: float = 0.0
report_path: Optional[str] = None
completed_subtasks: List[str] = Field(default_factory=list)
class AgentMemory(BaseModel):
"""Unified memory root object."""
research_topic: str
outer_iteration: int = 0
inner_iteration: int = 0
consecutive_errors: int = 0
working: WorkingMemory
episodic: List[IterationRecord] = Field(default_factory=list)
semantic: SemanticMemory = Field(default_factory=SemanticMemory)
The Inner Cognitive Loop (agent_loop.py)
The LoopEngine drives the primary PPAOR state machine. The execution block below illustrates how each phase feeds into the next, protected by runtime guards for max iterations and error limits:
import time
from typing import Tuple
class LoopEngine:
def __init__(self, config, tool_registry, llm_client):
self.cfg = config
self.tools = tool_registry
self.llm = llm_client
def run(self, task: str, memory: AgentMemory) -> Tuple[LoopState, AgentMemory]:
memory.inner_iteration = 0
memory.consecutive_errors = 0
loop_state = LoopState.RUNNING
while loop_state == LoopState.RUNNING:
# --- Safety Guard 1: Max Iteration Threshold ---
if memory.inner_iteration >= self.cfg.max_inner_iterations:
loop_state = LoopState.TIMEOUT
break
# --- Safety Guard 2: Error Resilience Boundary ---
if memory.consecutive_errors >= self.cfg.error_threshold:
loop_state = LoopState.FAILED
break
memory.inner_iteration += 1
# STAGE 1: PERCEIVE - Flush volatile memory & fetch episodic context
memory.working = WorkingMemory(
current_task=task,
retrieved_context=self._get_recent_context(memory)
)
# STAGE 2: PLAN - Ask LLM to determine action given current state
plan = self._plan_next_step(task, memory)
# STAGE 3: ACT - Safe execution through ToolRegistry
tool_result = self._execute_action(plan)
# STAGE 4: OBSERVE - Structured transformation of tool output
observation = self._format_observation(tool_result)
memory.working.raw_tool_output = observation
# STAGE 5: REFLECT - Compute quality metrics and decide loop continuation
reflection = self._reflect_and_score(task, observation, memory)
# Record cycle execution in episodic history
memory.episodic.append(IterationRecord(
iteration_id=memory.inner_iteration,
task=task,
action_type=plan.action_type,
action_args=plan.args,
success=tool_result.is_success,
observation=str(observation),
reflection_score=reflection.score,
timestamp=time.time()
))
# Evaluate stopping condition
if reflection.goal_achieved:
loop_state = LoopState.COMPLETE
elif not tool_result.is_success:
memory.consecutive_errors += 1
else:
memory.consecutive_errors = 0
return loop_state, memory
Deterministic Reflection & Stopping Logic
A core principle of loop engineering is preventing infinite execution loops by converting open-ended task evaluation into quantitative checks. The _reflect_and_score method calculates goal completion across objective criteria rather than relying purely on LLM preference:
class ReflectionOutcome(BaseModel):
score: float
goal_achieved: bool
extracted_facts: List[str]
def evaluate_quality(semantic_mem: SemanticMemory, target_threshold: float = 0.85) -> ReflectionOutcome:
score = 0.0
extracted = []
# Fact density criteria (Max +0.30)
fact_count = len(semantic_mem.facts)
if fact_count >= 3:
score += 0.30
elif fact_count > 0:
score += 0.10
# Summary synthesis criteria (Max +0.20)
if len(semantic_mem.summaries) >= 1:
score += 0.20
# Task coverage criteria (Max +0.20)
if len(semantic_mem.completed_subtasks) >= 2:
score += 0.20
# Final document generation criteria (Max +0.30)
if semantic_mem.report_path is not None:
score += 0.30
semantic_mem.quality_score = score
return ReflectionOutcome(
score=score,
goal_achieved=(score >= target_threshold),
extracted_facts=extracted
)
Application Configuration (.env)
The application parameterizes runtime behavior, tool reliability, and LLM providers via environment variables configured in .env. Key configuration variables control loop boundaries, simulation noise, and backend routing:
# Loop Engineering — Environment Configuration
# =============================================
# Copy this file to .env and fill in the values for your environment.
# Never commit the real .env file — it is listed in .gitignore.
# ── Loop control parameters ──────────────────────────────────────────────────
# Hard ceiling on the outer orchestrator loop (default: 8)
MAX_OUTER_ITERATIONS=8
# Hard ceiling on each inner task loop (default: 6)
MAX_INNER_ITERATIONS=6
# Minimum quality score [0.0–1.0] to trigger early exit (default: 0.85)
GOAL_QUALITY_THRESHOLD=0.85
# Consecutive tool errors before a loop is marked FAILED (default: 3)
ERROR_THRESHOLD=3
# Per-tool-call retry budget (default: 2)
MAX_TOOL_RETRIES=2
# ── Tool simulation ───────────────────────────────────────────────────────────
# Probability [0.0–1.0] of a simulated transient tool failure.
# Set to 0.0 for a clean demo run; values > 0 exercise retry/recovery paths.
TOOL_FAILURE_RATE=0.15
# ── Observability ────────────────────────────────────────────────────────────
# 0=silent | 1=milestones only | 2=per-iteration detail | 3=full debug
VERBOSITY=2
# ── Research topic (overrides CLI argument if set) ────────────────────────────
# RESEARCH_TOPIC=Quantum Computing in Healthcare
# ── Output directory ─────────────────────────────────────────────────────────
# Directory where generated research reports are saved (default: output/)
OUTPUT_DIR=output
# =============================================================================
# LLM BACKEND CONFIGURATION
# =============================================================================
# Set USE_REAL_LLM=true to connect to a real LLM.
# Then set BACKEND to select which provider to use:
# openai — OpenAI API or any OpenAI-compatible generic endpoint
# ollama — Local Ollama server
# llamacpp — Local llama.cpp llama-server
#
# When USE_REAL_LLM=false (the default) the built-in StubLLM is always used
# regardless of BACKEND — no external dependencies required.
# =============================================================================
USE_REAL_LLM=true
# Active backend when USE_REAL_LLM=true.
# Accepted values (case-insensitive): openai | ollama | llamacpp
BACKEND=llamacpp
# ── [BACKEND=openai] OpenAI / generic OpenAI-compatible endpoint ──────────────
# API key for the OpenAI service (required when BACKEND=openai).
OPENAI_API_KEY=your-openai-api-key-here
# Model name to request (default: gpt-4o-mini).
LLM_MODEL=gpt-4o-mini
# Optional base URL override for non-OpenAI endpoints (e.g. Azure, LM Studio).
# Leave blank to use OpenAI's default: api.openai.com
LLM_BASE_URL=
# ── [BACKEND=ollama] Local Ollama server ──────────────────────────────────────
# Start Ollama: ollama serve
# Pull a model: ollama pull llama3.2
#
# Base URL of the Ollama OpenAI-compatible endpoint (default shown).
OLLAMA_BASE_URL=http://localhost:11434/v1
# Model name to request from Ollama (default: llama3.2).
OLLAMA_MODEL=llama3.2
# ── [BACKEND=llamacpp] llama.cpp llama-server ─────────────────────────────────
# Start llama-server with Ministral-3B:
#
# llama-server \
# --model /path/to/Ministral-3B-Instruct-2410-Q4_K_M.gguf \
# --ctx-size 8192 \
# --n-gpu-layers 99 \
# --port 8080 \
# --host 127.0.0.1
#
# Flags:
# --ctx-size 8192 Context window — must match LLAMACPP_CTX_SIZE below.
# --n-gpu-layers 99 Offload all layers to GPU; use 0 for CPU-only.
# --port 8080 Must match the port in LLAMACPP_BASE_URL.
#
# Base URL of the llama-server OpenAI-compatible endpoint (default shown).
LLAMACPP_BASE_URL=http://127.0.0.1:8080/v1
# Model identifier sent in API requests.
# llama-server ignores this field but the openai client requires a non-empty value.
LLAMACPP_MODEL=ministral-3b
# Context window size — must match the --ctx-size flag used at server launch.
LLAMACPP_CTX_SIZE=8192
# Sampling temperature forwarded in every chat completion request (default: 0.2).
LLAMACPP_TEMPERATURE=0.2
# Maximum tokens to generate per completion (default: 512).
LLAMACPP_MAX_TOKENS=512
Application Execution Outputs
Below are three standard execution trace artifacts generated during testing, showing the differences between non-LLM stub execution and full LLM backend integration.
- Report Output (sample) (
report_quantum_computing_in_healthcare_20260818_075302.md):
Research Report: Quantum Computing in Healthcare
_Generated by Loop Engineering Agent — 2026-08-18 07:53:02_
---
## Executive Summary
**Gather Overview:** ['Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years. ', 'Pharmaceutical companies a.
**Find Key Facts:** ['Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years. ', 'Pharmaceutical companies a.
## Key Findings
1. Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years.
2. Pharmaceutical companies are using quantum annealers (D-Wave) to optimise molecular docking simulations for early-stage drug candidates.
3. Quantum-enhanced MRI could theoretically increase signal-to-noise ratios by 10x, enabling earlier detection of sub-centimetre tumours.
## Research Methodology
- Total iterations executed: 4 outer / 18 inner
- Subtasks completed: gather_overview, find_key_facts, identify_applications
- Final quality score: 70.0%
---
_This report was generated by the Loop Engineering Autonomous Research Summarizer._
- Application Trace Without LLM (test mode) (
without-llm.md): Running the application withUSE_REAL_LLM=falseactivates the deterministic built-in StubLLM. Notice how individual inner tasks timeout after hittingmax_inner_iterations (6)until reachingcompile_report, where the threshold is satisfied and an early exit triggers:
[LLM] Using built-in StubLLM (no API key required)
╔══════════════════════════════════════════════════════════╗
║ Loop Engineering — Autonomous Research Summarizer ║
╚══════════════════════════════════════════════════════════╝
Topic : Quantum Computing in Healthcare
Phases : gather_overview → find_key_facts → identify_applications → compile_report
Budget : 8 outer / 6 inner iterations
Threshold: 85%
LLM : stub
┌─ OUTER ITER 1 ─ dispatching subtask: 'gather_overview' (4 pending) ─────────────────
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Iteration [outer=1 inner=6] task='gather_overview'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[PERCEIVE] context_lines=3
[PLAN] action=web_search conf=0.75 args=['query']
[ACT] tool=web_search success=True attempts=1 elapsed=0.0ms
[OBSERVE] data_type=list data_len=3
[REFLECT] score=0.50 facts=3 continue=True
→ CONTINUE (score=0.50 < threshold=0.85)
⏱ TIMEOUT: reached max_inner_iterations (6) for task 'gather_overview'.
│ inner_state=TIMEOUT inner_iters=6 quality=0.50
└─ outer_state=RUNNING facts=3 score=0.50
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Iteration [outer=4 inner=2] task='compile_report'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[PERCEIVE] context_lines=3
[PLAN] action=write_report conf=0.95 args=['filename', 'content']
[ACT] tool=write_report success=True attempts=1 elapsed=0.4ms
[OBSERVE] data_type=str data_len=108
[REFLECT] score=1.00 facts=3 continue=False
✓ COMPLETE: quality threshold reached (score=1.00 ≥ 0.85) task='compile_report'
│ inner_state=COMPLETE inner_iters=2 quality=1.00
└─ outer_state=COMPLETE facts=3 score=1.00
✓ EARLY EXIT: global quality threshold (85%) met after subtask 'compile_report'.
════════════════════════════════════════════════════════════
ORCHESTRATION RESULT
════════════════════════════════════════════════════════════
Final state : COMPLETE
Quality score : 100.0%
Facts collected: 3
Subtasks done : gather_overview, find_key_facts, identify_applications, compile_report
Subtasks failed: none
Total iters : 20
Elapsed : 0.11s
Report path : /Users/alainairom/Devs/loop-engineering-1st/output/report_quantum_computing_in_healthcare_20260818_081638.md
- Application Trace With LLM (
withllm.md): Running with a live LLM endpoint (BACKEND=llamacpp, modelministral-3b) demonstrates runtime dynamic recovery. In this trace, when the LLM supplies malformed parameters or encounters tool execution exceptions (e.g., missing arguments or transient simulated failures), the loop catches the failure, records the observation, and continues without crashing:
[LLM] Backend=llamacpp model=ministral-3b url=http://127.0.0.1:8080/v1
╔══════════════════════════════════════════════════════════╗
║ Loop Engineering — Autonomous Research Summarizer ║
╚══════════════════════════════════════════════════════════╝
Topic : Quantum Computing in Healthcare
Phases : gather_overview → find_key_facts → identify_applications → compile_report
Budget : 8 outer / 6 inner iterations
Threshold: 85%
LLM : real
┌─ OUTER ITER 1 ─ dispatching subtask: 'gather_overview' (4 pending) ─────────────────
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Iteration [outer=1 inner=2] task='gather_overview'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[PERCEIVE] context_lines=1
[PLAN] Stripping unrecognised tool_args keys for 'fact_check': ['facts']
[PLAN] action=fact_check conf=0.80 args=[]
[ACT] tool=fact_check success=False attempts=1 elapsed=0.0ms
error: _tool_fact_check() missing 1 required positional argument: 'claim'
[OBSERVE] data_type=NoneType data_len=None
[REFLECT] score=0.30 facts=4 continue=True
→ CONTINUE (score=0.30 < threshold=0.85)
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Iteration [outer=3 inner=1] task='identify_applications'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[PERCEIVE] context_lines=3
[PLAN] action=text_summarize conf=0.95 args=['text']
[ACT] tool=text_summarize success=False attempts=3 elapsed=157.8ms
error: Tool 'text_summarize' failed after 3 attempts. Last error: Simulated transient failure on attempt 3.
[OBSERVE] data_type=NoneType data_len=None
[REFLECT] score=0.50 facts=4 continue=False
✓ COMPLETE: quality threshold reached (score=0.50 ≥ 0.85) task='identify_applications'
│ inner_state=COMPLETE inner_iters=1 quality=0.50
└─ outer_state=RUNNING facts=4 score=0.50
...
════════════════════════════════════════════════════════════
ORCHESTRATION RESULT
════════════════════════════════════════════════════════════
Final state : COMPLETE
Quality score : 50.0%
Facts collected: 4
Subtasks done : gather_overview, find_key_facts, identify_applications, compile_report
Subtasks failed: none
Total iters : 11
Elapsed : 32.34s
Report path : not written
════════════════════════════════════════════════════════════
Conclusion
Loop engineering shifts AI application design from unpredictable single-prompt pipelines toward dependable, closed-loop software architectures. Explicitly splitting state into volatile working memory, append-only episodic history, and persistent semantic knowledge gives models the context needed to self-correct upon tool failure. Combined with strict iteration bounds and quantitative reflection metrics, these agentic loops turn non-deterministic models into predictable enterprise tools. Special thanks to IBM’s Bob for building out this clear implementation — it serves as a solid foundation for deploying autonomous, resilient AI systems.
Thanks for reading 🛞
Links
- What is “loop engineering”: https://www.ibm.com/think/topics/loop-engineering
- A very interesting loop engineering github repo (courtesy to Cobus Greyling): https://github.com/cobusgreyling/loop-engineering
- Github repo for this post: https://github.com/aairom/loop-engineering-101
