How to Build a Personal AI Assistant with Claude Code (Voice, Email, Calendar, Phone)

A personal AI assistant that actually earns the name does five things: reads your email, knows your calendar, searches your files, drafts documents in your formats, and can act in the real world (including making phone calls). You can build all five on top of Claude Code and Anthropic's Claude Agent SDK using the Model Context Protocol (MCP), with per-action approval so nothing sends without your sign-off. This guide gives the exact architecture, the tool connections, the costs, and the guardrails that keep it safe.

Last verified: 2026-08-24 - Stack: Claude Code + Claude Agent SDK + MCP connectors + a voice platform. Ballpark cost: $20-60/month for personal use. Prices change often; recheck the vendor pages linked below.

Brain: Claude via the API (Sonnet 4.6 at $3/$15 per million input/output tokens works for most assistant traffic) (#pricing).
Hands: MCP connectors for Gmail, Google Calendar, Google Drive, Slack, and more (Claude Code MCP docs; connectors overview).
Voice and phone: a voice-agent platform such as Vapi, which charges a $0.05/min platform fee with model costs passed through at cost (Vapi pricing).

What does a Claude Code personal assistant actually consist of?

It is four layers: a model (the brain), an agent loop with tools (Claude Code or the Agent SDK), connected services via MCP (the hands), and a set of standing instructions plus memory files (the personality and context).

Claude Code already ships the hard part. The Claude Agent SDK gives you the same agent loop, tools, hooks, and context management that power Claude Code, programmable in Python or TypeScript. You are assembling, not inventing.

How do you connect email, calendar, and files?

Anthropic provides prebuilt connectors for Gmail, Google Calendar, Google Drive, Slack, GitHub, and Microsoft 365, plus a standard for any remote MCP server. In Claude Code you manage them with the /mcp command, and Google Workspace connectors authenticate through claude.ai's Settings, then appear in Claude Code automatically (MCP docs).

What the Google connectors can do, per Anthropic's support documentation:

Connector	Read	Write
Gmail	Search and read mail, list drafts	Send, reply, forward - approval required by default
Google Calendar	View events, find availability	Create, update, delete events, manage invites
Google Drive	Search and read Docs, Sheets, PDFs, Office files	Create folders, save generated files

That approval-by-default behavior is the safety feature that makes the whole thing viable: your assistant drafts the invoice email, you click send.

The build steps:

Install Claude Code and sign in (or set ANTHROPIC_API_KEY for API billing).
Connect Gmail, Calendar, and Drive at claude.ai under Settings > Connectors, or add remote MCP servers with claude mcp add.
Write a CLAUDE.md in a dedicated folder describing your assistant: who you are, your business, your invoice format, your tone, and which actions always need approval.
Create folders that act as its knowledge base - client records, proposal templates, past invoices. This is what lets a one-line request like "invoice the new lead $2,500" produce a correctly formatted document: the assistant reads your leads table and your existing invoice template.
Test each connector with a read-only question first ("what meetings do I have tomorrow?") before granting any write capability.

If you prefer a workflow-automation layer for scheduled or triggered jobs instead of a conversational assistant, the same MCP wiring works through n8n - we covered that setup in connecting Claude to n8n for AI agent automation.

Can it really make phone calls on your behalf?

Yes, with a voice-agent platform in front of the model. The pattern is: a phone number provisioned by the platform, speech-to-text on the caller's audio, your Claude-powered agent deciding what to say and which tools to call, and text-to-speech back to the caller.

Vapi is the developer-standard option here: a $0.05/minute platform fee with speech-to-text, LLM, and text-to-speech billed at cost (or free of Vapi's markup if you bring your own API keys), putting a realistic all-in call around $0.15 to $0.31 per minute depending on your model and voice choices (Vapi pricing, accessed 2026-08-24). A 90-second reservation call lands around $0.25 to $0.45.

Two rules make this safe in practice:

The agent announces itself. Program the opening line to state it is an AI assistant calling on your behalf. Several jurisdictions require disclosure of automated calls, and it is simply good manners.
Report back, never act silently. Have the agent end every call by summarizing the outcome to you (confirmed reservation, quoted price, follow-up needed) and offering to log it to your calendar or CRM.

How should you handle the "brain" and its cost?

Match the model to the job instead of running everything on the most expensive tier. Per Anthropic's pricing page (accessed 2026-08-24): Claude Haiku 4.5 is $1/$5 per million input/output tokens, Sonnet 4.6 is $3/$15, and Opus 4.8 is $5/$25. For an assistant, Sonnet-class handles routing, drafting, and summarizing; reserve Opus-class for genuinely hard reasoning.

You can also make the brain swappable. Running the assistant loop against a model router lets you change models per task - cheap model for calendar lookups, frontier model for contract review - the same pattern we described in our AI agent operating system guide. Prompt caching (up to 90% off cached input) matters here because your CLAUDE.md and knowledge files get resent constantly.

Realistic monthly budget for a solo operator:

Component	Light use	Notes
Claude API (or Max subscription)	$20-50	Cache aggressively
Voice calls (Vapi)	$5-15	A handful of short calls per week
Phone number	~$2-3	Per number per month
Scheduler/hosting	$0-5	A cron job on an existing machine is fine

What makes it feel like a chief of staff instead of a chatbot?

Three design choices, all cheap:

Standing instructions with personality and standards. The system prompt is where tone, humor, and house rules live. "Draft emails politely and never send without approval" is one sentence in CLAUDE.md.
A knowledge folder, not a database project. A leads spreadsheet, an invoices folder with your template, and a clients folder give the assistant 80% of what a $50k CRM integration would.
Reachable from your pocket. Wiring the assistant to a messaging surface (Telegram or Slack via MCP) means you can send a one-line request from your phone and get a voice-note-style summary back.

For a ready-made take on this always-on pattern, our Hermes Agent setup guide covers an open-source alternative architecture, and the broader blueprint sits in how to build a personal AI assistant in 2026.

What this means for you

Start with the smallest useful loop: Claude Code, the Gmail and Calendar connectors, and a CLAUDE.md describing your work. That alone answers "what's on my plate today" and drafts your replies. Add Drive next week, a messaging surface the week after, and a voice line only when you have a real call (appointments, vendor quotes) worth delegating. Each layer is an afternoon, not a project.

FAQ

Q: Is a Claude Code personal assistant free?
A: The software is free to install; the model time is not. You either pay Anthropic per token (Haiku 4.5 at $1/$5, Sonnet 4.6 at $3/$15 per million input/output) or use a Claude Pro/Max subscription. Most personal users land at $20-60 per month all-in.

Q: Can Claude send emails without my approval?
A: By default, no. The Gmail connector asks for your explicit approval before sending, replying, or forwarding; only organization owners on Team/Enterprise plans can change that behavior (Anthropic Help Center).

Q: Can the assistant create calendar events?
A: Yes. The Google Calendar connector supports creating, updating, and deleting events, plus managing invitations, on Pro, Max, Team, and Enterprise plans (Claude docs).

Q: How much does it cost to have the assistant make a phone call?
A: On Vapi, around $0.15 to $0.31 per minute all-in ($0.05 platform fee plus speech-to-text, LLM, and text-to-speech at cost), so a typical 90-second booking call costs roughly $0.25 to $0.45 (Vapi pricing, accessed 2026-08-24).

Q: Do I need to know how to code?
A: For the connector-based version, no; setup is clicking connect and writing instructions in plain English. A small amount of scripting helps for scheduled jobs (the Agent SDK supports headless claude -p runs you can cron) but is optional.

Q: Is my Gmail and Drive data used to train Claude?
A: Anthropic states it does not train models on Gmail, Drive, or Calendar connector data; data is encrypted in transit and at rest, and deleting the chat deletes the retrieved data (Anthropic Help Center).