Personal Project

A.L.F.R.E.D

All-Knowing Logical Facilitator for Reasoned Execution of Duties

A voice-enabled AI desktop assistant with multi-provider intelligence, semantic memory, and deep system integration.

Personal Project
Python / PySide6
View on GitHub

Overview

A.L.F.R.E.D is a desktop AI assistant inspired by the idea of having a truly intelligent, always-available digital companion. It combines voice interaction, multi-provider LLM intelligence, and real-world integrations into a polished desktop application with real-time audio visualization.

Unlike simple chatbot wrappers, A.L.F.R.E.D maintains persistent semantic memory using vector embeddings, so it remembers context across conversations and can retrieve relevant past interactions automatically.

Architecture

Voice Input
Whisper STT
→
LLM Router
OpenAI / Claude
→
Tool System
Calendar, Files, etc.
→
TTS Output
ElevenLabs
ChromaDB Vector Memory
OpenAI Embeddings · Persistent

Key Features

Multi-Provider LLM
Automatic fallback between OpenAI and Claude. If one provider is down or rate-limited, the system seamlessly switches to the other.
Semantic Memory
ChromaDB vector storage with OpenAI embeddings. Past conversations are searchable by meaning, not just keywords.
Voice Interface
Natural voice interaction with Whisper speech-to-text and ElevenLabs text-to-speech for human-quality responses.
PySide6 GUI
Desktop application with real-time audio waveform visualization, system dashboards, and a clean conversational interface.
System Integrations
Google Calendar, weather APIs, local file search, and system automation. A.L.F.R.E.D can take action, not just answer questions.
Tool System
Extensible tool architecture where new capabilities can be added as plugins without modifying the core conversation loop.

Challenges & Solutions

  • Latency: Voice interaction needs to feel instant. Implemented streaming responses and parallel TTS generation to minimize perceived delay.
  • Memory Relevance: Not all past context is useful. Tuned embedding similarity thresholds and added recency weighting to surface the most relevant memories.
  • Provider Reliability: API outages are unpredictable. Built a provider abstraction layer with health checks and automatic failover between OpenAI and Claude.
  • Audio Pipeline: Coordinating STT, LLM, and TTS in real-time required careful async orchestration to prevent audio glitches and race conditions.

Tech Stack

Python PySide6 OpenAI API Anthropic API ChromaDB ElevenLabs Whisper Google Calendar API

Want to see more of my work?

All Projects