🎙️ DeskBar AI — Autonomous AI Voice Assistant & Smart Desktop Bar

(Evolved from upstream TuneBar)

DeskBar AI is an extensive architectural evolution of the TuneBar project for the official Waveshare ESP32-S3-Touch-LCD-3.49 development platform. What began as a palm-sized internet radio has been re-engineered into an autonomous AI Voice Assistant, smart desktop companion, and distributed telemetry station with high-fidelity acoustic processing, advanced power and breathing profile management, dynamic crowd-sourced battery discharge calibration, 6-axis digital spirit level, 6-language international radio, local SD & LAN media streaming, Home Assistant device control (LAN Devices), zero-bitmap procedural vector themes, and complete privacy controls.

---

📑 Quick Navigation

• 📦 Hardware Overview
• 📸 Interface Showcase (Authentic Hardware Captures)
• ⚡ Why DeskBar AI is Different
• ⚡ Quickstart & Flash Guide
• 📚 Detailed Technical Documentation
• 🛒 Where to Buy Genuine Hardware
• 📜 Credits & License
• ☕ Support DeskBar AI

---

📦 Hardware Overview

DeskBar AI runs on the 3.49-inch capacitive touch screen development board powered by the ESP32-S3R8 (dual-core 240 MHz, 16MB Flash, 8MB OPI PSRAM) with an integrated acoustic front-end (dual MEMS microphones + ES7210 ADC), audio codec (ES8311 DAC + NS4150B amplifier), 6-axis IMU, and RTC:

Complete pinouts, wiring diagrams, and silicon driver workarounds are documented in docs/HARDWARE_AND_DRAM.md.

---

📸 Interface Showcase

All captures shown below are 100% genuine uncompressed framebuffers extracted directly from the physical ESP32-S3 hardware via /api/screenshot.

Application Interface & Dynamic Panels	Feature Architecture & Operating Details
First-Boot Security & Master PIN (Cycles between Masked State & TRNG PIN Setup)	Privacy-First Hardware Cryptography & Dynamic TOTP • Interactive 2-Column Registration Modal: Configure a 4-digit Master PIN or rely on hardware TRNG. • Isolated SHA-256 Telemetry Table: TB_<hash> with prominent live preview. • Hardware-Bound Dynamic TOTP: 6-digit rolling code ([ xxx xxx ]) with renewal countdown timer. • 30-Day Authenticated Web Session: Zero personal data collected; complete offline air-gapped mode toggle.
Main Menu Launcher Carousel (Smooth horizontal scrolling across all 7 application cards)	Neon Card-Based Navigation Hub • Ultra-Crisp 640×172 Carousel: Horizontal swipe launcher featuring 7 custom neon application cards. • Visual Categories: CLIMATE (Cyan), RADIO (Pink), MUSIC (Purple), AI VOICE (Emerald), LAN DEVICES (Blue), SYSTEM (Gold), and SETTINGS (Slate). • Ambient Audio Breathing: Active card pulses gently in rhythm with background playback. • Inertial Touch Scrolling: Responsive touch drag with snappy center-screen snapping.
LAN Devices — Home Assistant Control	Local Smart-Home Remote (HTTP only, no cloud) • Auto-Discovery: Finds Home Assistant on your LAN via mDNS and shows live connection status. • Sign In With Username & Password: no long-lived token hunting; only a refresh token is stored, never your password (token entry still available). • Rooms & Toggles: Lights, switches, covers and fans grouped by Home Assistant area with on-screen switches, plus temperature/humidity readouts. • Responsive: all network work runs on background tasks; roomy 422 px keyboard with show/hide-password. • Details: docs/LAN_DEVICES.md.
Clock, Weather & Alarm Suite (Cycles through Retro Nixie Clock, Weather & Retro Alarm)	Ambient Time, Environmental Telemetry & Wake Station • Retro Nixie Clock Face: Driven by hardware PCF85063 RTC with battery telemetry and PWM sine-wave breathing. • Live Climate & AQI Station: Real-time observations with animated glyphs, °C/°F toggle, and US Air Quality Index (AQI & PM2.5). • Retro Alarm Clock: Emerald-green wake station driven by hardware RTC alarm register with acoustic chime.
Multilingual Global Internet Radio (Cycles through Hindi, English & German streaming with live telemetry)	International 6-Language Live Streaming & Stream Telemetry • 6 International Languages: Worldwide internet radio supporting HI, EN, ES, CN, DE, JA. • Real-Time Stream Quality Telemetry: Live top-row display of audio bitrate, download speed, and jitter buffer percentage with dynamic green/amber/red health indicators. • Curated Regional Catalogs: Instant switching between Indian stations and global stream directories. • Zero-Alloc PSRAM Prebuffering: 64 KB ring buffer with seamless background streaming while switching screens. • Custom Station Import: MicroSD /stations.csv hot-reloading.
Media Player Subsystem (SD & LAN) (Cycles between Local SD Playback & LAN Streaming)	Unified Dual-Engine Audio Player • Local SD FATFS Engine: Reads tracks from MicroSD with $O(1)$ fast PSRAM offset indexing. • DMA-Safe PSRAM Streaming: Internal DRAM bounce buffer for SPI SD DMA compatibility, eliminating PSRAM bus contention and memory panics. • Local Wi-Fi LAN Engine: Streams lossless audio directly over HTTP from home music servers. • Real-time waveform animation, progress scrubber, and volume control.
Autonomous AI Voice Assistant	Full-Duplex Voice Intelligence Pipeline • Acoustic Precision: Dual MEMS microphones with ES7210 ADC hardware clock divider calibration eliminating buffer popping. • Adaptive Hardware VAD: Dynamically measures room ambient noise floor on boot to prevent false triggers. • Sub-Second Streaming Pipeline: Groq Whisper STT (<200ms) and Llama 3.3 70B reasoning stream directly into I2S DMA pipeline without temporary flash writes. • Interactive touch trigger with acoustic playback via NS4150B amplifier.
System Utilities & App Launcher	5-in-1 Quick-Access Toolkit • Torch: Instant 100% full-screen pure white backlight flashlight. • System Info: Live DRAM and PSRAM heap gauges, FreeRTOS task stacks, uptime, and supply rails. • Spirit Level: Quick launcher for the 6-axis digital inclinometer. • Power Profile: Dedicated shortcut to the battery & breathing optimization suite. • Firmware OTA: Direct launch into the community release manager.
6-Axis Digital Spirit Level & Inclinometer (Cycles through Digital Pitch/Roll, Bullseye & Calibration)	Precision Motion & Tilt Sensor • QMI8658 6-Axis IMU: Real-time digital pitch and roll telemetry. • Dual Operating Views: Numeric inclinometer angle readout and a responsive 2D bullseye bubble level. • 8-Step Guided Calibration Wizard: Calibrates zero-point offsets in NVS across 3 spatial planes. • Dynamic auto-rotation display support based on gravity vector.
On-Device Firmware OTA Manager (Cycles through Repos, Catalog Carousel & Stepper)	Decentralized Multi-Repo Release Manager • Dynamic Community Discovery: Dynamic release catalog generation across multiple GitHub community repositories. • Multi-Partition Flashing: Automated rollback protection and SHA-256 binary validation. • Battery Safety Interlock: Voltage safety guards preventing flashing under low-battery conditions. • Swipe-based carousel with live changelog viewer and step-by-step progress stepper.
Power & Breathing Profile Manager (Cycles between Breathing Controls & Cloud Calibration)	Intelligent Battery & Energy Optimization • Sine-Wave PWM Breathing: Interactive scrollable manager for PWM floor/ceiling, ramp times, and dark holds. • Peripheral Power Gating: Selectively shuts down audio amp, IMU sensor, and Wi-Fi modem during dark breathing states. • Piecewise Linear Curve Fitting: Multi-point piecewise linear discharge curve fitting saved in NVS. • Crowd-Sourced Cloud Calibration: Pulls verified golden discharge curves from peer telemetry with source and timestamp verification.
System & Display Settings (Cycles in tab order: Wi-Fi, Screen, Radio, Music, Region, User Guide)	Complete Hardware & Configuration Control • Zero-Recompile MicroSD Hot-Reload: Update Wi-Fi networks, PINs, and API keys via /secrets.ini—no toolchain or compiling required. • Multi-SSID Wi-Fi Manager: Stores up to 10 network slots with live RSSI signal telemetry. • 7 Zero-Bitmap Procedural Themes: Math-generated vector themes consuming 0 bytes of flash image memory, keeping DRAM free. • Global Localization: Timezone offsets with IP geolocation and complete on-device touch user guide.

---

⚡ Why DeskBar-AI is Different

DeskBar-AI bridges the gap between commercial smart displays and open-source hardware by solving notorious ESP32-S3 silicon quirks:

⚡ Silicon & Audio Architecture	🔋 Mathematical Battery Science
• ES7210 Clock Jitter Calibration: Custom register MCLK/BCLK divider fixes eliminating audio popping. • DMA-Safe PSRAM Bounce Buffers: Zero bus contention or DRAM starvation during high-bitrate streaming. • Non-Resetting Serial Diagnostic: Custom monitor script preventing unwanted ESP32 DTR/RTS auto-resets.	• Piecewise Linear Curve Fitting: Replaces inaccurate linear estimates with true battery discharge physics. • Crowd-Sourced Cloud Sync: One-tap sync with golden peer discharge curves via telemetry. • Peripheral Power Gating: Shuts down amplifier, IMU, and radio during dark breathing intervals.
🎙️ Sub-Second Streaming AI Pipeline	🔓 Zero-Recompile Consumer Polish
• Sub-200ms Whisper & Llama 3.3 70B: Neural TTS streams straight into I2S DMA with zero flash wear. • Adaptive Hardware VAD: Dynamic ambient room noise floor calibration. • Air-Gapped Privacy: 100% offline mode with hardware TRNG PIN generation.	• MicroSD /secrets.ini Hot-Reloading: Change Wi-Fi, keys, and settings on SD—no IDE or recompiling. • On-Device GitHub OTA Manager: Browse releases, inspect changelogs on LCD, and flash safely. • Web Remote REST & SSE: 10 Hz real-time motion telemetry and uncompressed framebuffer streaming.

---

⚡ Quickstart & Flash Guide (Click to expand build & upload instructions)

### 1. Clone & Set Up Secrets Template ```bash git clone https://github.com/rahulraj80/DeskBar-AI.git cd DeskBar-AI # Set up secrets template cp sketch/include/secrets_example.h sketch/include/secrets.h ``` ### 2. Compile via PlatformIO ```bash pio run -d sketch ``` ### 3. Flash to Board via USB Connect the board to your PC via USB-C: ```bash # Upload firmware pio run -d sketch -t upload # Open safe non-resetting serial monitor python scripts/safe_serial_monitor.py listen --duration 20 ```

---

📚 Detailed Technical Documentation

To keep this overview concise, detailed architecture guides are organized in the docs/ directory:

• 🛠️ Environment Setup & Cross-Platform Build Guide: Full compilation, LittleFS packaging, and flashing guide for Windows, Linux, and macOS.
• 🎙️ Autonomous AI Voice Assistant & Self-Hosting Guide: Voice pipeline architecture, self-hosting FastAPI reference server, and community bandwidth notices.
• 🔒 Privacy, First-Boot Consent & Telemetry: Hardware TRNG PIN setup, SHA256 isolated database tables, and /secrets.ini configuration.
• 📖 Touch Gestures & Complete User Guide: Comprehensive gesture maps, screen switching hierarchies, and settings configuration.
• ⚡ Hardware Pinouts, Silicon Quirks & DRAM Optimization: Deep-dive into ES7210 clock divider fixes, DMA frame stride unpacking, and FreeRTOS PSRAM optimization.
• 🔊 Audio Pipeline & Microphone Hardware Master Reference: In-depth ES7210/ES8311 I2S register maps, gain staging, and DMA double-buffering.
• 🏠 LAN Devices — Home Assistant Control: Auto-discovery, username/password sign-in, room-grouped device toggles, and CLI.
• 🌐 Web Remote REST API & Serial CLI: Complete table of embedded HTTP REST endpoints and interactive USB CLI commands.
• 📐 6-Axis IMU, Spirit Level & Auto-Rotation: Inclinometer physics, bullseye bubble mechanics, and 8-step guided calibration wizard.
• 🔄 On-Device Multi-Repo Firmware OTA Manager: Split carousel UI, dynamic GitHub catalog generation, and battery safety guards.

---

🛒 Where to Buy Genuine Hardware

For makers looking to replicate this device:

• 🇮🇳 India (Hindi / HI): Evelta Electronics (Search SKU 061-32374 / Waveshare ESP32-S3 Touch LCD 3.49).
• 🌐 Global / North America (English / EN & Spanish / ES): Waveshare Official Store and official stores on AliExpress.
• 🇪🇺 Europe / DACH (German / DE): BerryBase and Welectron.
• 🇯🇵 Japan (Japanese / JA): Switch Science.

Community Notice: None of the above are affiliate links; links are provided strictly for maker convenience.

---

📜 Credits & License

• Original TuneBar project foundation by VaAndCob.
• Core audio functionality powered by the ESP32-audioI2S library.
• UI engine powered by LVGL 8.4.0.
• Advanced Audio Engineering, ES7210 driver fixes, DRAM optimization, power saving architecture, multilingual radio expansion, 6-axis instruments, and AI Voice Assistant by Rahul Raj (@rahulraj80).

This software and firmware codebase is licensed under the GNU General Public License v3.0 (GPL-3.0) (see LICENSE). Any hardware designs, UI assets, and media components derived from upstream TuneBar remain licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) license. Attribution to original author Va&Cob and contributing author Rahul Raj is required on all derivative distributions.

---

☕ Support DeskBar AI

DeskBar AI is free and open-source. Click here to support the Original Upstream Repo by VaAndCob

If you enjoyed DeskBar AI with the new autonomous voice assistant and desktop instrument bar features - please click below to support the continued development of DeskBar AI.

DeskBar AI is 100% free, open-source, and privacy-respecting. Your sponsorship directly funds:

• 🧠 AI Tokens & Inference: Groq Whisper STT and Llama 3.3 70B cloud inference for live community testing and development.
• 🔬 Hardware Lab Testing: Acoustic tuning for the ES7210 ADC, NS4150B amplifier thermal stress tests, and I2C/SPI bus stability.
• 🌐 Community Cloud Infrastructure: Maintaining the cloud OTA release catalog and crowd-sourced battery telemetry servers.

Every little bit helps keep the project active and maintained.

International supporters: click the card above to pay by card or PayPal via Razorpay. In India: scan the UPI QR in the card, or pay to rahul.raj@okicici.