Native Kotlin Android App + ESP32 Firmware BLE Audio Prototype (PoC)

START YOUR PROPOSAL WITH THE EXACT PHRASE: "Avo-Heart Developer Baseline Verified". If this phrase is missing, your bid will be instantly deleted as automated spam. # Phase 1 Product Requirement Document (PRD) and Freelancer Master Agreement **Project Name:** V1 Toy - "Avo-Heart" Proof of Concept (PoC) **Target Hardware Platform:** ESP32-WROOM-32E Dev Kit (38-Pin) **Target Mobile Ecosystem:** Android 10.0 (API Level 29) to Android 16+ Baseline **Minimum Connection Standard:** Bluetooth Low Energy (BLE) 5.0 **Power Profile:** 3x AAA Alkaline Batteries (Targeting 1+ Month Standby via Dynamic Power Scaling) --- ## SYSTEM CONTEXT AND OVERVIEW This Phase 1 Proof of Concept (PoC) establishes a bare-bones, low-latency, screen-free interactive audio data loop between a physical hardware peripheral array and an Android companion mobile application. The physical toy captures a child's speech via a digital microphone, locally processes a tactile button interrupt state, compresses the raw audio, and streams it instantly over Bluetooth Low Energy (BLE) to the mobile application. The Android application acts as the background processing engine, handling voice activity parsing, stream buffering, packet re-assembly, and downstream audio playback routing back to the toy's amplifier speaker array. [Physical Toy Setup] ---(BLE 5.0 Link: IMA-ADPCM 4:1)--- [Android App] --- ## 1. CRITICAL DELIVERY AND COMPILATION MANDATES (READ FIRST) * **Rule 1.1 (The 10-Minute Handoff Guarantee):** Final milestone release is strictly contingent upon the Product Owner successfully cloning the repository, opening it in standard Android Studio (for mobile) and PlatformIO/Arduino IDE (for firmware), and generating error-free binaries within 10 minutes using the supplied README.md instructions. The freelancer remains contractually responsible for resolving any environment, path, or linker dependency bugs. * **Rule 1.2 (Developer Hardware Procurement Guidance):** Within 48 hours of contract acceptance, the freelancer must provide a precise, clickable components shopping list tailored to the Product Owner's local market region (e.g., Amazon.in, Robocraze, or element14 India) for: 1. 1x ESP32-WROOM-32E Development Kit (38-pin variant). 2. 1x INMP441 I2S Digital Microphone Module. 3. 1x MAX98357A I2S Class-D Amplifier Breakout Board. 4. 1x 8-Ohm Micro Speaker. 5. 1x WS2812B Addressable LED Ring (8 or 12 LED layout). 6. 1x Tactile/Membrane Push Button Switch. 7. 1x Solderless Breadboard with standard jumper wires. * Deliverable: The freelancer must deliver a clear text or graphic markdown pin-connection matrix (e.g., INMP441 SCK to GPIO 14) so the Product Owner can construct an identical hardware testing bench. * **Rule 1.3 (Language and Dependency Constraints):** * Android Mobile App: Native Kotlin exclusively using Jetpack Compose (preferred) or clean, standard XML layouts. Keep the interface completely unstyled, using standard system components. Must use native Android BluetoothGatt and BluetoothGattCallback modules. No third-party BLE wrappers allowed. * Embedded Firmware: Standard Arduino framework or PlatformIO ecosystem compiled for C++. No proprietary or closed-source third-party dependencies are allowed. * **Rule 1.4 (IP Assignment):** Full intellectual property, copyrights, and source distribution rights transfer to the product owner immediately upon milestone release. No code reuse or sub-licensing rights are retained by the freelancer. --- ## 2. CORE EMBEDDED FIRMWARE FUNCTIONAL RESPONSIBILITIES The freelancer is responsible for writing a non-blocking, multi-threaded C++/Arduino firmware application on the ESP32 that manages the following hardware states and processing pipelines: * **BLE Device Discovery and Identification Protocol:** * Advertising Local Name: The ESP32 firmware must explicitly advertise using the local string identifier: "AvoHeart-PoC". * Service UUID Architecture: The system must utilize a designated 128-bit Universally Unique Identifier (UUID) for the primary service (Freelancer to generate a clean, standard UUID or use a placeholder string like "4fa72c30-e35b-11ee-b5f2-0800200c9a66"). * App Filtering Strategy: The Android companion app must implement a native ScanFilter targeting both the explicit Local Name and the Service UUID simultaneously to execute a quiet, background connection routine without requiring manual user selection inputs. * **I2S Hardware Dual-Channel Allocation:** To eliminate resource contention and audio DMA buffer collisions, the firmware must allocate distinct hardware I2S peripherals for the input and output streams: * I2S Channel 0: Dedicated exclusively to the MAX98357A Audio Amplifier output pipeline (Downstream). * I2S Channel 1: Dedicated exclusively to the INMP441 Digital Microphone input pipeline (Upstream). * Driver Rule: The drivers must not share memory space or interrupt handling lines, allowing rapid transitions between the two modes without requiring a full chip or peripheral reset. * **Local Audio Interaction Hint:** Immediately upon detecting a valid heart button press, the firmware must play a short, pre-cached local audio cue (e.g., a distinct chime or a brief verbal "Hello! I'm listening!") directly from the ESP32 internal SPIFFS/Flash memory to the speaker before opening the microphone recording channel. This local playback must complete in less than 500ms to serve as an instant sensory confirmation for the child. * **Audio Capture Pipeline (Upstream):** Configure I2S Channel 1 to interface with the INMP441 digital microphone at a sampling rate of 16kHz, 16-bit depth, mono channel. Implement a real-time IMA-ADPCM 4:1 compression algorithm to encode raw audio data into lightweight chunks on the fly. Stream compressed byte packets sequentially to the BLE stack using the custom Inbound Characteristic notification loop. * **Audio Playback Pipeline (Downstream):** Configure I2S Channel 0 to interface with the MAX98357A amplifier. Maintain an internal Circular Ring Buffer (Data Queue) in RAM to cache incoming BLE compressed chunks sent from the mobile application. Implement the matching IMA-ADPCM decoder engine to expand chunks back into raw PCM audio frames. Manage audio clocking strictly via a FreeRTOS hardware timer interrupt to prevent buffer underflows, clicks, or popping noises during speaker playback. * **Peripheral State Machine and LED Driver Engine:** Initialize the WS2812B LED halo driver to dynamically render the custom animation patterns mapped in Section 3. Implement a hardware interrupt service routine (ISR) with a 50ms software debounce protocol for the physical heart membrane switch to step between idle, capture, and playback states. * **Low-Power Management Routine:** Continuously poll the Analog-to-Digital Converter (ADC) monitoring pin connected to the raw battery array. Enforce a Light Sleep mode (scaling down BLE connection intervals) after 2 minutes of inactivity, and execute a Deep Sleep routine (under 15 microamps) after 5 minutes of total disconnection or upon reaching the 3.6V battery critical limit. --- ## 3. STRICT MILESTONE ACCEPTANCE CRITERIA Final milestone sign-off and payment release are strictly contingent on satisfying every objective test case outlined below on the Product Owner's hardware bench setup. ### Milestone 1: Hardware-to-Software Link Stability (25% Budget) * [ ] **Automated Pairing:** The physical ESP32 breadboard setup must discover and pair with the mobile application within 3.0 seconds of initial power-on via the locked Name/UUID payload scan filter. * [ ] **Connection Endurance:** The BLE link must maintain an open, error-free connection for a minimum of 60 continuous minutes while the smartphone screen is completely locked and asleep. * [ ] **Automatic Re-pairing:** When the hardware is disconnected manually (power cycle or out-of-range test) and brought back online, it must re-establish its operational link within 5.0 seconds without requiring an app restart. ### Milestone 2: The Squeeze-to-Magic Interaction Matrix (25% Budget) The 3-stage visual and functional feedback loop must map accurately to the following hardware states: | Current Stage | Physical Trigger Action | Required LED Behavior | Required Local/Stream Action | | --- | --- | --- | --- | | **1. Talk Phase** | Squeeze/Tap the membrane button | **Solid Soft Green LED** turned on instantly. | Plays 500ms local greeting file, then opens the INMP441 microphone and begins streaming raw audio packets over BLE. Interrupts active playback. | | **2. Process Phase** | Button release OR 1.5-second silence timeout | Green LED turns off; **Breathing/Pulsing Yellow LED** activates. | Microphone stream closes; app trims silence and routes data package to the intent engine. | | **3. Output Phase** | Audio data packet arrives back at mobile app | Yellow LED turns off; **Twinkling/Chasing Blue LED** activates. | App streams the response audio file back down; MAX98357A amp triggers the speaker cleanly. | * [ ] **Interaction Check:** Every single state change must sync perfectly with its designated LED color behavior within less than 50 milliseconds of the phase transition. ### Milestone 3: Audio Performance and Fidelity Standards (30% Budget) * [ ] **Round-Trip Latency:** The duration between the end of user speech (button release) and the initial note emitting from the speaker must not exceed 2.0 seconds when tested using local mock file loops on the device to bypass external network congestion variables. * [ ] **Fidelity and Continuity:** Playback of a standard 60-second compressed nursery rhyme must complete from start to finish without audio stuttering, clicks, pops, or dropped frames. * [ ] **Sample Normalization:** Audio captured by the INMP441 mic must register inside the app's diagnostic logs at a stable, standard sample rate, showing zero pitch distortion or playback speed anomalies. ### Milestone 4: Environmental Faults, Low-Power, and Handoff (20% Budget) * [ ] **Offline Fallback Validation:** Turn off the phone entirely. Power cycle the toy. Verify that after 30 seconds of searching (Yellow blinking), it enters the offline mode where pressing the heart successfully triggers the local default flash audio prompt. * [ ] **System Bluetooth Prompt:** Disable Bluetooth on an Android 10+ test device. Launch the app. Verify that a native system runtime prompt automatically appears within 1.0 second requesting permission to enable Bluetooth. * [ ] **Graceful Snapping/Disconnect:** While a rhyme is actively playing out loud from the toy's speaker, instantly turn off the phone's Bluetooth tray. Verify that the toy immediately silences the audio track cleanly (no pops, clicks, or permanent white noise hissing), streams the local connection loss speech file, and outputs a Solid Red visual signal for 3 seconds before shutting down. * [ ] **Multi-Device Handshake Override:** Force a drop on Phone 1. Open the App on Phone 2. Verify that Phone 2 can scan the peripheral, initiate a manual override via a physical squeeze of the heart switch, and completely assume operational control without firmware flashing. * [ ] **Deep Sleep Current Verification:** Connect a multimeter in series with the battery line. Leave the device disconnected or idle for 5 minutes. Verify that the device safely enters a sleep state drawing below 15 microamps to mathematically secure the 1-month battery life objective. --- ## 4. COMPREHENSIVE FUNCTIONAL SCENARIOS AND TECHNICAL HANDLING ### Scenario 4.1: Dynamic MTU Handshake and Packet Management * **Technical Requirement:** The Android app must programmatically request a Maximum Transmission Unit (MTU) size of 247 bytes immediately upon establishing a BLE link with the ESP32. * **Fallback Logic:** If a budget Android device refuses the 247-byte request and forces a smaller MTU (e.g., 23 bytes or 64 bytes), the app's BLE service must automatically slice the outgoing compressed audio stream into smaller sequential packets on the fly. It must append sequential packet headers to prevent out-of-order data corruption. ### Scenario 4.2: Audio Stream Queueing and Latency Jitter Protection * **Technical Requirement:** To mitigate connection drops or signal interference caused by physical obstacles, both the ESP32 firmware and the Android app must implement a Circular Ring Buffer (Data Queue). * **Playback Rule:** When streaming a nursery rhyme from the phone to the toy, the Android app must buffer the first 1.5 to 2.0 seconds of audio data into the ESP32's internal RAM before sending the explicit execution flag to activate the MAX98357A amplifier. If a packet drops mid-transmission, the ESP32 reads seamlessly from its local RAM queue while requesting a low-overhead background re-transmission. ### Scenario 4.3: Voice Activity Detection (VAD) and Ambient Noise Filtering * **Technical Requirement:** The system must protect backend resources by avoiding the transmission of dead silence or continuous household background noise. * **The Logic:** The Android app must monitor the incoming microphone stream in real time. If the audio level drops below a calibrated decibel threshold for 1.5 consecutive seconds, the app must immediately close the recording session, trim out the trailing silence, and pass the clean speech chunk to the processing engine. ### Scenario 4.4: Foreground Service Execution and OS Power Management * **Technical Requirement:** To prevent the Android operating system from forcefully sleeping or killing the BLE thread when the phone is locked, the companion application must execute within a persistent Android Foreground Service. * **The Logic:** The app must pin a low-priority, persistent media-streaming notification to the Android drop-down menu. This prevents background memory-clearing algorithms from severing the connection while the phone is asleep in a parent's pocket. ### Scenario 4.5: Automated Reconnection and State Recovery * **Technical Requirement:** If the connection drops because the toy is carried out of physical wireless range, the system must recover gracefully without user intervention. * **The Logic:** The mobile app must transition to a background scanning loop using the toy's explicit BLE UUID. The microsecond the toy's beacon signal is detected again, the app must re-pair, complete the MTU handshake, and return the system to an "Idling / Ready" state without freezing or crashing the application layer. ### Scenario 4.6: Wireless Disconnection and Tiered Failure Fallbacks * **No Phone Present / App Closed (Local Hardware Speech):** Upon power-on, the toy enters Advertising Mode for 30 seconds (indicated by a faint, slowly blinking Yellow light). If no paired phone responds within this window, the toy must fall back to Offline Play Mode. If the heart is squeezed while offline, the firmware plays a locally stored voice file from internal flash memory ("Let's wait for Mom's phone to connect!"). * **Sudden Mid-Sentence Disconnection (Local Hardware Speech):** If the child carries the toy completely out of range during active playback or recording, the ESP32 must instantly ground the audio output line, shut down the microphone, clear all active buffers, flash the LED ring Solid Red for 3 seconds, and play a local backup recovery file from SPIFFS flash memory: "Oops! I lost connection to the phone. Bring me closer!" before entering a low-power standby state. * **Cloud/Server Integration Processing Issues (App-Driven Speech):** If the toy streams audio successfully to the phone, but the phone's background internet connection drops or the intent processing engine returns an exception error, the Android App must intercept the failure. The app will compress and stream down a distinct error-dialogue block ("Oops, my brain is a little cloudy right now! Let's try asking again in a second.") to be broadcast out of the toy's speaker array, safely returning the system to the Ready state. ### Scenario 4.7: Multi-Device Takeover and Reset Logic (Lost Phone Protocol) * **The Logic:** The toy can only maintain an active pairing with one smartphone core at a time. If the paired phone is lost, switched off, or missing, the toy enters its Advertising State. * **The Takeover Handshake:** When a secondary device opens the App and requests a pairing, the toy flashes its LED halo ring yellow and plays an internal voice prompt: "Squeeze my heart to confirm new phone link." Upon a physical press of the heart membrane switch, the firmware clears its previous MAC bonding table, saves the new phone as the primary controller, and opens the operational pipeline. ### Scenario 4.8: Mid-Playback Interruption Override (Stop and Skip) * **The Logic:** If a child presses the physical heart button while the toy is actively playing back a nursery rhyme (Output Phase), the system must prioritize the new manual interaction. * **The Action:** The firmware must instantly stop the I2S Output DMA stream, clear the remaining audio queue in RAM, shut off the amplifier, and immediately jump back to the Talk Phase (Solid Green LED), playing the local "Hello" confirmation file and opening the microphone channel for new voice input within less than 50 milliseconds. --- ## 5. SYSTEM SCENARIO EXCEPTION AND INTEGRATION ARRAYS * **Rule 5.1 (Rapid Button Input):** The firmware must implement strict software debouncing (minimum 50ms) and lock out all hardware switch interrupts while the system is actively in the Processing phases. Repetitive button mashing must not freeze or crash the execution loop. * **Rule 5.2 (Audio Input Overload):** The application layer must gracefully intercept clipped or corrupted audio payloads. If an input stream fails to parse due to excessive amplitude or wireless frame corruption, the application must catch the exception and reset the toy to the Ready state without crashing the background service. * **Rule 5.3 (Hardware Brownout Protection and LDO Overhead):** The firmware's low-voltage monitoring routine must measure the raw battery input array prior to the LDO regulator. The critical low-power intercept threshold must be set to 3.6V to ensure the ESP32 retains enough overhead voltage to successfully execute the final low-power audio prompt, flash a "Low Battery" blink sequence, and transition cleanly into a deep-sleep state (under 15 microamps). * **Rule 5.4 (Codec Standardization):** The 4:1 compression engine utilized for both the upstream and downstream pipelines must adhere strictly to standard, open-source IMA-ADPCM formatting. Custom, proprietary, or closed compilation variations of the compression logic will be rejected. --- ## 6. TESTING APP (DIAGNOSTIC UTILITY PROFILE) The freelancer will provide a lightweight, single-screen diagnostic application to benchmark functional performance. * **Connection Interface:** Manual toggle button to Scan and Connect, displaying live logging for current connection state (DISCONNECTED, CONNECTING, CONNECTED) along with the negotiated MTU bandwidth value. * **Bidirectional Validation Stream Panel:** * *Upstream:* Visual text logger counting raw incoming audio packets in real-time when the toy's heart button is pressed, with an on-screen "Play Received Wave File" button to check mic performance via the phone's native speaker. * *Downstream:* A panel containing 3 test trigger modules ("Play Rhyme 1", "Play Rhyme 2", "Play 10-Second Continuous Sine Tone") that stream local compressed audio blocks over BLE down to the toy's speaker array. ---