Build an Autonomous Self-Developing Multi-Agent "Jarvis" Setup

Posted 4 weeks ago

Worldwide

Summary

Job Description Project Overview: I am seeking a highly skilled AI Systems Engineer / Python Developer to integrate a fully autonomous, local-first multi-agent AI environment on my computer. The objective is to build a "Jarvis-like" personalized system: it must be entirely voice-activated, coordinate multiple specialized AI "brains" working as a team, run software and terminal scripts autonomously, manage GitHub repositories, and use an Obsidian vault as its persistent memory bank. I have all the foundational pieces installed and accounts ready: OpenAI API, Gemini API, Anthropic API, Claude Code CLI, OpenAI Whisper (local setup), and Obsidian. I also have cloned OpenJarvis and MiniClaw, but I need an expert engineer to debug the environment, resolve system misconfigurations, and securely tie these components into a unified background system. Key Responsibilities & Architecture Goals 1 Debug & Configure the Core Frameworks: Audit, debug, and correctly initialize OpenJarvis and MiniClaw on my local environment. Ensure their local background daemons/schedulers run continuously and error-free. 2 Implement the Multi-Agent "Team" Logic: Configure OpenJarvis/Gemini as the "Orchestrator & Scholar" to handle high-level task breakdown, deep research, web scraping, and large file ingestion. Configure MiniClaw / Claude Code CLI as the "Operative" to execute terminal actions, manage file structures, interact with GitHub, and download/run third-party software. Configure an OpenAI (GPT-4o) layer to act as a system code reviewer/critic to safely parse agent scripts and intercept infinite execution bugs. 3 Voice Command Pipeline (Whisper Integration): Map a global keyboard shortcut to trigger my pre-installed local Whisper setup. The system must capture microphone input, transcribe it instantly, and automatically route the payload text as an actionable prompt directly into the agent team. 4 Obsidian Integration (The Brain Storage): Point the multi-agent system directly to a dedicated directory inside my Obsidian Vault. The AI team must dynamically read tasks from an index note, log its active memory/session summaries to markdown, and deposit research summaries directly into my knowledge graph. 5 Autonomy & Self-Development Loop: Leverage MiniClaw’s Metabolic Harvesting and OpenJarvis's trace-driven learning loops to enable self-modification. When the agent team writes a successful script to solve a repeated task, it must automatically convert that sequence into a permanent, reusable tool in its ecosystem. Technical Requirements Deep, proven experience engineering with local AI agent frameworks (OpenJarvis, MiniClaw, OpenClaw, or advanced custom multi-agent architectures). Expert-level comprehension of the Model Context Protocol (MCP) and Agent-to-Agent (A2A) communications.  High proficiency with the Claude Code CLI tool and programmatic shell interaction (⁠subprocess⁠, ⁠pexpect⁠, or Node terminal hooks). Native fluency with cloud AI APIs (Anthropic, OpenAI, Google Gemini) including structured JSON outputs and working with massive context windows. Strong background in Python, Bash/Zsh scripting, local process monitoring, and handling background daemons. Familiarity with local audio input streaming tools to bridge hardware recording with local Whisper CLI execution. Solid understanding of markdown formatting and semantic file indexing for Obsidian vault integration. Deliverables A fully operational, robustly integrated multi-agent background system customized to my local machine. A working, global system hotkey script linking my Whisper audio capture directly into the agent network. A cleanly mapped Obsidian folder configuration allowing the AI team to read tasks, log history, and store markdown research profiles natively. Detailed, readable markdown documentation outlining how to monitor the system logs, how to handle safe shell whitelisting (preventing destructive commands), and how to verify background daemon operations. When applying, please briefly describe your past experience working with Claude Code or Model Context Protocol (MCP), your familiarity with OpenJarvis/MiniClaw, and your technical strategy for linking a local script pipeline to an Obsidian vault. Computer: threadripper 9995wx, 512 gb ram, rtx pro 6000 blackwell 96 gb Windows OS My goal is to be able to download or have it connect to a bunch of different repository basically programs from GitHub that do cyber security OSINT and other things like that and be able to do what I need it to do. Basically, I can tell it download this software and connected to your brain or whatever it’s called and it will be able to do that and use it also to be able to connect to my websites through the github repo ect https://www.tiktok.com/t/ZTBAkrnTL/ https://www.tiktok.com/t/ZTBAkRdvJ/ Not hourly. Looking for someone to do offer of exactly the price it will cost. this will be full new build. i have stuff downloaded and installed ect but never figured out how to get it working Openjarvis isnt working. no miniclaw yet. open to changing things. api:claude, gemini, openai, ect When you reach out please tell me your budget. would like budget total price not hourly.

  • Less than 30 hrs/week
    Hourly
  • 1-3 months
    Duration
  • Expert
    Experience Level
  • Remote Job
  • Ongoing project
    Project Type
Skills and Expertise
Mandatory skills
AI Development
AI Agent Development
Activity on this job
  • Proposals:20 to 50
  • Last viewed by client:3 weeks ago
  • Interviewing:
    11
  • Invites sent:
    13
  • Unanswered invites:
    1
About the client
Member since May 28, 2026
  • USA
    Casa Grande8:45 AM
  • 1 hire, 1 active

Explore similar jobs on Upwork

Software DeveloperHourly‐ Posted 7 months ago
ASP.NET MVC
Django
Python
AngularJS
JavaScript
jQuery
WordPress
Google Chrome Extension
React
CRM Development
Microsoft Dynamics 365
Microsoft Dynamics CRM
Microsoft Dynamics Development
Microsoft PowerApps
Single Sign-On
Build Marketplace on TokopediaHourly‐ Posted 4 weeks ago
PHP
HTML5
JavaScript
Web Development

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo