Build an Autonomous Self-Developing Multi-Agent "Jarvis" Setup
Worldwide
Job Description Project Overview: I am seeking a highly skilled AI Systems Engineer / Python Developer to integrate a fully autonomous, local-first multi-agent AI environment on my computer. The objective is to build a "Jarvis-like" personalized system: it must be entirely voice-activated, coordinate multiple specialized AI "brains" working as a team, run software and terminal scripts autonomously, manage GitHub repositories, and use an Obsidian vault as its persistent memory bank. I have all the foundational pieces installed and accounts ready: OpenAI API, Gemini API, Anthropic API, Claude Code CLI, OpenAI Whisper (local setup), and Obsidian. I also have cloned OpenJarvis and MiniClaw, but I need an expert engineer to debug the environment, resolve system misconfigurations, and securely tie these components into a unified background system. Key Responsibilities & Architecture Goals 1 Debug & Configure the Core Frameworks: Audit, debug, and correctly initialize OpenJarvis and MiniClaw on my local environment. Ensure their local background daemons/schedulers run continuously and error-free. 2 Implement the Multi-Agent "Team" Logic: Configure OpenJarvis/Gemini as the "Orchestrator & Scholar" to handle high-level task breakdown, deep research, web scraping, and large file ingestion. Configure MiniClaw / Claude Code CLI as the "Operative" to execute terminal actions, manage file structures, interact with GitHub, and download/run third-party software. Configure an OpenAI (GPT-4o) layer to act as a system code reviewer/critic to safely parse agent scripts and intercept infinite execution bugs. 3 Voice Command Pipeline (Whisper Integration): Map a global keyboard shortcut to trigger my pre-installed local Whisper setup. The system must capture microphone input, transcribe it instantly, and automatically route the payload text as an actionable prompt directly into the agent team. 4 Obsidian Integration (The Brain Storage): Point the multi-agent system directly to a dedicated directory inside my Obsidian Vault. The AI team must dynamically read tasks from an index note, log its active memory/session summaries to markdown, and deposit research summaries directly into my knowledge graph. 5 Autonomy & Self-Development Loop: Leverage MiniClaw’s Metabolic Harvesting and OpenJarvis's trace-driven learning loops to enable self-modification. When the agent team writes a successful script to solve a repeated task, it must automatically convert that sequence into a permanent, reusable tool in its ecosystem. Technical Requirements Deep, proven experience engineering with local AI agent frameworks (OpenJarvis, MiniClaw, OpenClaw, or advanced custom multi-agent architectures). Expert-level comprehension of the Model Context Protocol (MCP) and Agent-to-Agent (A2A) communications. High proficiency with the Claude Code CLI tool and programmatic shell interaction (subprocess, pexpect, or Node terminal hooks). Native fluency with cloud AI APIs (Anthropic, OpenAI, Google Gemini) including structured JSON outputs and working with massive context windows. Strong background in Python, Bash/Zsh scripting, local process monitoring, and handling background daemons. Familiarity with local audio input streaming tools to bridge hardware recording with local Whisper CLI execution. Solid understanding of markdown formatting and semantic file indexing for Obsidian vault integration. Deliverables A fully operational, robustly integrated multi-agent background system customized to my local machine. A working, global system hotkey script linking my Whisper audio capture directly into the agent network. A cleanly mapped Obsidian folder configuration allowing the AI team to read tasks, log history, and store markdown research profiles natively. Detailed, readable markdown documentation outlining how to monitor the system logs, how to handle safe shell whitelisting (preventing destructive commands), and how to verify background daemon operations. When applying, please briefly describe your past experience working with Claude Code or Model Context Protocol (MCP), your familiarity with OpenJarvis/MiniClaw, and your technical strategy for linking a local script pipeline to an Obsidian vault. Computer: threadripper 9995wx, 512 gb ram, rtx pro 6000 blackwell 96 gb Windows OS My goal is to be able to download or have it connect to a bunch of different repository basically programs from GitHub that do cyber security OSINT and other things like that and be able to do what I need it to do. Basically, I can tell it download this software and connected to your brain or whatever it’s called and it will be able to do that and use it also to be able to connect to my websites through the github repo ect https://www.tiktok.com/t/ZTBAkrnTL/ https://www.tiktok.com/t/ZTBAkRdvJ/ Not hourly. Looking for someone to do offer of exactly the price it will cost. this will be full new build. i have stuff downloaded and installed ect but never figured out how to get it working Openjarvis isnt working. no miniclaw yet. open to changing things. api:claude, gemini, openai, ect When you reach out please tell me your budget. would like budget total price not hourly.
- Less than 30 hrs/weekHourly
- 1-3 monthsDuration
- ExpertExperience Level
- Remote Job
- Ongoing projectProject Type
Skills and Expertise
Activity on this job
- Proposals:20 to 50
- Last viewed by client:3 weeks ago
- Interviewing:11
- Invites sent:13
- Unanswered invites:1
About the client
- USACasa Grande8:45 AM
- 1 hire, 1 active
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by