Local Ollama / AI / LLM Infrastructure Expert — Open-Source Models
Worldwide
We are looking for an experienced AI/LLM infrastructure expert to consult on and potentially build a local, self-hosted AI environment for our business. Objective We currently rely heavily on LLM APIs for tools like OpenClaw and others and are expecting high token costs and API/rate limits. We want to evaluate and implement a local solution using hardware such as an Apple M3/M4 machine or NVIDIA GPU server or others and open-source models. Scope * Recommend the best hardware/configuration for our needs and budget. * Compare Apple Silicon vs. NVIDIA GPU solutions or others. * Set up and optimize local LLM inference. * Deploy models such as Llama and other appropriate open-source models. * Evaluate performance, scalability, context windows, and concurrent usage. * Recommend tools/frameworks such as Ollama, vLLM, or similar. * Help integrate the local models with our existing applications/APIs. * Provide a realistic cost comparison vs. OpenAI/Anthropic/API usage. * Design the solution so we can minimize dependence on external API limits and recurring token costs. Ideal Candidate * Strong experience deploying LLMs locally/on-premise. * Experience with open-source models. * Strong NVIDIA/CUDA and/or Apple Silicon experience. * Experience with Ollama, vLLM, Docker, APIs, and AI infrastructure. * Able to advise on both the architecture and actual implementation. We are open to starting with a paid consultation/architecture assessment and then moving to implementation if the proposed solution makes sense. Please provide examples of local LLM environments you have built, including the hardware, models, and approximate performance/cost savings.
$150.00
Fixed-price- ExpertExperience Level
- Remote Job
- Ongoing projectProject Type
Skills and Expertise
Activity on this job
- Proposals:20 to 50
- Last viewed by client:2 weeks ago
- Hires:1
- Interviewing:14
- Invites sent:18
- Unanswered invites:2
About the client
- United StatesSouthwick6:11 AM
- $26K total spent71 hires, 6 active
- 808 hours
- Media & EntertainmentSmall company (2-9 people)
Explore similar jobs on Upwork
How it works
Create your free profileHighlight your skills and experience, show your portfolio, and set your ideal pay rate.
Work the way you wantApply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
Get paid securelyFrom contract to payment, we help you work safely and get paid securely.
About Upwork
- 4.9/5(Average rating of clients by professionals)
- G2 2021#1 freelance platform
- 49,000+Signed contract every week
- $2.3BFreelancers earned on Upwork in 2020
Find the best freelance jobs
Growing your career is as easy as creating a free profile and finding work like this that fits your skills.
Trusted by