Multi-Angle 2D to 3D Pose Reconstruction App (Desktop)

Posted 2 days ago

Worldwide

Summary

Freelancer Needed: 2D Image to 3D Pose Detection Desktop Tool I am looking for a developer to build a stable and reliable application for a desktop tool that converts 2D image sequences from different camera angles into usable 3D pose/animation data. The app will not take images, but will need to import images from multiple camera folders (from mocap sessions). The core purpose of the tool is to use images from at least two different angles, detect the humanoid skeleton in each view, triangulate the body position, and rebuild the pose in 3D. The exact pose detection model is open to discussion and may involve MediaPipe, OpenCV, existing pose estimation models or another suitable computer vision approach. In each image there will be a 5cm x 5cm Aruco markers present. Accuracy for this is key, as well as restricting joint/bone lengths (initial thinking is to map the output to a mixamo rig) The project includes two main areas of work: the desktop dashboard and the pose detection/reconstruction workflow. The desktop app should allow the user to import image batches from multiple camera angles, organise and match them by frame, and review the detection results. The user should be able to move frame by frame, view the original camera images, see the detected skeleton/bones over each image with a confidence score for each joint, manually adjust incorrect joints, recalculate the 3D pose after adjustments, preview the 3D pose, and export the final result as a video, a BVH and FBX. There should also be an option to save the animation and revisit it later within the app. I already have a desktop app layout/design that I want to use as the template, so the UI does not need to be invented from scratch. The goal is to turn that layout into a practical working tool. The end-to-end flow should be: 1. Open the desktop app 2. Import image batches from at least two camera angles 3. Automatically match/order frames correctly across the camera views 4. Detect the humanoid skeleton in each image 5. Show skeleton/bones over the original images and a confidence score/RAG status 6. Triangulate the detected joints to create a 3D pose preview 7. Allow the user to refine incorrect joint positions manually 8. Recalculate the 3D pose after corrections 9. Preview the 3D pose/animation (example attached to this project brief) 10. Export the final output into a usable format for further animation work, Blender-compatible, as well as a video of the animation for quick sharing. The system is designed to pick up a humanoid skeleton, but the exact model may change from time to time. I also want the system to learn from each manual correction or adjustment where possible. For example, if the user continually adjusts a missed joint, that correction should be saved and fed back into the workflow so future detections can improve over time. The head tracking needs to be super simple. 5 point detection is too complicated when looking to adjust across many frames. Important Requirements - Desktop app, not just Python scripts - Simple one-click launch - Batch image import - Minimum two-angle image input - Frame matching across camera views - Frame-by-frame navigation - Skeleton/bone overlay on original images - Manual joint/bone correction - 3D pose triangulation/reconstruction - 3D pose recalculation after corrections - 3D preview - Export to a usable animation/3D format & mp4 - Clean workflow for a non-technical user - Uses the provided dashboard layout as a template - Correction data should be stored so the tool can improve with future use Questions for Applicants 1. What technical approach would you use to detect a humanoid skeleton from two camera angles and triangulate it into a usable 3D pose? Does your approach require the same model to be used every time? 2. How would you handle manual joint corrections so the 3D pose updates properly after the user adjusts the skeleton? 3. How would you design the correction-learning feedback loop so the tool can improve over time from user adjustments? 4. How would you ensure extreme movements are always captured, even if they have to be manually adjusted? For example gymnastics or martial arts movements. Budget Please provide two prices: 1. Cost for the core working prototype, everything included in the brief except for the machine learning feedback loop. 2. Cost for the above version plus the correction-learning feedback loop, where manual adjustments are saved and used to improve future detection results. My target budget for a working version is under $1,000, so please be clear about what can be delivered. There will be future iterations of this software with more features but I am keeping it relatively simple for v1

  • $1,000.00

    Fixed-price
  • Intermediate
    Experience Level
  • Remote Job
  • One-time project
    Project Type
Skills and Expertise
Mandatory skills
Desktop Application
Computer Vision
Activity on this job
  • Proposals:20 to 50
  • Last viewed by client:yesterday
  • Hires:
    1
  • Interviewing:
    1
  • Invites sent:
    0
  • Unanswered invites:
    0
About the client
Member since May 31, 2016
  • United Kingdom
    Huddersfield7:39 PM
  • $8.7K total spent
    86 hires, 3 active
  • 1 hour

Explore similar jobs on Upwork

Crestron Programming
Extron Programming
IDE
Microsoft Word for Mac Expert NeededHourly‐ Posted 4 weeks ago
Microsoft Word
Office 365
Microsoft Office
Word Processing

How it works

  • Post a job icon
    Create your free profile
    Highlight your skills and experience, show your portfolio, and set your ideal pay rate.
  • Talent comes to you icon
    Work the way you want
    Apply for jobs, create easy-to-by projects, or access exclusive opportunities that come to you.
  • Payment simplified icon
    Get paid securely
    From contract to payment, we help you work safely and get paid securely.
Want to get started? Create a profile

About Upwork

  • Rating is 4.9 out of 5.
    4.9/5
    (Average rating of clients by professionals)
  • G2 2021
    #1 freelance platform
  • 49,000+
    Signed contract every week
  • $2.3B
    Freelancers earned on Upwork in 2020

Find the best freelance jobs

Growing your career is as easy as creating a free profile and finding work like this that fits your skills.

Trusted by

  • Microsoft Logo
  • Airbnb Logo
  • Bissell Logo
  • GoDaddy Logo