Software for Visual SLAM

# Computer Vision / SLAM Engineer for Distributed Edge-to-Server Visual SLAM System ## Project Overview We are looking for an experienced computer vision and SLAM engineer to design and implement a **SLAM / visual odometry pipeline** for a resource-constrained edge device. The system will use a distributed architecture: * An edge device processes the camera stream and performs feature detection. * Detected features and the required metadata are transmitted over Wi-Fi to a more powerful server. * The server performs visual odometry, mapping, triangulation, loop closure, localization, and tracking recovery. The target edge hardware has limited computing power and may be based on a Cortex-A55-class processor device. It may be possible to add an NPU providing approximately 1 TOPS of acceleration. ## Key Tracking and Mapping Constraints Tracking must be based on purpose-built floor patterns that serve as visual landmarks. Two types of patterns are available: 1. Geometric patterns created by arranging black and blue floor tiles. 2. Geometric symbols created from colored tape and attached to the floor. See attached images for a birds-eye perspective of the floor patterns. Reference images and recorded videos of both pattern types will be provided for development and evaluation. The proposed solution must support at least one of these two pattern types for: * Visual tracking * Map creation * Loop closure * Localization against a stored map * Relocalization after temporary tracking loss Support for both pattern types is preferred but is not mandatory. Note that when choosing the tape pattern, then the natural texture of the floor must not be relied upon for tracking or localization. Only the yellow/black pattern can be used as landmarks. The system should therefore remain functional when the surrounding floor material, texture, color, lighting, or environment changes. Candidates should explain how their proposed approach distinguishes the intended landmarks from unrelated visual information. ## Camera Input The supplied videos are recorded using a calibrated monochrome monocular camera with 60FPS, 1280x800 and wide angle field of view. Camera intrinsic parameters will be provided. No stereo camera, depth sensor, or IMU data will be available during this phase. ## Offline Map Creation Map creation will be performed offline using the supplied video recordings. This stage may use computationally intensive processing, including: Landmark triangulation Bundle adjustment Pose-graph optimization Loop closure Map refinement These operations are not required to run in real time. After optimization, the completed map must be serialized and stored on disk. ## Real-Time Localization The system must be able to reload the stored map and operate in localization-only mode. During localization-only operation: The stored map must remain fixed. No new landmarks should be added. No bundle adjustment or map optimization should be performed. Camera pose must be estimated using only the supported floor patterns and the stored map. The system must automatically relocalize after temporary or sporadic tracking loss. Tracking must resume without manual initialization or rebuilding the map. Real-time tracking and localization must operate at a minimum of 30 FPS, with 60 FPS preferred, on the agreed edge-to-server hardware architecture. ## Scope of Work The selected engineer will be responsible for: * Designing the complete edge-to-server SLAM architecture. * Implementing robust detection of the features. * Ensuring that floor texture and unrelated environmental features are excluded from tracking. * Optimizing the feature-detection stage for resource-constrained edge hardware. * Evaluating whether CPU-based processing is sufficient or whether the optional NPU should be used. * Designing an efficient Wi-Fi data format and communication interface for transmitting features, descriptors, timestamps, camera information, and other required metadata. * Implementing server-side visual odometry and SLAM. * Implementing landmark triangulation and map optimization. * Implementing loop detection and loop closure. * Implementing relocalization when tracking is lost. * Implementing a map visualization of the camera, frames and detected features * Measuring accuracy, latency, bandwidth consumption, and computational performance. * Calculate the device’s position and orientation in real time. * Map detected features into a persistent 3D coordinate system. * Localize the device against the previously created 3D feature map. * Handle motion blur, changing viewing angles, partial occlusion, and lighting variations. * Recover from temporary tracking loss where possible. * Run efficiently on mobile or AR-class hardware. The solution may be developed as a new SLAM pipeline or by extending an existing open-source system such as ORB-SLAM2, OpenVSLAM, or another suitable framework. Candidates should explain which approach they recommend and why. ## Expected Deliverables 1. **System Architecture and Technical Design** * Complete design of the distributed edge-to-server SLAM pipeline * Definition of the responsibilities, interfaces, data flow, and processing requirements for the edge device and server 2. **Edge-Side Feature Detection** * Feature detector or segmentation pipeline that identifies suitable features belonging exclusively to the patterns. * Prevention of floor texture, background objects, and unrelated environmental features from being used for tracking * Optimized implementation for the selected edge hardware and, where applicable, the optional NPU 3. **Feature Serialization and Communication** * Efficient serialization of detected features, descriptors, timestamps, frame metadata, and camera-calibration data * Wi-Fi communication interface between the edge device and server * Appropriate handling of latency, packet loss, dropped frames, and temporary communication interruptions 4. **Server-Side Visual SLAM Pipeline** * Visual odometry and camera-pose estimation * Landmark triangulation and map creation * Loop detection and loop closure * Bundle adjustment and offline map optimization * Recovery from temporary tracking loss 5. **Map Storage and Localization** * Serialization of the optimized map to disk * Ability to reload and reuse a previously generated map * Localization-only mode without adding landmarks, performing bundle adjustment, or modifying the map * Automatic relocalization against the stored map after temporary tracking loss 6. **Visualization and Diagnostics** * Visualization of the estimated camera trajectory * Visualization of detected features, mapped landmarks, tracking status, and relocalization events * Diagnostic output suitable for identifying tracking and communication problems 7. **Performance Evaluation** * Evaluation of: * Tracking and localization accuracy * Tracking stability and drift * Relocalization success rate and recovery time * Edge-side feature-detection latency * Server-side processing latency * End-to-end system latency * Processing frame rate * Network bandwidth usage * Behavior under packet loss, network jitter, and temporary disconnections 8. **Test and Evaluation Package** * Repeatable test procedure for map creation, localization, tracking loss, and relocalization * Documented acceptance-test results 9. **Source Code and Documentation** * Clean, maintainable, and documented source code * Build and dependency instructions * Configuration and deployment instructions * Hardware integration instructions * Testing and troubleshooting documentation 10. **End-to-End Demonstration** * A working demonstration on the agreed target hardware * The demonstration must include: * Edge-side stripe-feature detection * Feature transmission over Wi-Fi * Offline map creation and optimization * Map serialization and reloading * Real-time localization against the stored map * Temporary tracking loss followed by successful relocalization * Visualization of the estimated trajectory and mapped features ## Provided data 1. Camera intrinsics 2. Images and videos of the environment recorded using a monocular camera (the intrinsics are given) ## Acceptance Criteria 1. The final delivery must include a repeatable demonstration using the supplied video data. The demonstration must show: * Initial map creation * Camera trajectory estimation * Simulated or naturally occurring tracking loss * Successful camera relocalization * Continued localization within the existing map ## Required Experience Applicants must be able to demonstrate strong professional or academic experience in: * Visual SLAM and visual odometry * Monocular or stereo camera geometry * Feature detection and description * Feature matching and tracking * Camera calibration * Epipolar geometry and triangulation * Pose estimation and bundle adjustment * Loop detection and loop closure * Place recognition and relocalization * Robust estimation methods such as RANSAC * C++ and Python * OpenCV * Real-time or embedded computer vision * Performance optimization on ARM-based hardware Experience with one or more of the following is highly desirable: * ORB-SLAM2 or ORB-SLAM3, OpenVSlam, Kimera, or other related SLAM frameworks * g2o, Ceres Solver, or GTSAM * Learned feature detectors or segmentation models * NPU deployment, quantization, and model optimization * Embedded Linux or microcontroller-based vision systems * Wi-Fi-based edge-to-server processing pipelines * Tracking line-based or color-constrained landmarks ## Application Requirements Please include the following in your proposal: * Examples of previous SLAM, visual odometry, or 3D computer-vision projects. * A description of your exact contribution to each relevant project. * Links to source code, publications, demonstrations, or technical documentation where available. * Your proposed technical approach for detecting and tracking only the purple stripes. * Your recommended SLAM framework or architecture and the reasoning behind your choice. * Your approach to relocalization and loop closure. * Your strategy for dividing computation between the edge device and the server. * Your experience optimizing computer-vision algorithms for limited hardware. * Any technical risks or feasibility concerns you identify. * Your availability and hourly rate. Please also provide an **estimated number of hours**, preferably divided into phases such as: 1. Architecture and feasibility assessment 2. Prototype feature detection 3. Edge-device optimization 4. Server-side SLAM implementation 5. Loop closure and relocalization 6. Hardware and network integration 7. Testing, optimization, and documentation ## Engagement Model This is a **time-and-materials project**. Work will be billed hourly. We expect the project to be divided into measurable milestones, beginning with a technical feasibility and architecture phase. Each phase should produce testable deliverables before proceeding to the next stage. Applicants must provide an initial estimate of the required hours. The estimate may be refined after reviewing sample camera footage, hardware specifications, camera parameters, expected movement, stripe geometry, and accuracy requirements. ## Potential Follow-On Work Successful completion of this project may lead to a follow-on engagement. The next phase is expected to include: * Integration of IMU data to create a visual-inertial odometry or visual-inertial SLAM system * Improved robustness during rapid motion, motion blur, and temporary visual tracking loss * Sensor synchronization and camera–IMU calibration * Optimization for the final target hardware * Conversion of the prototype into a reliable, maintainable, and production-ready system * Performance validation, testing, documentation, and deployment support Candidates with experience in visual-inertial SLAM, embedded systems, hardware acceleration, and productionizing computer-vision pipelines are especially encouraged to apply.