3d capture lidar

Expert iOS ARKit/LiDAR Engineer to Fix 3D Scanning and Measurement Drift Project summary We are looking for an expert native iOS developer with strong practical experience in ARKit, iPhone LiDAR, 3D reconstruction, computer vision and photogrammetry. We have an existing iOS application that scans apartments using an iPhone or iPad equipped with LiDAR. The application currently generates: A colored point cloud A 3D mesh A 2D apartment floor plan Wall and room measurements Individual room areas Total apartment area An interactive 3D viewer The application is functional, but it currently has accuracy, reconstruction quality, drift and 3D-viewer problems. We need an expert to audit the existing code, identify the root causes and implement measurable improvements. This is a technical 3D computer-vision assignment, not a general iOS UI-development project. Current problems The main problems include: Accumulated measurement drift while scanning multiple rooms Misalignment when returning to an area that was scanned earlier Duplicate or overlapping walls Inaccurate wall dimensions Incorrect room and apartment-area calculations 2D floor plans that do not always match the 3D scan Noisy or incomplete point clouds Rough, unoptimized architectural meshes Uneven walls, floors and ceilings Poorly reconstructed corners, doors, windows and openings Low-quality furniture reconstruction Chairs and thin furniture appearing melted, damaged or incomplete Blurry, stretched or incorrectly aligned textures Bugs in model loading, camera controls and 3D navigation Performance problems with larger apartment models Crashes or excessive memory consumption in the 3D viewer Project objective The objective is to repair and improve the existing application so that it produces two aligned models. Measurement model The measurement model must contain clean and dimensionally reliable: Walls Floors Ceilings Doors Windows Openings Room boundaries Closed room polygons Wall measurements Room areas Total apartment area All measurements must be calculated from the optimized architectural geometry, not from a decorative visual mesh. Visual model The visual model must provide: Clean architectural geometry Attractive wall, floor and ceiling surfaces High-quality textures Improved furniture presentation Consistent lighting and materials Smooth and stable viewing performance The measurement model, point cloud, visual mesh and 2D floor plan must remain aligned in the same coordinate system and metric scale. Scope of work 1. Audit the existing application The first task is to build and investigate the existing Xcode project. The developer must: Build and run the current application. Reproduce the reported problems. Review the ARKit session configuration. Review camera, RGB, depth, IMU and timestamp synchronization. Review coordinate systems and transformation matrices. Confirm that metric units are used consistently. Review how LiDAR depth is converted into world-space points. Review the current point-cloud fusion method. Review the current mesh-generation method. Review wall, room and opening detection. Review the 3D-to-2D floor-plan conversion. Review room-area calculations. Review RGB texture generation. Review the 3D viewer and rendering architecture. Profile CPU, GPU and memory usage. Identify crash conditions and memory leaks. Determine whether the current image-frame pipeline is true photogrammetry, RGB texturing or only tracking support. Preserve existing failed scans as regression tests. The audit must result in a written technical report explaining: Problems reproduced Root causes identified Components that can remain Components that require correction Components that may require refactoring Proposed implementation approach Expected technical limitations Proposed testing procedure Realistic delivery schedule Major development should begin only after the audit findings are reviewed. 2. Correct tracking and measurement drift The developer must repair or implement a reliable capture and tracking pipeline using: ARKit camera poses LiDAR depth Depth-confidence information RGB camera frames Camera intrinsics and calibration data IMU/device-motion information Accurate timestamps Device orientation ARKit tracking-quality information The solution should include: Monitoring of ARKit tracking quality User warnings when tracking becomes unreliable Pausing or limiting data integration during poor tracking Guidance to maintain sufficient scan overlap Selection and storage of useful keyframes Detection of previously scanned areas Relocalization Loop-closure detection Pose-graph optimization or an equivalent correction method Rejection of incorrect loop closures Geometric registration where appropriate Rebuilding or reintegrating the point cloud after pose correction Rebuilding the mesh after pose correction Detection and merging of duplicated walls Preservation of correct metric scale The developer may recommend additional accuracy controls such as: Printed coded targets Known-distance measurements Wall-to-wall constraints Floor and ceiling constraints Manual control points for difficult scans If the current application modifies or replaces ARKit camera poses incorrectly, that behavior must be removed or corrected. 3. Improve point-cloud reconstruction The point-cloud pipeline should be reviewed and improved using appropriate methods such as: Depth-confidence filtering Distance filtering Temporal filtering Statistical outlier removal Radius outlier removal Voxel downsampling Correct world-coordinate transformations RGB and depth alignment Normal estimation Multi-frame fusion Duplicate-point removal Registration-quality scoring Preservation of architectural edges The developer must evaluate whether the project should use: TSDF fusion Surfel fusion Voxel-based fusion A hybrid method Another justified reconstruction approach The final point cloud must: Use the correct metric scale. Remain aligned with the floor plan and mesh. Avoid obvious duplicate walls. Avoid severe floating noise. Preserve useful architectural detail. Remain manageable on the supported iPhone. Export successfully in at least one agreed point-cloud format. 4. Correct 3D-to-2D floor-plan generation The floor plan must be generated from optimized architectural geometry rather than from the raw mesh outline. The developer must implement or repair: Floor-plane detection Ceiling-plane detection Wall-plane detection Grouping of observations belonging to the same wall Stable plane fitting Wall-intersection calculation Corner generation Door detection Window detection Opening detection Multi-room alignment Closed room-polygon construction Invalid or incomplete room detection Wall-length calculation Room-dimension calculation Individual room-area calculation Total apartment-area calculation Measurement-confidence reporting Net room area should be calculated from the finished internal wall faces unless another measurement convention is agreed upon. The application should automatically identify incomplete or low-confidence rooms rather than silently reporting inaccurate results. A limited manual correction tool may be proposed for cases where full automation cannot produce a reliable result. 5. Generate a clean architectural mesh The final mesh should look deliberately modeled rather than like an untreated LiDAR scan. The developer must implement or improve: Removal of floating triangles Removal of duplicated geometry Removal of severe spikes Planar correction of walls Planar correction of floors and ceilings Clean wall intersections Clean floor and ceiling connections Clean door and window openings Mesh-normal correction Removal of inverted surfaces Controlled hole filling Mesh simplification Level-of-detail generation Mobile performance optimization Mesh cleanup and simplification must not change the dimensions of the metric architectural model. Before applying the final visual treatment to an entire apartment, the developer must provide one completed sample room for visual approval. 6. Produce high-quality textures and materials The visual pipeline should include: Selection of sharp RGB keyframes Correct projection of RGB images onto optimized geometry UV generation Texture-atlas generation Texture seam reduction Exposure correction Color correction Reduction of stretched textures Reduction of blurry textures Appropriate wall, floor and ceiling materials Mobile-optimized texture sizes Suitable lighting and shadows Ambient-occlusion support where practical The visual textures must stay aligned after geometry and pose optimization. 7. Improve chairs and other furniture Raw iPhone LiDAR often produces poor geometry for chairs, thin legs and complex furniture. The developer should not attempt to solve this only by smoothing the damaged raw mesh. The preferred workflow is: Detect or classify the furniture category. Estimate its position, rotation and dimensions. Remove or hide the damaged raw furniture geometry. Select an appropriate high-quality model from an approved asset library. Scale the model to match the detected dimensions. Position and orient it correctly. Allow the user to select another replacement model when necessary. Initial furniture categories should include: Chairs Sofas Tables Beds Desks Cabinets Wardrobes Televisions Kitchen units Toilets Sinks Replacement furniture will be used for attractive visualization. It must not affect room measurements or apartment-area calculations. Any third-party models, materials or software dependencies must be properly licensed for commercial use and disclosed before integration. If exact reconstruction of a particular furniture item is required, the developer may propose a separate guided photogrammetry or object-capture workflow. 8. Confirm and improve photogrammetry functionality The application may already save RGB frames during scanning. The selected expert must determine how these frames are currently used. If photogrammetry is proposed or claimed, the developer must explain the implementation of: Image quality and keyframe selection Feature detection and matching Camera-pose estimation Structure-from-Motion Bundle adjustment Multi-view reconstruction LiDAR and photogrammetry fusion Texture generation Simply extracting frames from a video does not constitute a complete photogrammetry pipeline. The developer should recommend whether apartment-scale photogrammetry should run: On the iPhone On a Mac On a backend processing service As an optional high-quality mode Not at all, if it does not provide sufficient benefit Any backend or paid processing service requires approval before implementation. 9. Repair the 3D viewer The developer must diagnose and fix reproducible viewer problems involving: Crashes Memory leaks Excessive CPU usage Excessive GPU usage Slow loading Incorrect scale Incorrect orientation Missing textures Black surfaces Incorrect normals Z-fighting Camera clipping Orbit controls Pan controls Zoom controls Models appearing outside the camera view Freezing when switching models Large apartment models The final viewer should provide: Orbit Pan Zoom Reset camera Fit model to screen Top view Perspective view Room selection Point-cloud visibility control Measurement-model visibility control Visual-model visibility control Basic measurement inspection The expected minimum performance target is stable interactive viewing at approximately 30 FPS on the agreed test device for the largest agreed apartment model. 10. Testing and validation Testing will use three real apartments with different layouts. Each apartment should be scanned multiple times using the same documented procedure. Ground-truth measurements will be collected using a suitable laser distance meter. The developer must provide: A repeatable testing procedure Baseline results from the current application Final results after repairs Measurement-error tables Area-error results Loop-closure results Repeatability results CPU, GPU and memory results Before-and-after screenshots or videos A list of remaining limitations Target acceptance criteria The following are proposed MVP acceptance targets. Any necessary changes must be explained and agreed during the audit milestone. Measurement accuracy At least 90% of tested wall measurements below 5 metres must be within ±3 cm of ground truth. At least 90% of tested wall measurements from 5–10 metres must be within ±5 cm. Individual room-area error must be within ±3%. Total net apartment-area error must be within ±2%. Repeated scans of the same apartment must produce total areas within 2% of one another. Drift Returning to the starting area should result in no more than 5 cm of closure error in the agreed test case. Previously scanned rooms must not create visibly duplicated copies. There should be no clearly visible duplicate architectural wall more than 3 cm from the corresponding wall. Incorrect loop closures must not seriously deform the apartment. Floor plan Completed rooms must form valid closed polygons. Connected rooms must use consistent walls and openings. Measurements must remain consistent after saving and reopening. Low-confidence results must be visibly identified. Visual model Walls should look straight and clean. Floors and ceilings should look flat. Corners and openings should be properly formed. There should be no major spikes, double walls or inverted surfaces. Textures should not be widely black, stretched or incorrectly projected. Supported furniture categories should use clean visual representations. Viewer The viewer must complete a continuous 30-minute test without crashing. All agreed camera controls must work. Repeatedly opening and closing models must not cause uncontrolled memory growth. The largest agreed model must remain usable on the test device. Deliverables The developer must deliver: Complete updated source code Working Xcode project All new Swift, Objective-C++, C++, Metal and shader source files Dependency and license list Audit report Architecture documentation Coordinate-system and unit documentation Reconstruction-pipeline documentation Measurement-pipeline documentation Test procedure Baseline and final test results Regression-test instructions Reproducible build instructions Deployment instructions Known-limitations list Thirty days of defect correction for work included in this contract No undisclosed proprietary SDK, paid cloud service or restrictive software dependency may be added without approval. Milestones Milestone 1 — Technical audit: $500 Completed when: The existing project builds. Main problems are reproduced. A root-cause report is delivered. The repair plan is approved. Milestone 2 — Tracking and drift correction: $1,750 Completed when: Tracking-quality handling is implemented. Loop closure or an equivalent correction is demonstrated. Revisited-room duplication is corrected. Agreed drift and repeatability tests pass. Milestone 3 — Measurements and 3D quality: $1,500 Completed when: Wall and room extraction is corrected. Measurement and area tests pass. A clean architectural mesh is generated. The furniture-visualization workflow is demonstrated. The sample-room visual target is approved. Milestone 4 — Viewer, performance and exports: $750 Completed when: Reproducible viewer bugs are fixed. Performance and stability tests pass. Agreed export formats work at the correct scale. Milestone 5 — Final handover: $500 Completed when: Complete source code is delivered. The Xcode build is reproducible. Documentation and test results are delivered. Final acceptance testing is completed. Required experience Applicants must have demonstrated professional experience with: Native iOS and Swift ARKit and iPhone LiDAR 3D computer vision Point-cloud and mesh processing SLAM, relocalization or loop closure 3D rendering and performance optimization Photogrammetry or RGB texture reconstruction Experience with the following is highly desirable: RoomPlan RealityKit SceneKit Metal OpenCV Open3D or PCL GTSAM or Ceres C++ integration with Swift TSDF or surfel fusion ICP and point-cloud registration USD/USDZ, GLB, OBJ or PLY Working arrangement The selected developer will work independently on the existing codebase and communicate progress through Upwork. We will provide: Access to the existing source code after the contract begins Existing problem scans Reproduction information Test-device information Ground-truth apartment measurements Feedback on the sample visual model We need the selected expert to provide technical guidance regarding the best reconstruction, loop-closure, furniture and texture approaches. All communication and payments will remain on Upwork in accordance with Upwork’s rules.