Firmware Audit: C++ on NVIDIA Jetson Edge Device

Vectech builds AI systems for vector surveillance, specializing in mosquitoes and ticks. Our flagship product, IDX, is a lab tool used by vector control organizations to image specimens and identify their species with computer vision. This engagement concerns the C++ firmware of that device. We are seeking a developer or firm to audit the firmware on our deployed IDX fleet: an independent assessment of correctness, robustness, failure behaviour and security posture, with prioritised recommendations. WHY THIS MATTERS The IDX device has been in production for 6 years without an external reliability assessment. As our operations expand, some devices have shown variability in the field, and field failures have required replacing units, which is expensive. Critically: these devices sit at third-party sites on networks we do not control, have no field serial console, and are expensive to recover. A device that fails before its command loop starts cannot be reached at all without manual intervention. We want the assessment weighted toward that operating profile rather than toward defect count. IN SCOPE - Boot and initialisation, including every failure path before the device is remotely reachable - The camera and capture pipeline - Concurrency, memory safety and lifetime correctness across threads - Network behaviour under degraded, filtered and stalled conditions - OTA update and provisioning - Failure modes: what is observable remotely, what self-heals, what needs physical intervention - Security review of network-facing and privileged paths, proportionate to a firmware audit and not a penetration test OUT OF SCOPE - Hardware, electrical, thermal and mechanical design - The ML model, its training and accuracy - The cloud backend and web app, except where firmware depends on them - Implementing the remediation WHAT WE PROVIDE (on NDA, at kickoff) - The firmware repository with tagged history and a written orientation to its layout and build - Toolchain and build tooling - A completed internal static audit with per-finding severity, confidence, evidence and fix ordering - Release history and per-device version inventory - Anonymised field reports with symptoms and resolutions - Provisioning tooling as needed to understand device identity and configuration - An engineer for design-intent questions - Test hardware On the internal audit: it is supplied so you understand what our internal investigation concluded. We want your independent judgement of it - what you would rank differently, what you think is wrong, what it misses. Forming your own conclusions on the highest-risk subsystems before reading it is welcome. REQUIRED EXPERTISE All required unless noted. A proposal that cannot evidence an area is unlikely to succeed. Embedded Linux on NVIDIA Tegra - Jetson / L4T: device tree, bootloader-supplied configuration, kernel module packaging, BSP versioning - Cross-compilation for aarch64 against a target sysroot, at working proficiency, not just able to read - systemd internals: unit ordering, restart policy and rate limiting, watchdog, journald. The target runs an older systemd; check recommendations against its feature set - Diagnosing a headless device with no network output and no console - Flash and SD behaviour: wear, write amplification, durability under power loss, what survives a reboot Camera, ISP and media - GStreamer: read, modify, instrument and debug a pipeline, including one not delivering buffers - The NVIDIA Argus stack (nvarguscamerasrc, Argus daemon): session lifecycle, daemon state, failure modes - MIPI CSI sensor and ISP behaviour. Exposure, gain and white balance are fixed rather than automatic, and image-quality faults are among the reported symptoms - Separating artefacts introduced by the sensor, the ISP, the encoder and our own processing Modern C++ - C++17 at auditing rather than authoring level; roughly 15 modules as shared libraries - Concurrency and lifetime correctness are the priority. The process runs an MQTT event loop, a capture worker, a multi-worker upload pool, an embedded interpreter thread, a Bluetooth thread and a watchdog. Ownership across threads, object versus callback lifetime, promise/future misuse and data races are the defect classes most likely present and least likely to be caught by reading - UB and memory safety, including the discipline to establish reachability rather than reporting a suspected site as certain - Dynamic analysis on aarch64: sanitizers, valgrind or equivalent, on-device or emulated Mixed-language runtime - Embedded CPython via the C API: GIL discipline, interpreter lifecycle, threading constraints - Error handling across the C++/Python boundary. How this goes wrong matters more than general Python skill. The device interpreter is 3.6-era and shipped Python must stay compatible Networking and cloud device management - MQTT v5: session persistence and expiry, reconnection semantics, will messages, client-identity rules - AWS IoT Core: shadows, policies, certificate auth, session semantics. Fleet-scale operational failure modes valued - libcurl in a multithreaded process: timeout semantics, connection versus transfer behaviour, retry design, thread-safety of global init - TLS/mTLS including clock dependence - NetworkManager (libnm) and D-Bus, including GLib/GObject ownership conventions - BLE/BlueZ/GATT. Bluetooth provisioning is the only pre-network path onto a device, so its correctness gates recoverability - Network fault injection: packet loss, DNS interference, captive portals, DPI middleboxes, and connections that handshake then stall. One of the most valuable skills in this list OTA and fleet management - Update mechanism design: atomicity, interruption, rollback, failure partway. The highest-consequence area in the codebase - the failure outcome is an unbootable device - Package signing and verification - The fleet could span multiple firmware versions concurrently, and devices do not necessarily pass through intermediate releases. Analysis must be version-aware and assume no particular prior release ML runtime - familiarity only. TensorRT engines are built and run on-device; enough to reason about build cost, load-time failure and startup impact. Model, training and accuracy expertise not required. Security - proportionate. Threat modelling a root process at a third-party site reachable over a cloud broker; secret handling at rest; input validation on privileged network-facing paths. Methodology and judgement - weighted heavily. Harder to evidence than the technical items, and matters more. - Auditing fielded devices that are expensive to recover, where a low-probability unrecoverable failure outranks a frequent cosmetic one - Reproduction-first: distinguishing "I reproduced this" from "I believe this follows from the code". We want a meaningful proportion of findings reproduced - Ranked, dependency-aware recommendations rather than a severity-labelled list - Assessing the risk of a fix, not only of the defect. With no rollback and limited remote recovery, some corrections are more dangerous than the problem. We expect that reasoning in your findings - Calibrated confidence. Fewer findings you are confident in, plus an explicit list of what you could not determine, beats a long list of mixed quality. We will act on these, so overstated certainty is harmful Valuable but not required: measurement or scientific instrumentation, or any domain where a silently wrong output is worse than a visible failure - much of this device's risk has that shape. Fleet operations experience. Devices in institutional networks with restrictive egress. LevelDB or similar. DELIVERABLES D1. Findings report. Per finding: description, affected version(s), evidence, reproduction status and method, severity and confidence with reasoning, risk of implementing the fix, consequence of not doing so. D2. Reproduction artefacts we can re-run, including any fault-injection tooling built during the engagement. D3. Independent verification of the supplied internal audit - disputed findings, recommendations you consider harmful. D4. Verification recommendations - what testing would have caught these, and the cheapest worthwhile version. D5. Explicit list of what you could not determine, and what access or time would close each item. D6. Walkthrough with our engineering team, plus 30 days of follow-up availability. PROPOSAL FORMAT Eight focused pages beat forty of boilerplate. Please cover: - Understanding of the problem (max 2 pages) - Approach: how you will decide what to reproduce versus reason about - Named individuals who will do the work, with allocation, and the prior projects that evidence each area of the Required Expertise section - Phase plan, milestones and dependencies on us - Assumptions, risks and exclusions - Two references, ideally embedded or fielded-device work Work begins under NDA. Please note in your proposal that you are able to sign one.