You will get Edge AI on Rockchip RK3568: fast INT8 inference with proven accuracy
Project details
I developed an RK3568-based edge AI deployment project for real-time YOLO food detection on a smart refrigerator system. The work covered the full pipeline from PyTorch training and ONNX export to RKNN INT8 PTQ conversion, rknn-toolkit-lite2 board inference, GStreamer camera input, and real-time visualization.
What sets this project apart is that the optimization was driven by real board-side evidence, not estimation. I resolved key INT8 deployment issues including zeroed class channels, preprocessing mismatch, and real-scene calibration problems. Using staged benchmarking and RKNN layer profiling, I identified the true bottleneck: the YOLO DFL Softmax operator running on CPU. By moving DFL to confidence-filtered post-processing, inference p50 was reduced from 53.58 ms to 33.70 ms, and end-to-end FPS improved from 17.1 to 25.9. Accuracy was preserved on 93 annotated refrigerator images, reaching 1.00/1.00 precision and recall with 0 background false positives.
What sets this project apart is that the optimization was driven by real board-side evidence, not estimation. I resolved key INT8 deployment issues including zeroed class channels, preprocessing mismatch, and real-scene calibration problems. Using staged benchmarking and RKNN layer profiling, I identified the true bottleneck: the YOLO DFL Softmax operator running on CPU. By moving DFL to confidence-filtered post-processing, inference p50 was reduced from 53.58 ms to 33.70 ms, and end-to-end FPS improved from 17.1 to 25.9. Accuracy was preserved on 93 annotated refrigerator images, reaching 1.00/1.00 precision and recall with 0 background false positives.
Machine Learning Tools
NumPy, Open Neural Network Exchange, OpenCV, Python, PyTorchWhat's included
| Service Tiers |
Starter
$5
|
Standard
$10
|
Advanced
$15
|
|---|---|---|---|
| Delivery Time | 1 day | 1 day | 2 days |
Number of Revisions | 2 | 2 | 3 |
Model Validation/Testing | - | - | - |
Model Documentation | - | - | - |
Data Source Connectivity | - | - | - |
Source Code | - | - | - |
Optional add-ons
You can add these on the next page.
Fast Delivery
+$10
Model Validation/Testing
(+ 1 Day)
+$5
Model Documentation
(+ 1 Day)
+$3About Li
Edge AI & Computer Vision Engineer | C++ | CUDA | TensorRT
Chengdu, China - 3:07 am local time
I develop and optimize high-performance AI inference pipelines for edge devices and desktop systems.
My focus is deploying PyTorch and ONNX models as efficient, stable inference applications using C++, TensorRT, ONNX Runtime, CUDA, and OpenCV.
I solve the engineering challenges between a trained model and a production-ready application, including operator compatibility, numerical accuracy, inference latency, memory usage, and hardware-specific performance.
🔧 WHAT I CAN HELP WITH
📦 AI Model Deployment
PyTorch-to-ONNX and TensorRT conversion, inference pipeline development, and integration into C++ applications.
⚡ Inference Performance Optimization
FP16/INT8 quantization, INT8 calibration, operator fusion, memory optimization, and CPU/GPU performance bottleneck analysis.
🎯 Edge AI and Computer Vision
Performance-oriented object detection, image processing, YOLO deployment, and video inference on NVIDIA Jetson and Linux devices.
📊 Benchmarking and Profiling
Reproducible measurement of latency, throughput, memory usage, and accuracy, with clear comparisons before and after optimization.
🛠️ Debugging and Stability
Unsupported operator analysis, output mismatch investigation, runtime error diagnosis, fallback strategies, and inference pipeline debugging.
💻 TECH STACK
Languages: C++, Python
Inference: TensorRT, ONNX Runtime
Acceleration: CUDA, OpenCL, SIMD, ARM NEON
Computer Vision: OpenCV, YOLO, image and video processing
Deep Learning: PyTorch, model conversion and quantization
Platforms: Linux, NVIDIA Jetson, ARM, x86
Engineering: CMake, Git, profiling, benchmarking and debugging
📈 MEASURABLE RESULTS
Each optimization is benchmarked against agreed targets for latency, throughput, memory usage, numerical accuracy, and runtime stability.
📩 TO GET STARTED
Please share:
1. Your model format
2. Target hardware
3. Sample input and expected output
4. Current performance
5. Desired latency, throughput, memory, or accuracy target
I will review the requirements and propose a practical deployment and optimization plan.
Steps for completing your project
After purchasing the project, send requirements so Li can start the project.
Delivery time starts when Li receives requirements from you.
Li works on your project following the steps below.
Revisions may occur after the delivery date.
Review model & deployment requirements
Review your model, task type, class labels, preprocess settings, target platform, and FPS/latency goals. Confirm scope and delivery format before conversion starts.
Export & validate baseline model
Export or fix ONNX/TFLite if needed. Run host-side sanity checks on input/output shapes, class mapping, and a few sample images to establish a correct baseline.