Multi-Camera Test Platform for ModalAI
ModalAI, a leader in autonomous drone and robotics solutions, collaborated with tinyVision.ai to develop a modular and scalable camera testing system aimed at automating and accelerating their production camera validation workflows. The goal was to reduce test times to enable scaling up camera production, manual labor, and ensure consistent quality.
The collaboration led to the successful design and deployment of a multi-camera testing platform using tinyVision.ai’s low-power, FPGA-based tinyCLUNX system, significantly reducing camera testing time and increasing throughput while maintaining precision and flexibility. A later phase added focus-assist tooling and support for additional sensors as ModalAI scaled up camera testing to meet an accelerated timeline for the U.S. War Department's Drone Dominance Gauntlet program.
Client Background
ModalAI Inc. develops autonomous flight and navigation systems for drones and robots. With a focus on compact, embedded intelligence, ModalAI integrates cutting-edge vision, communication, and processing technologies into their platforms. As their camera production volumes increased, they sought a robust testing infrastructure that could scale with their product roadmap and maintain high standards of reliability.
The Challenge
ModalAI needed a camera testing system that could:
-
Support simultaneous testing of up to four cameras
-
Work with diverse sensor modules
-
Minimize latency and initialization time for real-time validation
-
Interface cleanly with existing ModalAI hardware using their adapter footprint
-
Be easily maintained and upgraded for future sensors
-
Give test operators reliable, repeatable focus assistance across sensor types to reduce manual calibration error
-
Key technical constraints included FPGA resource limits for multi-camera streaming, USB3.2 data throughput ceilings (3.5Gbps), and the need for rapid, repeatable test sequences with minimal host-side software dependencies.
Our Solution
tinyVision.ai designed and delivered a custom, production-grade camera testing platform using the tinyCLUNX33 SoM based on the Lattice CrossLinkU-NX33 FPGA. The solution was architected with modularity and reusability at its core:
Hardware Platform:
- tinyCLUNX33 SoM with 14 differential pairs supporting MIPI and USB3.2
- Adapter boards for ModalAI sensors
Firmware & RTL:
- Zephyr-based drivers for optical sensors
- Custom FPGA RTL for real-time MIPI to UVC streaming
- Focus-assist algorithm that guides operators to the optimal lens focus point during test
Integration:
- USB UVC pipeline optimized for <100ms image latency
Key innovations included tightly packed RTL design to overcome FPGA resource limitations, modular adapter cards enabling sensor swap without mainboard redesign, and a focus-assist algorithm that turns manual lens calibration into a guided, repeatable step in the test flow.
Development Process
The development followed a phased model:
Phase 1 (Production Scaling):
- Designed hardware for 4-camera test systems
- Implemented host image logging and multi-camera control features
Phase 2 (Sensor Expansion & Focus Assist):
- Added support for IMX412 and IMX664 2-lane MIPI sensors alongside the existing dual AR0144 test path
- Developed a focus-assist algorithm and unified Linux GUI that guides test operators to the optimal focus point for each sensor
- Ported Zephyr drivers for the new sensors and validated real-time MIPI-to-UVC streaming at the higher data rates they require
Throughout the project, tinyVision.ai maintained agile iterations with ModalAI engineers, using early place-route analysis to mitigate FPGA resource risks.
Results & Impact
- Testing throughput increased many times over their earlier manual test which included assembling a drone to test the camera and increased further with simultaneous 2-camera testing
- 2s camera initialization and <100ms image latency enabled near real-time validation
- The Focus-assist algorithm now guides operators to optimal focus across AR0144, IMX412, and IMX664 sensors, cutting manual calibration error
- Development time reduced by leveraging reusable hardware modules
- Hardware BOM cost optimized with in-house design and manufacturing
- ModalAI now has a production-ready platform adaptable for future sensors
Qualitative feedback from ModalAI engineers highlighted the robustness and ease of use of the system in production environments.
Technologies Used
- FPGA Platform: Lattice CrossLinkU-NX33 on tinyCLUNX33 SoM
- Processors: RISCV (Zephyr RTOS)
- Camera Modules: AR0144, IMX412, IMX664 MIPI image sensors
- Interfaces: MIPI (1/2/4-lane), USB 3.2 Gen 1 (UVC)
- Software: Zephyr RTOS
- Development Tools: Lattice Radiant, Zephyr SDK, custom FPGA IP
These platforms were selected for their small form factor, high-speed I/O, and flexibility in RTL customization.
Conclusion / Key Takeaways
The ModalAI collaboration showcases tinyVision.ai’s ability to deliver tailored, scalable embedded vision solutions — including camera tuning and ISP work like focus assistance, not just board bring-up. From rapid prototyping to production deployment, tinyVision.ai’s engineering team provided:
- Expertise in low-power, high-performance vision hardware
- Efficient multi-camera system architecture
- Adaptable firmware and FPGA design
- Camera tuning and focus-assist algorithm development
By building on a modular design philosophy and leveraging their tinyCLUNX platform, tinyVision.ai enabled ModalAI to significantly streamline their camera validation pipeline.
Looking to accelerate your embedded vision project? Contact our engineering team to discuss your requirements.
Executive Summary
tinyVision.ai partnered with ModalAI to develop a modular, high-performance camera testing platform using the tinyCLUNX SoM. Supporting simultaneous validation of multiple sensors, the system enabled ModalAI to increase test throughput, reduce latency, and future-proof their validation process. A later phase added a focus-assist algorithm and support for additional MIPI sensors, helping ModalAI scale camera production and testing. The solution leveraged Lattice FPGA technology, custom adapter boards, and Zephyr-based drivers for a complete end-to-end platform.
Connect us