Drones have been widely applied in the entertainment industry (such as TV program/film production), amateur photography, and even become a popular and trendy toy. With the ability to reach complex areas, their applications are gradually expanding into professional scenarios such as industrial inspection, logistics delivery, and security surveillance. But do you know that the core component supporting drone operations is the vision system? Before delving deeper into this topic, we will first clarify the definition of drones, outline their diverse application scenarios, and analyze the logic behind their rapid popularization.
Types and Applications
Drones belong to the category of Unmanned Aerial Vehicles (UAVs), also known as Unmanned Aircraft Systems (UASs), and in some cases, Remotely Piloted Aircraft (RPAs). They do not require onboard human pilots and can operate autonomously through various systems for navigation.
Drones are classified into three types: fixed-wing drones, single-rotor/multi-rotor drones, and hybrid-rotor drones. Each type serves different purposes, highly aligned with their intended application scenarios.
Fixed-wing drones are typically used for heavy-load transportation and long-endurance flight missions, with deployment scenarios including Intelligence, Surveillance, and Reconnaissance (ISR) missions, combat operations, loitering munitions deployment, mapping, and scientific research activities.
Single-rotor/multi-rotor drones have the broadest range of applications, covering industrial scenarios such as conventional warehousing, equipment inspection, and even logistics delivery. Due to the diverse scenarios these drones are suitable for, they require highly optimized electromechanical solutions to meet different needs.
Hybrid-rotor drones combine the advantages of the above two types, featuring Vertical Take-Off and Landing (VTOL) capabilities, thus offering more flexible application scenarios, especially in space-constrained areas. It is not surprising, therefore, that most logistics delivery drones choose this type.
Drone Motion and Navigation Systems
Drones are equipped with various sensors for motion and navigation, including accelerometers, gyroscopes, and magnetometers (collectively known as Inertial Measurement Units, or IMUs), barometers, etc. They employ various algorithms and technologies, such as optical flow (using depth sensors), Simultaneous Localization and Mapping (SLAM), and visual odometry. Although these sensors perform well, they often struggle to achieve the required precision and accuracy within reasonable cost and optimal size constraints. This issue is exacerbated during long-endurance flights, leading to the need for expensive batteries or reduced flight times due to battery charge-discharge cycle limitations.
Drone Vision Systems
Image sensors complement the functionality of the aforementioned sensors, significantly enhancing performance and transforming drones into highly accurate and precise devices. Vision systems mainly consist of two types of components: gimbals (often also referred to as payloads) and Visual Navigation Systems (VNS).
Gimbal – Provides First-Person View (FPV); typically integrates multiple image sensors covering a wide electromagnetic spectrum (in special cases, including ultraviolet sensors, while conventional CMOS image sensors cover the 300nm – 1000nm band, Short-Wave Infrared (SWIR) sensors extend to 2000nm, and Medium-Wave Infrared (MWIR) and Long-Wave Infrared (LWIR) sensors cover bands above 2000nm).
Visual Navigation System (VNS) – Used to provide navigation guidance, target recognition, and obstacle avoidance functions; typically composed of low-cost, low-resolution image sensors that, combined with IMU and other sensor data, construct a complete autonomous navigation solution through computer vision technology.
The Importance of Vision Systems
As mentioned in the previous section on uses and applications, drones can operate in both indoor and outdoor environments. These scenarios are often challenging, featuring not only a wide range of lighting variations but also visibility limitations in dust, fog, smoke, and pitch-black conditions. Drone systems need to process image data using a large number of Artificial Intelligence (AI) and Machine Learning (ML) algorithms, while also utilizing data provided by the aforementioned technologies. All of this must be achieved while enabling this highly optimized device to operate with low power consumption and achieve long-distance or long-endurance operations.
The data input into these algorithms must be of high fidelity and rich in detail, although in some usage scenarios, providing only necessary information is sufficient for efficient processing. The training time for AI/ML needs to be shortened, while the inference process should be rapid and possess high accuracy and precision. Regardless of the environment in which the drone operates, image quality must be guaranteed to meet these requirements.
Sensors that merely capture scenes and submit information for processing are far from sufficient to support high-quality operations of these devices and, in most cases, even fail to achieve their deployment objectives. The ideal sensor should possess the following capabilities: achieving miniaturization while retaining complete details of the area of interest; having a wide dynamic range to cope with bright and dark light in the same frame; minimizing or eliminating parasitic effects in the image; solving visibility issues caused by dust, fog, and smoke; and assisting image processing with high depth resolution. Such sensors will greatly contribute to transforming drones into highly optimized devices.
These capabilities minimize the resource scale required for image reconstruction, analysis, and accelerated decision-making processes, including processing cores, Graphics Processing Units (GPUs), on-chip or off-chip memory, bus architectures, and power management. This also reduces the Bill of Materials (BOM) cost of the entire system, especially considering that today’s drones can easily carry more than 10 image sensors. Additionally, with the same resource allocation, deeper analysis and more complex decision-assistance algorithms can be implemented, enabling drones to gain a differentiated advantage in competitive fields.
OMAGINE specializing in ODM PCB design, PCB assembly, open source hardware related modules and sourcing service.