BlackBerry QNX provides guidance on minimizing jitter, latency in robotics

Robots need to precisely synchronize for manufacturing applications such as assembly, welding, and materials handling. BlackBerry QNX recently released a whitepaper on “Optimizing Robotic Precision: Unleashing Real-Time Performance with Advanced Foundational Software Solutions.”

The document provides guidance to manufacturers on reducing jitter in high-speed robotic motion. Otherwise, it can lead to misaligned components, defective products, and a decrease in throughput and efficiency, said the company.

Founded in 1980, QNX supplies commercial operating systems, hypervisors, development tools, and support and services for critical embedded systems. Acquired by BlackBerry in 2010, the Ottawa, Canada-based unit serves industries including aerospace and defense, automotive, heavy machinery, industrial controls, medical, and robotics.

Register now.

BlackBerry QNX explains purpose of whitepaper

Louay Abdelkader, senior product manager at QNX, replied to the following questions about the whitepaper from The Robot Report:

Who is the target audience for this whitepaper?

Abdelkader: Our QNX whitepaper is to advise and inform those responsible for building the software that goes into automated guided vehicles (AGVs), autonomous mobile robots (AMRs), robot motion controllers. [They’re also responsible for] teach pendants for robot control, data collection and processing, mapping, image analysis, path planning, obstacle avoidance and autonomy.

For example, it is relevant to software engineers, developers and leaders, product/program managers, and other technical and non-technical audiences.

What are the conventional approaches for mitigating jitter and latency in robotics, and how do they fall short?

Abdelkader: As the needs for fully or even partially autonomous systems increase, the software stack becomes the centerpiece of these systems, and as such, they become more feature-rich and complex. Because these systems are safety-critical and require very reliable and deterministic behavior, “hard” real-time requirements become prevalent.

The limitations of general-purpose operating systems like Linux become more apparent due to its lack of safety certifications and hard real-time behavior.

There are several reasons why the switch from general-purpose OS that comes with “soft” real-time behavior to hard real-time OS (RTOS) makes sense. These include:

• Determinism and timing guarantees – Hard RTOS provide strict timing guarantees on response times and execution deadlines.
• Complexity management – As software in robots becomes more complex, ensuring that your OS can handle mixed-criticality tasks is essential. With Hard RTOS, there are features and capabilities that allow designers to use mixed-critical software in the same stack and ensuring that the right separation is provided to avoid cross-contamination in case of faults or safety/security-related events that could occur during the deployment of the system.
• Safety – Hard RTOS come with safety and security features baked in, because they are utilized in safety and mission-critical systems. And some hard RTOS, like the ones QNX provides, come with not only the safety features and capabilities, but also the safety certifications needed for specific environments including industrial automation.
• Fault tolerance and reliability – In safety-critical applications, such as collaborative robot applications and surgical robots, fault tolerance and reliability are paramount. Hard RTOS are often designed to be very robust with high mean time between failures, as well as mechanisms to handle faults and ensure continued operation even in the event of hardware failures or unexpected events. It is particularly effective with a microkernel architectural design where the kernel and OS services are separated. This ensures that if an OS service fails, it doesn’t contaminate the kernel and cause it to be affected or crash, which could bring the system down.

What are some examples of real-time operating systems for the “soft,” “firm,” and “hard” approaches?

Abdelkader: In the hard real time behavior, strict time constraints with guaranteed response times are expected. Missing a deadline is not an option, as the consequences are nothing short of catastrophic, especially in highly safety-critical applications.

Consider an AMR navigating a high-traffic warehouse, for example. Any delay in its ability to respond to obstacles and change direction could lead to collisions, potentially causing damage to goods and posing a safety risk to personnel. 

Soft real-time behaviors introduce a measure of flexibility where a system’s operations degrade if it cannot meet specific timing requirements. While these systems aim to meet deadlines, they can occasionally tolerate minor deviations without disastrous outcomes.

In an industrial setting, vision systems for inspection play a role here. These systems ensure the quality and accuracy of manufactured products, where minor delays in inspection may affect production efficiency but not result in severe consequences. 

The firm real-time behaviors are akin to soft real-time but with a slight difference. Data arriving after the deadline is often deemed invalid. A prime example in robotics is automated 3D printing systems.

In additive manufacturing, if a layer isn’t deposited precisely on time, it can result in defects in the final product. While minor deviations might not be catastrophic, they could lead to the rejection of a printed part, which can harm production efficiency and waste materials.

We’ve heard that artificial intelligence can get away from deterministic and rigid approaches to robotic reactions. Is that true yet?

Abdelkader: In robotic systems, low latency and jitter are an essential component for both AI and non-AI applications. In applications for robots, real-time control will continue to require low latency (minimal delay in processing) and low jitter (consistency in timing) to ensure safe and deterministic operation.

Determinism ensures that responses to sensor inputs or environmental changes happen predictably and within a specified timeframe. Where AI models are deployed on the edge, on devices with limited computation resources, deterministic processing can also help optimize resource utilization and ensure timely responses without unpredictable delays.

RTOS promises to improve robot reliability

How might QNX improve reliability and safety? Would multiple systems be needed while a component is restarted?

Abdelkader: The QNX RTOS is a hard real-time OS built with the microkernel architecture, renowned for its inherent security, safety, and reliability. This architecture isolates the kernel, which is the most important component of the OS, in its own memory space and operates the system services in their own memory space outside the kernel, which provides additional isolation and safety barriers within the OS.

By reducing complexity and potential failure points, QNX facilitates thorough verification through rigorous testing, including fault injection testing and formal methods. Faults within individual components are contained, enabling dynamic restarts without system-wide impact or shutdowns.

In safety-critical embedded systems, where the maxim “No safety without security” holds true, the microkernel’s small footprint enhances security practices and restricts privileged access. Moreover, its modular design allows for customization tailored to specific application requirements, making QNX an ideal choice for robotics systems prioritizing robust safety and reliability.

Does this RTOS require additional power or sensing? Are there minimum requirements? How ruggedized does it need to be?

Abdelkader: The QNX RTOS are found in microprocessors with memory management units running on either an Intel x86 or ARMv8 or ARMv9 processors. When comparting the size of QNX to Linux, the number of software lines of code is significantly smaller in the kernel.

As a result, it typically requires less memory and processing power to operate and bring up the kernel efficiently with the added benefits of improving performance where real-time applications require predictability and consistent behavior.

With its smaller kernel codebase, it also means that there are fewer – and smaller – potential points of vulnerability. This  helps enhance the robustness and reliability of the system, since the kernel is the most important component of the OS.

Time-tested in numerous applications around the world, you’ll typically see QNX deployed in applications from slower-paced environments like nuclear power plants or an ocean buoy, to faster-paced environments like industrial robots and automotive.

QNX can integrate with other systems

Does the QNX architecture rely on any connectivity to the cloud, fleet managers, or other robots? Can it be used with them?

Abdelkader: The QNX architecture itself is primarily designed for embedded systems and real-time applications where reliability, safety, and security are a priority. It operates independently of cloud connectivity or fleet management systems.

Although QNX itself does not provide built-in cloud-connectivity features, it can be integrated with cloud services, fleet management systems or additional software layers. Developers can implement cloud connectivity with middleware such as ROS or applications that support communication protocols like MQTT or OPC UA.

This process allows QNX-based devices to interact with cloud services for data storage, remote monitoring, digital twinning, etc.

QNX can also be integrated into fleet management systems through software applications that handle tasks such as device tracking, telemetry data collection, and fleet optimization. This integration involves developing software components that communicate with QNX devices.

As it relates to inter-robot communication, robots using QNX can use standard communication protocols like TCP/IP to collaborate, share data, and coordinate tasks effectively.

What do robotics developers and suppliers need to know about integrating such systems in their products? How much work is required on their side?

Abdelkader: QNX provides POSIX compliant RTOS, which simplifies development for those familiar with POSIX compliant OS like Linux. This computability means that developers can use existing knowledge and tools, streamlining the integration process.

Moreover, QNX offers both safety certified products like the QNX OS and QNX hypervisor. The RTOS ensures deterministic behavior for safety-critical applications, while the hypervisor allows consolidation of hardware onto one SoC through software – allowing developers to build safety and non-safety applications on the same platform.