In surgical robotics, buying time can save lives, explains BlackBerry QNX

Imagine this: In the blink of an eye—approximately 100 milliseconds—your brain has already processed visual information, allowing you to react to what you see in real time. However, in the world of surgical robotics, the blink of an eye is a lifetime. It’s simply not fast or good enough.

Consider the precision required to navigate a scalpel through delicate tissues, avoid vital organs and blood vessels, and respond to any sudden patient movements. A delay or miscalculation, even by 100 milliseconds, could mean the difference between life and death.

For this reason, surgical robotic systems must operate with extraordinary speed and precision, often needing to perform actions and respond to any event in the range of low single-digit milliseconds.

But let’s break this down even further. In critical scenarios, like stopping a bleeding vessel or making an incision near a sensitive nerve, every microsecond counts. A surgeon relies on the robotic system to translate their hand movements instantaneously into action, without delay, jitter, or hesitation, and react to events such as patient movement or one of the sensors failing.

If the system takes too long to respond or if there’s any inconsistency in timing — known as jitter — the outcome is not guaranteed or inconsistent and that, in itself, could be catastrophic. There are strict timing requirements to manufacture surgical robotics. Failing to meet them could cause unintended damage, prolong procedures, or increase the risk of complications.

Register now.

Haptic, visual systems support real-time integration

Modern surgical robotics systems are moving to combine advanced visualization tools with haptic feedback to provide a comprehensive sensory experience for the surgeon. The integration of stereoscopic UHD (ultra-high-definition) vision systems and haptic feedback mechanisms allows surgeons to see and feel the surgical environment as if they were directly interacting with the patient’s tissues.

The reliability of these sensory systems is crucial for robotic surgery. If a process were to get hung up, delayed, or jitter—whether due to system overload, software or hardware issues, or resource contention—it would lead to significant issues or a lack of trust in the system itself. For example:

Visual delays: A delayed camera feed could inhibit the real-time visual information provided to the surgeon to navigate and make precise movements.

Even a “blink of an eye” lag could impair the surgeon’s ability to accurately perceive the surgical field. This visual lag may cause the surgeon to make an incorrect motion or misjudge the spatial relationships between tissues, potentially leading to accidental damage or errors in the procedure.

Haptic latency: Similarly, latency in tactile response could disrupt the surgeon’s sense of touch, preventing them from feeling the texture, resistance, and tension of tissues and instruments in real-time. Any delay in haptic feedback could cause the surgeon to receive late tactile information, leading them to apply too much or too little force, which could result in potential tissue damage or improper manipulation of instruments.

The combination of these systems must operate seamlessly in real time to ensure that the surgeon receives immediate and accurate feedback from both visual and tactile sources. This level of precision and accuracy is only possible when the software and hardware are perfectly synchronized, ensuring low latency and minimal jitter across all processes.

OS and hardware have a symbiotic relationship

To achieve the level of precision and speed required in surgical robotics, it’s not just about powerful hardware or an advanced operating system (OS), or complex applications potentially using artificial intelligence. It’s also about how well-integrated and responsive all of these elements are, and how they work together.

The relationship between software and hardware is akin to the synergy between a skilled surgeon and their instruments. Even the most advanced tool is only as effective as the hand that guides it. In the same way, a high-performance hardware with advanced CPU and GPU functionalities requires an equally sophisticated operating system to maximize their potential.

In surgical robotics, innovations like stereoscopic UHD vision systems and haptic feedback generate enormous amounts of data that must be processed in real-time. The GPU handles the heavy lifting of processing the high-definition video feed, providing the surgeon with an immersive and detailed view of the surgical field.

Meanwhile, the CPU is tasked with managing the influx of data, coordinating various processes, and ensuring smooth communication between system components.

However, for this intricate dance between the CPU and GPU to succeed, the OS must effectively manage these resources effectively so the complex surgical applications that are running on top can utilize the underlying hardware effectively, reliably and deterministically. The OS needs to ensure that both the CPU and GPU operate in harmony, processing data efficiently and in real time.

Without a robust and real-time OS to synchronize these components, the system could falter, unable to meet the demands of modern surgery.

The importance of low latency and low jitter in operating systems

This is where the importance of a real-time operating system (RTOS) comes into play. For example, an RTOS like BlackBerry QNX OS 8.0 isn’t just about managing different tasks in parallel as quickly as possible—it’s also about ensuring that every task is executed with the utmost precision, accuracy, and speed.

The RTOS must be finely tuned to work in harmony with the hardware and user applications, ensuring that the system can handle multiple high-priority tasks simultaneously, with minimal latency and jitter.

By minimizing latency and jitter, the RTOS effectively buys more time for complex surgical applications to process critical information and make real-time decisions.

Unexpected delays or issues introduced by the RTOS will have a cascading effect, amplifying the total delay across the entire system. This impact will degrade the overall system performance, potentially leading to life-threatening situations in a surgical environment not to mention a lack of trust in the surgical system.

Therefore, maintaining low latency and jitter is not just about performance; it’s about ensuring that the system performs its life-saving functions consistently without compromise.

Real-time operating systems: The heartbeat of surgical robotics

In surgical robotics, the coupling of hardware and software is not just important; it’s critical. This synergy ensures that the system can manage tasks as efficiently as possible, leaving room for software applications to run without compromising performance.

In such systems, handling interrupts is of the utmost importance. It signals when a process or an event needs urgent attention such as a sensor failing and handling it as quickly as possible ideally in microseconds is a necessity.

This is why an RTOS designed specifically for this purpose is essential, capable of handling such critical tasks and interrupts with minimal jitter. This buys time for the surgical software to respond to such interrupts and in some cases enter a “fail-safe” state.

Real-time performance is the future of robotics

The importance of a high-performance RTOS in the surgical environment cannot be overstated. It is the backbone that allows these systems to operate with the precision and reliability that surgeons and patients alike depend on.

But the need for such robust, real-time performance isn’t limited to surgical robots. Given the advanced capabilities of modern RTOS, one must wonder: Why aren’t advanced RTOS deployed everywhere, from industrial robots that require precise, fault-tolerant operation on the factory floor to drones that must navigate complex environments with split-second timing?

As the field of robotics continues to evolve across various industries, the adoption of advanced RTOS will be key to pushing the boundaries of what’s possible, ensuring not just the success of surgical procedures, but also the reliability and safety of robotics in manufacturing, logistics, defense, and beyond.

About the author

Winston Leung is a senior manager at BlackBerry QNX.

Founded in 1980, QNX supplies commercial operating systems, hypervisors, development tools, and support and services for critical embedded systems. Acquired by BlackBerry in 2010, the Ottawa, Canada-based unit serves industries including aerospace and defense, automotive, heavy machinery, industrial controls, medical, and robotics.

Editor’s note: This article is posted with permission.