Every time a sonographer glides a transducer across a patient's skin, the screen displays a reconstructed image born from millions of microscopic echoes. For decades, this process has relied on a fundamental simplification of physics: the assumption that ultrasound waves travel through human tissue at a constant, uniform speed. In reality, the human body is a chaotic map of varying densities, and by forcing raw data through a rigid reconstruction pipeline, clinicians lose a vast amount of nuanced information. The industry has long accepted this loss as a necessary trade-off for real-time imaging, but the bottleneck was never the physics—it was the compute.
The Architecture of Direct Signal Learning
NVIDIA, in collaboration with Siemens Healthineers, has introduced NV-Raw2Insights-US to dismantle this traditional pipeline. Rather than accepting a pre-processed image, the NV-Raw2Insights-US model ingests the raw signals captured directly by the ultrasound probe. The core innovation lies in the model's ability to learn the complex interactions between acoustic waves and biological tissue in real-time. Instead of relying on a global average for the speed of sound, the AI generates a patient-specific sound speed map, effectively tailoring the imaging process to the unique anatomy of the individual on the table.
This shift transforms a multi-stage computational burden into a single AI inference step. By bypassing the traditional reconstruction phase, the system can correct image distortions and optimize focus with a level of precision that was previously computationally prohibitive. To enable the broader research community to build upon this foundation, NVIDIA has released the project via a GitHub repository, providing the model weights and the datasets necessary to replicate and extend the findings.
Solving the High-Bandwidth Hardware Bottleneck
While the AI model provides the intelligence, the primary obstacle to implementing such a system in a clinical setting has always been the data deluge. Clinical-grade scanners generate an immense volume of raw channel data that typically exceeds the bandwidth capabilities of standard interfaces, making it nearly impossible to extract this data for external AI processing without significant latency or loss.
To bridge this gap, NVIDIA implemented the Holoscan Sensor Bridge, an open-source FPGA IP designed to stream high-bandwidth sensor data directly to a GPU. The hardware chain is a sophisticated exercise in edge engineering: the system utilizes an Altera Agilex-7 development kit to intercept the output of a Siemens Healthineers ACUSON Sequoia diagnostic scanner. By leveraging a technique known as Data over DisplayPort, the raw signals are streamed from the scanner, passed through the FPGA, and transmitted via Ethernet to an NVIDIA IGX edge AI computing platform.
This infrastructure represents a fundamental pivot from static algorithms to an adaptive AI pipeline. The entire system is deployed on the NVIDIA Holoscan platform and is optimized for accelerated inference on Blackwell-architecture GPUs. The loop is closed when the AI-derived sound speed estimates are fed back into the ultrasound scanner, allowing the hardware to optimize the image focus in real-time based on the AI's insights. The result is a system that does not just process an image, but actively understands the physical properties of the patient's body to improve the diagnostic quality of the scan.
This technology currently remains in the research and development phase and has not yet received regulatory clearance for commercial sale in the United States or other global markets.
