Computational Imaging

Motion in Magnetic Resonance Imaging (MRI) scans causes image corruption and remains a barrier to clinical imaging. We propose Beat Pilot Tone (BPT), a simple serendipitous system exploitation that turns any MRI receiver chain into a radio frequency (RF) motion sensing system that can operate at arbitrary frequencies (up to several GHz). Our contact-free system can be implemented on any MRI scanner regardless of field strength. Through electromagnetic field simulations and experiments, we explain BPT’s novel mechanism: two or more transmitted RF tones form motion-modulated standing wave patterns that are sensed by the same receiver coil arrays used for MR imaging. These waves are incidentally mixed by intermodulation and digitized simultaneously to the MRI data. BPT achieves an order of magnitude greater sensitivity to motion than other methods in detecting and separating common motion types (respiratory, bulk, cardiac, and head motion) in volunteers. Moreover, BPT offers tunable sensitivity to motion based on the transmit frequencies; at microwave frequencies, BPT can detect millimeter-scale vibrations (ballistocardiograms). With multiple antennas and frequency-multiplexing, BPT can operate as a multiple-input multiple-output (MIMO) system. BPT significantly expands the capability of any MRI system, paving the way toward multi-modality, motion-robust, and simultaneous RF and MR imaging. Joint work with Suma Anan

Advanced long-range imaging systems are often equipped with adaptive optics to compensate for the phase distortions caused by atmospheric effects. At the core of this compensation process is the wavefront estimation problem where we need to know the phase of the distortion from the magnitude of a complex signal. While this in many senses is similar to a phase retrieval problem, a phase retrieval method can accept trivial ambiguity in a solution (e.g., conjugate flip) whereas a wavefront estimation needs a unique solution. I will discuss our recent findings about the conditions under which a wavefront estimation will have a unique global solution. I will also demonstrate conceptually how wavefront estimation can be achieved using a single measurement, without any iterative algorithm.

Diffusion models can learn strong image priors and use them to solve inverse problems, but they are computationally expensive and require lots of training data. We propose a method that trains diffusion models on patches of images and combines them to obtain an image prior for the entire image. By zero padding the original image and shifting the patch tiling scheme, we avoid boundary artifacts and obtain a score function of the whole image without inputting the whole image into the neural network. This diffusion model can be trained more quickly, requires less data and less memory, while still having generative capabilities. Our flexible framework allows for the trained network to be used with previously established sampling algorithms. We demonstrate the approach by using our method to solve inverse problems including CT reconstruction, deblurring, and superresolution.Work with Jason Hu, Bowen Song, Xiaojian Xu and Liyue Shen. See arXiv 2406.02462.

Sampling theory inspires a lot of approaches in signal processing society. In particular, reconstructions of an analog signal from discretized samples were quite important for us to understand a system. As deep learning era has begun, we are surrounded by tremendous digital datasets and now is the time to reconstruct such accumulated digital datasets up to their preferences. In this talk, I will reshape a typical DAC(digital-to-analog) into a neural networks to handle continuous representations. This will be done by implicit neural representation models. Single image super resolution (SISR) and image warping are demonstrated with the proposed regime.

Unknown-view tomography (UVT) reconstructs a 3D density map from its 2D projections at unknown random orientations. A line of work starting with Kam (1980) employs the method of moments (MoM) to solve UVT in the frequency domain, assuming that the orientations are uniformly distributed. Such approaches bypass the estimation of individual viewing orientations that traditional MAP estimation requires. Kam’s autocorrelation method leads to an underdetermined system, with missing orthogonal matrices. In this talk, we extend the previous orthogonal matrix retrieval (OMR) methods to jointly recover the density map and the orthogonal matrices such that spatial constraints, such as nonnegativity, can be easily incorporated. This is enabled by the closed-form expressions for spatial autocorrelation features. Numerical results show that the new approach is more robust and performs significantly better than the previous state-of-the-art OMR approach in the typical low-SNR scenario of 3D UVT.

Computational imaging aspires to bring together mathematics, physics, computer science, and applications to solve frontier problems in imaging. While it has made great progress and proved its potential, an enormous array of amazing new methods has gone unused in real applications. In particular, model-based iterative reconstruction (MBIR) is an example of a method with established value that is not widely used.

In this talk, we take a hard look at why MBIR is not used and formulate a strategy to address these limitations in MBIRJAX, a software package that combines novel theoretical, software, and interface design to reduce barriers to entry for scientific, industrial, and commercial applications.

******Biography******

Charles A. Bouman is the Showalter Professor of Electrical and Computer Engineering and Biomedical Engineering at Purdue University. He received his B.S.E.E. degree from the University of Pennsylvania, M.S. degree from the University of California at Berkeley, and Ph.D. from Princeton University in 1989. He is a member of the National Academy of Inventors, a Fellow of the IEEE, AIMBE, and SPIE, and an Honorary Member of the IS&T. He is the recipient of the 2021 IEEE Signal Processing Society, Claude Shannon-Harry Nyquist Technical Achievement Award, the 2014 Electronic Imaging Scientist of the Year award, and the IS&T’s Raymond C. Bowman Award; and in 2020, his paper on Plug-and-Play Priors won the SIAM Imaging Science Best Paper Prize. He has served as the IEEE Signal Processing Society’s Vice President of Technical Directions, Editor-in-Chief of the IEEE Transactions on Image Processing, Vice President of Publications for the IS&T Society, and he led the creation of the IEEE

The Wellness session will be led by Devin O’Rourke, a specialist in the field of organizational development who focuses on tailoring mindfulness tools for professionals. The session will be oriented specifically for supporting researchers in computational disciplines.
We are offering this opportunity as a way to help ensure your experience at the Workshop is not only informative and engaging but uplifting as well. All are welcome to attend! No special clothing, equipment, etc. is required.

Computational Imaging in general has high compute requirements, this talk intends to provide a starting point to advance computational imaging towards real-time.  Today’s image sensors, as most compute technologies advance at Moore’s law pace in terms of pixel size, however, this drives the image processing requirements to advance at a quadratic rate, fast outpacing compute performance – this requires a paradigm shift in compute architectures to compensate and take advantage of image sensors advancements.  Specifically, the data bottleneck present in current architectures is examined as a limiting factor – this talk explores potential solutions that could bridge this increasing gap and could offer computational imaging a path to go online.

New-generation imaging systems have enabled high-dimensional signal sensing employing optical coding theory and high-performance inverse problems algorithms. State-of-the-art methods employ deep learning approaches to properly design optical systems jointly with neural networks that perform decoding tasks. This talk presents a comprehensive overview of deep optical coding design and novel learning paradigms providing new insights on optimal optical coding performance such as regularization techniques and knowledge distillation strategies

Our Diffusion in Style (ICCV 2023, WACV 2024) method adapts Stable Diffusion to any desired style using only a small set of target images. It is based on the key observation that the style of the images generated by Stable Diffusion is tied to the initial latent tensor,i.e., the initial noise the images are generated with. Not adapting this initial latent tensor to the style makes fine-tuning slow, expensive,and impractical, especially when only a few target style images are available. In contrast, fine-tuning is much easier if the initial latenttensor is also adapted. Our Diffusion in Style is orders of magnitude more sample-efficient and faster. It also generates more pleasing images than existing approaches.

There is growing interest in learning effective models for computational imaging from even limited training data. First, we will dive into the deep image prior (DIP) approach that recovers images from limited or corrupted measurements by learning neural networks directly at reconstruction time with no training data. Typical DIP suffers from severe overfitting and spectral bias effects. We present an analysis of when and how DIP recovers images from limited imaging data by studying the training dynamics of the network in the kernel regime. We then propose a novel self-guided DIP scheme that learns both the network and its input and exploits a crucial denoiser regularization. Self-guided DIP surpasses prior unsupervised and popular supervised deep learning methods for MRI or CT reconstruction from limited measurements, while using no training data.

Second, our studies show that deep image reconstruction models trained with supervision can be sensitive to tiny measurement perturbations, scan-time changes, anomalies/pathologies, etc., which cause unstable, low-quality reconstructions. To address this, we propose smoothed unrolling (SMUG), which incorporates a randomized smoothing-based robust learning operation into a deep unrolled reconstruction architecture. We theoretically show SMUG’s robustness to perturbations and our experiments indicate its efficacy for MRI reconstruction with respect to diverse perturbations and training-testing changes.

We will conclude by briefly highlighting other promising directions such as using generative diffusion models to improve robustness of deep learning based imaging, learning sparse networks in DIP, and characterizing the effectiveness of meditation and analyzing its neural mechanisms using EEG.