A toolkit for generating virtual brightfield images of histological and immunohistochemical stains from multiplexed data with AI-based channel selection and image enhancement

Abstract:: Multiplex imaging provides valuable insights into the functional and spatial organization of cells and tissues. However, traditional brightfield histopathology imaging remains important and may be required alongside multiplex imaging. We introduce a generalized framework to generate virtual brightfield images from multiplexed data, thereby reducing the need for additional tissue preparation and alignment with the multiplex images. Our approach uses a physically based stain model that simulates the light absorption of stains through the tissue. A channel selection strategy, using a lookup table or Large Language Model (LLM), allows for the mapping of molecular markers to their corresponding stain colors. To further enhance image quality, we integrate a deep learning-based upsampling and denoising model, trained on real brightfield images. We evaluated the methods on several modalities including mass-spectrometry based imaging mass cytometry and fluorescence based multiplex imaging. The results demonstrate that our method produces virtual brightfield images that are of similar quality as real brightfield images, are quantifiable and of diagnostic quality. We also show that LLMs are able to consistently determine appropriate channels in the multiplex image.

Authors:: T Whitmarsh, M Al Sa’d, EG Solares, A Molaeinezhad, MO Irfan, C Mulvey, M Paez-Ribes, A Fatem, W Cope, K Hua, G Hannon, D Bressan, N Walton

Themes