SpectrAI


"Artificial intelligence for spectroscopy and spectral imaging."



Learn More

Overview

Spectroscopy and spectral imaging have widespread applications across various scientific disciplines. SpectrAI represents a collection of tailored augmentation techniques and specialized deep learning architectures in an open-source framework. The package is build on the popular PyTorch library and designed to expedite the progress of AI in spectroscopy and spectral imaging. We welcome any contributions to SpectrAI.

  • Comprehensive framework: SpectrAI offers a suite of tools and functionalities specifically tailored for working with spectral data, ensuring a seamless deep learning experience.
  • Pre-processing and augmentation: SpectrAI comes equipped with a wide range of built-in spectral data pre-processing and augmentation methods, allowing you to optimize your data for training neural networks.
  • Neural Networks for spectral data: SpectrAI provides specialized neural networks for various spectral data tasks, including spectral (image) denoising, spectral (image) classification, spectral image segmentation, spectral image super-resolution, and transfer learning.

Getting started

Here are four examples of various applications

Spectral denoising and reconstruction

By applying spectral denoising, we can enhance downstream data analysis and potentially reduce data acquisition times. Here we demonstrate the effectiveness of SpectrAI in denoising Raman spectra. We utilized a dataset of low and high SNR Raman spectra from MDA-MB-231 human breast cancer cells and trained a ResUNet model.

Docs

Spectral image super-resolution reconstruction

To showcase the capability spectral image super-resolution, we used a dataset of intraoperative hyperspectral images of human brains for brain cancer detection. For training a hyperspectral residual channel attention network (RCAN) model. With SpectrAI, we demonstrate spectral image super-resolution, preserving both spatial and spectral details for enhanced analysis and applications.

Docs

Spectral semantic segmentation

Image segmentation finds applications in diverse fields like cell biology and remote sensing. To showcase SpectrAI's capabilities in spectral image segmentation, we analyzed a large hyperspectral image dataset (AeroRIT). After training a UNet model we demonstrates the efficient use of SpectrAI for spectral image segmentation.

Docs

Transfer learning for spectral denoising and reconstruction

Transfer learning enables effective generalisation of models. Training of models from scratch can be prohibitively time-consuming and expensive. Here, we demonstrate that transfer learning using an existing neural network model trained on a large data can achieve high-quality results within a short timeframe.

Docs

Datasets

Develop your own models using our data or other freely available spectral datasets.

MDA-MB-231 low SNR and high SNR Raman spectra

Example SpectrAI application: Denoising

Source: https://github.com/conor-horgan/DeepeR

This dataset consists of 172,312 pairs of low SNR (0.1 s spectral integration time) and high SNR (1 s spectral integration time) spectra from 11 MDA-MB-231 cells.

Link to data

MDA-MB-231 hyperspectral Raman images

Example SpectrAI application: Super resolution

Source: https://github.com/conor-horgan/DeepeR

This dataset consists of 169 Raman images of MSA-MB-231 cells acquired on a confocal Raman microscope using 532 nm excitation laser

Link to data

AeroRIT

Example SpectrAI application: Segmentation

Source: https://github.com/aneesh3108/AeroRIT/tree/master


A. Rangnekar, N. Mokashi, E. J. Ientilucci, C. Kanan and M. J. Hoffman, "AeroRIT: A New Scene for Hyperspectral Image Analysis," in IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 11, pp. 8116-8124, Nov. 2020, doi: 10.1109/TGRS.2020.2987199.

This hyperspectral image offers a scene overlooking Rochester Institute of Technology captured using a hyperspectral camera

Link to data

HSI Human Brain Database

Example SpectrAI application: Super resolution

Source: https://hsibraindatabase.iuma.ulpgc.es/


Fabelo H., Ortega S., Szolna A., Bulters D., Pineiro J. F., Kabwama S., ... & Ravi D. (2019) In-Vivo Hyperspectral Human Brain Image Database for Brain Cancer Detection. IEEE Access, 7, 39098-39116

The dataset consists of 36 hyperspectral images acquired during brain surgeries from 22 patients. The images are on average 439 × 400 pixels with 826 spectral bands between 400 nm and 1000 nm.

Link to data

About Us

SpectrAI was started by Dr Conor Horgan and Dr Mads Bergholt at the Label-free Bioimaging Laboratory, King’s College London to establish an community of AI research and for the development and exchange of best practices for AI in spectroscopy and spectral imaging.

If you find this project helpful in your work, please cite the following articles:

Conor C. Horgan, Magnus Jensen, Anika Nagelkerke, Jean-Phillipe St-Pierre, Tom Vercauteren, Molly M. Stevens, and Mads S. Bergholt, "High-throughput molecular imaging via deep learning enabled Raman spectroscopy", Analytical Chemistry 2021, 93, 48, 15850-15860.

Conor C. Horgan and Mads S. Bergholt, "spectrai: a deep learning framework for spectral data.", Journal of Spectral Imaging, Volume 11, Article ID a7, (2022)

Contact Information

King's College London
Label-free Bioimaging Laboratory
Floor 17, Tower Wing
Great Maze Pond, London SE1 9RT
Email: mads.bergholt@kcl.ac.uk