Unsupervised identification of cell type proportions and cell type-specific gene expression in spatial transcriptomics

Thomas M. Menino Convention & Exhibition Center 

An essential first step in the analysis of spatial transcriptomics data is to assign cell types to each spatial location. This process is complicated by the presence of cell type mixtures on individual spatial locations. The best performing cell type identification algorithms are based on supervised methods that rely on a reference dataset to estimate cell type expression profiles. However, finding a high quality annotated single-cell RNA-seq (scRNA-seq) reference dataset is difficult and often impossible. Here, we address this challenge by developing an unsupervised factor-based statistical method for identifying cell types in spatial transcriptomics datasets, which we call Reference-free Inference of Cell types and Expression (RICE). We model gene expression as a linear mixture of cell type-specific gene expression profiles, and both cell type proportions and cell type-specific gene expression are estimated via maximum likelihood within our probabilistic model. We demonstrate, in several Slide-seq and MERFISH spatial transcriptomics datasets, RICE's accuracy in estimating both cell type proportions and cell type-specific gene expression. We show that RICE achieves comparable accuracy to state of the art supervised methods when a scRNA-seq reference is available, while it can outperform these methods when the reference is less reliable due to cell type-specific platform effects. We further show that our sparse factor modeling approach outperforms existing non-sparse unsupervised factor-base methods. We distribute RICE within the R package \url{https://github.com/dmcable/spacexr}.