AI Engineer Talk

From Tokens to Cells

The engineering challenges of foundation models for single‑cell biology

Akram Baharlouei
Altos Labs
Outline
01
Why single cells
02
What to measure
03
Why it's hard: heterogeneity
04
Modeling cells
05
Takeaways
From Tokens to CellsAltos Labs
SECTION 01

Why single cells

Rejuvenation, the path to a virtual human, and a way to bend the cost curve of medicine.

01 · Why single cells

Yamanaka factors (OSKM)

In 2006, Shinya Yamanaka discovered that just four proteins — Oct3/4 · Sox2 · Klf4 · c-Myc — can reset any adult cell to an embryonic-like state. Nobel Prize 2012.

Different type Same type Embryonic Young Old Full reprogramming — type + age Partial reprogramming — age only iPSC pluripotent young skin cell aged skin cell
FULL →
Regenerative medicine
Erases age + cell identity. iPSCs can become any tissue type.
PARTIAL →
Cellular rejuvenation
Resets age only. Short OSKM bursts — cell keeps its identity.
From Tokens to CellsAltos Labs
Sarah Teichmann & the Human Cell Atlas From Tokens to Cells · Altos Labs
01 · Why single cells

AI at every stage of drug development

1
Research &
Development
3–6 years
Target identification
Compound screening
Lead optimization
↓
AI REPLACES
Protein engineering models
Scientific reasoning agents
›
2
Preclinical
Studies
1–2 years
In vitro studies
In vivo animal models
Toxicity testing
↓
AI REPLACES
Virtual cells
Virtual tissues & organs
In silico toxicity
›
3
Clinical
Trials
4–7 years
Phase I, II & III trials
Dosage & safety monitoring
Patient recruitment
↓
AI REPLACES
Virtual humans
In silico trials
Synthetic patient cohorts
›
4
Review &
Approval
1–2 years
Rx
Safety & efficacy review
Regulatory approval
Post-market surveillance
↓
AI REPLACES
Digital twins
Adverse event prediction
Continuous monitoring
From Tokens to CellsAltos Labs
SECTION 02

What to measure

Different ways to read a cell — and why we focus on one.

02 · The data

Many ways to read a cell

One cell — many measurement modalities, each capturing different biology.

Morphology cell shape & structure Genomics DNA sequence Epigenomics chromatin access scRNA-seq ★ transcriptome Proteomics protein output Spatial location in tissue
scRNA-SEQ
OUR FOCUS
Transcriptome
~20,000-gene vector per cell · high throughput · millions of cells
GENOMICS
The DNA code
Static sequence · same in every cell · genome-wide variants
EPIGENOMICS
Chromatin accessibility
Which DNA regions are open · histone modifications
PROTEOMICS
Protein output
Actual functional molecules · low throughput · expensive
SPATIAL
Location in tissue
Gene expression + tissue coordinates · cell neighborhoods
MORPHOLOGY
Cell shape & structure
Size, shape, organelle layout · light & electron microscopy
IMAGING
Fluorescence & microscopy
Protein localization, live-cell imaging · high-content screening
From Tokens to CellsAltos Labs
02 · The data  ·  NeurIPS 2025

scGeneScope: treatment-matched scRNA-seq & Cell Painting

627,704 RNA profiles  ·  716,767 cell images
28 treatments  ·  U2-OS cell line
28 Distinct Chemical Treatments 5 Pairs of Biological Replicates Cell Painting or scRNA-seq Independent or Joint Sampling Treatment Identification Task OH Br HO O N N CH₃ O N OMe OH HO A DMSO B A F G B C E D H I A DMSO B A F G B C E D H I Fix, Stain, Image, and Segment Cells Cell Painting 10x Chromium scRNA-seq scRNA-seq Imaging Model Cell Painting features Multimodal Model imaging + RNA features scRNA-seq Model gene expression features O OH CH₃ N O Cl NH₂ F OMe HO
SECTION 03

Why it's hard: heterogeneity

Two “identical” cells rarely read the same — and not all of it is signal.

03 · Heterogeneity

Technical: variation that isn't biology

batch‑effect
Two labs, two machines, two days — different numbers for the same biology.
dropout
Expressed genes read as zero — absence you can't trust.
depth
Sequencing depth varies cell to cell, rescaling every count.
ambient‑RNA
Free‑floating transcripts contaminate every droplet.
platform
Different chemistries and protocols aren't directly comparable.

The trap: confounds look exactly like signal. Disentangling them is half the modeling problem.

From Tokens to CellsAltos Labs
04 · Modeling cells  ·  an open question

Quantum computing might be a better fit for biology in the future

A cell's state lives in a ~20,000-dimensional gene space. Classical computers search it sequentially. Quantum computers explore many states at once — which might be a natural match. Nobody knows yet.

THE INTUITION

Quantum state spaces scale exponentially with dimension — on paper, that matches the combinatorial complexity of gene co-expression. Molecular simulation is also a natural quantum application.

WHERE WE ACTUALLY ARE

Hardware is noisy, small-scale, and nowhere near the qubit counts needed. No quantum algorithm has shown a real advantage on biological data.

From Tokens to CellsAltos Labs
SECTION 04

Modeling cells

From the obvious choice — transformers — to what actually fits the data.

04 · Modeling cells

The input: a cells × genes count matrix

GAPDH
ACTB
CD3D
CD19
MKI67
TP53
PTPRC
VIM
EPCAM
SOX2
···
Cell_0001
47
31
0
0
0
12
0
0
0
0
···
Cell_0002
0
0
83
0
0
0
38
0
0
0
···
Cell_0003
22
8
0
71
0
0
0
0
0
0
···
Cell_0004
0
0
0
0
94
0
0
0
27
0
···
Cell_0005
0
14
0
0
0
0
0
59
0
0
···
⋮
⋮
⋮
⋮
⋮
⋮
⋮
⋮
⋮
⋮
~20,000 genes (columns)
ROWS
Cells
Thousands to millions per experiment. Each row is one destroyed cell.
COLUMNS
Genes ~20,000
Every known human gene. No natural ordering.
VALUES
Integer UMI counts
80–95% zeros. Sparse, non-negative, not normally distributed.
From Tokens to CellsAltos Labs
04 · Modeling cells

First instinct: treat genes like tokens

WHAT WE TRIED
scGPT, Geneformer & friends

Rank or tokenize expression, run the language‑model playbook, self‑supervise on millions of cells with masked / next‑gene prediction.

TRAINING DATA
scGPT 33M human cells · CellxGene Census · 51 tissues
Geneformer 29.9M cells · 100+ datasets · 30 organs
WHY IT STALLS
  • No order. Genes aren't a sequence — position is meaningless.
  • Continuous, not categorical. Expression is a magnitude; tokenizing discards it.
  • Mostly zeros. A ~20k‑dim, sparse vector wastes the objective on absence.

The token objective optimizes the wrong thing — it discards the magnitudes that carry the biology. → sc‑gene‑scope

From Tokens to CellsAltos Labs
04 · Modeling cells

The deeper problem: the latent bottleneck loses information

Real cell
~20,000 genes
→
Encoder
→
Latent vector
compressed · lossy
→
Decoder
→
Reconstruction
≠ the original
Downstream on the embedding
Tasks run on the latent — inheriting whatever it failed to keep.
Reconstruction is wrong
The rare, sparse signal that matters most is the first thing compressed away.
So: preserve the information
Build models that keep the full expression — which leads us to flow matching.
From Tokens to CellsAltos Labs
04 · Modeling cells

Flow matching: transport a distribution from noise to data

THE ODE
dX_t/dt = u_t(X_t)
A neural network learns the velocity field u_t that drives each point from Gaussian noise to the data distribution.
STRAIGHT PATHS
x_t = (1−t)·x₀ + t·z
Linear interpolation between noise and data. No stochastic noise mid-trajectory — unlike diffusion, paths are straight.
FOR CELLS
Sample any cell state
Draw noise → follow u_t → land in the distribution of perturbed cell states. Works directly in full 20k-gene space.
From Tokens to CellsAltos Labs
04 · Modeling cells

PrimeFlow: drop the embedding bottleneck

THE USUAL PIPELINE
expression → encoder → latent (lossy) → decoder → prediction
The bottleneck compresses away the rare, sparse signal that matters most.
PRIMEFLOW
raw expression vectors → flow matching → predicted distribution
No autoencoder, no latent — the model sees the real cell, nothing thrown away.

Sharper perturbation prediction, better‑preserved heterogeneity — at the cost of working in a high‑dimensional space.

From Tokens to CellsAltos Labs
SECTION 05

Takeaways

Where we are, what works, and what's still hard.

05 · Conclusion

Three things to take away

01
Single cell is the right unit of study
Cellular rejuvenation, a path to a digital human, and a better drug development cycle — none of this is possible without resolving biology at the single-cell level.
02
scRNA-seq scales — other modalities still lag
We have high-throughput transcriptomics at millions of cells. Proteomics, spatial, and epigenomics are catching up — technological advances in those modalities will reshape what's learnable.
03
Flow matching fits the data — but we need more of it
Transformers hit a ceiling on continuous, unordered gene expression. Flow matching better captures the full distribution of cell states. The next frontier: more and better data to generalize across cell types and organisms.
From Tokens to CellsAltos Labs
From our team
PAPER 01
sc‑gene‑scope
Why token‑style objectives underperform on gene expression.
openreview.net →
QR
PAPER 02
PerturBench
A benchmark for predicting the effect of perturbations.
arxiv.org/2408.10609 →
QR
PAPER 03
PrimeFlow
Flow matching directly on raw single‑cell expression.
arxiv.org/2604.13986 →
QR

Thank you

Questions & discussion.

Akram Baharlouei
Altos Labs
APPENDIX

Extra slides

04 · Modeling cells · the task

What does a perturbation do to a cell?

01 · Drug / OSKM
Small molecule
delivered
Enters the cell and targets specific genes
›
02 · Gene activation
Target genes
activated
Drug binds and switches on specific gene programs
›
03 · Transcription
mRNA
produced
Gene sequence copied into messenger RNA
›
04 · Translation
Protein
produced
mRNA translated into functional protein by ribosomes
›
before after
05 · Expression shift
Gene expression
rewired
Protein changes the cell's expression program
Perturbation
response task
expression profile
before treatment
+
perturbation identity
drug / gene
→
?
→
expression profile
after treatment
From Tokens to CellsAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Multimodal integration circuit

PROFILES SET NEURAL NET ENCODER AGGREGATION OUTPUT ViT-H ResNet CLIP Cell Painting set neural net N profiles → 1 imaging encoder ViT-H / ResNet-50 scVI PCA scGPT scRNA-seq set neural net N profiles → 1 rna-seq encoder scVI / PCA same treatment structured aggregation late fusion · concat + MLP Treatment Classification 28 chemicals · balanced accuracy
Each modality encoded independently · set neural net aggregates multiple profiles · late fusion combines modalities
From Tokens to CellsAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Benchmark results

Model
Within
Bal.Acc
Within
Macro-F1
Held-out
Bal.Acc
Held-out
Macro-F1
UNIMODAL · scRNA-seq
scVI ★
0.517
0.497
0.268
0.466
PCA
0.510
0.473
0.207
0.203
Geneformer†
0.281
0.248
0.196
0.122
scGPT†
0.357
0.378
0.189
0.341
UNIMODAL · Cell Painting
ViT-H†
0.279
0.264
0.208
0.194
ResNet-50†
0.262
0.243
0.213
0.187
MULTIMODAL · Multi-profile
AvgPool+scVI
0.637
0.663
0.581
0.534
Transformer+ViT-H
0.316
0.293
0.219
0.181
† zero-shot (no fine-tuning)
Generalization gap
Even top models fail to generalize across laboratory batches — large drop from within to held-out split.
Simpler is often better
scVI (domain-trained VAE) matches or beats Geneformer and scGPT zero-shot. Scale ≠ domain fit.
Multimodal synergy elusive
Combining modalities helps within-experiment but often hurts held-out generalization. Alignment is hard.
From Tokens to CellsAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Model progression: three tiers

Tier 1
Single Profile
1 cell encoder embed y
1 cell → encoder → embed → classify
7 imaging encoders
7 RNA encoders
Simple · fast · noisy
Tier 2
Multi-Profile
N cells enc enc enc aggregator embed y
N cells → encoders → set function → classify
AvgPool · DeepSet · Set Transformer
Per-modality aggregation
Permutation-invariant · captures spread
Tier 3
Multimodal
scRNA-seq agg+enc Cell Painting agg+enc late fusion label
RNA + Imaging paths → structured aggregation
Single profile or multi-profile per modality
Late fusion (concat + MLP)
Treatment-matched pairing
From Tokens to CellsAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Single-profile encoder zoo

Cell Painting · Imaging Encoders
Classical CNN
ResNet-50 / 152†
ImageNet-pretrained CNN. Standard deep-learning baseline for cell imaging.
ResNet-SO-CLIP†
ResNet backbone + CLIP contrastive pre-training. Zero-shot visual encoder.
Vision Transformer
ViT-L / ViT-H†
Large/Huge Vision Transformer. Best supervised imaging encoder in dataset.
ViT-2†
Alternate ViT variant. Best held-out generalization among imaging encoders.
ViT-H-CLIP†
ViT-H backbone + CLIP pre-training. Zero-shot but underperforms supervised ViT.
Cell Imaging Foundation
OpenPhenom†
Phenomics-specific foundation model. Underperforms simpler supervised encoders zero-shot.
scRNA-seq Encoders
Linear Baseline
PCA†
Principal component analysis on log-normalized counts. Simple linear baseline.
Domain-trained VAE
scVI ★
Variational autoencoder trained on scRNA-seq. Trained on data → best unimodal performer.
scVI-1 / scVI-2†
scVI variants pre-trained on external datasets (not scGeneScope). Zero-shot.
scRNA Foundation Models (zero-shot)
UCE†
Universal Cell Embeddings. Pre-trained on 36M cells across species.
Geneformer†
Transformer pre-trained on 30M cells via rank-based gene tokenization.
scGPT†
Generative pre-trained transformer. Gene-order invariant masked modeling.
From Tokens to CellsAltos Labs · † = zero-shot
APPENDIX · scGeneScope · NeurIPS 2025

Multi-profile aggregation methods

Method 01
Average Pooling
e₁ e₂ e₃ ⋮ eₙ 1/N Σ eᵢ element-wise mean z
· No learned parameters
· Permutation-invariant by definition
· Fast · deterministic
· Loses distributional shape
Method 02
DeepSet
e₁ e₂ e₃ ⋮ eₙ φ(·) φ(·) φ(·) φ(·) Σ ρ(·) z ρ ( Σᵢ φ(eᵢ) )
· Learned φ (per-element MLP)
· Permutation-invariant by construction
· Universal approx. for set functions
· Zaheer et al., NeurIPS 2017
Method 03
Set Transformer
e₁ e₂ e₃ ··· eₙ ISAB (self-attn · M inducing pts) reduces O(N²) → O(NM) · M ≪ N ISAB (second attention block) PMA (pool by multihead attention) k=1 seed vector → 1 output
· Attention over cells → learns which matter
· Inducing points reduce O(N²) cost
· Most expressive of the three
· Lee et al., ICML 2019
From Tokens to CellsAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Unimodal results

Model
Within Bal.Acc
Within F1
Held-out Bal.Acc
Held-out F1
CELL PAINTING · SINGLE PROFILE
ViT-H†
.279
.264
.208
.194
ViT-2†
.267
.248
.224
.195
ResNet-50†
.262
.243
.213
.187
ResNet-152† / ViT-L†
.257–.270
.236–.259
.211–.214
.190–.197
CELL PAINTING · MULTI-PROFILE
AvgPool + ViT-L†
.471
.455
.258
.226
DeepSet + ViT-L†
.436
.431
.250
.209
AvgPool + ViT-H†
.387
.374
.295
.219
scRNA-seq · SINGLE PROFILE
scVI ★
.517
.497
.268
.466
PCA†
.510
.473
.207
.203
scGPT†
.357
.378
.189
.341
Geneformer†
.281
.248
.196
.122
UCE† / scVI-1†
.223–.319
.204–.290
.202–.280
.146–.220
scRNA-seq · MULTI-PROFILE
DeepSet + scVI† ★
.637
.633
.581
.534
AvgPool + scVI†
.632
.616
.504
.548
Transformer + PCA†
.615
.604
.456
.448
AvgPool + PCA†
.599
.556
.179
.164
W = Within-Experiment · H = Held-out-Experiment · † zero-shot · ★ bestAltos Labs
APPENDIX · scGeneScope · NeurIPS 2025

Multimodal results

Model (RNA enc + Imaging enc)
Within Bal.Acc
Within F1
Held-out Bal.Acc
Held-out F1
SINGLE PROFILE PER MODALITY
scVI + ViT-H† ★
.504
.502
.517
.501
scVI + ViT-L†
.500
.485
.517
.509
PCA + ViT-H†
.502
.492
.225
.211
PCA + ViT-L†
.510
.496
.257
.300
scGPT + ViT-H†
.399
.399
.431
.422
scGPT + ViT-L†
.413
.411
.418
.416
MULTI-PROFILE PER MODALITY · AvgPool aggregation
AvgPool + scVI + ViT-H† ★
.619
.659
.544
.512
AvgPool + scVI + ViT-L†
.466
.440
.328
.288
AvgPool + scGPT + ViT-H†
.495
.475
.294
.250
AvgPool + PCA + ViT-L†
.473
.489
.374
.329
AvgPool + PCA + ViT-H†
.420
.377
.426
.384
AvgPool + scGPT + ViT-L†
.473
.487
.374
.329
Best multimodal held-out (0.544) < best unimodal RNA held-out (0.581 DeepSet+scVI). Adding imaging does not reliably help generalization.
† zero-shot · ★ best in category · Late fusion: concat + MLPAltos Labs