sc_tools.pp — Preprocessing#

Modality-aware preprocessing with GPU auto-detection (rapids-singlecell, falls back to scanpy).

import sc_tools.pp as pp

# One-call recipe
pp.preprocess(adata, modality="visium", integration="scvi", batch_key="library_id")

# Or step-by-step
pp.backup_raw(adata)
pp.normalize_total(adata)
pp.log_transform(adata)
pp.pca(adata)
pp.cluster(adata, resolution=0.8)

Recipe#

sc_tools.pp.preprocess(adata, modality='visium', batch_key='library_id', integration='scvi', spatial_clustering=None, n_top_genes=2000, resolution=0.8, filter_patterns=None, use_gpu='auto', copy=False, **kwargs)[source]#

Modality-aware preprocessing pipeline.

Dispatches to a recipe based on modality. Each recipe handles normalization, feature selection, batch integration, dimensionality reduction, and clustering appropriate for the data type.

Parameters:

  • adata (AnnData) – Annotated data with raw counts/intensities in X.

  • modality (str) – Data modality: "visium", "visium_hd", "xenium", "cosmx", or "imc".

  • batch_key (str) – Column in adata.obs for batch/library identification.

  • integration (str) – Integration method: "scvi" (default), "harmony", "cytovi" (IMC only), or "none".

  • spatial_clustering (str | None) – If "utag", run UTAG spatial-aware clustering after standard Leiden.

  • n_top_genes (int) – Number of highly variable genes to select (not used for IMC).

  • resolution (float) – Leiden clustering resolution.

  • filter_patterns (list[str] | None) – Gene name patterns to exclude. None uses modality defaults.

  • use_gpu (str | bool) – "auto" (detect), True, or False.

  • copy (bool) – If True, operate on a copy and return it.

  • **kwargs (Any) –

    Extra arguments passed to integration and recipe-specific steps. Recognized keys:

    • strategy: explicit ScaleStrategy instance (default: auto-select).

    • n_latent, n_layers, n_hidden, max_epochs, early_stopping: passed to run_scvi/run_cytovi.

    • n_neighbors: passed to neighbors() (default 20).

    • n_comps: number of PCA components (default 50).

    • cofactor: arcsinh cofactor for IMC (default 5).

    • skip_log1p: if True, skip log1p in Xenium recipe (default True).

    • max_dist, utag_resolutions: passed to run_utag.

    • hvg_flavor, hvg_batch_key: passed to HVG selection.

Returns:

Preprocessed data with clustering, embeddings, and (optionally) batch-corrected latent space.

Return type:

AnnData

Normalization#

sc_tools.pp.backup_raw(adata)[source]#

Save a copy of the current adata to adata.raw (no-op if already set).

Parameters:

adata (AnnData) – Annotated data matrix. Modified in place.

Return type:

None

sc_tools.pp.normalize_total(adata, target_sum=10000.0, inplace=True, **kwargs)[source]#

Library-size normalize counts per cell.

Wraps scanpy.pp.normalize_total (or rapids_singlecell.pp.normalize_total on GPU).

Parameters:

  • adata (AnnData) – Annotated data with raw counts in X.

  • target_sum (float | None) – Target total counts per cell after normalization.

  • inplace (bool) – If True, modify adata in place.

  • **kwargs (Any) – Passed to the backend normalize_total.

Returns:

Modified adata if inplace=False, else None.

Return type:

AnnData or None

sc_tools.pp.log_transform(adata, base=None, inplace=True, **kwargs)[source]#

Apply log1p transformation.

Wraps scanpy.pp.log1p (or rapids_singlecell.pp.log1p on GPU).

Parameters:

  • adata (AnnData) – Annotated data (typically after normalize_total).

  • base (float | None) – Logarithm base. None for natural log (default).

  • inplace (bool) – If True, modify adata in place.

  • **kwargs (Any) – Passed to the backend log1p.

Return type:

AnnData | None

sc_tools.pp.scale(adata, max_value=10, zero_center=True, inplace=True, **kwargs)[source]#

Zero-center and scale features to unit variance.

Wraps scanpy.pp.scale (or rapids_singlecell.pp.scale on GPU).

Parameters:

  • adata (AnnData) – Annotated data.

  • max_value (float | None) – Clip values to this maximum after scaling (default 10).

  • zero_center (bool) – If True, center each gene to zero mean.

  • inplace (bool) – If True, modify adata in place.

  • **kwargs (Any) – Passed to the backend scale.

Return type:

AnnData | None

sc_tools.pp.arcsinh_transform(adata, cofactor=5, inplace=True)[source]#

Arcsinh transform for mass cytometry (IMC) protein data.

Applies arcsinh(X / cofactor) element-wise. This is the standard normalization for CyTOF / IMC data (NOT log1p).

Parameters:

  • adata (AnnData) – Annotated data with raw protein intensities in X.

  • cofactor (float) – Scaling factor before arcsinh. Standard value is 5 for CyTOF/IMC.

  • inplace (bool) – If True, modify adata.X in place.

Returns:

Modified adata if inplace=False, else None.

Return type:

AnnData or None

sc_tools.pp.filter_genes_by_pattern(adata, patterns=None, exclude=True, case_sensitive=False)[source]#

Remove (or keep) genes matching regex patterns in place.

Parameters:

  • adata (AnnData) – Annotated data. Modified in place.

  • patterns (list[str] | None) – List of regex patterns. Defaults to ["^MT-", "^RP[SL]", "^HB[^(P)]"] (mitochondrial, ribosomal, hemoglobin).

  • exclude (bool) – If True (default), remove matching genes. If False, keep only matching genes.

  • case_sensitive (bool) – If False (default), patterns are case-insensitive.

Return type:

None

Integration#

All integration functions are soft dependencies; install hints are printed if the required package is missing.

sc_tools.pp.run_scvi(adata, batch_key='library_id', n_latent=30, n_layers=2, n_hidden=128, max_epochs=400, early_stopping=True, use_gpu='auto', layer=None, **kwargs)[source]#

Run scVI batch integration.

Trains an scVI model on raw counts and stores the latent representation in adata.obsm['X_scVI']. Model parameters are recorded in adata.uns['scvi_params'].

Requires scvi-tools: pip install sc-tools[deconvolution]

Parameters:

  • adata (AnnData) – Annotated data with raw counts in X (unnormalized). Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • n_latent (int) – Dimensionality of the latent space.

  • n_layers (int) – Number of hidden layers.

  • n_hidden (int) – Number of units per hidden layer.

  • max_epochs (int) – Maximum training epochs.

  • early_stopping (bool) – If True, stop training when validation loss plateaus.

  • use_gpu (str | bool) – "auto" (default), True, or False.

  • layer (str | None) – Layer in adata.layers to use. None uses X.

  • **kwargs (Any) – Passed to scvi.model.SCVI().

Return type:

None

sc_tools.pp.run_scanvi(adata, batch_key='library_id', labels_key='celltype', n_latent=30, max_epochs=200, use_gpu='auto', unlabeled_category='Unknown', scvi_model=None, **kwargs)[source]#

Run scANVI semi-supervised integration.

Initializes from a pre-trained scVI model (or trains one) and refines using cell type labels. Stores latent representation in adata.obsm['X_scANVI'].

Requires scvi-tools: pip install sc-tools[deconvolution]

Parameters:

  • adata (AnnData) – Annotated data with raw counts. Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • labels_key (str) – Column in adata.obs with cell type labels.

  • n_latent (int) – Dimensionality of the latent space.

  • max_epochs (int) – Maximum training epochs for scANVI fine-tuning.

  • use_gpu (str | bool) – "auto" (default), True, or False.

  • unlabeled_category (str) – Label for unlabeled cells (default "Unknown").

  • scvi_model (Any | None) – Pre-trained scvi.model.SCVI model. If None, one is trained.

  • **kwargs (Any) – Passed to SCANVI.from_scvi_model().

Return type:

None

sc_tools.pp.run_harmony(adata, batch_key='library_id', basis='X_pca', key_added='X_pca_harmony', **kwargs)[source]#

Run Harmony integration on a PCA embedding.

Wraps scanpy.external.pp.harmony_integrate. Stores corrected embedding in adata.obsm[key_added].

Requires harmonypy: pip install harmonypy

Parameters:

  • adata (AnnData) – Annotated data with PCA computed (obsm[basis]). Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • basis (str) – PCA embedding key in obsm.

  • key_added (str) – Key for the corrected embedding in obsm.

  • **kwargs (Any) – Passed to harmony_integrate.

Return type:

None

sc_tools.pp.run_combat(adata, batch_key='library_id', key_added='X_pca_combat', n_pcs=50, **kwargs)[source]#

Run ComBat batch correction followed by PCA.

Wraps scanpy.pp.combat(). Corrects adata.X in-place, then re-runs PCA and stores the result in adata.obsm[key_added].

Parameters:

  • adata (AnnData) – Annotated data (log-normalized). Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • key_added (str) – Key for the PCA embedding of corrected data in obsm.

  • n_pcs (int) – Number of PCs to compute after correction.

  • **kwargs (Any) – Passed to scanpy.pp.combat.

Return type:

None

sc_tools.pp.run_bbknn(adata, batch_key='library_id', neighbors_within_batch=3, **kwargs)[source]#

Run BBKNN batch-balanced k-nearest-neighbors graph construction.

Wraps bbknn.bbknn(). Modifies the neighbor graph in obsp and runs UMAP, storing coordinates in adata.obsm['X_umap_bbknn'].

BBKNN is graph-based and does not produce a latent embedding. For benchmarking, the UMAP coordinates are used as a proxy.

Requires bbknn: pip install bbknn

Parameters:

  • adata (AnnData) – Annotated data with PCA in obsm['X_pca']. Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • neighbors_within_batch (int) – How many nearest neighbors to find per batch.

  • **kwargs (Any) – Passed to bbknn.bbknn.

Return type:

None

sc_tools.pp.run_scanorama(adata, batch_key='library_id', key_added='X_scanorama', **kwargs)[source]#

Run Scanorama integration.

Wraps scanorama.integrate_scanpy() or scanpy.external.pp.scanorama_integrate. Stores corrected embedding in adata.obsm[key_added].

Requires scanorama: pip install scanorama

Parameters:

  • adata (AnnData) – Annotated data (log-normalized). Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • key_added (str) – Key for the corrected embedding in obsm.

  • **kwargs (Any) – Passed to the integration function.

Return type:

None

sc_tools.pp.run_cytovi(adata, batch_key='library_id', n_latent=20, max_epochs=300, use_gpu='auto', **kwargs)[source]#

Run CytoVI for mass cytometry / IMC protein data integration.

CytoVI is a totalVI-inspired model from scvi-tools designed for protein marker data. Stores latent representation in adata.obsm['X_cytovi'].

Requires scvi-tools: pip install sc-tools[deconvolution]

Parameters:

  • adata (AnnData) – Annotated data with protein intensities. Modified in place.

  • batch_key (str) – Column in adata.obs for batch correction.

  • n_latent (int) – Dimensionality of the latent space.

  • max_epochs (int) – Maximum training epochs.

  • use_gpu (str | bool) – "auto" (default), True, or False.

  • **kwargs (Any) – Passed to the model constructor.

Return type:

None

sc_tools.pp.integrate.run_imc_phenotyping(adata, batch_key='sample', roi_key='roi', z_score_per='roi', z_score_cap=3.0, key_added='X_pca_imc_phenotyping', n_pcs=50, **kwargs)[source]#

Run ElementoLab IMC phenotyping batch correction pipeline.

Reproduces the batch correction from imc.ops.clustering.phenotyping() (ElementoLab/imc). Pipeline: log1p -> per-ROI z-score (capped) -> global scale -> ComBat -> scale -> PCA -> BBKNN (batch-aware neighbors).

Stores corrected PCA in adata.obsm[key_added] and batch-aware neighbor graph in obsp.

Parameters:

  • adata (AnnData) – Annotated data with raw or arcsinh-normalized intensities. Modified in place (X is overwritten with corrected values).

  • batch_key (str) – Column in adata.obs for batch correction (ComBat + BBKNN).

  • roi_key (str) – Column in adata.obs for per-ROI z-scoring. Falls back to batch_key if not present.

  • z_score_per (str) – Z-score within "roi" (default) or "sample" groups.

  • z_score_cap (float) – Cap z-scores at ±this value. Default 3.0.

  • key_added (str) – Key for the corrected PCA embedding in obsm.

  • n_pcs (int) – Number of PCs to compute after correction.

  • **kwargs (Any) – Passed to scanpy.pp.combat.

Return type:

None

Dimensionality Reduction and Clustering#

sc_tools.pp.pca(adata, n_comps=50, use_highly_variable=True, **kwargs)[source]#

Run PCA on the data.

Wraps scanpy.tl.pca (or rapids_singlecell.tl.pca on GPU).

Parameters:

  • adata (AnnData) – Annotated data. Modified in place.

  • n_comps (int) – Number of principal components.

  • use_highly_variable (bool) – If True and adata.var['highly_variable'] exists, use only HVGs.

  • **kwargs (Any) – Passed to the backend pca.

Return type:

None

sc_tools.pp.neighbors(adata, n_neighbors=20, use_rep=None, **kwargs)[source]#

Compute K-nearest neighbors graph.

Wraps scanpy.pp.neighbors (or rapids_singlecell.pp.neighbors on GPU). Auto-detects use_rep: X_scVI > X_cytovi > X_pca_harmony > X_pca.

Parameters:

  • adata (AnnData) – Annotated data. Modified in place.

  • n_neighbors (int) – Number of neighbors.

  • use_rep (str | None) – Representation to use. Auto-detected if None.

  • **kwargs (Any) – Passed to the backend neighbors.

Return type:

None

sc_tools.pp.umap(adata, **kwargs)[source]#

Compute UMAP embedding.

Wraps scanpy.tl.umap (or rapids_singlecell.tl.umap on GPU).

Parameters:

  • adata (AnnData) – Annotated data with neighbor graph computed. Modified in place.

  • **kwargs (Any) – Passed to the backend umap.

Return type:

None

sc_tools.pp.leiden(adata, resolution=0.8, key_added='leiden', **kwargs)[source]#

Run Leiden clustering.

Wraps scanpy.tl.leiden (or rapids_singlecell.tl.leiden on GPU).

Parameters:

  • adata (AnnData) – Annotated data with neighbor graph computed. Modified in place.

  • resolution (float) – Clustering resolution. Higher values yield more clusters.

  • key_added (str) – Key in adata.obs to store cluster labels.

  • **kwargs (Any) – Passed to the backend leiden.

Return type:

None

sc_tools.pp.cluster(adata, resolution=0.8, use_rep=None, n_neighbors=20, key_added='leiden', run_umap=True, random_state=0, **kwargs)[source]#

Convenience: neighbors + leiden + umap in one call.

Parameters:

  • adata (AnnData) – Annotated data. Modified in place.

  • resolution (float) – Leiden resolution.

  • use_rep (str | None) – Representation for neighbor graph. Auto-detected if None.

  • n_neighbors (int) – Number of neighbors.

  • key_added (str) – Key for cluster labels in adata.obs.

  • run_umap (bool) – If True (default), compute UMAP embedding.

  • random_state (int) – Random state for Leiden reproducibility (D-14, PRV-05).

  • **kwargs (Any) – Extra kwargs passed to neighbors().

Return type:

None

sc_tools.pp.run_utag(adata, max_dist=20, slide_key='library_id', clustering_method='leiden', resolutions=None, key_added='utag', **kwargs)[source]#

Run UTAG spatial-aware clustering.

UTAG builds an adjacency graph from spatial coordinates, performs message passing to aggregate neighborhood features, then clusters to identify microanatomical domains. Runs after standard Leiden as a complementary spatial-aware annotation.

Requires utag package: pip install git+https://github.com/ElementoLab/utag.git@main

Parameters:

  • adata (AnnData) – Annotated data with obsm['spatial']. Modified in place.

  • max_dist (float) – Distance threshold for cell adjacency. 10-20 for IMC, 10-100 for transcriptomics.

  • slide_key (str | None) – Batch key for multi-image processing. None for single image.

  • clustering_method (str) – Clustering method (“leiden” or “parc”).

  • resolutions (list[float] | None) – List of resolutions to explore. Defaults to [0.5, 0.8, 1.0].

  • key_added (str) – Prefix for cluster labels in adata.obs.

  • **kwargs (Any) – Passed to utag.utag.

Return type:

None