UCALM
UCALM: Unsupervised adaptive Clustering-based Association and Layered Meta-integration. It is designed for GWAS in heterogeneous populations where hidden population structure can dilute association signals or create residual confounding.
Overview
UCALM combines three steps in one framework: unsupervised subgroup inference, subgroup-specific GWAS and layered meta-integration. The first step infers genetically homogeneous subgroups directly from genotype data. The second step runs association testing within each inferred subgroup. The final step integrates subgroup-level association evidence so that shared, ancestry-correlated and population-specific signals can all contribute to the final result.
| Problem in heterogeneous cohorts | UCALM strategy |
|---|---|
| Pooled GWAS can estimate an average genetic effect across heterogeneous samples, attenuating or masking subgroup-specific association signals. | Perform subgroup-specific GWAS within inferred genetically homogeneous subgroups. |
| Predefined population labels are sometimes unavailable, often subjectively defined, and may not accurately reflect the underlying genetic structure. | Infer subgroups directly from genotype data without requiring predefined labels. |
| A single meta-analysis model may have limited ability to accommodate multiple complex effect architectures simultaneously, reducing sensitivity to heterogeneous association signals. | Combine fixed-effect, MR-MEGA and ACAT components into a final UCALM P-value. |
When UCALM Helps
Use UCALM when the study cohort is genetically heterogeneous and the subgroup structure is not fully captured by existing metadata. It is most useful when samples come from multiple ancestries, breeds, breeding lines, companies, regions or admixed backgrounds, or when conventional GWAS shows inflation, masked peaks or inconsistent signals across known groups.
- Use the individual-level data path when MVP needs to infer latent population structure and run association testing within inferred subgroups.
- Use the summary-level data path when subgroup GWAS has already been run and you only need the UCALM layered integration step.
- Use conventional pooled GLM or MLM when the cohort is genetically homogeneous and subgroup-specific effects are not expected.
Path 1: Individual-level Data
Use --ucalm for the full individual-level data path. This command starts from PLINK genotype files
and phenotype records, infers subgroups and then performs subgroup GWAS followed by UCALM integration.
Input: PLINK 1 binary dataset prefix with --bfile (.bed/.bim/.fam) or PLINK 2 PGEN
dataset prefix with --pfile (.pgen/.pvar/.psam), a phenotype file with --pheno, the target
trait column with --pheno-pos, and optional covariates, MAF, thread and clustering parameters. See Genotype Files,
Phenotype File and
Covariates in Phenotype File.
mvp --ucalm \
--bfile demo \
--pheno demo.phe \
--pheno-pos 2 \
--npc-cluster 10 \
--use-umap \
--use-kdtree \
--threads 32 \
--out demo- Analysis steps: genomic relationship matrix (GRM) construction, eigendecomposition of the GRM to derive PCs, variance-weighted PCs, UMAP, AutoHDBSCAN, subgroup GWAS and UCALM layered integration.
- Clustering parameters: set the number of PCs used for clustering (
--npc-cluster), use a KD tree for nearest neighbor search (--use-kdtree), and adjust the UMAP neighborhood size (--umap-nn) and minimum distance parameter (--umap-md) to control the local and global structure and the compactness of the embedding. - Output: the individual-level data path writes cluster assignments, UMAP coordinates, subgroup variance summaries and a final association table. See Outputs for the file list.
Path 2: Summary-level Data
Use the summary-level data path when subgroup GWAS summary statistics have already been generated. In this path, UCALM skips genotype clustering and directly performs layered meta-integration across population-level summary statistics.
Input: at least two harmonized summary statistics files. Each file must include a header and the columns
SNP CHR POS A1 A2 MAF BETA SE PVAL N. Marker IDs, chromosome and position, effect allele coding, reference genome build
and sample-size definitions must be consistent across files.
Requires: summary statistics prepared as described in Summary Statistics Input.
mvp --meta \
--sumstat pop1.sumstat,pop2.sumstat,pop3.sumstat \
--meta-method 1 \
--threads 32 \
--out demo- Analysis steps: MVP keeps SNPs shared across the summary statistics files, checks chromosome, position and allele coding against the first file, flips frequency and beta when A1/A2 are reversed, applies missing-value, MAF and minor allele count filters, and then performs UCALM layered integration.
- Default: UCALM uses a 1 Mb window by default to provide local allele frequency structure for the ancestry-correlated component.
- Output:
demo.assoc.txtcolumns:SNP,CHR,POS,A1,A2, subgroupMAFcolumns, subgroupBETA,SEandPVALcolumns, plus the finalUCALM_PVALcolumn.
Computational Acceleration
To improve computational efficiency, UCALM uses several acceleration strategies during clustering. The table below lists the main strategies and the parameters that control them.
| Scenario | Strategy | Parameter |
|---|---|---|
| PCs calculation | Compute only the leading k PCs from the GRM with truncated eigendecomposition via Spectra rather than a full decomposition. | Set k with --npc-cluster; the default is 10 in the tutorial examples. |
| UMAP neighbor graph | Build the UMAP graph from variance weighted PCs. Use HNSW when the retained PC feature dimension is greater than 50; otherwise use VP-tree with Euclidean distance. | The neighbor search backend is selected automatically from the retained PC feature dimension. |
| HDBSCAN nearest neighbor search | Use a KD tree to compute core distances and support nearest neighbor queries on the UMAP embedding during density based clustering. | Enable with --use-kdtree in the individual-level path. |
Outputs
Several files will be generated in the work directory:
demo.cluster.txt: the cluster results for all individuals. In the ambiguity column, 0 indicates a core sample and 1 indicates an ambiguous reassigned sample.
| id | cluster | ambiguity |
|---|---|---|
| IND1 | c1 | 0 |
| IND2 | c1 | 0 |
| IND3 | c2 | 0 |
| IND4 | c2 | 1 |
| IND5 | c3 | 0 |
demo.cluster.umap.txt: the UMAP results for all individuals.
| id | u1 | u2 |
|---|---|---|
| IND1 | -11.6093 | -13.7700 |
| IND2 | -11.3111 | -13.8723 |
| IND3 | -11.6836 | -13.7768 |
| IND4 | -11.4300 | -14.0521 |
| IND5 | -10.9065 | -14.1600 |
demo.c*.vars.txt: the estimated variance and heritability of all random effects in each subgroup.
| Trait | VarG | VarG_SE | VarE | VarE_SE | h2 | h2_SE | Convergence |
|---|---|---|---|---|---|---|---|
| trait1 | 0.982122 | 0.0701002 | 1.18056 | 0.0315849 | 0.454123 | 0.0214886 | TRUE |
demo.assoc.txt: the UCALM association results.
| SNP | CHR | POS | A1 | A2 | c1_MAF | c2_MAF | c3_MAF | c1_BETA | c1_SE | c1_PVAL | c2_BETA | c2_SE | c2_PVAL | c3_BETA | c3_SE | c3_PVAL | UCALM_PVAL | DOMINANT |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SNP1 | 1 | 112 | T | C | 0.119588 | 0.026840 | 0.000362 | -0.051802 | 0.069109 | 0.453549 | 0.207191 | 0.336108 | 0.537821 | -0.083324 | 0.683110 | 0.416479 | 0.446716 | homogeneous |
| SNP2 | 1 | 370 | A | G | 0.432343 | 0.409509 | 0.343677 | -0.050887 | 0.051815 | 0.326102 | 0.152516 | 0.118324 | 0.197870 | 0.005187 | 0.200672 | 0.244406 | 0.463292 | ancestry-correlated |
| SNP3 | 2 | 482 | C | T | 0.017861 | 0.001534 | 0.005715 | 0.040391 | 0.136479 | 0.767278 | -2.063310 | 1.375000 | 0.133952 | -0.040607 | 0.052340 | 0.187197 | 0.765387 | population-specific |
| SNP4 | 3 | 827 | G | A | 0.119588 | 0.051881 | 0.019908 | -0.051802 | 0.069109 | 0.453549 | -0.555076 | 0.336108 | 0.537821 | 0.779909 | 0.472023 | 0.412295 | 0.446716 | ancestry-correlated |
| SNP5 | 5 | 1550 | G | C | 0.123328 | 0.109231 | 0.106686 | 0.040391 | 0.136479 | 0.767278 | 0.207191 | 0.336108 | 0.537821 | 0.059925 | 0.163459 | 0.265092 | 0.201753 | homogeneous |
UCALM_PVAL is the final layered meta-integration P-value. DOMINANT reports the dominant
UCALM component label for the marker: homogeneous, ancestry-correlated or population-specific.
Result Checks
Use the checks below before interpreting UCALM outputs.
| Check | Why it matters | Action |
|---|---|---|
| Genome build and position | Position mismatches can refer to different loci across builds. | Lift all genotype and summary files to the same reference genome build. |
| Allele harmonization | Reversed A1/A2 labels flip beta signs and allele frequency. | Harmonize A1, A2, allele frequency and BETA before summary integration. |
| Subgroup sample size | Very small subgroups reduce power and can make estimates unstable. | Inspect subgroup counts and interpret sparse subgroup signals cautiously. |
| Missing final results | Markers may be absent from one file or fail MAF or sample-size filters. | Check marker IDs, allele coding, MAF, sample size and shared marker overlap. |