NVIDIA BioNeMo recipes: LoRA fine-tunes ESM2-3B, Evo2-1B

NVIDIA published BioNeMo Recipes on Jun 15, 2026, showing how Low-Rank Adaptation, LoRA, enables parameter-efficient fine-tuning of billion-parameter biological foundation models. The recipes demonstrate adapting ESM2-3B and Evo2-1B on a single NVIDIA RTX 6000 Blackwell Workstation Edition GPU, training only a small fraction of the model weights while preserving or improving task accuracy.

How LoRA and the BioNeMo Recipes work

LoRA keeps the pretrained backbone frozen and adds small low-rank adapter matrices in parallel to target weight matrices, so training updates affect only those adapter parameters rather than the full model. The Recipes present this pattern using familiar PyTorch, Hugging Face, and PEFT primitives and include examples that attach adapters to fused QKV projections. The documentation shows an example PEFT configuration with r=8, lora_alpha=16 and target_modules set to ["layernorm_qkv"].

The Recipes also integrate NVIDIA performance tooling. The team used NVIDIA Transformer Engine and sequence packing in THD format to reduce padding overhead. In the ESM2-3B case study, enabling TE and THD packing allowed the full fine-tuning workflow to run on one RTX 6000 Blackwell Workstation Edition GPU in under one hour. Switching from the padded BSHD format to packed THD raised tokens-per-second throughput by 5.5x in the reported setup.

Two case studies: ESM2-3B for PSSP, Evo2-1B for splice-site classification

ESM2-3B, a 3-billion-parameter protein language model, was adapted for protein secondary structure prediction by adding a per-residue classification head and training LoRA adapters while keeping the backbone frozen. On the Porter 6 test set the ESM2-3B plus LoRA model achieved Q3 and Q8 accuracies of 84.80% and 74.30% respectively, reported as the mean over the top five validation checkpoints. Those scores compare to Porter 6 at 84.56%/74.18%, NetSurfP-3.0 at 82.92%/71.84%, and SPOT-1D-LM at 84.30%/74.09% on the same benchmark.

Evo2-1B, a DNA foundation model built on striped Hyena blocks and a small number of attention layers, was adapted for splice-site classification by attaching adapters to attention, MLP, and Hyena-mixer layers. Using the splice_sites_all task from the Nucleotide Transformer downstream-tasks dataset, the Evo2-1B plus LoRA workflow raised splice-site classification accuracy from a head-only baseline of 52.3% to 96.6%, while training only 1.42% of the model parameters.

Both case studies use the same LoRA recipe pattern despite differing modalities, task types, and underlying architectures. The Recipes include code, data loader, loss, and optimizer setup so practitioners can reproduce or customize the workflows.

Why it matters

LoRA lowers the resource barrier to adapt billion-parameter biological models by reducing the number of trainable parameters and the optimizer state that must be stored. That lets teams fine-tune large protein and DNA models on a single workstation GPU while achieving competitive or superior accuracy. Sequence packing and TE acceleration further shift the bottleneck away from raw compute and toward efficient data representation, enabling practical iteration on biological tasks without large GPU clusters.

What to watch

Look for external reproductions of the ESM2-3B Q3/Q8 numbers on the Porter 6 splits and for broader adoption of THD sequence packing in other BioNeMo workflows. Also watch whether LoRA adapter placements beyond fused QKV and Hyena-mixer layers maintain the same gains across more genomic and proteomic downstream tasks.