# A new way to fine-tune LLMs just dropped

- Channel: [bycloud](https://www.youtube.com/channel/UCgfe2ooZD3VJPB6aJAnuQng)
- Published: 2026-04-27
- Source video: [Watch on YouTube](https://youtube.com/watch?v=lLkE9w1NJs0)
- Read on ReadTube: https://www.read.tube/p/videos/lLkE9w1NJs0
- Language: Original (source language)

## Summary

**Evolution Strategies Return to LLM Fine-Tuning**

The video explains why evolution strategies, long considered impractical for deep learning, are reappearing as a viable way to fine-tune large language models. It argues that while they are weak for next-token pretraining, they fit sparse reinforcement-learning settings where only a single outcome reward is available, and recent papers show they can scale to billion-parameter models and compete with or outperform GRPO in some benchmarks.

The video traces the history of evolution strategies from an old optimization idea that seemed too noisy and expensive for deep neural networks to a method that is now showing fresh promise in LLM fine-tuning. Early versions struggled because they had to explore huge parameter spaces and could not handle millions of interdependent weights, but OpenAI’s 2017 work made the approach scalable by using simple Gaussian perturbations and averaging many random trials to recover useful update directions.

The speaker draws a sharp distinction between two training regimes. For pretraining with next-token prediction, evolution strategies are a poor fit because gradients provide dense, token-level supervision, while evolution strategies compress everything into a single reward signal. But for RL-style fine-tuning, where feedback is often sparse and only arrives at the level of a full response, the method becomes much more appealing because it does not need token-level credit assignment and can treat the model as a black box.

Recent papers are presented as the key evidence that this comeback is real:
- Evolution Strategies at Scale reportedly showed that response-level rewards can work on billion-parameter models.
- The method uses parameter-space exploration rather than only sampling different outputs from the same model.
- Surprisingly small populations, such as 30 perturbations, were enough to estimate useful uphill directions.
- The speaker argues this works because large models have smoother local neighborhoods and because coarse RL rewards align across many parameters.

The second major advance is Eggroll, which attacks the systems bottleneck. Instead of perturbing full weight matrices, it uses low-rank LoRA-style perturbations so many candidate updates can be evaluated much more cheaply, largely by swapping adapters rather than rerunning full separate models. That makes ES far more hardware-friendly, more parallelizable, and more memory efficient while still producing a rich final update after averaging perturbations.

The video concludes that Eggroll and related ES methods are not clearly universal replacements for GRPO or other RL fine-tuning methods, but they look very promising where wall-clock efficiency, memory use, and sparse rewards matter most. The main takeaway is that evolution strategies are no longer just a historical curiosity; they may be becoming a practical tool for large-model optimization again.