Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance 1AXXV9ZT11
| URI | m2://res/1AXXV9ZT11 |
|---|---|
| Kind | preprint |
| Creators | Arda Fazla, Antesh Upadhyay, Ege C. Kaya, M. Berk Sahin, Abolfazl Hashemi |
| Date | 2026-10-01 |
| Subjects | arxiv cs.LG, arxiv math.OC |
| Scope | border |
Expressions
e1: version, preprint
| Date | 2026-10-01 |
|---|---|
| Ids | arXiv 2610.02355 |
Items in this resource
The mentions of the inventory that name this resource. Each name links to the card of its inventory row.
| Item | Kind | Where | Source |
|---|---|---|---|
| Convergence of SGDM with Dynamic Batching | statement | theorem 3 | arxiv |
| SGD Upper Bound with Dynamic Batching | statement | theorem 2 | arxiv |
| Unrestricted algorithm reduction of the activated BG-$a$ chain | statement | lemma 5 | arxiv |
| Lower bound under BG-$a$ noise | statement | theorem 1 | arxiv |
| SGDM Upper Bound with Dynamic Batching | statement | theorem 3 | arxiv |
| Masked oracle properties from Lemma 3 | statement | lemma 3 | arxiv |
| Deterministic chain properties (Lemma 2 of restated | statement | lemma 1 | arxiv |
| Range bound | statement | lemma 2 | arxiv |
| Deterministic chain | definition | definition 3 | arxiv |
| Expected SFO cost for $a>2$ | statement | proposition 1 | arxiv |
| Lemma 6 of restated | statement | lemma 4 | arxiv |
| Lower bound under smoothness | statement | theorem 1 | arxiv |
| Zero-respecting algorithm | definition | definition 2 | arxiv |
| Convergence of SGD with Dynamic Batching | statement | theorem 2 | arxiv |
| Progress function | definition | definition 1 | arxiv |
| Bernoulli masked chain oracle | definition | definition 4 | arxiv |
Catalogue records
- arxiv: 2610.02355 (work)