AdamNX: An Adam improvement algorithm based on a novel exponential decay mechanism for the second-order moment estimate
Zhu, Xiao, Min · cs.LG,stat.ML · 2026-07-24 · 原文 · 证据态 未学
This paper studies the exponential decay mechanism of the second-moment estimate in Adam. We propose AdamNX and a time-varying second-moment decay rate that gradually weakens the correction applied to the update scale. Under the assumptions used in our analysis, this mechanism makes the updates approach momentum-SGD-like behavior during the training plateau phase. We report results on the image-classification, object-detection, and semantic-segmentation tasks, configurations, and comparators included in this paper. These results do not establish multi-seed statistical effects, flatness, or generalization beyond the reported tasks. Our code is open-sourced at https://github.com/mengzhu0308/AdamNX.
讲义
讲义·推断 依据「原文」自动生成的结构化摘要（推断），非原文表述；以原文为准。
1. 人话版
This paper studies the exponential decay mechanism of the second-moment estimate in Adam.
We propose AdamNX and a time-varying second-moment decay rate that gradually weakens the correction applied to the update scale.
2. 领域脉络
本文类目：cs.LG、stat.ML，属于其所在研究脉络的最新进展。
3. 机制拆解
Under the assumptions used in our analysis, this mechanism makes the updates approach momentum-SGD-like behavior during the training plateau phase.
We report results on the image-classification, object-detection, and semantic-segmentation tasks, configurations, and comparators included in this paper.
These results do not establish multi-seed statistical effects, flatness, or generalization beyond the reported tasks.
4. 证据与数字
Our code is open-sourced at https://github.com/mengzhu0308/AdamNX.
5. 反例与边界
摘要未声明局限与反例——这是需要警惕的信号，精读时先问边界。
6. 跨领域连接与意外收获
横跨 2 个类目（cs.LG、stat.ML），关注其在你兴趣板块间的迁移面。
7. 可复用方法
把本文机制与你手头项目对照，找一个两周内能验证的最小实验。
8. 术语表
精读时把不熟的术语记入此处，作为下次回忆的锚点。
主动回忆
先合上内容自己复述，再点「显示」核对，然后如实自评。评分即时进 FSRS 排程。
讲义·推断 依据「原文」自动生成的结构化摘要（推断），非原文表述；以原文为准。
1. 人话版
This paper studies the exponential decay mechanism of the second-moment estimate in Adam.
We propose AdamNX and a time-varying second-moment decay rate that gradually weakens the correction applied to the update scale.
2. 领域脉络
本文类目：cs.LG、stat.ML，属于其所在研究脉络的最新进展。
3. 机制拆解
Under the assumptions used in our analysis, this mechanism makes the updates approach momentum-SGD-like behavior during the training plateau phase.
We report results on the image-classification, object-detection, and semantic-segmentation tasks, configurations, and comparators included in this paper.
These results do not establish multi-seed statistical effects, flatness, or generalization beyond the reported tasks.
4. 证据与数字
Our code is open-sourced at https://github.com/mengzhu0308/AdamNX.
5. 反例与边界
摘要未声明局限与反例——这是需要警惕的信号，精读时先问边界。
6. 跨领域连接与意外收获
横跨 2 个类目（cs.LG、stat.ML），关注其在你兴趣板块间的迁移面。
7. 可复用方法
把本文机制与你手头项目对照，找一个两周内能验证的最小实验。
8. 术语表
精读时把不熟的术语记入此处，作为下次回忆的锚点。