Bachelor's thesis talk. Tao is advised by Severin Reiz.
Previous talks at the SCCS Colloquium
Tao Xiang: Extending a Newton-CG Second-order Optimizer to Natural Language Processing
SCCS Colloquium |
We first introduce a new second-order optimizer: Newton-CG, which includes how it works and its advantages and disadvantages compared to other first-order and second-order optimizers theoretically. We then introduce the experiment including the machine translation problem and the transformer model. Finally, we display the experiment results.