Open in app
Jonathan Hui
18.1K Followers
About

Sign in

18.1K Followers
About
Open in app
Jonathan Hui

Jonathan Hui

·Jan 3

F is a second-order derivative. Comparing with gradient descent which is the first-order derivative, F is expensive to compute. That results in many research papers that try to approximate it (even reduce it back to the first-order derivative).

Image for post
Image for post
May I ask what do you mean by NPG as second order optimization method is expensive?
1

Jason Dou

More from Jonathan Hui

Deep Learning

More From Medium

TensorFlow Generative Model Examples

Jonathan Hui

TensorFlow Sequence to Sequence Model Examples

Jonathan Hui

TensorFlow & Keras

Jonathan Hui

TensorFlow BERT & Transformer Examples

Jonathan Hui

TensorFlow Keras Preprocessing Layers & Dataset Performance

Jonathan Hui

TensorFlow NLP Classification Examples

Jonathan Hui

Graph Convolutional Networks (GCN) & Pooling

Jonathan Hui

TensorFlow Computer Vision & Deep Learning Examples

Jonathan Hui

About

Help

Legal

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store