This page collects together the posts for the graduate course on optimization I taught at Princeton in the Spring 2013. This material has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published “Foundations and Trends in Machine Learning, Vol. 8: No. 3-4, pp 231-357, 2015” (see here for the free version):
Part I: Computational complexity of optimization
Post 1: Introduction
Post 2: The ellipsoid method
Post 3: LP, SDP, and Conic Programming
Post 4: Interior Point Methods
Post 5: IPMs for LPs and SDPs
Post 6: Lagrangian duality
Post 7: Classification, SVM, Kernel Learning
Post 8: Turing Machines
Post 9: P vs. NP, NP-completeness
Post 10: Getting around NP-hardness
Bonus lectures by Amir Ali Ahmadi: Sum of Squares (SOS) Techniques: An Introduction, Part I, and Part II
Part II: Oracle complexity of optimization
Post 11: Oracle complexity, large-scale optimization
Post 12: Oracle complexity of Lipschitz convex functions
Post 13: Oracle complexity of smooth convex functions
Post 14: Nesterov’s Accelerated Gradient Descent
Post 15: Strong convexity (added in 2014: Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization)
Post 16: Conditional Gradient Descent and Structured Sparsity
Post 17: ISTA and FISTA
Post 18: Mirror Descent, part I/II
Post 19: Mirror Descent, part II/II
Post 20: Mirror Prox
Part III: Optimization and Randomness
Post 21: Noisy oracles (for stochastic approximation and acceleration by randomization)
Post 22: Acceleration by randomization for a sum of smooth and strongly convex functions
Post 23: Optimization with bandit feedback
Final exam
Post 24: An SDP based on Grothendieck’s inequality; Coordinate Descent
By huay crypto September 28, 2022 - 3:39 am
Reading this content makes me feel that it has given me more knowledge. Thank you for making such a good article. Let’s see it.
By Mark Kotov September 24, 2022 - 9:14 am
Thanks for the post. Training is very important. We are learning something all our life. And this is great because how else to get skills and new knowledge. After all, the world is constantly changing and does not stand still. https://flirtymania.plus/ar.html
By AmeliaJerry September 22, 2022 - 8:36 am
Your post is too good. I really Appreciate it. Thanks for sharing this blog with us. Customized Lip Gloss Boxes
By Chris Essen September 18, 2022 - 9:03 am
Schädlingsbekämpfung muss durch Profis gemacht werden.
By Sharon September 13, 2022 - 6:11 am
Amazing blog, please keep on writing this. I love to read such engaging and informative blogs, I look forward for more of your blogs.
Student Accommodation Melbourne
By Franz Wächter September 12, 2022 - 12:00 pm
Glas ist der beste Werkstoff in Deutschland!