This page collects together the posts for the graduate course on optimization I taught at Princeton in the Spring 2013. This material has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published “Foundations and Trends in Machine Learning, Vol. 8: No. 3-4, pp 231-357, 2015” (see here for the free version):
Part I: Computational complexity of optimization
Post 1: Introduction
Post 2: The ellipsoid method
Post 3: LP, SDP, and Conic Programming
Post 4: Interior Point Methods
Post 5: IPMs for LPs and SDPs
Post 6: Lagrangian duality
Post 7: Classification, SVM, Kernel Learning
Post 8: Turing Machines
Post 9: P vs. NP, NP-completeness
Post 10: Getting around NP-hardness
Bonus lectures by Amir Ali Ahmadi: Sum of Squares (SOS) Techniques: An Introduction, Part I, and Part II
Part II: Oracle complexity of optimization
Post 11: Oracle complexity, large-scale optimization
Post 12: Oracle complexity of Lipschitz convex functions
Post 13: Oracle complexity of smooth convex functions
Post 14: Nesterov’s Accelerated Gradient Descent
Post 15: Strong convexity (added in 2014: Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization)
Post 16: Conditional Gradient Descent and Structured Sparsity
Post 17: ISTA and FISTA
Post 18: Mirror Descent, part I/II
Post 19: Mirror Descent, part II/II
Post 20: Mirror Prox
Part III: Optimization and Randomness
Post 21: Noisy oracles (for stochastic approximation and acceleration by randomization)
Post 22: Acceleration by randomization for a sum of smooth and strongly convex functions
Post 23: Optimization with bandit feedback
Final exam
Post 24: An SDP based on Grothendieck’s inequality; Coordinate Descent
By John Crap February 2, 2023 - 8:05 pm
Wow! This is good idea and i just love it and thanks for sharing!
Explore our New Valentine’s Special Outfit
By ebet February 1, 2023 - 10:09 pm
You have written content that is very interesting. Every time I read your article It gave me new knowledge.
By Alice Aguilar February 1, 2023 - 6:50 am
One of the better posts I’ve read in a long time! aurora roofers
By Anna Watts February 1, 2023 - 6:23 am
Only someone who went to Princeton could write something so amazing. Thank you so much for sharing! firewood for sale denver
By Luis Wood February 1, 2023 - 6:21 am
Thank you so much for sharing this. Where can I find more articles like this? My brain is fueling! firewood for sale Denver
By Kevin N. February 1, 2023 - 6:20 am
SUPERB WRITING COMBINED WITH A MASTER MIND RESULTS IN THIS! tile installers Denver
By Oliver Stevens February 1, 2023 - 6:13 am
I really enjoy this website. I can’t wait to share this with my friends. garage builders Denver
By Benjamin Riley February 1, 2023 - 5:59 am
Princeton produces the best minds in this country and this blog is one of many examples. Thank you for your in depth research to ensure everything was cited and fact checked. fence company denver
By Royal K. February 1, 2023 - 5:58 am
What first made you so interested in this subject? You right with such passion!
fence company colorado springs
By James S. February 1, 2023 - 5:43 am
Fascinating observations. I can tell you are a sharp mind! fence company arvada
By Angela M. February 1, 2023 - 5:16 am
One of the coolest things I’ve stumbled upon. I can’t wait to read more from you – I’m certain you just gained a lifelong follower! concrete contractors arvada
By Jordan Hudson February 1, 2023 - 5:13 am
Great content thanks for sharing! why do lizards do push ups
By Tyler February 1, 2023 - 5:10 am
A complex subject that is extremely entertaining once fully grasped! Keep at it. concrete colorado springs
By John February 1, 2023 - 5:09 am
One of the most interesting things I’ve read in a long time – thanks for sharing! concrete contractors denver
By guriksha January 31, 2023 - 10:07 am
What an Amazing Article… Very Very Helpful Information Sir
By James January 28, 2023 - 8:31 pm
Thanks for sharing this article regarding Parking spaces on campus, it was quite insightful. Hoping to see more articles. Meanwhile, refer
By Article Terrain January 26, 2023 - 12:43 pm
Hi there, Thanks For Sharing The Amazing List. It is very helpful and informative.
By pankaj January 26, 2023 - 2:16 am
i will certainly come back again. This is an amazing blog and i have seen many of your articles on the net. But I have to say that you’ve outdone yourself this time! The article was definitely well-written, and it made my day. Thanks for sharing this great info with us !
https://einfonet.in/procure-to-pay-cycle-in-sap/
By Manoj January 26, 2023 - 2:14 am
Yes i am completely concurred with this article and i simply need say this article is extremely decent and exceptionally useful article.
https://esarkariyojna.in/instant-e-pan-card-kaise-banaye/
By Vitamix 5200 January 25, 2023 - 9:02 pm
I just found this good blog and have high hopes for it to continue. Thank You. Vitamix 5200 blenders
By breville bes870xl January 25, 2023 - 8:56 pm
Thanks a lot for sharing this nice article, hope to read more informativa posts like this one around here, Others are inescapably hard: the travelling salesman problem
By pankaj January 22, 2023 - 2:08 am
I just found this good blog and have high hopes for it to continue. Keep up the great work, its hard to find good ones. I have added to my favorites. Thank You.
By Mannu January 22, 2023 - 2:07 am
What an Amazing Article… Very Very Helpful Information Sir
By jhon K January 21, 2023 - 8:17 am
You share informative post I love to read your article, I hope you will share another good post. Read more home related blogs at hereBest Kitchens dubai
By Hotel in Dubai January 20, 2023 - 7:20 am
https://www.rovehotels.com/en/
By taniya January 19, 2023 - 10:40 am
https://news.ostan-qz.ir/content/ostan/news/39971
By taniya January 19, 2023 - 10:21 am
https://qazvin.ostan-qz.ir/content/ostan/news/78102
By yaran January 19, 2023 - 10:13 am
https://news.ostan-qz.ir/content/ostan/news/45065
By taniya January 19, 2023 - 10:12 am
https://news.ostan-qz.ir/content/ostan/news/45065
By salina January 19, 2023 - 9:29 am
https://ostan-qz.ir/content/ostan/general-content/297/18097
By yalda January 19, 2023 - 9:20 am
https://ostan-qz.ir/content/ostan/general-content/297/18097