ORF523: The complexities of optimization

This page collects together the posts for the graduate course on optimization I taught at Princeton in the Spring 2013. This material has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published “Foundations and Trends in Machine Learning, Vol. 8: No. 3-4, pp 231-357, 2015” (see here for the free version):

Part I: Computational complexity of optimization

Post 1: Introduction

Post 2: The ellipsoid method

Post 3: LP, SDP, and Conic Programming

Post 4: Interior Point Methods

Post 5: IPMs for LPs and SDPs

Post 6: Lagrangian duality

Post 7: Classification, SVM, Kernel Learning

Post 8: Turing Machines

Post 9: P vs. NP, NP-completeness

Post 10: Getting around NP-hardness

Bonus lectures by Amir Ali Ahmadi: Sum of Squares (SOS) Techniques: An Introduction, Part I, and Part II

Part II: Oracle complexity of optimization

Post 11: Oracle complexity, large-scale optimization

Post 12: Oracle complexity of Lipschitz convex functions

Post 13: Oracle complexity of smooth convex functions

Post 14: Nesterov’s Accelerated Gradient Descent

Post 15: Strong convexity (added in 2014: Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization)

Post 16: Conditional Gradient Descent and Structured Sparsity

Post 17: ISTA and FISTA

Post 18: Mirror Descent, part I/II

Post 19: Mirror Descent, part II/II

Post 20: Mirror Prox

Part III: Optimization and Randomness

Post 21: Noisy oracles (for stochastic approximation and acceleration by randomization)

Post 22: Acceleration by randomization for a sum of smooth and strongly convex functions

Post 23: Optimization with bandit feedback

Final exam

Post 24: An SDP based on Grothendieck’s inequality; Coordinate Descent

338 Responses to "ORF523: The complexities of optimization"

By John Crap February 2, 2023 - 8:05 pm

Wow! This is good idea and i just love it and thanks for sharing!
Explore our New Valentine’s Special Outfit

By ebet February 1, 2023 - 10:09 pm

You have written content that is very interesting. Every time I read your article It gave me new knowledge.

By Alice Aguilar February 1, 2023 - 6:50 am

One of the better posts I’ve read in a long time! aurora roofers

By Anna Watts February 1, 2023 - 6:23 am

Only someone who went to Princeton could write something so amazing. Thank you so much for sharing! firewood for sale denver

By Luis Wood February 1, 2023 - 6:21 am

Thank you so much for sharing this. Where can I find more articles like this? My brain is fueling! firewood for sale Denver

By Kevin N. February 1, 2023 - 6:20 am

SUPERB WRITING COMBINED WITH A MASTER MIND RESULTS IN THIS! tile installers Denver

By Oliver Stevens February 1, 2023 - 6:13 am

I really enjoy this website. I can’t wait to share this with my friends. garage builders Denver

By Benjamin Riley February 1, 2023 - 5:59 am

Princeton produces the best minds in this country and this blog is one of many examples. Thank you for your in depth research to ensure everything was cited and fact checked. fence company denver

By Royal K. February 1, 2023 - 5:58 am

What first made you so interested in this subject? You right with such passion!
fence company colorado springs

By James S. February 1, 2023 - 5:43 am

Fascinating observations. I can tell you are a sharp mind! fence company arvada

By Angela M. February 1, 2023 - 5:16 am

One of the coolest things I’ve stumbled upon. I can’t wait to read more from you – I’m certain you just gained a lifelong follower! concrete contractors arvada

By Jordan Hudson February 1, 2023 - 5:13 am

Great content thanks for sharing! why do lizards do push ups

By Tyler February 1, 2023 - 5:10 am

A complex subject that is extremely entertaining once fully grasped! Keep at it. concrete colorado springs

By John February 1, 2023 - 5:09 am

One of the most interesting things I’ve read in a long time – thanks for sharing! concrete contractors denver

By guriksha January 31, 2023 - 10:07 am

What an Amazing Article… Very Very Helpful Information Sir

By James January 28, 2023 - 8:31 pm

Thanks for sharing this article regarding Parking spaces on campus, it was quite insightful. Hoping to see more articles. Meanwhile, refer

By Article Terrain January 26, 2023 - 12:43 pm

Hi there, Thanks For Sharing The Amazing List. It is very helpful and informative.

By pankaj January 26, 2023 - 2:16 am

i will certainly come back again. This is an amazing blog and i have seen many of your articles on the net. But I have to say that you’ve outdone yourself this time! The article was definitely well-written, and it made my day. Thanks for sharing this great info with us !
https://einfonet.in/procure-to-pay-cycle-in-sap/

By Manoj January 26, 2023 - 2:14 am

Yes i am completely concurred with this article and i simply need say this article is extremely decent and exceptionally useful article.
https://esarkariyojna.in/instant-e-pan-card-kaise-banaye/

By Vitamix 5200 January 25, 2023 - 9:02 pm

I just found this good blog and have high hopes for it to continue. Thank You. Vitamix 5200 blenders

By breville bes870xl January 25, 2023 - 8:56 pm

Thanks a lot for sharing this nice article, hope to read more informativa posts like this one around here, Others are inescapably hard: the travelling salesman problem

By pankaj January 22, 2023 - 2:08 am

I just found this good blog and have high hopes for it to continue. Keep up the great work, its hard to find good ones. I have added to my favorites. Thank You.

By Mannu January 22, 2023 - 2:07 am