This page collects together the posts for the graduate course on optimization I taught at Princeton in the Spring 2013. This material has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published “Foundations and Trends in Machine Learning, Vol. 8: No. 3-4, pp 231-357, 2015” (see here for the free version):

**Part I: Computational complexity of optimization**

Post 1: Introduction

Post 2: The ellipsoid method

Post 3: LP, SDP, and Conic Programming

Post 4: Interior Point Methods

Post 5: IPMs for LPs and SDPs

Post 6: Lagrangian duality

Post 7: Classification, SVM, Kernel Learning

Post 8: Turing Machines

Post 9: P vs. NP, NP-completeness

Post 10: Getting around NP-hardness

Bonus lectures by Amir Ali Ahmadi: Sum of Squares (SOS) Techniques: An Introduction, Part I, and Part II

**Part II: Oracle complexity of optimization**

Post 11: Oracle complexity, large-scale optimization

Post 12: Oracle complexity of Lipschitz convex functions

Post 13: Oracle complexity of smooth convex functions

Post 14: Nesterov’s Accelerated Gradient Descent

Post 15: Strong convexity (added in 2014: Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization)

Post 16: Conditional Gradient Descent and Structured Sparsity

Post 17: ISTA and FISTA

Post 18: Mirror Descent, part I/II

Post 19: Mirror Descent, part II/II

Post 20: Mirror Prox

**Part III: Optimization and Randomness**

Post 21: Noisy oracles (for stochastic approximation and acceleration by randomization)

Post 22: Acceleration by randomization for a sum of smooth and strongly convex functions

Post 23: Optimization with bandit feedback

**Final exam**

Post 24: An SDP based on Grothendieck’s inequality; Coordinate Descent

## By Hvac Colorado Springs July 26, 2021 - 1:55 am

Great blog! An algorithm is a p approximate algorithm if the performance ratio is bounded by the function p in input size

## By Vedonlyöntibonukset July 13, 2021 - 9:47 am

Very good written!

## By Simon JOhansson July 13, 2021 - 9:45 am

Very interesting to read!

## By istanbul evden eve nakliyat July 6, 2021 - 7:40 am

İstanbul’un en iyi evden eve nakliyat şirketleri.

## By David Allen June 10, 2021 - 2:39 pm

Thanks for the compilation of posts from this course. Optimization methods are at the heart of many computer science problems. For example, in machine learning, the optimization problem must be solved every time you set up some model of algorithms from the data, and the practical applicability of the machine learning method itself depends on the efficiency of solving the corresponding optimization problem. I think you should post this collection on Facebook, where she can see and appreciate a fairly large number of students. I see similar posts there from time to time and noticed that in most cases such posts had almost 62 thousand likes! I am sure that in order to achieve such indicators, the authors of these posts used the services of https://viplikes.net/ to quickly increase the number of likes.

## By Concrete Fort Collins June 9, 2021 - 8:34 pm

Thanks. Hope one day you record your lectures for the world to view and benefit from

## By UusimmatKasinot June 3, 2021 - 1:27 am

Part II was the most beneficial for my schooling. I hope you can add up additional studies.

## By Concrete Boulder Co May 24, 2021 - 9:58 am

Absolutey! where the resources are labor, raw materials, etc. and production targets must be achieved. Very interesting blog,

## By Buzsu Code April 13, 2021 - 4:24 pm

thank you Nice blog

## By Jianchao Huang October 23, 2015 - 7:47 am

Excellent blog! Thanks for posting!

## By eşya depolama March 10, 2015 - 4:33 pm

Thanks. Hope one day you record your lectures for the world to view and benefit from

## By Hog August 18, 2014 - 12:29 pm

Great blog!

## By fetullah bayır February 16, 2014 - 3:57 am

Nice post. Many thanks for sharing!

## By Evden eve nakliyat September 9, 2013 - 12:57 pm

Nice post. Many thanks for sharing!

## By Nat Padmanabhan June 17, 2013 - 4:41 pm

Great blog! Thanks for pointing to the accelerated gradient method. Hope one day you record your lectures for the world to view and benefit from!

## By Stephen Becker May 27, 2013 - 7:59 am

Well written and covers a timely choice of topics. Very nice! Thanks for taking the time to write this up.

## By Matthieu Durut April 19, 2013 - 11:50 am

Very interesting blog, thanks for this