This page collects together the posts for the graduate course on optimization I taught at Princeton in the Spring 2013. This material has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published “Foundations and Trends in Machine Learning, Vol. 8: No. 3-4, pp 231-357, 2015” (see here for the free version):
Part I: Computational complexity of optimization
Post 1: Introduction
Post 2: The ellipsoid method
Post 3: LP, SDP, and Conic Programming
Post 4: Interior Point Methods
Post 5: IPMs for LPs and SDPs
Post 6: Lagrangian duality
Post 7: Classification, SVM, Kernel Learning
Post 8: Turing Machines
Post 9: P vs. NP, NP-completeness
Post 10: Getting around NP-hardness
Bonus lectures by Amir Ali Ahmadi: Sum of Squares (SOS) Techniques: An Introduction, Part I, and Part II
Part II: Oracle complexity of optimization
Post 11: Oracle complexity, large-scale optimization
Post 12: Oracle complexity of Lipschitz convex functions
Post 13: Oracle complexity of smooth convex functions
Post 14: Nesterov’s Accelerated Gradient Descent
Post 15: Strong convexity (added in 2014: Nesterov’s Accelerated Gradient Descent for Smooth and Strongly Convex Optimization)
Post 16: Conditional Gradient Descent and Structured Sparsity
Post 17: ISTA and FISTA
Post 18: Mirror Descent, part I/II
Post 19: Mirror Descent, part II/II
Post 20: Mirror Prox
Part III: Optimization and Randomness
Post 21: Noisy oracles (for stochastic approximation and acceleration by randomization)
Post 22: Acceleration by randomization for a sum of smooth and strongly convex functions
Post 23: Optimization with bandit feedback
Final exam
Post 24: An SDP based on Grothendieck’s inequality; Coordinate Descent
By VenuePro October 21, 2021 - 7:22 am
VenuePro is made up of 14 comprehensive modules that deliver all aspects of effective venue and event management, and whilst consulting with industry experts and global venue and space owners, is constantly in further development. venue management software uk
By Snozone UK October 12, 2021 - 10:43 am
when it comes to choosing indoor skiing london for yourself, it’s entirely dependent on what you are searching for.
By TBIMauritius October 12, 2021 - 2:49 am
There are occupation permits, residence permits, a permanent residency program and even a Premium visa that allows foreign nationals to come to Mauritius to live for a year with very few conditions. We also cover the different ways for foreign nationals to get Mauritian citizenship and decipher the many immigration regulations. We try to answer a few key questions and topics on the subject. Citizenship Of Mauritius
By UFA October 7, 2021 - 12:51 am
Thanks for sharing a great article.
You are providing wonderful information, it is very useful to us.
Keep posting like this informative articles.
Thank you.
By top cars in USA October 6, 2021 - 9:52 am
Thanks for sharing your wonderful and helpful content with us. Looking forward to see more good posts from you in future.
By Kasinoarvostelut October 5, 2021 - 10:35 am
Thank you. Very intresting topic and post. I belive this will benefit many peoples greatly.
By outdoor champ October 4, 2021 - 12:46 pm
Another type of fuel, the kerosene lanterns are great for campers that aren’t afraid to fuel their own gear. These lanterns aren’t as easy to use and they come in a few different styles
By emma September 2, 2021 - 3:10 am
rial has been reorganized (some parts have been cut, some have been extended) into a monograph which got recently published
By Paul Shishechkin September 1, 2021 - 10:52 am
Thanks for making the course available to public! Do you mind if I post a link to your page on my Instagram? You can do that too using this service https://krootez.com so you can find more readers for your blog.
By Online Typing Certificate August 31, 2021 - 6:17 am
Nice post.
By vkpak August 17, 2021 - 3:15 am
nice post
By Jose Ellis August 5, 2021 - 7:00 am
Thank you for sharing this post! Do you want to make detailed video about it? You can get some likes for your video on youtube from this site https://soclikes.com
By Hvac Colorado Springs July 26, 2021 - 1:55 am
Great blog! An algorithm is a p approximate algorithm if the performance ratio is bounded by the function p in input size
By Vedonlyöntibonukset July 13, 2021 - 9:47 am
Very good written!
By Simon JOhansson July 13, 2021 - 9:45 am
Very interesting to read!
By istanbul evden eve nakliyat July 6, 2021 - 7:40 am
İstanbul’un en iyi evden eve nakliyat şirketleri.
By David Allen June 10, 2021 - 2:39 pm
Thanks for the compilation of posts from this course. Optimization methods are at the heart of many computer science problems. For example, in machine learning, the optimization problem must be solved every time you set up some model of algorithms from the data, and the practical applicability of the machine learning method itself depends on the efficiency of solving the corresponding optimization problem. I think you should post this collection on Facebook, where she can see and appreciate a fairly large number of students. I see similar posts there from time to time and noticed that in most cases such posts had almost 62 thousand likes! I am sure that in order to achieve such indicators, the authors of these posts used the services of https://viplikes.net/ to quickly increase the number of likes.
By Concrete Fort Collins June 9, 2021 - 8:34 pm
Thanks. Hope one day you record your lectures for the world to view and benefit from
By UusimmatKasinot June 3, 2021 - 1:27 am
Part II was the most beneficial for my schooling. I hope you can add up additional studies.
By Concrete Boulder Co May 24, 2021 - 9:58 am
Absolutey! where the resources are labor, raw materials, etc. and production targets must be achieved. Very interesting blog,
By Buzsu Code April 13, 2021 - 4:24 pm
thank you Nice blog
By Jianchao Huang October 23, 2015 - 7:47 am
Excellent blog! Thanks for posting!
By eşya depolama March 10, 2015 - 4:33 pm
Thanks. Hope one day you record your lectures for the world to view and benefit from
By Hog August 18, 2014 - 12:29 pm
Great blog!
By fetullah bayır February 16, 2014 - 3:57 am
Nice post. Many thanks for sharing!
By Evden eve nakliyat September 9, 2013 - 12:57 pm
Nice post. Many thanks for sharing!
By Nat Padmanabhan June 17, 2013 - 4:41 pm
Great blog! Thanks for pointing to the accelerated gradient method. Hope one day you record your lectures for the world to view and benefit from!
By Stephen Becker May 27, 2013 - 7:59 am
Well written and covers a timely choice of topics. Very nice! Thanks for taking the time to write this up.
By Matthieu Durut April 19, 2013 - 11:50 am
Very interesting blog, thanks for this