Author Archives: Sebastien Bubeck

Guest post by Julien Mairal: A Kernel Point of View on Convolutional Neural Networks, part II

This is a continuation of Julien Mairal‘s guest post on CNNs, see part I here. Stability to deformations of convolutional neural networks In their ICML paper Zhang et al. introduce a functional space for CNNs with one layer, by noticing … Continue reading

Posted in Machine learning | 4 Comments

Guest post by Julien Mairal: A Kernel Point of View on Convolutional Neural Networks, part I

    I (n.b., Julien Mairal) have been interested in drawing links between neural networks and kernel methods for some time, and I am grateful to Sebastien for giving me the opportunity to say a few words about it on … Continue reading

Posted in Machine learning | Leave a comment

Optimal bound for stochastic bandits with corruption

Guest post by Mark Sellke. In the comments of the previous blog post we asked if the new viewpoint on best of both worlds can be used to get clean “interpolation” results. The context is as follows: in a STOC … Continue reading

Posted in Machine learning, Optimization, Theoretical Computer Science | 2 Comments

Amazing progress in adversarially robust stochastic multi-armed bandits

In this post I briefly discuss some recent stunning progress on robust bandits (for more background on bandits see these two posts, part 1 and part 2, in particular what is described below gives a solution to Open Problem 3 … Continue reading

Posted in Machine learning, Optimization, Theoretical Computer Science | 2 Comments

Nemirovski’s acceleration

I will describe here the very first (to my knowledge) acceleration algorithm for smooth convex optimization, which is due to Arkadi Nemirovski (dating back to the end of the 70’s). The algorithm relies on a -dimensional plane-search subroutine (which, in … Continue reading

Posted in Optimization | 8 Comments

A short proof for Nesterov’s momentum

Yesterday I posted the following picture on Twitter and it quickly became my most visible tweet ever (by far):     I thought this would be a good opportunity to revisit the proof of Nesterov’s momentum, especially since as it … Continue reading

Posted in Optimization | 11 Comments

Remembering Michael

It has been a year since the tragic event of September 2017. We now know what happened, and it is a tremendously sad story of undiagnosed type 1 diabetes. This summer at MSR Michael was still very present in our … Continue reading

Posted in Uncategorized | 1 Comment

How to generalize (algorithmically)

A couple of months ago I taught an introduction to statistical learning theory. I took inspiration from two very good introductory books on SLT: “Foundations of ML”, and “Understanding Machine Learning: From Theory to Algorithms”. I also covered some classical … Continue reading

Posted in Machine learning | Leave a comment

Some updates

The blog has been eerily silent for most of 2018, here is why: The main culprit is definitely the COLT 2018 chairing. This year we received a surprise 50% increase in number of submissions. This is great news for the … Continue reading

Posted in Announcement | 2 Comments

Youtube channel

I just started a youtube channel. The hope is that this will be a companion to the blog. Some of the posts will “graduate” into videos (e.g., the first set of videos will correspond to an expanded version of bandit … Continue reading

Posted in Announcement | 2 Comments