<strong>Math</strong> <strong>5311</strong> – <strong>Gateaux</strong> <strong>differentials</strong> <strong>and</strong> <strong>Frechet</strong> <strong>derivatives</strong>Kevin LongJanuary 26, 20091 Differentiation in vector spacesThus far, we’ve developed the theory of minimization without reference to <strong>derivatives</strong>. We’ve even been able tocompute minimizers of quadratic forms without using <strong>derivatives</strong>, by proving that the minimizer of a positivedefinite quadratic form must be the solution to the algebraic equation Kx = f . However, in computations it’sconvenient <strong>and</strong> efficient to use <strong>derivatives</strong>.We’re developing the theory of minimization of functions set in arbitrary vector spaces, so we need to developdifferential calculus in that setting. In multivariable calculus, you learned three related concepts: directional<strong>derivatives</strong>, partial <strong>derivatives</strong>, <strong>and</strong> gradients. In arbitrary vector spaces, we will be able to develop a generalizationof the directional derivative (called the <strong>Gateaux</strong> differential) <strong>and</strong> of the gradient (called the <strong>Frechet</strong>derivative). We won’t go deeply into the theory of these <strong>derivatives</strong> in this course, but we’ll establish the basicdifferentiation rules.A reference for differentiation in infinite-dimensional vector spaces, done at the level of this course, is D. R.Smith, Variational Methods in Optimization, Dover, 1998. A careful development of the finite-dimensional casecan be found in M. Spivak, Calculus on Manifolds, J. Munkres, Analysis on Manifolds, or Rudin, Principles of<strong>Math</strong>ematical Analysis. If you’re interested, E. W. Cheney’s Analysis for Applied <strong>Math</strong>ematics gives a treatment of<strong>Gateaux</strong> <strong>and</strong> <strong>Frechet</strong> <strong>derivatives</strong> at a level one notch above the level of this course.1.1 The <strong>Gateaux</strong> differentialThe <strong>Gateaux</strong> differential generalizes the idea of a directional derivative.Definition 1. Let f : V → U be a function <strong>and</strong> let h ̸= 0 <strong>and</strong> x be vectors in V. The <strong>Gateaux</strong> differential d h f isdefinedf (x + ɛh) − f (x)d h f = lim.ɛ→0 ɛSome things to notice about the <strong>Gateaux</strong> differential:• There is not a single <strong>Gateaux</strong> differential at each point. Rather, at each point x there is a <strong>Gateaux</strong> differentialfor each direction h. In one dimension, there are two <strong>Gateaux</strong> <strong>differentials</strong> for every x: one directed“forward,” one “backward.” In two of more dimensions, there are infinitely many <strong>Gateaux</strong> <strong>differentials</strong>at each point!• The <strong>Gateaux</strong> differential is a one-dimensional calculation along a specified direction h. Because it’s onedimensional,you can use ordinary one-dimensional calculus to compute it. Your old friends such as thechain rule work for <strong>Gateaux</strong> <strong>differentials</strong>. Thus, it’s usually easy to compute a <strong>Gateaux</strong> differential evenwhen the space V is infinite dimensional.1