In algebraic geometry, Serre duality is a duality present on non-singular projective algebraic varieties V of dimension n (and in greater generality) for vector bundles and the more general coherent sheaves. It shows that a cohomology group Hⁱ is the dual space of another one, Hⁿ⁻ⁱ. If the variety is defined over the complex numbers, this is therefore quite distinct from Poincaré duality, which relates Hⁱ to H²ⁿ⁻ⁱ because as a manifold V has dimension 2n.

The case of algebraic curves was already implicit in the Riemann-Roch theorem. For a curve C the coherent groups Hⁱ vanish for i > 1; but H¹ does enter implicitly. In fact the basic relation of the theorem involves L(D) and L(K−D), where D is a divisor and K a divisor of the canonical class. After Serre we recognise l(K−D) as the dimension of H¹(D), where now D means the line bundle determined by the divisor D. That is, Serre duality in this case relates groups H⁰(D) and H¹(KD*), and we are reading off dimensions (notation: K is the canonical line bundle, D* is the dual line bundle, and juxtaposition is tensor product of line bundles).

In this formulation the theorem can be rearranged to read as a calculation of the Euler characteristic of a sheaf

h⁰(D) − h¹(D),

in terms of the genus of the curve, which is

h¹(C,O_C),

and the degree of D. It is this expression that can be generalised to higher dimensions.

Serre duality of curves is therefore something very classical; but it has interesting light to cast. For example, in Riemann surface theory, the deformation theory of complex structures is studied classically by means of quadratic differentials (namely sections of L(K²)). The deformation theory of Kunihiko Kodaira and D. C. Spencer identifies deformations via H¹(T), where T is the tangent bundle sheaf K*. The duality shows why these approaches coincide.

The origin of the theory lay in Serre's earlier work on several complex variables. In the generalisation of Alexandre Grothendieck, Serre duality becomes a part of coherent duality in a much broader setting. While the role of K above in general Serre duality is played by the determinant line bundle of the cotangent bundle, when V is a manifold, in full generality K cannot just be a single sheaf in the absence of some hypothesis of non-singularity on V. The formulation in full generality uses a derived category and Ext functors, to allow for the fact that K is now represented by a chain complex of sheaves. The statement of the theorem is recognisably Serre's, however.