{
  "aliases": null,
  "backlinks": null,
  "body": "\n## Proof\n\nHere $^*$ is the [[conjugate transpose|0EX045E2BP]], and $\\C^k$ carries the inner product $\\ip{x}{y} = y^* x$.\n\n1. A square matrix is [[unitary|0D6RMJ10M9]] exactly when its columns form an [[orthonormal family|09VYHEF2VT]]. Indeed the $(i, j)$ entry of $W^* W$ is $w_i^* w_j = \\ip{w_j}{w_i}$ for the columns $w_i$ of $W$.\n2. By [[the Gram matrix result|0AM050KCAW]], $A^* A$ is a positive semidefinite Hermitian $n \\times n$ matrix.\n3. By the [[spectral theorem|0DGZBQWZWN]], $A^* A = V_0 D_0 V_0^*$ with $V_0$ unitary and $D_0$ real diagonal.\n4. Choose a [[permutation matrix|0Q5SM60HTH]] $\\Pi$ such that $D = \\Pi^T D_0 \\Pi$ is diagonal with entries $\\lambda_1 \\ge \\lambda_2 \\ge \\cdots \\ge \\lambda_n$. This permutes the diagonal entries of $D_0$ into decreasing order.\n5. The matrix $\\Pi$ is real with $\\Pi^T \\Pi = I$, so it is unitary. By the [[one-sided inverse theorem|0753Y2X5K6]], $\\Pi \\Pi^T = I$. Let $V = V_0 \\Pi$. It is unitary, since $V^* V = \\Pi^T V_0^* V_0 \\Pi = \\Pi^T \\Pi = I$.\n6. Then $V D V^* = V_0 \\Pi \\Pi^T D_0 \\Pi \\Pi^T V_0^* = A^* A$. Also $V V^* = I$ by the one-sided inverse theorem.\n7. Let $v_1, \\dots, v_n$ be the columns of $V$. They are orthonormal by step 1. From $A^* A V = V D$ we get $A^* A v_i = \\lambda_i v_i$.\n8. For each $i$, $\\lambda_i = \\lambda_i v_i^* v_i = v_i^* A^* A v_i = (A v_i)^* (A v_i) \\ge 0$.\n9. Since $D = V^* (A^* A) V$ and $V^* = V^{-1}$, the matrices $A^* A$ and $D$ have the [[same characteristic polynomial|0TYMASG33Y]]. So $\\lambda_1 \\ge \\cdots \\ge \\lambda_n$ are the eigenvalues of $A^* A$, with multiplicity.\n10. By the definition of [[singular values|0K2TX12X58]], they are $\\sigma_i = \\sqrt{\\lambda_i}$, using the [[square roots of nonnegative reals|068BZBXE0C]]. So $\\sigma_1 \\ge \\cdots \\ge \\sigma_n \\ge 0$.\n11. Let $r$ be the number of $i$ with $\\lambda_i \u003e 0$. Then $\\lambda_i \u003e 0$ for $i \\le r$, and $\\lambda_i = 0$ for $i \u003e r$.\n12. For $i \\le r$, let $u_i = A v_i / \\sigma_i \\in \\C^m$. By step 7, $u_i^* u_j = v_i^* A^* A v_j / (\\sigma_i \\sigma_j) = \\lambda_j\\, v_i^* v_j / (\\sigma_i \\sigma_j)$.\n13. This is $0$ for $i \\ne j$, and $\\lambda_i / \\sigma_i^2 = 1$ for $i = j$. So $u_1, \\dots, u_r$ is an orthonormal family in $\\C^m$.\n14. By the [[existence of an orthonormal basis|0Y85GYEGSF]], it extends to an orthonormal basis $u_1, \\dots, u_m$ of $\\C^m$. This basis has $m$ vectors, by the [[dimension theorem|0WK0Y8RH1M]] and [[the dimension of the coordinate space|0QK7FPE6VE]]. So $r \\le m$.\n15. Let $U$ be the $m \\times m$ matrix with columns $u_1, \\dots, u_m$. By step 1, $U$ is unitary.\n16. For $i \u003e r$, step 8 gives $(A v_i)^* (A v_i) = \\lambda_i = 0$. A vector $y$ with $y^* y = \\sum_k \\abs{y_k}^2 = 0$ is zero, so $A v_i = 0$.\n17. Let $\\Sigma$ be the $m \\times n$ matrix with $\\Sigma_{ii} = \\sigma_i$ for $i \\le \\min(m, n)$ and all other entries $0$.\n18. Column $j$ of $U \\Sigma$ is $\\sum_i \\Sigma_{ij} u_i$. This is $\\sigma_j u_j$ if $j \\le m$, and $0$ if $j \u003e m$.\n19. Column $j$ of $AV$ is $A v_j$. For $j \\le r$, we have $j \\le m$ and $A v_j = \\sigma_j u_j$ by step 12.\n20. For $j \u003e r$, $A v_j = 0$ by step 16. Column $j$ of $U\\Sigma$ is also $0$, since either $j \u003e m$ or $\\sigma_j = 0$.\n21. By steps 18 to 20, $AV = U \\Sigma$. By step 6, $A = A V V^* = U \\Sigma V^*$.\n22. The diagonal entries of $\\Sigma$ are $\\sigma_1 \\ge \\cdots \\ge \\sigma_{\\min(m,n)} \\ge 0$. If $m \u003c n$, the remaining singular values $\\sigma_i$ with $i \u003e m$ are $0$, because $r \\le m$.\n23. So $A = U \\Sigma V^*$ with $U$ and $V$ unitary, $\\Sigma_{ij} = 0$ for $i \\ne j$, and decreasing diagonal entries given by the singular values of $A$.\n\n## Idea\n\nThe matrix $A^* A$ is Hermitian, so it has an orthonormal basis of eigenvectors $v_i$. The images $A v_i$ are automatically orthogonal, with lengths $\\sigma_i$. Normalising them and completing to an orthonormal basis gives $U$.\n",
  "commit": "4a3290d299c8b2cd6493fce2296920ba7f82d942",
  "front": {
    "kind": "proof",
    "msc": [
      "15A18",
      "15A23"
    ],
    "name": "Proof of the singular value decomposition from the spectral theorem",
    "proves": "000NPTMX6J",
    "sources": [
      {
        "locator": "Proof of existence, based on the spectral theorem",
        "url": "https://en.wikipedia.org/wiki/Singular_value_decomposition"
      }
    ],
    "tag": "31AT4VJF68",
    "uses": [
      "0AM050KCAW",
      "0DGZBQWZWN",
      "0D6RMJ10M9",
      "0EX045E2BP",
      "0Q5SM60HTH",
      "0753Y2X5K6",
      "0TYMASG33Y",
      "0K2TX12X58",
      "09VYHEF2VT",
      "0Y85GYEGSF",
      "0WK0Y8RH1M",
      "0QK7FPE6VE",
      "068BZBXE0C"
    ]
  },
  "kind": "proof",
  "link": "https://math2.tamnd.com/node/31AT4VJF68/",
  "links": [
    {
      "field": "proves",
      "target": "000NPTMX6J"
    },
    {
      "field": "uses",
      "target": "0AM050KCAW"
    },
    {
      "field": "uses",
      "target": "0DGZBQWZWN"
    },
    {
      "field": "uses",
      "target": "0D6RMJ10M9"
    },
    {
      "field": "uses",
      "target": "0EX045E2BP"
    },
    {
      "field": "uses",
      "target": "0Q5SM60HTH"
    },
    {
      "field": "uses",
      "target": "0753Y2X5K6"
    },
    {
      "field": "uses",
      "target": "0TYMASG33Y"
    },
    {
      "field": "uses",
      "target": "0K2TX12X58"
    },
    {
      "field": "uses",
      "target": "09VYHEF2VT"
    },
    {
      "field": "uses",
      "target": "0Y85GYEGSF"
    },
    {
      "field": "uses",
      "target": "0WK0Y8RH1M"
    },
    {
      "field": "uses",
      "target": "0QK7FPE6VE"
    },
    {
      "field": "uses",
      "target": "068BZBXE0C"
    }
  ],
  "markdown": "https://math2.tamnd.com/node/31AT4VJF68.md",
  "msc": [
    "15A18",
    "15A23"
  ],
  "name": "Proof of the singular value decomposition from the spectral theorem",
  "prov": null,
  "role": "",
  "source": "https://github.com/tamnd/math2/blob/main/nodes/31A/31AT4VJF68.md",
  "status": null,
  "tag": "31AT4VJF68",
  "uri": "m2://31AT4VJF68",
  "url": "https://math2.tamnd.com/wiki/proof-of-the-singular-value-decomposition-from-the-spectral-theorem/",
  "writing": {
    "tag": "31AT4VJF68",
    "kind": "proof",
    "level": "W0",
    "next": [
      {
        "what": "an inventory row",
        "note": "the row of the inventory that the node accounts for",
        "have": 0,
        "want": 1
      }
    ]
  }
}