The Cartesian Cafe

The Cartesian Cafe is the podcast where an expert guest and Timothy Nguyen map out scientific and mathematical subjects in detail. This collaborative journey with other experts will have us writing down formulas, drawing pictures, and reasoning about them together on a whiteboard. If you’ve been longing for a deeper dive into the intricacies of scientific subjects, then this is the podcast for you. Topics covered include mathematics, physics, machine learning, artificial intelligence, and computer science. Content also viewable on YouTube: www.youtube.com/timothynguyen and Spotify. Timothy Nguyen is a mathematician and AI researcher working in industry. Homepage: www.timothynguyen.com, Twitter: @IAmTimNguyen Patreon: www.patreon.com/timothynguyen

https://cartesiancafe.podbean.com

subscribe
share






episode 4: Tai-Danae Bradley | Category Theory and Language Models


Tai-Danae Bradley is a mathematician who received her Ph.D. in mathematics from the CUNY Graduate Center. She was formerly at Alphabet and is now at Sandbox AQ, a startup focused on combining machine learning and quantum physics. Tai-Danae is a visiting research professor of mathematics at The Master’s University and the executive director of the Math3ma Institute, where she hosts her popular blog on category theory. She is also a co-author of the textbook Topology: A Categorical Approach that presents basic topology from the modern perspective of category theory.

In this episode, we provide a compressed crash course in category theory. We provide definitions and plenty of basic examples for all the basic notions: objects, morphisms, categories, functors, natural transformations. We also discuss the first basic result in category theory which is the Yoneda Lemma. We conclude with a discussion of how Tai-Danae has used category-theoretic methods in her work on language modeling, in particular, in how the passing from syntax to semantics can be realized through category-theoretic notions.

Patreon: https://www.patreon.com/timothynguyen

Originally published on July 20, 2022 on YouTube: https://youtu.be/Gz8W1r90olc

Timestamps:

  • 00:00:00 : Introduction
  • 00:03:07 : How did you get into category theory?
  • 00:06:29 : Outline of podcast
  • 00:09:21 : Motivating category theory
  • 00:11:35 : Analogy: Object Oriented Programming
  • 00:12:32 : Definition of category
  • 00:18:50 : Example: Category of sets
  • 00:20:17 : Example: Matrix category
  • 00:25:45 : Example: Preordered set (poset) is a category
  • 00:33:43 : Example: Category of finite-dimensional vector spaces
  • 00:37:46 : Forgetful functor
  • 00:39:15 : Fruity example of forgetful functor: Forget race, gender, we're all part of humanity!
  • 00:40:06 : Definition of functor
  • 00:42:01 : Example: API change between programming languages is a functor
  • 00:44:23 : Example: Groups, group homomorphisms are categories and functors
  • 00:47:33 : Resume definition of functor
  • 00:49:14 : Example: Functor between poset categories = order-preserving function
  • 00:52:28 : Hom Functors. Things are getting meta (no not the tech company)
  • 00:57:27 : Category theory is beautiful because of its rigidity
  • 01:00:54 : Contravariant functor
  • 01:03:23 : Definition: Presheaf
  • 01:04:04 : Why are things meta? Arrows, arrows between arrows, categories of categories, ad infinitum.
  • 01:07:38 : Probing a space with maps (prelude to Yoneda Lemma)
  • 01:12:10 : Algebraic topology motivated category theory
  • 01:15:44 : Definition: Natural transformation
  • 01:19:21 : Example: Indexing category
  • 01:21:54 : Example: Change of currency as natural transformation
  • 01:25:35 : Isomorphism and natural isomorphism
  • 01:27:34 : Notion of isomorphism in different categories
  • 01:30:00 : Yoneda Lemma
  • 01:33:46 : Example for Yoneda Lemma: Identity functor and evaluation natural transformation
  • 01:42:33 : Analogy between Yoneda Lemma and linear algebra
  • 01:46:06 : Corollary of Yoneda Lemma: Isomorphism of objects = Isomorphism of hom functors
  • 01:50:40 : Yoneda embedding is fully faithful. Reasoning about this.
  • 01:55:15 : Language Category
  • 02:03:10 : Tai-Danae's paper: "An enriched category theory of language: from syntax to semantics"


Further Reading:

  • Tai-Danae's Blog: https://www.math3ma.com/categories
  • Tai-Danae Bradley. "What is applied category theory?" https://arxiv.org/pdf/1809.05923.pdf
  • Tai-Danae Bradley, John Terilla, Yiannis Vlassopoulos. "An enriched category theory of language: from syntax to semantics." https://arxiv.org/pdf/2106.07890.pdf


fyyd: Podcast Search Engine
share








 August 22, 2022  2h25m