Also the (provocatively titled) "Let's Remove Quaternions from every 3d Engine" [1]
Spoiler alert: rotors are mechanically identical to quaternions, while being easier to understand. If you understand rotors, you understand quaternions. You can fit the laws you need to understand rotors on a business card.
Plus, rotors abstract to higher and lower (well, there's only one plane and its two respective orientations in 2d, but still) dimensions.
Complex numbers as planes (bivectors in GA parlance) has been the most mind-opening mathematical concept I've been exposed to in the last decade. The associated geometric product has helped me better understand concepts (like "handedness") that troubled me during undergrad engineering.
I had never even heard of rotors! Thanks for this. I watched that video. The video doesn't really explain how it extends to higher dimensions tho, that I could discern.
I wonder how/if any of this can be applied to LLMs 'Semantic Space'. As you might know, Vector Databases are used a lot (especially with RAG - Retrieval Augmented Generation) mainly for Cosine Similarity, but there is a 'directionality' in Semantic Space, and so in some sense we can treat this space as if it's real geometry. I know a TON of research is done in this space, especially around what they call 'Mechanistic Interpretability' of LLMs.
> The video doesn't really explain how it extends to higher dimensions tho, that I could discern.
The neat thing is that it "extends" automatically. The math is exactly the same. You literally just apply the same fundamental rules with an additional basis vector and it all just works.
MacDonald's book [1] proves this more formally. Another neat thing is there are two ways to prove it. The first is the geometric two-reflections-is-a-rotation trick given in the linked article. The second is straightforward algebraic manipulation of terms via properties of the geometric product. It's in the book and I can try to regurgitate it here if there's interest; I personally found this formulation easier to follow.
If you really want your mind blown, look into the GA formulation of Maxwell's laws and the associated extension to the spacetime (4d) algebra, which actually makes them simpler. That's derived in MacDonald's book on "Geometric Calculus" [2]. There's all kinds of other cool ideas in that book like a GA formulation of the fundamental law of calculus from which you can derive a lot of the "lesser" theorems like Green's law.
Take all of this with a grain of salt. I'm merely an enthusiast and fan, not an expert. And GA unfortunately has (from what I can tell) some standardization and nomenclature issues (e.g. disagreement over the true "dot product" among various similar but technically distinct formulations)
> I wonder how/if any of this can be applied to LLMs 'Semantic Space'.
Yeah, an interesting point. Geometric and linear algebra are two sides of the same coin; there's a reason why MacDonald's first book is called _Linear and_ Geometric Algebra. In that sense, Geometric Algebra is another way of looking at common Linear Algebra concepts where algebraic operations often have a sensible geometric meaning.
Interesting ideas there thanks. I do know about that Maxwell derivation that involves Minkowski space, Lorentz transform consistency, etc, although I haven't fully memorized how it works, so that I can conjure up how it works from memory. I don't really think in equations, I think in visualizations, so I know a lot more than I can prove with math. You're right it's mind-blowing stuff for people like us that are interested in it.
Spoiler alert: rotors are mechanically identical to quaternions, while being easier to understand. If you understand rotors, you understand quaternions. You can fit the laws you need to understand rotors on a business card.
Plus, rotors abstract to higher and lower (well, there's only one plane and its two respective orientations in 2d, but still) dimensions.
Complex numbers as planes (bivectors in GA parlance) has been the most mind-opening mathematical concept I've been exposed to in the last decade. The associated geometric product has helped me better understand concepts (like "handedness") that troubled me during undergrad engineering.
1. https://marctenbosch.com/quaternions/