> MIA Meeting: Low dimensional embeddings of words and documents (and how they might apply to single-cell data)
Leland McInnes
Tutte Institute for Mathematics and Computing
Over the last decade the field of Natural Language Processing (NLP) has been overtaken by neural networks and deep learning. The latest models and algorithms, from word2vec to Google’s Universal Sentence Encoder and BERT, can perform seemingly magical feats and provide powerful tools to understand and analyze text documents.
This talk will seek to pick apart word and document embedding techniques from NLP, removing the neural networks and instead working with linear algebra and dimension reduction on large sparse matrices of counts that is very similar to a large amount of single-cell data. Such an approach can achieve results on par with neural network approaches, but are both simpler to understand the inner workings of, and are generalizable to much more diverse domains than NLP, including, hopefully, single-cell research. In particular all we really need to apply these kinds of techniques are large sparse matrices of counts and some notion of locality or cooccurrence of the feature columns of that matrix. The goal of this talk is to hopefully open up some of the ideas that have been so effective in NLP and make them usable and accessible to a wider audience working with more diverse types of data, and initiate a conversation about what is required to make such techniques useful for the single-cell community.
I switched from pinboard to diigo around 2 years ago. I love the "Annotate Page" functionality. I use it on all my bookmarks to quickly save the parts that I found useful or interesting. It is particularly useful when bookmarking HN/Reddit discussions, newsletters / link aggregations like High Scalability Newsletter. The hightlights are available when I open the page later so I know what I liked. Furthermore, the highlights show up when browsing the bookmarks on diigo, which makes it really great for research.
Peter Norvig also has two implementations of Prolog in Paradigms of AI Programming (first one is interpreted, second is complied) in Common Lisp. The book is highly recommended because Norvig's code is very elegant and his exposition is wonderful.
Allegro Prolog [1] is based on Norvig's implementation, although I'm sure they've done a fair bit of optimizing.
You should totally do it... it's fantastic. Norvig makes it quite clear in the beginning that he expects you to know atleast some lisp beforehand (the "intro" chapter is a bit sparse), so you might want to brush you on your CL skills before you dive in.
> MIA Meeting: Low dimensional embeddings of words and documents (and how they might apply to single-cell data)
Leland McInnes Tutte Institute for Mathematics and Computing
Over the last decade the field of Natural Language Processing (NLP) has been overtaken by neural networks and deep learning. The latest models and algorithms, from word2vec to Google’s Universal Sentence Encoder and BERT, can perform seemingly magical feats and provide powerful tools to understand and analyze text documents. This talk will seek to pick apart word and document embedding techniques from NLP, removing the neural networks and instead working with linear algebra and dimension reduction on large sparse matrices of counts that is very similar to a large amount of single-cell data. Such an approach can achieve results on par with neural network approaches, but are both simpler to understand the inner workings of, and are generalizable to much more diverse domains than NLP, including, hopefully, single-cell research. In particular all we really need to apply these kinds of techniques are large sparse matrices of counts and some notion of locality or cooccurrence of the feature columns of that matrix. The goal of this talk is to hopefully open up some of the ideas that have been so effective in NLP and make them usable and accessible to a wider audience working with more diverse types of data, and initiate a conversation about what is required to make such techniques useful for the single-cell community.