I worked on RNN and CNN models for text a few years back and I really liked the idea of character or byte-based models but it seems that these fasttext-like token encodings won.