Warsaw U, OpenAI and Google’s Hourglass Hierarchical Transformer Model Outperforms Transformer Baselines | Synced

A team from the University of Warsaw, OpenAI and Google Research proposes Hourglass, a hierarchical transformer language model that operates on shortened sequences to alleviate transformers’ huge c...