This review examines RoFormer and the rotary position embedding technique used by modern large language models. The repository contains the reading notes and presentation materials from my Transformer course.
← Mini Projects
Mini project~1 min read
RoFormer Review
A technical review of RoFormer and rotary position embedding, prepared for a Transformer course paper presentation.