Unveiling In-Context Learning: Provable Training Dynamics and Feature Learning in Transformers
In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its theoretical foundations remain elusive due to the complexity of transformer architectures.