Meta AI develops compact language model for mobile devices [VentureBeat]

Key innovations in Meta’s MobileLLM include prioritizing model depth over width, implementing embedding sharing and grouped-query attention and utilizing a novel immediate block-wise weight-sharing technique.

AI,ai models,AI, ML and Deep Learning,API,category-/Science/Computer Science,GPT-4,language models,large language models,LLaMA 2,LLMs,Meta,meta ai,Meta AI Research (FAIR),MobileLLM,PyTorch,small language models (SLMs)

PyTorch