In recent research papers, Apple has unveiled groundbreaking developments in its generative artificial intelligence (AI), showcasing a significant leap forward in on-device AI capabilities. These advancements include running Large Language Models (LLMs) on device memory and the introduction of HUGS (Human Gaussian Splats) technology for generating animated 3D avatars from short monocular videos.
On-Device LLMs: Apple’s researchers are tackling the challenge of enabling on-device AI capabilities without compromising speed or performance. In a paper titled “LLM in a flash: Efficient Large Language Model Inference with Limited Memory,” they detail an innovative approach that utilizes flash memory to run memory-intensive LLMs efficiently.
Key Techniques:
- Windowing: The recycling technique reduces data transfer between flash memory and RAM by reusing processed data, saving time and energy.
- Row-Column Bundling: This reading technique enhances data retrieval speed from flash memory by reading larger data chunks simultaneously, resulting in significant speedups on CPUs and GPUs.
Implications:
This breakthrough has significant implications for Apple users, potentially allowing future iPhones to run advanced AI models on-device, enhancing Siri’s capabilities, and enabling new AI-driven features in photography, augmented reality, and beyond.
HUGS Technology:
In another paper, Apple introduces HUGS, a novel method for generating animated 3D avatars from short monocular videos. HUGS leverages 3D Gaussian splatting and a statistical body shape model, SMPL, to create animatable human avatars with realistic details like clothing and hair.
Novel Neural Deformation Module:
HUGS incorporates a novel neural deformation module using linear blend skinning to realistically animate Gaussians, avoiding artifacts while reposing avatars. The speed and efficiency of HUGS make it up to 100 times faster in training and rendering compared to previous methods.
Generative AI Roadmap:
Apple’s generative AI roadmap includes integrating these advancements into Siri, aiming for a smarter assistant capable of handling complex queries. The company is also developing its own generative AI model, “Ajax,” with 200 billion parameters, indicating a high level of complexity for language understanding and generation.
Future Integration:
According to reports, Apple plans to roll out generative AI features on the iPhone and iPad with iOS 18 in late 2024. This strategic move suggests a deeper integration of AI into Apple’s ecosystem, promising users a new level of interactivity and engagement with their devices.
