On-device AI is the latest offering for bringing powerful models to users, while protecting sensitive data and improving latency.
On-device cannot completely replace and replicate the work you do on the cloud. After all, servers are incredibly powerful and able to hold large, complex models and still deliver results fast. However, on-device can unlock some amazing possibilities alongside any existing server-side setup.
- Privacy and security: On-device AI lets you work with data locally, which greatly impacts your ability to work with sensitive data and keep it safe and private. You can offer AI features to users with end-to-end encryption.
- Greater availability: On-device AI can help achieve greater availability to your users. Your users' devices can shoulder some of the processing load in exchange for more access to AI features. If your product offers a premium service, you could consider a free tier with on-device AI features to help your customers get a glimpse of what the premium service provides.
On-device isn't always the right choice, so before we move further, we want to remind you of some best practices:
- Design your features with graceful fallbacks and run benchmarks on your target devices. Not every device will be able to act as an AI powerhouse.
- Build for specific use cases. On-device AI works best for specific use cases. The models are inherently smaller than what's typically found in server-side AI. Break your process down into targeted steps and make use of pre- and post-processing, so that smaller models can still deliver the best possible response.
- Be strategic about download requirements. AI models can be large, which could lead to a large use of mobile data and device storage. Make sure you're building a useful feature for your users and that you have a responsible serving and caching strategy.
Share your feedback
We want to hear about what use cases most interest you and your feedback on our approach. You can share feedback and request to join the early preview program to test with local prototypes.
Your contribution can help us make AI a powerful, yet practical, tool for everyone.