Google unveils more Gemini and Android functionality today at Google I/O
Today in Mountain View during the Google I/O Keynote, Sameer Samat from Google presented the next step in the evolution of bringing Google's AI tools directly to users through deeper integrations within Android. Through this deeper integration at the operating system level, Google promises a world where they've reimagined how we interact with our devices.
Of all the developments in 2024 so far, Circle to Search appears to be one of the biggest hits (for everyone but me it seems, since it still isn't available on my Pixel 8 Pro, but that's another topic.) Ever since the rollout earlier this year in partnership with Samsung, Circle to Search has been heralded as the new interface when using your phone and unlocking the power of Google Search and is available on more than 100 million devices and Google claims to be on track to double that by the end of 2024.
Today it was announced that Circle to Search now can help students with their homework, much to their teacher's chagrin. When working on a math or physics problem on their phone or tablet, students can circle the problem they're stuck on and it will reveal step-by-step instruction on how to solve it. This is available starting today, just in time for finals! As the year goes, they'll be adding support for more complex problems involving formulas, diagrams, graphs and more.
I have to admit this is pretty cool and something I wish I had back when I was in school. That said, I have to imagine schools and teachers around the world are cringing now about how this will be used (and abused) but students in an academic world already compromised by AI term papers and the like.
The emergence of Gemini as the Assistant replacement and dominant force in Android continued as Google revealed the deeper integrations to Android to empower Gemini to have and even better understanding of what's on your screen, what app you're using and the contexts beneath them.
During the keynote, they showed how Gemini will become even more ubiquitous in assisting daily tasks on Android. By enabling Gemini's overlay above apps there's a wide array of things you can do now. Need that perfect image for your texts or email? Use Gemini to generate an image that easily copies and pastes into your message. Watching a video and want more info? Just use Gemini so you can get all the answers you want. Have a huge PDF that you don't have time to read or need to find info quickly? Use Gemini's new "Ask This PDF" function to dig into that document and get the answers you need. These updates will be rolling out over the next few months.
Google flaunted the fact that Android is the first mobile OS with a built-in, on-device foundation model with Gemini Nano. Coming later this year on Pixel devices (to be named later), they'll continue to develop this to ensure the experience on your device remains private to only you and at the same time, unlock new functionalities with Gemini Nano with Multimodality. What does that mean? Well with it, you'll be able to use your device to process text but also understand context in images, sounds and spoken language.
Gemini Nano with Multimodality will bring new features like TalkBack, which wiill provide users with vision impairments the ability to get richer and clearer descriptions of images very quickly thanks to being on-device. Additionally, Gemini Nano with Multimodality will help fight back against scams via phone calls. Google reveal they're currently testing functionality to provide real-time alerts during a call if it believes the call is a scam. Pretty wild when you think about it and see the rising number of scams to get people's password and bank account information. This feature could help many unsuspecting users who aren't as tech savvy or risk averse as power users. Google is going to be revealing more details on this later this year.
These announcements show how committed to Gemini and AI and Android Google is, with direct user impact. In addition to these, Google promises even more functionality for Developers and as Google I/O continues tomorrow, we'll be hearing more about Android 15 and other updates to the Google Ecosystem
