Connecting the home
Strategy, UX & UI Design and development
FrieslandCampina, the Dutch dairy giant, engaged us to demonstrate how they could potentially make significant commercial gains and drive a cutting edge customer experience via nascent technologies that are increasingly being used to manage the home such as voice assistants, smart fridges as well as exploring new routes to online shopping for the FMCG category.
They gave us a smart fridge and 9 days to come up with working examples of how they can do something innovative with commercial impact in the space.
While connected appliances have featured in the news and recently at CES, there are few truly meaningful brand executions. With this in mind, we set out to create a product that would solve real challenges and fit well with existing consumer needs and behaviour.
Our first prototype was a simple one - an app that would show you what you had in your fridge by leveraging the fridge’s in-built cameras. A user could confirm the ingredients and then search for recipes based on these inputs. As a first-pass solution, this was a step in the right direction but we obviously wanted to push the envelope much further by looking at how we could remove user friction and also introduce voice UI into the equation.
The natural next step was to build an Alexa skill that would allow us to link the Amazon Echo device to the smart fridge. Our aim was to create a naturalistic interaction between users and Alexa, but to do this we had to overcome several hurdles.
Firstly, we had to get a list of ingredients from the fridge without requiring user input. We used the fridge's built in cameras again but this time, we leveraged image recognition in order to translate the image data into a recognisable list of items stored in a database.
Now, in everyday scenarios, fridges are obviously opened and closed multiple times throughout the day. Everytime this happens, this could mean that the fridge’s inventory has changed. So, every time the fridge door was closed, we prompted the fridge’s cameras to take a picture and check for changes before updating our list accordingly. We then integrated with a 3rd party API to power the recipe recommendations to completely remove any user friction.
Now, when we asked, “Alexa, what’s in my fridge?”, our prototype springs into life and reads out the exact contents of your fridge, following up with a “Would you like a recipe suggestion?”. Users can then ask for a recipe based on a specific items in the fridge, or in a moment of spontaneity, asked to be surprised. Alexa then proceeds to read out the full list of ingredients needed for the selected recipe, saving the user both time and effort.
The next challenge was to see if we could introduce a potential commercial benefit via e-commerce - i.e. what if the user was missing a particular ingredient for a recipe? The commercial benefit in this case would be the ability to proactively suggest a certain brand, thereby reducing cognitive demand via a default state, and therefore drive seamless conversion. We were able to demonstrate that this was possible by successfully integrating with a user’s retail basket of choice. This means they could use their voice to add a particular product to basket - ready for checkout with their next online shop.
The moral of the story?
- Proved the viability and potential commercial impact for Frieslandcampina in the connected home.
- Successfully integrated the popular Siemens smart fridge (RRP circa £800) with Alexa to create a fully functional demo in 9 days.
- The demo was a a central piece at an annual innovation day intended to get the rest of the business planning for future opportunities.