Delivering the Best Version 1

Posted by Matt Sisto & Gracie Clemens

Here at Hi Mum! Said Dad, we hold a monthly Lunch & Learn where team members present on something the team can learn from and apply to our work. Last month Matt Sisto, Head of Delivery, outlined the purpose, outcomes and process of user testing prototypes. Here are some of the key takeaways to consider.

Objective Setting / Test Requirements

To define the anticipated outcomes you need to ask a simple question, what are you trying to achieve with user testing? Whether that’s validating user experience with gauging utility and usefulness, qualitative and quantitative results, or understanding sentiment towards the product, objective setting provides a benchmark against which results can be measured. Often, the project team are so close to the product that we all have to hypothesis about what the results will be, but many times we’re surprised.

Test Scenarios

The purpose of creating test scenarios is to narrow the focus of testing. In most products there are hygiene functions, for example logging in, registration, FAQs, etc. These elements often follow the norm, so to test something that’s already widely accepted and used in the market is nonsensical. Therefore we focus on the features that:

a) Are unique to the product and experience.

b) Will make a big difference in whether they’ll use the product again.

By narrowing the search, we also make the production of the prototype easier and less time consuming; meaning we do not over-invest in the prototype before testing.

Something else to consider when creating the test scenarios is to ensure they’re relatively open, testing core functionality without leading the user. This is imperative to get the most value out of the testing. If you, as the facilitator of the session, lead the user step-by-step through a flow, the insight from the consumer would be minimal. A scenario example might be, ‘you’ve heard about a product and now you want to buy it’.

User Profile and Recruitment

We build products and each normally has a target audience, a demographic of users that the product will resonate most with. However, you want to test the product with not just the people that will be die-hard advocates, but also those that typically wouldn’t use the product. This will give you a wide and varied range of feedback. So take a mix of customers and potential customers, as wide as your budget will allow. The more feedback we get, the more we’ll learn and the better the final product will be.

Agenda and Session Plan

As always, it’s so important to have structure to the facilitated sessions. As the facilitator, I need it to be clear on how the day and each session will run. This is equally as important for the participants because it allows them to follow the same scenarios and process every time. This ensures that the data we collect is comparable and mitigates the risk of bias. Taking the learning’s on one thing from the majority will give you some great quantifiable data, which then makes it easier to decide what you change, why and how.


Like any good Lunch & Learn, at the end of Matt’s presentation he gave the team the chance to ask questions:

What was the biggest challenge in your most previous user-testing project?

Matt’s response: “There are various platforms that you can use to help support your prototyping and user testing process. Some of these platforms have limitations and we have to be cognisant of this during the process to ensure that it doesn’t impact the data and feedback we get.“

What was the biggest success?

Matt’s response: “We (Client and Agency) were able to validate our overall strategy and approach for the product; more importantly, we discovered some quick wins and improvements that will improve the first release of the product. These could have taken a lot longer to fix later down the line.”

To sum it all up, we‘ve developed a process that ensures the first versions of products we launch are optimum viable products. However, first versions are the inception, they’re the product of the first phase of design and development process. Just compare iOS1 to iOS9!

Originally published at