Participant selection for testing is crucial
The importance of having relevant participants for usability tests cannot be understated. Unless participants represent potential or actual users, their feedback is essentially invalid and the testing is wasted.
We experienced this after doing multiple rounds of testing on usertesting.com, where we quickly realized that even though we set up very specific screeners, we weren’t getting relevant participants. With more niche user groups, it’s worthwhile to create a group you can trust and depend on, and offer an incentive to if necessary – something we are in the process of doing for this and future projects.
Prototyping has its limits
For more complex interactions, rudimentary prototyping only goes so far to convey what the experience is like. With the sticky breadcrumb design, for instance, the prototyping experience and scrolling interactions for mobile in particular just couldn’t be replicated effectively on a desktop.
In these cases, what made the most sense was to get enough feedback from the basic prototype to support the design and then have development implement the design while making a note to do more robust testing post-launch.