Testing potential solutions for novice investors

Note: Some of the information on this project is considered confidential. I've redacted my employer's name, name of specific products, and generalized most data and findings.

Context

On October 2024, the product area I was aligned with was tasked with improving the experience for novice investors. After I conducted discovery research, the team got to planning. They conducted a workshop, which helped them identify areas of work, including new solutions. However, they needed user research to determine if the new solutions made sense from a customer perspective.

What I did

Ground ideas in user needs

My work for this project started before I identified the need for a new study. After I conducted discovery research for this team, I actively participated in product and design planning. I brought up research where relevant. During the ideation workshop, I helped base the design problem statement on real user needs.

Plan Research

The team decided on 6 potential solutions. When overwhelm and the urge to pick one emerged, I urged them not to create visuals yet.

I asked the team to write succinct descriptions of the ideas, so we could further evolve them with customer feedback and eliminate the ideas that did not resonate.

I also emphasized that we were not ranking ideas, as they were not fully developed. We were simply identifying which ideas (if any) had potential to solve real customer problems.

Before crafting a detailed study plan, I shared an outline of what we would do via Miro.

This helped create alignment about the purpose of the research, its benefits and limitations, and how we might utilize the findings.

In the Miro I suggested unmoderated testing, so we could get the results with the necessary speed and ensure questions were asked consistently for each idea.

Recruit participants

I conducted research with 16 novice investors via UserTesting. I used the UserTesting panel because customer typically took longer, and at this stage we weren't necessarily interested in existing customers, we just wanted to know if this approach worked for research

Conduct Research & Analysis

To reduce cognitive load on participants, concepts were divided into 2 groups of 3.

This meant that 8 participants saw 3 of the 6 ideas. This approach worked well because were were not ranking ideas. I paired with my colleague Rui Wang to build studies with identical screener questions and tasks. Later we each analyzed one group using the same spreadsheet.

Each concept asked participants to talk through their understanding of the idea.

This later helped designers further develop the solution, by helping them marry their professional understanding of design capabilities and customer expectations.

Participants also rated their perceived usefulness of each concept. These were used ratings as a qualitative measure.

We tracked individual ratings on a spreadsheet, but did not add them in the same way you would for a quantitative study. Rather we focused on why they thought the concept was useful or why they thought it was not useful. This helped me thematically determine if the idea made sense to pursue from a customer perspective.

Delivered Insights

I shared findings by providing:

• A summary of reactions to concepts divided into four areas: success, challenges, and opportunities.
• Reels high-lighting key customer responses.
• A table of recommendations, classifying concepts in 3 categories:
  • Move Forward: We can carry this idea to the next step. Participants agree that the idea provides value with little or no concerns.
  • Revise: Changes are needed before moving forward. Participants agree on what could improve the idea.
  • Hold: This idea does not appear to meet a need for this audience. Participants did not find enough value in this idea or were completely split on its usefulness and/or likelihood to adopt.

Impact

• Concepts that did not resonate were deprioritized, including one which the team was already scoping.
• Concepts that resonated or somewhat resonated were further evaluated from a feasibility and viability stand point. One of these was revised, usability tested, and put to development.