Tyro Growth Funding Pilot Study

My report for this project is confidential and findings are still being used today. This research was used to refine the Growth Funding product and process before general release.

Because I can’t share details of the report, I’ve outlined my process and results the below, and shared examples using information that has been changed or is publicly available.

Process

Meeting with stakeholders to review business model and hypotheses

I met with Product Management, Credit, Sales, and Marketing, to generate hypotheses regarding the product. This was done in a workshop to create business model canvases for the product and identify which parts of the model were untested.We then had a session where everyone contributed hypotheses using test cards and discussed them. We made the cards visible, and allowed everyone, including engineers, to contribute their ideas.

Sample test card using a hypothetical situation:

Planning pilot experiments

Next, I organized hypotheses and planned experiments that could be completed as part of pilot. I gathered all hypotheses and categorized them by themes, such as “price acceptance”. I determined how to test and monitor each hypothesis during pilot, and discussed with credit, product, sales, and marketing about who would be responsible for monitoring each, because not all hypotheses would be reported on by me as Customer Researcher. For example, some credit hypotheses needed to be reported on by credit, while tests that involved feedback in sales calls needed to be reported on by sales. I recorded the hypotheses, tests, measurement, success criteria, and who was responsible into a table visible to everyone on Confluence.

I met with the Product Managers and sales to prioritize what we needed to learn before general release. These were hypotheses that would have large effects on our product offering or credit policy, or information we needed to report to the Australian Prudential Regulation Authority.

Once pilot launched, I began conducting pilot research and monitoring experiments Some of my research involved interviewing new customers about their experience accepting and using the loan. For these customers, I interviewed them as soon as possible after accepting the loan, and then interviewed them after they had been using a product for a month. I updated results on my page. I also monitored relevant customer data and checked in other groups who were reporting information each week to make sure everything was up to date. Keep in mind that each research method was tailored to the specific hypothesis we were testing.

Sharing findings

I reported customer research results was part of a larger pilot report for the board and government bodies. At the top, I outlined each hypothesis and whether it was Valid, Invalid, or Unknown (not enough data to present a finding) Below that, for each hypothesis I summarized: The test method used The resulting data collected The conclusion, which included whether it was Valid, Invalid, or unknown ,a short summary of any helpful context and the recommendation based on the finding. Here is an example from the report using information that has been made public

Impacts

  • Drove product decisions for general release launch and beyond, for example changing the minimum daily repayment percentage offered, and other functionality
  • Gave a clear picture of what would be accepted and loved by customers, and what needed to change to make the product successful, greatly reducing the risk of the launch
  • Included recommendations of how to make our other product offerings more attractive to loan customers, such removing the requirement to use Xero from our bank account
  • Demonstrated the evidence and rationale for these recommendations to the management team and board