“Data-driven.” It’s a buzzword used in business contexts ranging from company mission statements to applicant resumes. WillowTree Staff Analytics Architect Jenny Kreizman prefers the term “data-informed”.
“The phrase ‘data-driven’ can be alienating because it’s easy to be misled by data. I prefer ‘data-informed’ because it reflects the cross-disciplinary collaboration that’s essential to building great products,” Jenny advises. “WillowTree’s analytics and optimization practice brings a level of nuance that you might not see from a typical, in-house analytics team. We’re powering the ideation phase and informing what our designers and developers ultimately build. Many of us have a background in strategy and are adept at translating complex data to higher-level strategic business conversations.”
Whereas traditional analytics agencies or in-house teams might rush to incorporate analytics too late in the product development lifecycle, data practitioners at WillowTree have a seat at the table during preliminary conversations with engineering, product, and leadership teams. It’s a primary differentiator of our data discipline that drives measurable and meaningful ROI for our clients.
Before those strategic conversations take place, Jenny often hears clients refer to vanity metrics such as “average session duration” or “scroll depth” to draw generalizations about user sentiment and behavior. She’s encountered assumptions like, “Because time-on-site is so high, users must be engaged with our content.” Or, “If people aren’t scrolling to the bottom of a page, there must be too much content, or users must have scroll fatigue.” But neither conclusion is necessarily accurate, as Jenny points out.
“Our analytics team implements custom event tracking to create a detailed understanding of the ‘what’ of user behavior. We recognize it’s critical to partner with colleagues in research, strategy, and design to understand and hypothesize about the ‘why,’” Jenny says.
Jenny and her cross-functional WillowTree colleagues transform these hypotheses into outcomes-based measurement strategies that are personalized to client needs. They ensure that clients can define and measure the metrics that actually matter in order to drive tangible business results. And it’s critical that this process takes place during the early phases of product development.
"Having unequivocal analytics in place to evaluate your product is a great equalizer. This data can neutralize what may be unfounded opinions, and foster a more creative, democratic environment for conversations.” — Jenny Kreizman, staff analytics architect
Senior Director of Analytics and Optimization Jeremy Stern recalls one particularly memorable workshop with a client that was having trouble measuring ROI after a website redesign.
“We all got in a room together and broke down their 10-step process for users to set up an account. I asked them, ‘Where does this flow start?’ And the three stakeholders put sticky notes on three different screens. Then I asked, ‘Where does this flow end?’ And they put sticky notes on three different screens again.”
What did this activity reveal for Jeremy and the client? Each in-house team had a different definition of “conversion,” which led to different interpretations of the product’s success. Some identified conversion as the quickest possible check-out process, while others saw conversion as successful upsells.
If two different teams have distinct goals, it can cause confusion — especially if you don't realize this misalignment until the end. Contradictory cross-disciplinary feedback inevitably obscures measurement and lengthens the optimization process.
“That's one of the big benefits of our initial outcomes workshops,” Jeremy explains. “We get people in a room together saying, ‘Here is what is most important for me in this app.’ That way, at the start of the project, we can hear if different teams have openly contradictory outcomes that will impact how they assess ROI and prioritize product changes over time.”
“That’s what makes WillowTree’s analytics consulting role unique from an in-house analyst,” Jenny says. “Having data at your fingertips is one thing, it’s another to interpret what you have in a meaningful and productive way.”
Part of Jenny’s role as an analytics architect involves thoughtful negotiations and problem-solving with software engineers. And, it’s done in real-time, rather than after the product is built.
She partnered with a WillowTree client in the wealth management space to implement an analytics solution while the product was still being piloted. With early access to data regarding prospects’ interactions with the client’s platform, the project team could make several optimizations that ultimately increased conversion rates when the product became available nationwide.
“When we observed that a high percentage of users dropped off between key steps in the product journey, for example, we combined those multiple steps into one. All of our features are rolled out as A/B tests, so we were able to verify that this optimization led to an uptick in leads and an additional return for the client,” Jenny says.
“So as we’re constructing the product, we’re weighing in on how its structure applies to analytics,” Jenny continues. “You have the same people who understand the tracking taxonomy who are also pulling the data and informing future analysis. An analytics practitioner is attached to the whole lifecycle of a product.”
She pictures these processes as a “T” shape. Product development, product management, and engineering represent one arm of the T; on the other arm are research, strategy, and product design. Analytics and optimization, the area of deep specialization, unite all of these disciplines. It’s a growing practice that facilitates collaboration and value for WillowTree clients and teams alike.
“In addition to the measurable ROI we see through optimization, for example, our work offers clients what I’d call a ‘softer’ return on investment — problem-solving. It gives them a shared language to use when conflict arises.” — Jeremy Stern, senior director of analytics and optimization
