Critical Elements of a Data Driven Product Organization in E-Commerce

Ankit Mangal, Director, Advanced Analytics and Insights, Wayfair

As more consumers are relying on e-commerce platforms for purchasing their groceries, home decor, electronics and more, matured platforms are collecting data to understand customer behavior and leverage it to generate insights that help improve customer experience leading to higher purchase rate, long term customer loyalty and eventually higher market share. Millions of active and engaged customers browsing hundreds of products during their purchase cycle, generates petabytes of data capturing what customers are looking for, how they are finding it and more importantly what's preventing them from making the purchase at that moment; it could range from product selection, pricing, what other customers are saying about the quality of product in the reviews, how quickly it will it be delivered, is the website intuitive to navigate and so on.

Data and insights backed decisions could really be a strategic advantage if envisioned and executed right. Most e-commerce companies in their early years, including Wayfair, start with a hybrid team with a group of business intelligence experts who do a bit of everything, ranging from managing infrastructure/ ETLs, building appropriate data models, developing dashboards, setting up and analyzing experiments and generating insights to drive business strategy. However as they scale, cost of suboptimal execution becomes more expensive as constant tool migrations, unstable ETLs, poorly setup experiments could really slow down a rapidly growing business. Late 2017, Wayfair started setting up a vision of evolving our analytics with more defined verticals within Product Analytics that can support a business eyeing 100Bn revenue in the next few years.

First, the foundation of any product analytics organization is the data engineering team that can define what customer behavior to capture in the data, how to model it for efficient storage and deliver insights at speed of thought in a stable and reliable environment. This needs careful selection of tag management tools, cloud based storage platforms and highly skilled engineers that can balance the speed of agile environments with right long term decisions.

Second, an analytics team, a hybrid of technical and business analysts that can take on 3 major responsibilities.

1. Defining Relevant-Actionable- Measurable (RAM) KPIs to assess business performance, and build an intuitive self service reporting suite. Understand relationship between metrics in Mutually-Exclusive- Collectively-Exhaustive (MECE) tree format to investigate declines or upticks efficiently.

2. `Develop an experimentation strategy to understand how customer behavior changes with different product designs and improved journeys. This includes not only clearly defining hypotheses with focus on learning but also identifying the right metrics to measure the impact and right methodology for experiment. Analytics team should have multiple methods in their repertoire to optimize learning under real world challenges i.e. if low traffic, try Bayesian vs frequentist; move fast by rolling out variations that are good long term solutions and leverage Synthetic Controls to measure impact post-hoc, leverage Multi-Arm Bandits for running multiple variations without impacting customers significantly or Multivariate testing to understand interaction between variations.

3. Utilize data science methods for customer segmentation, feature modeling to identify levers to optimize, topic model customer feedback to identify better solutions etc to proactively drive product strategy.

Third, having a talented data science team with a mix of data scientists and machine learning engineers could really provide competitive advantage to any retail tech company. Data Scientists should take on some of the most complex business problems to develop most sophisticated and real time solutions with massive impact on customer and business. This could include Machine Learning recommendation algorithms that optimize customer journey in real time, operations research projects that optimize driver routes to shorten delivery times while controlling for cost, AI chatbots and more.

Fourth, the analytics products team can help procure the right tools to support the data science and analytics team and help establish best practices across the organization.

As we are heading towards the specialized role based org-design, it's evident that an agile environment is extremely crucial where each business has a squad designed to focus on specific customer problems with representatives from each vertical embedded in the business unit to ensure balance in speedy execution and domain expertise.

In addition to right vision and setup for analytics organization, advanced tool stack and streamlined processes can help scale retail tech companies while continuing to solve most important customer problems with data driven solutions.