I presented on Backwards Engineering – planning Machine Learning (ML) deployment in reverse. Data shows that a traditional data-first approach to analytics is not generating much value for companies and I urged the audience instead to adopt a decisions-first mindset. The last mile – getting ML models embedded into production systems – is critically important for analytic value and yet it is hard and often neglected. To succeed the data shows that you need to:
- Make the project about the decision, not the analytic
- Integrate decisions, not scores
- Mix and match decision technology
You have to be sure your analytic will be actionable, before you build it.
There were lots of other great presentations in the business track that I moderated. Here are a few of my takeaways:
- Theresa Kushner, formerly Senior Vice President, Performance Analytics Group, Dell EMC presented on How to Overcome Barriers to Predictive Deployments. She had great stories showing how important it is to involve users from the beginning, investing in building trust and working well with IT among other things. I loved her three T’s – Trust, Teach, Technology. She also emphasized beginning at the edge – at the decision – and focusing on continuous improvement – two things we have likewise found are critical. Her summary slide was great:
- Tom Warden, Senior Vice President, Chief Data and Analytics Officer, EMPLOYERS talked about how to Engage Everyone Who Will Touch the Analytic Model in the Development Process. In particular, Tom pointed out that it is not enough for data scientists to engage senior management – they must also engage front line employees who “know how we make money”. He also urged them to “listen, then speak” and to practice humility! Great advice.
- Anne G. Robinson, Chief Strategy Officer, Kinaxis discussed Searching for Unicorns: Attracting and Retaining Analytics Talent. She emphasized the importance to data science teams of business translation and of partnering with domain experts. It’s often hard to get domain expertise and data science skills in the same person – hence the need for translation. We like to use decision modeling on analytic projects precisely for this reason – the decision model represents a visual map of the domain expertise that the analytics team can use to frame their work.
- Lily Quinto-Banton, Associate Director of Data Science, Humana Military presented on How a Single Question and a Healthy Dose of Skepticism Inspired Us to Use Analytics to Rethink Strategy. Her presentation really showed the importance of persistence, experimentation and lateral thinking in developing an analytic solution.
- Robert M. Horrobin, AVP of Advanced Analytics and Planning, Retirement Solutions Division, Pacific Life presented about Doubling Down-How Pacific Life Accelerated Analytics Adoption. I was particularly struck by their investment in a community of practice:
I also thought his pitfall list was great – he talked about the importance of solving the right problems, of integrating with the business and of thinking about implementation up front among other things. Plus, he had a great shout-out to CRISP-DM, a framework we really like too.
- Leslie Barrett, Senior Software Engineer, Bloomberg LP discussed Machine Learning Evaluation from Start to Finish. She had lots of great tips like using simulation before deployment to give context to your ML model and making sure you understand the value/cost of false positives, false negatives, true positives and true negatives before you pick the “best” model! She also had a great concept about an indifference curve, pointing out that many ML teams keep investing in improving the quality of their model long past the point where improved quality has any utility to the business. A great technical view of critical business issues!
- Bill Franks, Chief Analytics Officer, International Institute for Analytics talked about The Ethics of Analytics. He talked about considering if something is legal, ethical and acceptable to your customers. It must be all three to be viable. He also emphasized the need to apply this framework when defining problems, looking at data, picking models and using them – throughout the lifecycle. Many of our clients worry about this and we have found that showing how an analytic or ML model will be used by developing a decision model goes a long way to resolving ethics concerns or at least enables a clear and coherent discussion of any potential issues.
- Eric J. Felsberg, Principal, Long Island Office, Jackson Lewis P.C. gave a talk that sounded like the intro to a funny story: A Lawyer, an Employer and a Data Scientist Walk into a …, in which he pointed out that well established frameworks for considering if your approach to HR issues is legal – considering bias as well as disparate impact – can and should be applied to ML and analytics too.
- Nathan Susanj, Vice President, Data Science Manager, Wells Fargo discussed why the old model of Analytics Consulting is Dead. He had a great story about the impact electricity had on factories and how it did not replace steam until factories could be re-designed around electricity. He argues, and I would agree, that data science is to BI as electricity was to steam – you can’t simply plug data science into your existing approach to BI and analytics, it needs a new approach – one focused on developing insight that can be used by machines and deploying that insight in automated decisioning applications designed from the beginning to automate a solution to a business goal. DecisionsFirst indeed!
- Finally, Shingai Manjengwa, CEO, Fireside Analytics Inc. wrapped things up by showing how to Build a Pipeline of Data Science Talent with Existing Talent. She has developed some great self-paced data science classes and urged companies to find those in their organization that are teaching themselves analytics and data science skills, provide broad training and literacy opportunities that are highly contextual (rooted in the company’s business) and to develop cross-functional teams. Great advice! You don’t need to hire unicorns; you can develop the skills and roles that a strong data science practice needs.
We also held a lunch and learn with our friends at Datarobot: How to Operationalize Your Analytics to Maximize the Business value of ML. Using DecisionsFirst™ Modeler, our decision modeling tool, Datarobot, a business rules management system (IBM’s ODM), and a dashboard we showed how you can focus on the right problem, automate developing ML insight and get this insight across the last mile and into a business learning environment.
Predictive Analytics World 2019 was a great event – we met lots of great people and had many fascinating conversations about digital decisioning, decision modeling, operationalizing analytics and maximizing the ROI of ML investments.
If you haven’t been to Predictive Analytics World, or if it has been a few years, plan to go next year! Details will appear at https://www.predictiveanalyticsworld.com/ and there’s an email list you can sign up for to receive updates.