Driving AI Success for Enterprise

By Jeremy Fajardo

Technology’s ubiquity is difficult to ignore these days and analytics — one of its key enablers — is an industry agnostic driver whose impact is probably best characterized by its explosive growth (and fair share of hype). Despite that ubiquity (or maybe because of it), trying to grasp the full scope of the latest trends, pitfalls and insights feels like an impossible task. In May last year, our team ran a conference to try and get a pulse on the industry from as many perspectives as we could fit into a single day.

Here are a few key learnings:

  1. Data science as a platform and service is a common approach that many AI firms take to drive value for large enterprises.

  2. End-to-end design is a luxury and so generating quick-wins are key to success in larger companies

  3. That being said, it’s very important to get familiar with technical debt when deploying even intermediate solutions.

1. Data science as a platform and service

In Toronto’s garment district, the AI firm Dessa occupies a refurbished warehouse where cofounder Stephen Piron’s grandmother used to work as a seamstress during the early days of the industrial revolution. It was a time where machines were being leveraged to augment the way humans worked and had a profound impact on society and the way we lived — echoing our current digital age with the applications of ML and AI.

The portfolio of Dessa and AI firms like it, showcase the breadth of possibilities analytics can have in multiple industries. At the ABD 2018 conference, Piron highlighted the blueprinting process his team follows and the “Dragon’s Den” style pitches they employ to filter ideas for AI applications internally. An engineer at heart, he noted the shortcomings of open-source out-of-the-box solutions and the subsequent pivot his team made towards building a software platform for enterprise AI (aptly named, Foundations).

The SaaS model is a common one for many AI firms as it provides an onboarding mechanism for enterprises to quick start AI projects with clients but Piron is well aware that advanced analytics cannot be sold as a turnkey solution.

Oft quoted figure from the well-circulated paper : “Hidden Technical Debt in Machine Learning Systems”

Oft quoted figure from the well-circulated paper : “Hidden Technical Debt in Machine Learning Systems”

Many enterprise companies do not have this ideal infrastructure and lack the resources to build it quickly. This reality is what makes the service and product approach attractive for small AI firms to pitch. By simplifying the integration, companies like Dessa or, fellow Toronto firm, Rubikloud can achieve a small amount of lock-in when onboarding an enterprise client.

And that lock-in — no matter how small — will make the race for market penetration interesting in the next 1–2 years. Rubikloud’s approach of focusing on a single industry (retail) is one way to get ahead. By ceding other opportunities (ie. the financial sector) to its competition, they can focus their solutions and their brand in one sector.

There will also be an interesting push and pull as enterprises themselves try to balance which projects to dole out to the AI firms and which ones to keep in their back pockets for their own growing data science teams. For enterprises more mature in their analytics journey, the movements and solutions offered by AI startups can provide a benchmark for up-to-date applications.

2. End-to-end design is a luxury

Enterprise analytics as a process, might look something like this:

  1. Data Generating Process

  2. Cleaning / Feature Engineering

  3. Analytics Engine

  4. User Interface

  5. Decision

Being able to design and optimize every stage of this workflow is a luxury that enterprise organizations don’t always have. Additionally, every stage can look different for every company and every stage has has its own pitfalls. Here’s two that we’ve heard come up several times:

  1. Not having clear expectations of what the analytics engine can do

  2. Mistaking data availability as the sole responsibility of the “Analytics” team

Mistake 1. Attributing a failure to derive an impactful decision at the end as the sole failure of the analytics engine.

Mitigation 1. Have a clear measurement strategy for your approach and a strong preliminary understanding of your data.

Conduct a thorough data exploration — not only on the data itself, but also audit the process that generates it. Outline all the caveats about the process with respect to the inferences you’re hoping to make once the model is in production. Some examples:

  • Are the users generating our data who we think they are? Do they skew a particular way?

  • Do we have 1-2 years worth of data? Is that enough given our industry?

Thinking of these questions and trying to answer them will help guide your measurement strategy and most importantly, help you scope the minimum amount of value you can get and scope your expectations against that.

In other words, is even the “dumbest” version of our would-be model, still providing some amount of incremental value?

Mistake 2. Data engineering + data science == new data creation.

Mitigation 2. Conduct a data feasibility assessment.

Is the business asking for an insight on something you don’t have the data to comfortable infer from?

Data cleanliness and building efficient pipelines is where data engineers can help. Generating insights and communicating to the business is where the new fangled data scientists can work their magic. But data availability / feasibility can’t be fixed by efficient ETL or the Most Sophisticated Algorithm; it is a cross-functional, business and strategy problem.

Data enrichment as a strategic consideration when making business decisions is often not something large enterprises are actively doing (yet). In fact, many organizations struggle to find value in the data they do have.

If this is you or someone you know, consider the following motivating questions:

  • What are the other processes adjacent to or related to the one which you are sourcing your data? Can you access / buy / acquire those sources / products / businesses?

  • For product-driven companies: how can you design features / encourage new forms of interaction to get more robust info from your user base?

3. Parting thoughts on technical debt in ML processes

There is one last thing we hear often within the discussions of analytics for enterprise. That is the notion of the quick win project to help kickstart analytics. We’ll be diving deeper into this topic with future articles. But for now, we just want to highlight one concept: technical debt.

For those not familiar with the software engineering concept, it’s relevance to ML and AI is important. The idea is that agility in engineering often comes at the cost of technical debt. Speed now, begets future (more painful) work. There are strategies and frameworks in software engineering to mitigate technical debt but many of them don’t account for the host of ML/AI specific debt that comes from advanced analytics processes.

If you’re an analytics professional in a large enterprise trying to secure buy-in for future analytics investment, be aware of moving too quickly or taking your quick-win project farther than it needs to (or should go).

It’s perfectly ok if your company’s first foray into advanced analytics gets scrapped before seeing widespread deployment.

Even if you have to go back to the drawing board, if you’re doing so with better and proper resources than when you started that first project, that’s kind of a win.

For curious readers, we highly recommend the paper Hidden Technical Debt in Machine Learning Systems which serves as an excellent primer into the details of what makes technical debt exponentially tricky with ML.

Author. Jeremy is a data analyst by day and something else witty by night. Say hi on LinkedIn.

Analytics By Design is a Toronto based community of analytics practitioners looking to understand the design, development and implementation of ML/AI strategy and technology.

Follow us on Twitter to stay up to date on new articles! And check out our website for more content.

Jeremy FajardoSummaryComment