Menu

A Technical Leader's Field Guide to ML/AI Investment

Devlin Liles // June 9, 2021

AI

Are we ready for disruption from artificial intelligence (AI) or Machine Learning (ML)? If not, how do we get ready? Many technologists and technology leaders are struggling with these questions today, so we want to dig into the underlying pieces of the puzzle and offer some practical advice on where to start. There is a lot of buzz around artificial intelligence and machine learning, and their ability to revolutionize the world. It is important to remember that we have been here before, multiple times. Each turn of the AI hype wheel has yet to live to the full promise, but it has changed the way people live worldwide. 

Artificial Intelligence is a space packed with acronyms and wild packs of jargon ready to leap onto the unwary. To break down this space effectively, we need to cut through some of this jargon to the root of what each of them means. We are going to focus on where AI falls into the business ecosystem and are aiming for a high-level path for understanding the remainder of the discussion.

null

We look at machine learning as a part of the software development and business intelligence spaces because the automated responses, reactions, or software processes are all built on data centric recommendations and decisions. As we continue to dive into this area, we can explore how this works with natural language processing (NLP), image processing, video processing, complex time series data, and other areas. 

Let’s take a short stroll down memory lane of AI for a few minutes. Have you heard of the Turing Test? If so, you are at least familiar with the focus of the first steps in AI. Alan Turing published a paper in 1950 around thinking machines, and how to test their intelligence. This foundational idea created a surge of peaks and valley as AI research would burst forward and then stagnate. Why is that? It has to do with processing speed and data storage. Once those two bottlenecks move far enough, we tend to advance on the research side of AI in big steps. 

Unfortunately, the path from research advancement to reality in business or day to day life is much harder due to the human and economic factors. Hundreds of millions of dollars of research create the leap forward, but it then takes billions in investments to make technology pervasive. That is the fundamental problem with mass media attention to the capabilities of AI in research environments.

The Hype Train Leaves the Station Before Reality has Decided to Buy a Ticket

At Improving, we like to focus on the reality of benefit rather than Resume Driven Development. This typically means we are asking a couple of questions.

 1.     How do we discern the areas and levels of investment that will bring the most value to our teams and organizations? 

AI, like performance enhancement of an application, is a place that you can spend infinite amounts of time and money for very little return. We strive to avoid that, so we tend to focus on value to the business first. A rule of thumb that guides well is this: Can you describe the value return clearly in less than 5 sentences? If the answer is no, then it is likely not refined enough yet to start working on it. Most great leaps forward, or revolutionary changes to a business comes from a clear, compelling vision.

With AI it is often the leveraging of a dataset that the company already has access to or ownership of in a new way. If we knew X about Y, we could do Z more or better. 

First, we want to explore what you are looking to learn or decide. If it is in the areas that AI is great at currently or will be soon (hype danger here), then you are likely in a good place on this one. If what you are looking to learn is much more amorphous or is not clearly tied to an outcome, there be dragons. 

Second let’s explore what you want to do more or better. The direct impact to one of the four major return areas is important. That is, does it directly lead to more revenue, less cost, faster delivery, or greater innovation? Faster delivery or more innovation are often viewed as paths to more revenue. If you are listening to hype these will be unclear, but for the practical they are typically very direct. AI, like performance enhancement of an application, is a place that you can spend infinite amounts of time and money for very little return. We strive to avoid that, so we tend to focus on value to the business first. A rule of thumb that guides well is this: Can you describe the value return clearly in less than 5 sentences? If the answer is no, then it is likely not refined enough yet to start working on it. Most great leaps forward, or revolutionary changes to a business comes from a clear, compelling vision.

Imagine some of these situations:

  •  If we knew when a worker had fallen, we could provide faster and better care and reduce the severity of workplace incidents. 
    ----- The lessons that you want the AI to learn are solidly in the realm of doable with camera feeds of the facility available for analysis. There are even service based offerings that help to accelerate the system development.

  • The return is a safer work environment, lower operational cost for accidents, and lower turn-over. All have direct financial impact that can be measured over time.

  • If we knew which loans would default, then we could reduce our lending risks.
    ----- This lesson is also doable and depending on what data you have available for large numbers of loans you can be very accurate. However, take care with bias, ethics, and explicability. (more on this in a moment)
    ----- Lowering the default rate and lending risk removes losses and lowers operational costs. This also has a direct drive to cost reduction.
  •  If we knew which young baseball players would make it to the majors, we could recruiter earlier and invest more.
    ----- You can likely train a model to learn some lessons to provide percentage likelihood on outcome. However, the further the data is from the outcome the worse the predictability is. You could spend a LOT of money chasing this without significantly moving your success ratio.
    ----- The result of faster delivery (in this case scouting and coaching) increases costs over the long term (more coaching for more years). If this is not also tied to a big increase to success rate, then money is likely to be wasted.
  •  If we knew which employees would be most successful on the job based on their interview performance, then we could hire only the best.
    ----- This is a tough lesson for AI to learn, and many have tried. It is not impossible, but we have a data issue. The amount of performance data, detailed interview data, previous history, skills evaluation, etc. required to find the determining factors would be extensive. Good, clean, and plentiful data in this space is not typically available.
    ----- The result is also a space that gets into some trouble with ethics, bias, and explicability.

2.    Where does failure live along the path? Are there common pitfalls to avoid?

There are many pitfalls in this space, but we will focus ones that we see most here. Firstly, we see many companies start AI projects without a clear understanding of the value they are trying to get out of the investment. This lack of clarity often leads to trying to boil the ocean, or to chasing each new idea without bringing the last to completion. Defining the goals and intent ahead of time gives a great measure of success and a clear finish line. It is imperative to the success of the initiative to not skip this step.

Secondly, we often see companies under-estimate the level of data work in AI projects. We often see that the data accuracy, cleanliness, availability, granularity, metadata enhancement, and much more are lacking to get reliable results from the machine learning models. This can drastically increase the cost and decrease the effectiveness of an AI project.

We see the last pitfall in application development and AI projects, but the impact is magnified in AI/ML projects. It is the problem of developing a application or model without operationalizing how this is brought to production. Often ML-Ops, like DevOps, is an afterthought. When dealing with unstructured data, models developed by data scientist, and shifting needs, a solid path to testing and getting the resulting model into usable form for users is critical.

Hopefully you have found guidance, some tips, and a path forward through this article. If you would like to have a conversation about your specific struggle, please reach out and we will setup some time with our AI practice lead to help chart a path forward.

If you are an application developer and interested in the AI space but don’t know enough, reach out. We are delivering AI for application developer courses in Q3 and will share a significant discount code with anyone that reaches out.

Most Recent Thoughts

How can we help on your next project?

Let's Talk

Like what you see?

Join Us
Top