Why Treating AI Projects Like Traditional Software Development Limits Your Team’s Potential for Success

Having worked in and led many Artificial Intelligence (AI) and software engineering teams, I’ve noticed major misunderstandings about how these teams work, in particular, the assumption that these processes are the same.

Although some believe that the development of AI is the same as the development of standard software or maybe even less complex because “the AI can figure it out” — the reality of course is quite different.

This article discusses the fundamental differences and provides actionable ways to help businesses adopt AI effectively.

Key Differences Between AI and Traditional Software Development

Firstly, when I discuss AI development I will combine some of the standard approaches for AI, machine learning and data science since there are many key similarities between these, all of which are in contrast to the traditional software development process.

AI projects are different from traditional software development in terms of approach, workflow, and success criteria. The following summaries many of the critical differences:

Development Lifecycle: Iterative Exploration vs. Linear Execution

AI Workflow

AI projects follow iterative frameworks (like CRISP-DM, which is typically used in data science projects), which emphasise discovery and adaptation.

Here I have used a similar workflow to CRISP-DM, but adding an explicit monitoring step (which emphasises its importance when developing AI and machine learning models). Although this can also be important within data science, in some circumstances the output is a set of reports or insights rather than a model.

In the case of an AI model, there is an increased need for monitoring, which can then retrigger the need for an updated model (starting the cycle again). The need for AI models to be retrained is not a failing of the model, it is a natural process to capture new patterns that are prevalent (that may not have existed in the data the model was trained on).

1. Problem Definition: State the objectives of the project in relation to business objectives (e.g., “Reduce customer churn by 20%”).
2. Data Preparation: Data cleaning, preprocessing and exploration (e.g., how to deal with missing values in the sales data).
3. Model Development: Experiment with algorithms (for example, neural networks versus support vector machines).
4. Evaluation: Validate model based on data it has never seen before (and optimise hyperparameters).
5. Deployment: It is released to production (for example, provide an API for real time predictions)
6. Monitoring: Monitor model performance and track any deviations from what is expected. Allow for regular model retraining (e.g., quarterly updates).

Software Development Workflow

Traditional projects follow methodologies such as the Software Development Lifecycle (SDLC):

1. Planning: Defining the scope, features, costs, and time frames.
2. Analysis & Design: Developing UI/UX prototypes and system architecture.
3. Development: Developing code (for instance, React for the front end, Node.js for the back end).
4. Testing: Checking whether it meets the set criteria.
5. Deployment: It is released to app stores or servers.
6. Maintenance: To fix bugs and make updates available to users

Software development follows a less complex flow: the main difference here is that the forward progression is continuous compared to the AI workflow where at several points in the development it can go back to an earlier stage in the process.

Why AI Projects Demand Iteration

Problem Definition Uncertainty

• Challenge: Complex problems are often not easy to define and scope (e.g., fraud detection involves patterns and behaviours that are constantly evolving).

• Solution: The problem should be refined as the data is investigated (in conjunction with the stakeholders).

Data Uncertainty

• Challenge: Data flaws or prejudice may not appear until the middle of the project (for example, lacking patient demographics data in a healthcare database).

• Solution: Feasibility studies and iterative data audits should be conducted.

Exploratory Model Development

• Challenge: There is no single ‘best fit’ algorithm (for instance, a fraud detection model might require the use of decision trees at one stage and graph neural networks at another)).

• Solution: Multiple approaches should be prototyped in the MVP phases.

Probabilistic Outcomes

• Challenge: Models can be brittle with respect to changes in the data coming from the real world (for instance, a fraud detection model may not cope well with a new type of fraud).

• Solution: Schedule retraining cycles and monitor performance.

Recommendations for Success

1. Prioritise Data Strategy Over Immediate Model Building

Build Robust Data Infrastructure:

• Data collection, cleaning and labelling should be automated
• Implement version control systems to track dataset iterations and make sure you can reproduce results

Conduct Pre-Project Audits:

• Aim to find out any issues within the data before modelling.

2. Build Cross-Functional Teams

Essential Roles:

• Data Engineers: Design pipelines for scalable data processing.
• Data Scientists/AI Engineers: Design, build and test effective AI/ML models
• Domain Experts: Provide feedback (e.g., clinicians for healthcare AI).

Break Down Silos:

• Hold regular meetings between technical teams and business stakeholders to align priorities.

3. Embrace Phased Development

Start with an MVP:

• Focus on a narrow, high-impact use case (e.g., “Predict equipment failure for one production line”).
• Use the results of the MVP to justify further investment.

Iterate Incrementally:

• Adjust the models based on feedback

4. Plan for Model Decay and Drift

Monitor Continuously:

• Track important performance metrics (for example accuracy, latency)
• Set automated alerts based on performance (e.g., “Retrain if precision drops below 90%”).

Budget for Retraining:

• It should be assumed that models will need to be retrained
• According to the use case, it may require more or less frequent retraining.
• For instance, a fraud detection model will need to be trained regularly to detect new patterns

5. Prioritise Explainability and Transparency

Choose Interpretable Models:

• Use simpler models (for example decision trees) in regulated industries (such as healthcare) unless a highly complex model is justified.

Document Rigorously:

• Maintain logs of model versions, training data, and hyperparameters

6. Align AI Goals with Business Outcomes

Avoid “AI for AI’s Sake”:

• If the solution can be achieved with a similar software solution or a set of rules, then pursue the simpler option
• It begins with a clear problem statement linked to business KPIs (for instance, “Decrease customer churn by 15%”).

Measure Holistic ROI:

• Indirect benefits are also included (for instance, customer satisfaction) together with accuracy metrics.

7. Build Ethical Guardrails Early

Test for Fairness:

• There are packages such as fairlearn or AIF360 to assess the models’ fairness across subgroups, for instance, age, gender, geography.

Compliance Integration:

• Comply with regulations such as GDPR, EU AI Act from the beginning.

Conclusion: Transforming Uncertainty into Strategic Advantage

While AI development and software engineering can often serve different purposes, they each have their own strengths and can also complement each other.

Software engineering excels at creating deterministic systems for well defined problems, while AI development handles pattern based problems which can not be programmed in a conventional manner.

The exploratory nature of AI development is not a flaw: it’s simply what’s needed to handle complex and uncertain problems.

For organisations who are new to AI development, success is achieved through an understanding and acceptance of the iterative process rather than attempting to implement traditional software development approaches.

If you’d like to find out more about me, please check out www.paulferguson.me, or connect with me on LinkedIn.

---

AI Development vs Software Engineering: Key Differences Explained was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

​Why Treating AI Projects Like Traditional Software Development Limits Your Team’s Potential for SuccessHaving worked in and led many Artificial Intelligence (AI) and software engineering teams, I’ve noticed major misunderstandings about how these teams work, in particular, the assumption that these processes are the same.Although some believe that the development of AI is the same as the development of standard software or maybe even less complex because “the AI can figure it out” — the reality of course is quite different.This article discusses the fundamental differences and provides actionable ways to help businesses adopt AI effectively.Key Differences Between AI and Traditional Software DevelopmentFirstly, when I discuss AI development I will combine some of the standard approaches for AI, machine learning and data science since there are many key similarities between these, all of which are in contrast to the traditional software development process.AI projects are different from traditional software development in terms of approach, workflow, and success criteria. The following summaries many of the critical differences:Core Differences — image created by authorDevelopment Lifecycle: Iterative Exploration vs. Linear ExecutionAI WorkflowAI projects follow iterative frameworks (like CRISP-DM, which is typically used in data science projects), which emphasise discovery and adaptation.Here I have used a similar workflow to CRISP-DM, but adding an explicit monitoring step (which emphasises its importance when developing AI and machine learning models). Although this can also be important within data science, in some circumstances the output is a set of reports or insights rather than a model.In the case of an AI model, there is an increased need for monitoring, which can then retrigger the need for an updated model (starting the cycle again). The need for AI models to be retrained is not a failing of the model, it is a natural process to capture new patterns that are prevalent (that may not have existed in the data the model was trained on).Problem Definition: State the objectives of the project in relation to business objectives (e.g., “Reduce customer churn by 20%”).Data Preparation: Data cleaning, preprocessing and exploration (e.g., how to deal with missing values in the sales data).Model Development: Experiment with algorithms (for example, neural networks versus support vector machines).Evaluation: Validate model based on data it has never seen before (and optimise hyperparameters).Deployment: It is released to production (for example, provide an API for real time predictions)Monitoring: Monitor model performance and track any deviations from what is expected. Allow for regular model retraining (e.g., quarterly updates).AI Development Workflow — image created by authorSoftware Development WorkflowTraditional projects follow methodologies such as the Software Development Lifecycle (SDLC):Planning: Defining the scope, features, costs, and time frames.Analysis & Design: Developing UI/UX prototypes and system architecture.Development: Developing code (for instance, React for the front end, Node.js for the back end).Testing: Checking whether it meets the set criteria.Deployment: It is released to app stores or servers.Maintenance: To fix bugs and make updates available to usersSoftware Development Workflow — image created by authorSoftware development follows a less complex flow: the main difference here is that the forward progression is continuous compared to the AI workflow where at several points in the development it can go back to an earlier stage in the process.Why AI Projects Demand IterationProblem Definition Uncertainty• Challenge: Complex problems are often not easy to define and scope (e.g., fraud detection involves patterns and behaviours that are constantly evolving).• Solution: The problem should be refined as the data is investigated (in conjunction with the stakeholders).Data Uncertainty• Challenge: Data flaws or prejudice may not appear until the middle of the project (for example, lacking patient demographics data in a healthcare database).• Solution: Feasibility studies and iterative data audits should be conducted.Exploratory Model Development• Challenge: There is no single ‘best fit’ algorithm (for instance, a fraud detection model might require the use of decision trees at one stage and graph neural networks at another)).• Solution: Multiple approaches should be prototyped in the MVP phases.Probabilistic Outcomes• Challenge: Models can be brittle with respect to changes in the data coming from the real world (for instance, a fraud detection model may not cope well with a new type of fraud).• Solution: Schedule retraining cycles and monitor performance.Recommendations for Success1. Prioritise Data Strategy Over Immediate Model BuildingBuild Robust Data Infrastructure:Data collection, cleaning and labelling should be automatedImplement version control systems to track dataset iterations and make sure you can reproduce resultsConduct Pre-Project Audits:Aim to find out any issues within the data before modelling.2. Build Cross-Functional TeamsEssential Roles:Data Engineers: Design pipelines for scalable data processing.Data Scientists/AI Engineers: Design, build and test effective AI/ML modelsDomain Experts: Provide feedback (e.g., clinicians for healthcare AI).Break Down Silos:Hold regular meetings between technical teams and business stakeholders to align priorities.3. Embrace Phased DevelopmentStart with an MVP:Focus on a narrow, high-impact use case (e.g., “Predict equipment failure for one production line”).Use the results of the MVP to justify further investment.Iterate Incrementally:Adjust the models based on feedback4. Plan for Model Decay and DriftMonitor Continuously:Track important performance metrics (for example accuracy, latency)Set automated alerts based on performance (e.g., “Retrain if precision drops below 90%”).Budget for Retraining:It should be assumed that models will need to be retrainedAccording to the use case, it may require more or less frequent retraining.For instance, a fraud detection model will need to be trained regularly to detect new patterns5. Prioritise Explainability and TransparencyChoose Interpretable Models:Use simpler models (for example decision trees) in regulated industries (such as healthcare) unless a highly complex model is justified.Document Rigorously:Maintain logs of model versions, training data, and hyperparameters6. Align AI Goals with Business OutcomesAvoid “AI for AI’s Sake”:If the solution can be achieved with a similar software solution or a set of rules, then pursue the simpler optionIt begins with a clear problem statement linked to business KPIs (for instance, “Decrease customer churn by 15%”).Measure Holistic ROI:Indirect benefits are also included (for instance, customer satisfaction) together with accuracy metrics.7. Build Ethical Guardrails EarlyTest for Fairness:There are packages such as fairlearn or AIF360 to assess the models’ fairness across subgroups, for instance, age, gender, geography.Compliance Integration:Comply with regulations such as GDPR, EU AI Act from the beginning.Conclusion: Transforming Uncertainty into Strategic AdvantageWhile AI development and software engineering can often serve different purposes, they each have their own strengths and can also complement each other.Software engineering excels at creating deterministic systems for well defined problems, while AI development handles pattern based problems which can not be programmed in a conventional manner.The exploratory nature of AI development is not a flaw: it’s simply what’s needed to handle complex and uncertain problems.For organisations who are new to AI development, success is achieved through an understanding and acceptance of the iterative process rather than attempting to implement traditional software development approaches.If you’d like to find out more about me, please check out www.paulferguson.me, or connect with me on LinkedIn.AI Development vs Software Engineering: Key Differences Explained was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.  data-science, ai-development, machine-learning, ai, artificial-intelligence Towards Data Science – MediumRead More

Add to favorites