1. AI use case versus business problem

A business problem describes an undesirable outcome

Too many insurance policies are not being renewed.

An AI use case describes how a model may assist:

Predict which active policies are at high risk of not renewing
30 days before expiry so the retention team can intervene.

The second statement is clearer because it includes:

  • Prediction target
  • Prediction timing
  • Intended user
  • Intended action

2. Define the unit of prediction

The unit of prediction is the individual thing for which the model produces one result.

Examples:

Requirement Prediction unit
Predict policy renewal One expiring policy
Predict exam malpractice One exam session or event
Route support tickets One support ticket
Predict server failure One server during a defined period
Predict customer churn One customer account
Classify CSR question One user question

This is similar to identifying the aggregate root or primary entity in application design.

If the unit is unclear, dataset rows will also be unclear.

3. Define the target or label

The target is what the model should predict.

For policy renewal:

WillRenewWithin30Days = true or false

The label must have a precise definition.

This is weak:

Customer is interested

What counts as interested?

This is better:

Renewed = a successful renewal payment recorded
on or before 30 days after the original expiry date.

Different teams must calculate the label in exactly the same way.

4. Define the prediction moment

The prediction moment is when the model will receive data and produce its result.

Example:

30 days before policy expiry

Only information available at that moment can be used as features.

Suppose today is 1 September and the policy expires on 1 October. We can use:

  • Historical claims up to 1 September
  • Complaints recorded up to 1 September
  • Past renewals
  • Payment history up to 1 September

We cannot use:

  • Renewal payment made on 28 September
  • Receipt generated after renewal
  • Final renewal status

Those future values cause leakage.

5. Define the prediction output

A model might return:

WillRenew = false
Probability = 0.78

The probability can help prioritize action.

For example:

Risk score Operational action
0.80–1.00 Call within 24 hours
0.60–0.79 Send personal follow-up
0.40–0.59 Send automated reminder
Below 0.40 Normal renewal workflow

These thresholds are business decisions, not universal mathematical truths.

6. Define the business action

A prediction without an action is only a report.

Ask:

When the model predicts X, who will do what—and by when?

Example:

When non-renewal risk exceeds the approved threshold,
create a retention task for the assigned POSP and show
the reasons available to the system.
 
The action may be:
  • Create a task
  • Prioritize a work queue
  • Request human review
  • Send a notification
  • Block an unsafe automated action
  • Ask for additional information
  • Generate a recommendation

High-impact decisions should generally retain appropriate human oversight.

7. Business KPI versus model metric

A business KPI measures business value.

Examples:

  • Renewal rate
  • Revenue retained
  • Fraud loss prevented
  • Support-resolution time
  • Manual review workload
  • Candidate complaints

A model metric measures predictive behaviour.

Examples:

  • Accuracy
  • Precision
  • Recall
  • F1 score
  • Mean absolute error
  • Response groundedness

A model can improve its technical metric without improving the business KPI.

Example:

Model accuracy rises from 80% to 86%
but the retention team cannot contact additional customers.
 
The model improved technically, but business performance may remain unchanged.

8. Establish a baseline

A baseline is the current or simplest method that the AI solution must outperform.

Possible baselines:

  • Existing business rule
  • Manual team performance
  • Majority-class prediction
  • Previous month’s KPI
  • A simple statistical method
  • Random selection

For renewal prediction, the current rule might be:

Contact every policy with a complaint in the previous 90 days.
Before building ML, measure:
  • How many policies it identifies
  • How many identified policies actually fail to renew
  • How many non-renewals it misses
  • How much staff time it consumes

If the ML model cannot outperform that rule meaningfully, it may not justify its cost.

9. False positive and false negative

For non-renewal risk:

False positive

Model predicts: Customer will not renew
Actual result: Customer renews

Possible cost:

  • Unnecessary call
  • Unneeded discount
  • Staff time
  • Customer annoyance

False negative

Model predicts: Customer will renew 
Actual result: Customer does not renew
Possible cost:
  • Lost policy
  • Lost revenue
  • Missed retention opportunity

The more expensive error should influence metric and threshold selection.

10. When AI is not appropriate

Prefer deterministic code when:

  • Rules are complete and stable.
  • The answer must be exact.
  • Historical examples are insufficient.
  • No reliable label exists.
  • The process cannot tolerate probabilistic output.
  • A simple SQL query already solves the problem.
  • There is no action that follows the prediction.
  • The AI solution costs more than the expected benefit.
PreviousNext