Scenario

Precision:
Of the policies we marked high-risk, how many actually did not renew?

Recall:
Of all policies that did not renew, how many did our rule identify?

Current requirement:

“Build AI to improve insurance policy renewals.”

We will convert it into an implementable ML project.

Step 1: Write the problem definition

For every active policy approaching expiry, predict the probability
of non-renewal 30 days before its expiry date.

The retention team will use the probability to prioritize follow-up.
The solution should increase the renewal rate without increasing the
team’s daily contact capacity.

Step 2: Define the ML task

Step 3: Represent it in C#

public sealed record MlProblemDefinition(
    string Name,
    string BusinessProblem,
    string PredictionUnit,
    string PredictionMoment,
    string Target,
    string TaskType,
    string ModelOutput,
    string IntendedUser,
    string BusinessAction,
    string BusinessKpi,
    string Baseline,
    string FalsePositiveCost,
    string FalseNegativeCost);

Create the definition:

var renewalProblem = new MlProblemDefinition(
    Name: "Policy Non-Renewal Risk",
    BusinessProblem:
        "Too many eligible policies expire without renewal.",
    PredictionUnit:
        "One active policy approaching expiry.",
    PredictionMoment:
        "30 days before the policy expiry date.",
    Target:
        "Whether the policy fails to renew within the approved window.",
    TaskType:
        "Binary classification.",
    ModelOutput:
        "Non-renewal probability between 0 and 1.",
    IntendedUser:
        "Retention executive or assigned POSP.",
    BusinessAction:
        "Create and prioritize a retention follow-up task.",
    BusinessKpi:
        "Renewal rate and retained premium revenue.",
    Baseline:
        "Contact policies with a complaint in the previous 90 days.",
    FalsePositiveCost:
        "Unnecessary contact, discount or staff effort.",
    FalseNegativeCost:
        "Lost policy and missed retention opportunity.");

Console.WriteLine(renewalProblem);

This is documentation expressed as code. It can later become part of:

• Experiment metadata
• Model registry description
• Project documentation
• Approval workflow
• Monitoring configuration

Step 4: Measure the business baseline with SQL

Assume a historical view contains:

PolicyId
HadComplaintInPrevious90Days
Renewed
PremiumAmount

The current baseline predicts non-renewal when the policy had a recent complaint:

SELECT
    COUNT(*) AS TotalPolicies,

    SUM(CASE
        WHEN HadComplaintInPrevious90Days = 1
        THEN 1 ELSE 0
    END) AS PredictedHighRisk,

    SUM(CASE
        WHEN HadComplaintInPrevious90Days = 1
         AND Renewed = 0
        THEN 1 ELSE 0
    END) AS CorrectHighRiskPredictions,

    SUM(CASE
        WHEN HadComplaintInPrevious90Days = 1
         AND Renewed = 1
        THEN 1 ELSE 0
    END) AS FalsePositives,

    SUM(CASE
        WHEN HadComplaintInPrevious90Days = 0
         AND Renewed = 0
        THEN 1 ELSE 0
    END) AS FalseNegatives
FROM PolicyRenewalHistory;

Meaning:

CorrectHighRiskPredictions:
The rule selected a policy that actually did not renew.

FalsePositives:
The rule selected a policy that renewed anyway.

FalseNegatives:
The rule did not select a policy that later failed to renew.

Step 5: Calculate simple baseline rates

WITH Baseline AS
(
    SELECT
        CASE
            WHEN HadComplaintInPrevious90Days = 1 THEN 1
            ELSE 0
        END AS PredictedNonRenewal,

        CASE
            WHEN Renewed = 0 THEN 1
            ELSE 0
        END AS ActualNonRenewal
    FROM PolicyRenewalHistory
)
SELECT
    CAST(
        SUM(CASE
            WHEN PredictedNonRenewal = 1
             AND ActualNonRenewal = 1
            THEN 1.0 ELSE 0
        END)
        /
        NULLIF(
            SUM(CASE
                WHEN PredictedNonRenewal = 1
                THEN 1.0 ELSE 0
            END),
            0
        )
        AS DECIMAL(10,4)
    ) AS BaselinePrecision,

    CAST(
        SUM(CASE
            WHEN PredictedNonRenewal = 1
             AND ActualNonRenewal = 1
            THEN 1.0 ELSE 0
        END)
        /
        NULLIF(
            SUM(CASE
                WHEN ActualNonRenewal = 1
                THEN 1.0 ELSE 0
            END),
            0
        )
        AS DECIMAL(10,4)
    ) AS BaselineRecall
FROM Baseline;

For now, understand them simply:

We will study these metrics mathematically in a later lesson.

Step 6: Add operational constraints

Suppose:

Expiring policies per day: 1,000
Maximum calls per day: 200

The model cannot simply label 800 policies as high-risk. The business cannot act on all of them.

A better operational output is:

Rank all 1,000 policies by risk
        ↓
Send the top 200 actionable cases
        ↓
Track contact outcome and renewal

Model design must respect real capacity.

Step 7: Define success before training

Example acceptance conditions:

1. Beat the approved baseline on the chosen evaluation metrics. 
2. Fit within 200 daily manual contacts. 
3. Improve renewal rate in a controlled pilot. 
4. Do not use information created after the prediction moment. 
5. Provide enough context for a human to review the recommendation. 
6. Log the model version, input time, score and resulting action.

Do not promise a specific improvement percentage until historical data has been analyzed and a realistic pilot has been designed.