AI Modeling

A model that ranks applicants by risk is only half the job. At some point, you have to draw a line — and how you draw it matters as much as the model itself. Score cutoffs come up constantly in conversations with clients and prospects, and the questions around them are often underappreciated. Where should the line be? What's driving that decision? And once you've set it, when do you look at it again? This post walks through how we think about cutoffs: what they're actually do

We've written a lot about specific aspects of credit modeling — how to choose features, what AUC actually measures, when to retrain. But we haven't spent much time on the bigger picture: if you've never done a custom model build before and you're considering one, what does the process actually look like? How long does it take? And where does it tend to go sideways? This post is for companies that have relied on off-the-shelf scores or externally developed policies and are now

Ask a lender how they plan to use their new model, and you'll often get an answer about approval rates, cutoffs, or pricing tiers. What you hear less often — but should — is a clear explanation of what the model's raw output actually represents and what it doesn't. That gap matters. How you interpret a score shapes everything downstream: how you set cutoffs, how you price, and how you know when something has gone wrong. And these are questions worth working through before a m

It’s a question we hear often during model builds: “Are we overfitting to a weird period in the data?” Sometimes the concern is macro — COVID, stimulus, rate shocks. Other times it’s more internal — a change in product, channel, or geography. Either way, the underlying issue is the same: Does the data we trained on actually reflect the environment we’re about to operate in? That’s not always an easy question to answer. But it’s one you have to ask. What “Overfitting” Means in

In many of our projects, we’ll build two candidate models that both look strong on paper. Similar AUC. Similar stability. Both clearly predictive. And yet — when we start comparing scores more closely — they don’t line up. The disagreement isn’t always dramatic. It doesn’t show up as one model approving and the other declining across the board. It’s subtler than that. Borrowers shift between deciles. Score correlations aren’t as high as expected. ROC curves have similar AUCs

Large language models (LLMs) are everywhere right now. They write emails, summarize documents, draft code, screen résumés, and answer questions with remarkable fluency. It’s not surprising that lenders are asking whether the same tools could be used to make—or support—credit decisions. At Ensemblex, we’re excited about LLMs. We use them internally, we track their progress closely, and we expect them to influence how analytical work gets done over time. But we’re also cautious

A common question we hear from lenders goes something like this: “My approval rate is already really high. I’m already letting most applicants through — so what’s the point of building a better model?” It’s a fair question. If your approval rate is 80% or even 90%, the gain from a better rank-ordering model might seem marginal at first glance. But in our work with dozens of lenders across product types, we’ve seen this situation again and again — and we’ve learned that better

It’s a question we get from nearly every client with a credit model in production. Once you’ve launched a model, how often should you revisit it? Is there a fixed schedule you should follow? Or is it only necessary when something breaks? As with many modeling questions, the answer is: it depends. There are some clear principles we use to guide retraining cadence — and some concrete signs that it’s time to act. What we mean by retraining Let’s start by clarifying what we mean

“How many features should I have in my model?” It sounds like a simple question. But like so many modeling decisions—especially in the credit space—the honest answer is: it depends . We often hear this question from internal teams and executives alike. It comes up when a team is building its first in-house model, when it's looking to upgrade an existing scorecard, or even when trying to explain why their current model looks the way it does. At Ensemblex, we've built models fo