What Makes a Good Feature and How Do We Decide What Goes Into a Model?

A credit model is only as good as the inputs it's built on. Feature selection — deciding which variables go into a model and which don't — is one of the most consequential parts of the build process.

This post walks through how we think about features: where they come from, what makes one worth including, the compliance guardrails that shape the universe of candidates, and how we narrow down from a large starting set to the lean, stable feature set that ends up in a final model.

The Three Buckets

When we're considering what features might go into a credit model, we generally think in three categories. Which of these are available — and how useful each turns out to be — depends heavily on the product, the market, and the borrower population.

Bureau data. Past credit performance is the most predictive signal for future credit performance. That holds across nearly every geography, product type, and borrower segment we've worked with. Bureau data is the workhorse. In the US, that means detailed information from Experian, Equifax, or TransUnion. In other markets, the depth and structure of bureau data varies considerably — some geographies have rich positive and negative reporting; others collect only negative information, which changes what's available and how useful it is. But in almost every context, bureau data is the starting point.

Alternative data. Over the last decade or so, a range of data sources have emerged that sit outside traditional bureau reporting and can add meaningful lift — particularly for segments of the population that are underserved by conventional credit data. This category covers a lot of ground, and it's worth being precise about what it actually includes, because "alternative data" gets thrown around loosely. Bank transaction data — sourced through open banking connections like Plaid — is one of the more powerful examples: it's somewhat orthogonal to bureau data and can surface income patterns, cash flow behavior, and financial stability signals that bureau data doesn't capture. Other sources in this bucket include buy now pay later performance, income verification data, payday loan history, and in some markets, utility or telco payment behavior. The utility of any given alternative data source depends on the specifics — the data source itself, the product, and the population — and the right response to "we use alternative data" is always to ask what that actually means.

Proprietary and internal data. For lenders with an existing customer relationship, internal data can be among the most powerful inputs available. A bank that already has mortgage, credit card, or deposit account information on a loan applicant has a meaningful informational advantage. Similarly, lenders offering a second product to existing customers can draw on prior repayment behavior that isn't reflected anywhere in bureau data. This information is unique to the lender, it's not available to competitors, and it tends to be genuinely additive to what bureau and alternative sources provide.

As a general pattern: for high-FICO borrowers in a mature credit market, bureau data alone often gets you quite far. As you move down the credit spectrum, or into markets where bureau data is thinner, alternative and proprietary sources become more important for differentiating risk.

Compliance First

Before evaluating any feature on its predictive merits, you need to know whether you can use it. In the US, this means working within the Fair Credit Reporting Act (FCRA) framework — which governs what data sources are permissible for credit decisioning — and the fair lending rules that prohibit the use of protected class characteristics and restrict the use of variables that function as proxies for them.

In practice, established data providers are usually able to tell you upfront whether a given source is FCRA-compliant and cleared for credit decisioning. That takes care of the most straightforward cases. The harder questions arise with newer data sources, where the regulatory guidance is less settled.

Our approach, regardless of market: we want to be satisfied that the data was collected fairly and with appropriate consent, and that there's an intuitive, defensible link between the feature and repayment behavior before we consider using it in a lending decision. This isn't just a compliance checkbox — it's also a modeling principle. Features that we can't explain in terms of why they should predict risk make us uncomfortable, and for good reason.

One practical implication of this is adverse action. When a model declines an applicant, you need to be able to explain why — in terms the borrower can understand and that can withstand regulatory scrutiny. That means every feature in the model needs to be explainable. A feature that adds a few basis points of AUC but can't be articulated sensibly in an adverse action notice probably doesn't belong in the model. Even in markets outside the US where regulatory constraints are lighter, we use the US standard as a guidepost. Features we don't fully understand and can't explain clearly tend not to make the cut.

What Makes a Feature Good

Once you've established that a feature is permissible, the evaluation shifts to performance — and there are a few dimensions that matter.

Predictive power. The baseline question: does the feature actually correlate with the outcome you're trying to predict? Traditional approaches use information value to measure univariate predictive power. More modern approaches use the model itself to surface which features are adding lift — not just in isolation, but in the presence of other features. Both matter. A feature can look strong on its own but add nothing to a model that already captures the same signal through other inputs.

Stability over time. This one is easy to underestimate. A feature that was strongly predictive three years ago may not be today — either because the underlying data has changed or because its relationship with the target variable has shifted. When you build a model, the model has no awareness of time; it learns whatever patterns are present in the training data. If a feature's behavior changed over the period the model was trained on, the model may be picking up on those time-related shifts rather than the feature's actual credit signal. Evaluating stability — both in the feature's values and in its performance against bad rates across different time periods — is essential before including it.

Coverage. How often is the feature actually populated for applicants in the dataset? A highly predictive feature with significant missing data creates practical problems: it either forces assumptions about what to do when the value is absent, or it effectively applies only to a subset of the population. Neither is ideal. Coverage needs to be evaluated alongside predictive power, not treated as an afterthought.

Consistency between development and production. We've written about this in the context of model monitoring, but it applies equally at the feature level during development. A feature constructed from a retrospective bureau pull may not arrive in exactly the same form in a live production environment — different timing logic, different handling of missing values, different data structures. Features that look clean in training data sometimes behave differently in production, and understanding that risk before the model goes live is important.

How We Narrow Down

Starting from a broad universe of candidate features and arriving at a final model set is a process of systematic reduction. The goal — and this is a design principle we hold consistently — is the smallest set of features that captures most of the available predictive lift.

That might sound like leaving performance on the table, but in practice it isn't. Bureau data alone — something like Experian's Premier Attributes set — includes roughly 2,000 features. Many of those carry highly redundant information. They're correlated with each other, they measure similar things in slightly different ways, and including all of them adds complexity without adding signal. By working through correlation analysis, importance screening, and stepwise feature addition while monitoring AUC, you can typically get from 2,000 candidates down to a few hundred — and end up with a model that performs as well or better than one that uses the full set.

The benefits of a leaner model compound over time. Fewer features means simpler implementation, easier adverse action documentation, lower data procurement costs, and more tractable ongoing monitoring. A model with 50 features is easier to understand and maintain than one with 500. If something goes wrong in production, it's far easier to diagnose. These aren't minor conveniences — they're meaningful operational advantages.

We've seen the costs of the opposite approach firsthand. In one engagement with an installment lender that had a sophisticated internal data science team, we found that their decisioning models were collectively drawing on thousands of variables across multiple combined models. It was expensive — each data source carried operational cost — and the complexity made monitoring difficult. When we rebuilt with our own approach, we matched or exceeded their predictive performance using roughly a tenth of the features. The reduction didn't cost anything in model quality; it bought a lot in manageability.

A related pattern we've noticed: it's not uncommon to find that some features in a model have zero importance — they aren't actually contributing to predictions at all. They made it into the model at some point, were never removed, and have been sitting there adding cost and complexity without doing any work. Getting disciplined about this during the build — and not including features unless they're demonstrably earning their place — pays dividends throughout the model's life.

The Bottom Line

Feature selection isn't a mechanical step that happens before the real modeling work begins. It's a core part of what determines whether a model is good — not just on paper, but in production, over time. The features you include shape what the model can learn, how stable it will be, how explainable its decisions are, and how easy it is to monitor and maintain. Getting this right requires thinking carefully about data sources, compliance, individual feature quality, and the tradeoffs between predictive power and complexity. The teams that do this well tend to end up with models that are not just more accurate, but more durable.