top of page
Search
  • Writer's pictureLeland Burns

Ensemblex’s comprehensive approach to explainability goes beyond modeling techniques to deliver explainable models that consistently pass rigorous validation and reviews by government regulators, including ECOA compliance. Multiple clients successfully use these models today, across business lines and geographies.

In May, the CFPB issued a circular that clarified its stance on machine learning (ML) models:

“…ECOA and Regulation B do not permit creditors to use complex algorithms when doing so means they cannot provide the specific and accurate reasons for adverse actions.”

This shouldn’t be a surprise. Explainability has been the standard since the ECOA was enacted. New technology—in this case ML—doesn’t change that requirement. It just means that lenders need to be mindful of how they build ML underwriting models.

In the wake of the CFPB’s circular, much of the discussion around explainability has concerned the precise techniques used to generate reliable adverse action reasons for ML models. In our view, explainability is more than a calculation. It’s a foundational principle that’s built into our entire development process.

If you’re considering incorporating ML into your underwriting model, or you’re concerned about your model’s explainability, here are some ideas to keep in mind:


1. Start with approved variables


Delivering a compliant model means starting with a list of variables that are FCRA-compliant and intuitive predictors of credit.

Much of the buzz around ML and other advanced technologies often centers on the use of “big data,” and an ever-growing range of data sources. However, when it comes to underwriting credit risk, boring is better for data.

We work with our clients to derive maximum power from data, but that data comes from the client’s own business and other industry-respected sources. We’ve proved time and again that it’s possible to build an incredibly powerful machine learning model without drawing from exotic data sources.


2. Reduce complexity


We never chase complexity for its own sake. While ML models can ingest more data than other traditional techniques, a cornerstone of our modeling approach is carefully balancing simplicity with performance.

Throughout model development, we iteratively assess all available data across multiple dimensions, to ensure that we are including the minimal number of variables needed to deliver desired performance improvements. Oftentimes, only a handful of variables deliver the bulk of the predictive power. Removing less effective variables will reduce the complexity of the model — and potentially the cost — while making the model easier to explain.


3. Get comfortable explaining how the model works


Lenders should always be able to explain how their models work, especially if they’re developed by an outside firm. At Ensemblex, we partner with clients throughout the entire process, so that they’re intimately familiar with all aspects of our model and its decisions.

That partnership starts with variable selection and data preparation. We carefully consult with them through all design and technical choices and then conduct a rigorous partnered analysis into all model decisions. For clients with little previous ML experience, we take extra care to explain our methods at each stage of the process and to tie our modeling decisions to meaningful business metrics. The result is an ML model that our clients feel comfortable owning and explaining.


4. Balance advancement with understanding


As the techniques and algorithms for both ML development and explainability have evolved in recent years, we’ve stayed at the forefront of understanding and deploying those tools in our own models and on behalf of clients. This includes using Shapley values for both general model explainability and applicant-level adverse action reasons, a technique upon which much of the conversation around explainability is now centered. Based on Game Theory, Shapley’s algorithm defines the additive contribution of each variable to an applicant’s model score, relative to a given benchmark.

With that said, we work hard to balance the explainability alongside both industry acceptance and our client’s understanding. While we applaud those at the cutting edge of further refining ML explainability tools, we’re careful to make sure any techniques we employ are clearly understood by our clients and sanctioned by industry professionals and regulators. This measured approach has allowed us to launch and manage multiple ML underwriting models, as both business owners and consultants, with zero enforcement actions from regulators.

Bottom line: Make explainability the core of the process, not a feature of the model.


ML models can be safely and transparently developed and explained with an array of established tools, as long as you have the right partner to guide you along the way. We at Ensemblex are uniquely qualified to be that partner. If there’s anything we can do to help, please don’t hesitate to reach out.


A few years ago, I had a conversation with the CEO of a promising credit startup that crashed and sold for parts, just one year after a big raise.


“We raised too much money,” he said. “I thought we could figure out the economics later.”


It turns out the team had pursued an aggressive growth strategy — acquiring lots of customers without focusing on lifetime value and chasing several new opportunities at once.


That decision proved to be fatal for his company. We see a lot of companies that have been doing the same over the last several years. In an environment where the next raise appears to be a given, that strategy can work – for a while.


We’re entering a time when a number of high-growth fintechs are going to experience the sudden impact of a changing economy. Investors are pulling back. Consumers are feeling increasing pain on a number of fronts. The combination means, for some, the runway may be shorter than they expected. Fintechs need to learn how to drive profitability. Now.


My goal today is to help managers assess their position and make the most of the runway they have left. Here are some of the questions managers should be asking themselves, along with some advice based on a few decades of experience in credit, both as a consultant and an operator.


How are we doing?

Don’t look at your P&L. Look at your unit economics.


At Capital One we often talked about “speedboating.” Growing businesses are often outrunning (speedboating) their metrics, especially in the credit industry where lenders have high upfront costs and recoup their investment over long periods of time. Speedboating can make a sustainable business look bad on a P&L statement (e.g. acquisition costs outpace earnings) or a bad business look good (e.g. revenues precede losses). The only way to understand whether your business model is sustainable is to look at the unit economics.


What drives our economics?

Identify your key profit drivers and use them to manage your business. For example, if you have a buy now, pay later (BNPL) business, you probably care a lot about first pay defaults and repeat rates. You may be able sustain a loss on the first loan (due to high upfront costs) if you get enough profitable repeats. If not, you need to bring the economics of the first loan into the black.


How resilient are our economics?

The ability to absorb “hits” is crucial to building a sustainable business. We apply stress scenarios to the key drivers. For example, if repeats stay at the same level, what’s the highest FPD rate that the business can tolerate? Once you understand the bounds, diligently monitoring the performance of those key metrics is key to surviving.


How do we guide the business to profitability?

Credit companies never start out profitable. Acquisition channels take time to figure out. Operating costs improve with scale. Attrition takes work to solve. All of this should improve with time, capital, and lots of hard work.


Lifetime value bridges are a great way to visualize that work, one step at a time.


Figure - Lifetime Value Bridge


To make the bridge work, each step must be realistic and actionable. With some steps, cost savings are relatively easy to assess and realize. For example, you may have contracts that give discounts for scale. Other steps will be much tougher. Reducing attrition, for instance, is typically much easier to imagine than to actualize.


How do we organize the team to deliver results?

Create teams that are focused on delivering results against each step in the bridge. For example, if lowering attrition is one of the steps in the bridge, one team should be assigned to the task of identifying opportunities, executing tests, and delivering results. In this way, each team is focused on a single task and accountable for delivering results.


How do we get buy-in with leadership?

You can’t build a profitable business on the back of unprofitable products. Unit economics provide a great framework for that conversation. The entire team should be able to take in the data and internalize the steps they need to take to drive the business to profitability.


Get started today

The environment has changed. Investors are reigning in investments, valuations are coming down, and a new era is beginning. Like 2009, strong businesses will survive, compelling new businesses will start, and innovation will continue. This environment creates opportunities for managers who can adapt to the new reality.


Focus on unit profitability, manage your economic drivers, understand your resilience, and use a bridge. In the end, a profitable business is a sustainable business. And the easiest raise, is the one you don’t need to do.


You’re not alone. You’re part of a team. You have investors. They all have networks. Ensemblex is a part of those networks and we’re here to help.

  • Writer's pictureMike Armstrong


Shawn and I started Ensemblex just over three years ago to help companies start, scale, and improve their credit products. Today, companies around the world partner with Ensemblex to develop game-changing products, expand into new markets, and safely lend billions of dollars to customers. It’s been quite a journey.

One thing is clear: Interest in hands-on, expert credit advice goes well beyond what Shawn and I expected when we were starting out. With that in mind, we’ve been making investments to support our clients’ continuing growth, including expanding the team, adding new capabilities, and more.

As we think about the future of credit, there are really three themes that have influenced the investments we are making.

1. Rapid innovation in emerging credit markets.

A growing number of our clients operate in markets where credit isn’t widely available, including Brazil, Mexico, and India.

Low credit availability is a double-edged sword. On one hand, you’re marketing to a population that doesn’t already have a wallet full of cards. On the other hand, you don’t have access to a lot of the infrastructure and guardrails that exist in more developed credit markets.

In other words, acquisition costs are lower, but the cost of learning is potentially much, much higher. That’s where Ensemblex has been able to help. We’ve deployed some of the same tools that Shawn and I used to manage sophisticated, large-scale credit operations at CapitalOne.

These tools, along with the establishment of a robust analytic culture, have allowed startups to accelerate their growth strategies — adding new customers, new revenue, and new profits — while reducing the risk and learning costs associated with new credit markets.

2. The continuous evolution of credit products.

How many credit products are there? It depends on how you count, I guess. When you look at the product landscape through the eyes of a credit expert, there aren’t that many — cards, lines, and loans of varying types and lengths. That’s about it.

But when you read the news, you see something completely different. Founders are combining old products with new technologies, targeting poorly served segments, and building hyper-growth companies. When I look at the product landscape this way — through the eyes of founders and innovators — the opportunities are endless.

That’s why we’ve expanded the team here at Ensemblex to include experts who can help founders with everything they need to get a new product off the ground, including setting up bank partnerships, selecting technology partners, acquiring and servicing customers, and much more.

Startups use our experience to get to market faster, along with all of the tools, services, and experience they’ll need to build a thriving business.

3. The next generation of talent.

All of this innovation has attracted a new generation of talent — sharp, skilled professionals who are interested in building the future of credit.

Over the years, we’ve had the pleasure of working with brilliant leaders and founders around the world, who are taking on massive opportunities in their local markets. Partnering with these founders has been one of the highlights of my career.

At Ensemblex, we are increasingly working with clients to put together teams and talent to build novel products, profitable businesses, and engines for economic growth. If you’re looking for a partner to develop a new idea or enter a new market, we’d love to talk. Likewise, if you’re looking for something new and have the right experience, we’d love the opportunity to connect you with one of our portfolio companies.

I should note here that Ensemblex is hiring as well. We are always on the lookout for proven experts who would like the opportunity to develop a global view of credit, partnering with founders and innovators to launch new products and develop new businesses.

We’ll have much more to share with you all in the coming months. Until then, if there’s anything that we can help you with, please don’t hesitate to reach out to Shawn or myself.


bottom of page