eXp Realty Moves to Enterprise Microservices Architecture

When does it make sense to move to microservice architecture? This is a pressing question many organizations have as they evaluate their existing software technology architecture in a bid to become more agile.

As an industry, real estate is local. Every state does the same type of business but in different ways. For eXp Realty, the sixth largest real estate broker in the US and first cloud-based real estate firm in the world, this meant updating its mission-critical business application, eXp Enterprise.

eXp agents, contractors and staff use eXp Enterprise, a cloud-based application developed on the Mendix platform, to conduct core business activities like onboarding and deal registration. eXp initially built the app on a monolithic architecture.

To remain competitive, however, Steve Ledwith, VP of Engineering at eXp Realty, identified the need to re-architect the application from a monolithic architecture to microservices. This re-architecture was a massive undertaking for the company, considering eXp does business in all 50 states; but by establishing a vision, getting buy-in from all stakeholders, and enabling its developers, eXp moved this business-critical application to microservices.

The Need for an Enterprise Level Change

Over the years, eXp has grown its business from 2,500 agents to 18,000. As such, so did eXp Enterprise’s features, user base, and hosting requirements, resulting in a heavy technical debt that Ledwith needed to address.

The Right Way or Right Now?

At the time of launch, the eXp Enterprise had 30 features. In fewer than two years, eXp added 1,700 features to the application with 2,000 users using the app daily. Managing these features with a userbase of that size called for what Ledwith dubbed “right now” approaches or ad-hoc developments to the app in order to meet immediate business needs. Ledwith sums up this problem best, saying:

“So, instead of doing it the right way, we did it right now and there was a lot of lack of input,” Ledwith says.

For a database domain model, the Mendix-way prescribes a clean model – one or two entities and modules. But eXp’s agent database domain model that carries information about the agents was tightly coupled.

Let’s look at some numbers:

• In one area of the database, eXp had over 6,000 connections.
• The agent entity had more than 140 attributes. The best practice is 20.
• The entity had seven calculated attributes. So anytime a user loaded agent information, the app calculated about 126,000 fields for 18,000 agents.
• The process involved in a real estate transaction is diverse, ranging from getting documents and contracts signed, to compliance tracking, generating audit files, and more. With five to six minutes to process a real estate transaction, across roughly 100 transactions per day, eXp had people working all day from Monday to Sunday to just process these transactions.

Hosting Challenges: Here Comes Thanos

eXp Enterprise called for heavy cloud resources. eXp initially hosted the application on the largest container available on Mendix Cloud v3. With 2,000 users per day, and a myriad of features and functionality, they soon outgrew its capacity.

To resolve this, eXp migrated the app to Magneto, the largest container on Mendix Cloud v4. At 90% of the cloud resource usage, Magneto seemed to work. But eXp quickly exhausted this container, too. Soon thereafter, Mendix created Thanos, a cloud container Ledwith used to host eXp Enterprise and its enterprise microservices architecture.

Pushing the Cloud Foundry Limits

When Mendix upgraded Cloud Foundry for the Mendix platform, it went smoothly for all the Mendix users, except for eXp Realty, whose application broke Cloud Foundry.

eXp would spin up the application, and because of the number of features, connections, and users the app would fail. As Ledwith says, “Our application would spin up, we’d get all our users on it and then it would crash. And we’d restart it and they would do it again about every two hours.”

According to Ledwith, the root cause analysis he received from the Mendix Cloud team stated that 1,000 threads were way beyond what was considered reasonable on the Mendix Platform, so it wasn’t caught in testing. eXp Enterprise had, at its peak, 1,250 threads running all day every day. Ledwith needed to reconsider refactoring the application to bring the number of threads down.

From “Right Now” to “Right Way” — Microservices

An engineering team of roughly 80 developers – 30 of which are full-time professional Mendix developers—supported the massive growth in eXp Enterprise. While eXp could have continued to build their team to support the application, they soon realized this process wasn’t scalable. eXp also tried other options like buying more cloud containers, refactoring, and trying the hub and spoke architecture to improve the situation, but neither of them proved to be a long-term solution.

Management gave Ledwith’s team two weeks to research and present up with a viable solution. They settled on microservices.

Right Way #1 | Get Buy-In

As Ledwith and his team were changing the technology infrastructure, it was important for them to understand the business pain-points and to get buy-in from not just the tech team or management but from teams including engineering, products, and subject matter experts.

In fact, Ledwith and his team spent almost two months on socializing the microservices idea with different business teams. The business teams’ input was vital for Ledwith to ensure the changes focus on what business truly needs and delivers value.

Right Way #2 | A Three-Pronged Approach

Ledwith came up with a three-headed strategy to implement microservices.

Strategic goals

• Help the business scale to 100,000 agents and enable innovation.
• Support thousands of third-party integrations for customer, listing, and financial data that a real estate company needs to help its agents sell.

Architectural Principles

• Help guide with architectural governance and reduce complexity – use the right tools for the right purpose for rapid development and feedback

Design and Delivery

• Limit shared code so different teams can make changes to their part of the apps without affecting other teams. If something was common to other teams, they used the internal app store to share the common module.
• Follow the CI/CD framework.

Right Way #3 | The Innovations Team

eXp enhanced the agile feature team, made up of a product manager, developers, and QA specialists, creating an innovation team that included members from the business. The business team was there to look at the bigger picture – how to deliver value, increase revenue, reduce churn, and look for opportunities to automate.

The innovation team owned the services they developed. Now, when the team goes to production, they don’t have a handoff to an operational team or a maintenance team, but rather, they own it throughout production and maintenance.

Right Way #4 | The Enterprise Microservices Mindset

Setting the right mindset was the most important ingredient for success throughout the re-architecture process, according to Ledwith. The delivery of enterprise-class software with real business value requires planning, talent, input, logic, feedback, and careful execution with a constant focus on what the business truly needs.