Bluestem Brands Migrates from Monolith to Microservices Efficiently with NGINX Plus

Bluestem Brands Migrates from Monolith to Microservices Efficiently with NGINX Plus

Enabling Improved Performance and Continuous Delivery

Situation

Bluestem Brands, Inc. is the parent to 16 fast-growing e-commerce retail brands. The company offers a unique mix of retail and payment options for a diverse set of customers with a wide range of financial needs. Bluestem Brands runs a separate site for each of its brands where customers shop for apparel, shoes, gifts, home accessories, and more.

Three of the Bluestem Brands sites – Fingerhut, Gettington, and Paycheck Direct – extend credit to shoppers, letting them pay for purchases over time. Shoppers log in, check the balances on their accounts, and shop. Until recently, Bluestem operated these three sites on a legacy architecture with a monolithic application for each brand. The machines that handled production traffic for the sites hosted both an Apache web server and a Tomcat application server. During peak shopping periods, such as the holiday season, the web servers quickly hit capacity. Even just a couple hundred connections on each machine caused it to stop accepting connections. Clients would retry their requests on another machine, which soon became overloaded and stopped accepting connections, and so on – until all machines were affected.

“A couple of holiday seasons ago, we were sitting in a war room. We could see our old boxes beginning to slow down. One box would get to a couple hundred connections and then would stop accepting connections, then the next box, and the next one, and so on. We could see them all slowly stop handling connections. We did a rolling wave of restarts just to keep those boxes going. Before the holidays, we had tried all sorts of different tricks to figure this out and prevent it from happening, but it didn’t work. After that year, we decided to do a re-architecture to improve performance overall, and during the holidays,” explains Dan Chamberlain, principal architect for the e-commerce team at Bluestem Brands.

Near the same time, the networking team at Bluestem Brands was on a journey to replace its hardware load-balancing technology, which wasn’t distributing traffic evenly. Sometimes the load balancer would send more traffic to one box and overwhelm it. Bluestem Brands tried a variety of load-balancing methods and techniques, but just couldn’t get the performance they needed.

It was clear that as Bluestem Brands’ business grew, the architecture that worked four years ago wasn’t working anymore. Committed to outstanding customer service and continuous improvement, Bluestem Brands needed to evolve the application architecture and underlying infrastructure to serve up high quality digital experiences for its customers.

Solution

Bluestem Brands decided to transform its application from a monolith to a modern, microservices architecture. The plan is for the new website frontend to include the HTML, CSS, JavaScript, and some domain logic for building up pages, while all the rest of the components become services. At the infrastructure level, Bluestem Brands replaced Apache and its legacy load-balancing technology to get the performance improvements they need. Bluestem Brands considered HAProxy and NGINX Plus as possible replacements. However, in addition to load balancing, the team also wanted to serve static content, which quickly narrowed the choice down to NGINX Plus.

“I didn’t want just a load balancer; I also wanted to serve static content, so the content caching feature was appealing to me. That narrowed it down to NGINX Plus right away. We’re iterative here, so we give things a try and see if it works. Feedback from the developers was positive, and I felt confident that because a lot of the Internet relies on NGINX, it would be a solid solution for us, too. So far, NGINX Plus has been working for us and has been a good decision.”

Bluestem Brands deployed NGINX Plus as a virtualized, on-premises load balancing and caching solution. The e-commerce team deployed two instances of NGINX Plus in an active-passive high availability pair for each brand, six total. Each brand uses the same infrastructure platform, but the runtimes are different – so the applications themselves are walled off. In the NGINX Plus tier, Bluestem Brands is both serving static content and proxying traffic to its app servers, which run Apache Tomcat for Java-based applications.

Results

> Simple and Efficient

Because NGINX Plus is a complete application delivery platform, it combines capabilities such as content caching, load balancing, and web serving in one easy to use software package. This allowed Bluestem Brands to deploy just one tool – NGINX Plus – to replace its previous load balancing and web serving solutions. And adding content caching was easy to do because it’s also included in the NGINX Plus feature set for no extra cost.

In addition, NGINX Plus can handle a large amount of traffic but is still lightweight. By moving to a simple architecture with load balancing and web serving out of the NGINX Plus tier, Bluestem Brands can efficiently handle all of the web traffic for each brand with just the active-passive NGINX Plus pair, without the need to stand up more and more web servers to handle the load.

> Worry-Free Load Balancing

With NGINX Plus, Bluestem Brands sees traffic more evenly distributed. With the old load balancing solution, numbers always looked off and it wasn’t clear what was causing the issue. Sometimes one machine would get 70 or 80 more connections than another, whereas with NGINX Plus the variance is just 1 or 2 connections. This provides confidence that the solution is working correctly, which eases the tension at the holidays and allows Bluestem Brands to move on to optimizing other parts of its business.

“At the holidays, NGINX Plus ran and we didn’t have to worry about it. During load testing before the holidays, we had tuned NGINX Plus and everything just worked. During the holidays, it was a very relaxed time for us with that part of the infrastructure. With NGINX Plus, we didn’t have any tension around whether or not our load balancer would perform,” notes Chamberlain.

Because NGINX Plus is working as required, the teams at Bluestem Brands can focus on other things. Chamberlain explains, “With NGINX Plus, we no longer argue about it if our load balancer is actually balancing the load. We don’t have to question the tool anymore, which is big for us. Here, everyone from our developers, to directors, and up to our CTO is very technical, so we like to pick and poke at problems until we figure them out. Once we turned on NGINX Plus, it just worked and we didn’t have to talk about load balancing anymore. You shouldn’t have to worry about if whether or not your load balancer is balancing requests. It’s a good thing to move on from. With NGINX Plus, we can move on from worrying about load balancing to solving other problems that are important for our business.”

> Content Easily Stays In Sync

Serving the right content to the right customer at the right time is important to Bluestem Brands, especially for the Fingerhut brand.

“Fingerhut customers get credit directly from us and then buy our products. A typical customer journey is that users log in, see how much credit they have open to buy products, and then go shopping. If the marketing content isn’t synced, we could be sending the wrong promotional content to our customers. Content needs to be in sync so that we’re providing the right content and best service to our customers,” explains Chamberlain.

Before NGINX Plus, Bluestem Brands was relying on rsync to keep data synced on all their machines. Every hour an rsync job had to be kicked off on each server to sync their new data. Even so, data could get out of sync pretty easily. Chamberlain notes, “The data would sometimes be out of sync and no one would know why, and it felt like we’d have to do something magical to make the data sync again.”

Now, by using NGINX Plus to cache static content, such as the promotional material appropriate for a given set of customers, Bluestem Brands is finding it much easier to keep data in sync. The simple architecture helps, too – the cached content is served from just the two NGINX Plus boxes in an active-passive pair.

> Great Tool for DevOps and Developers

NGINX Plus fits in well with Bluestem Brands’ DevOps processes. DevOps engineers set up the NGINX Plus instances so that changes to the configuration can be deployed via Jenkins. Developers can adjust the configuration on their own, as needed, and because deployments are automated, the changes can be live in production within seconds.

“Our whole deployment pipeline for NGINX Plus is automated. The developer pulls down a file from GitHub, makes changes, pushes it back up into GitHub and Jenkins deploys it. The process is fast and completely automated. It’s a great way to deploy changes,” says Chamberlain.

In addition, Bluestem Brands is already doing blue-green deployments and is moving towards doing canary releases with NGINX Plus. Chamberlain explains, “With the way we have NGINX Plus set up, it makes it easy for our developers to adopt a canary release model. NGINX Plus easily integrates with Jenkins, which means we have a simple push button to enable continuous delivery and to help us evolve our applications for our customers.”

> Enables a Smooth Migration

Bluestem Brands is using NGINX Plus to enable a smooth migration to its new application architecture. Aligned with its iterative workstyle, the e-commerce team at Bluestem Brands is migrating a set of pages at a time from the old system to the new system instead of doing a “big bang” migration, and is using the request routing features offered by NGINX Plus to direct requests to the appropriate system.

For example, if a user proceeds to checkout, NGINX Plus routes that request to the legacy system. But when a user uses the site’s search function, NGINX Plus routes that user to the new microservices that implement search.

The company uses location blocks in its NGINX Plus configuration to implement the migration from the legacy architecture to the new microservices. When testing shows that a new microservice works correctly under load, Bluestem Brands redirects all traffic from the legacy app to the microservice, and removes the old route.

“A migration is always a big, daunting task. With NGINX Plus, things are going well, and we’re making good progress. We expect to be fully migrated by mid-year,” says Chamberlain.

The migration project is a team effort at Bluestem Brands. A lot people contributed to making the migration to NGINX Plus successful, including Brian Brown, Kevan Ahlquist, and Spencer Peterson, just to name a few.

The Future

After the first three sites have fully migrated to NGINX Plus, Chamberlain expects that the other 13 sites at Bluestem Brands will become part of the same architecture.

“So far I’ve loved the performance of NGINX Plus. It’s easy to use, great for our environment and workflows, and I want to see us use it even more,” notes Chamberlain.

About Bluestem Group Inc.

Bluestem Group Inc. is a holding company whose businesses include Bluestem Brands, a multi-brand, online retailer of a broad selection of name-brand and private label general merchandise serving low- to middle-income consumers through 16 retail brands that include: Appleseed’s, Bedford Fair, Blair, Draper’s & Damon’s, Fingerhut, Gettington, Gold Violin, Haband, LinenSource, Norm Thompson, Old Pueblo Traders, PayCheck Direct, Sahalie, Solutions, Tog Shop, and Wintersilks. Complementing each brand is a large selection of merchandise with payment options that provide customers with the flexibility of paying over time. Bluestem Group is headquartered in Eden Prairie, MN. For additional information visit the Bluestem Group website at www.bluestem.com.

The post Bluestem Brands Migrates from Monolith to Microservices Efficiently with NGINX Plus appeared first on NGINX.

Source: Bluestem Brands Migrates from Monolith to Microservices Efficiently with NGINX Plus