ClearScore is a financial technology company in the United Kingdom. Founded in 2014, the company’s mission is to help people take control of their financial future by providing insight, guidance, and a unique customer experience. ClearScore’s solution provides consumers with free access to their credit score, deep analytics and reporting, money management and credit improvement guidance and coaching articles, and connection with credit providers.
The company is headquartered in London and currently serving over 12 million consumers.
ClearScore began as a tech startup in 2014. The marketplace responded and the business grew quickly. For the first three and a half years, ClearScore was primarily focused on velocity—deploying services quickly—and scaling operations. This rapid growth required the company to migrate from on-premise infrastructure, first to IBM Cloud and then to Amazon Web Services (AWS), to leverage the scalability of cloud.
But soon after the move to AWS, business grew beyond what the application and data platform could handle. As a result, led by the Site Reliability Engineering (SRE) team, ClearScore built a new application and data platform on AWS to handle the increased volume. This effort required them to run two parallel infrastructures—the legacy infrastructure and the next-generation infrastructure—for one year.
Operating two infrastructures for that amount of time was expensive, and senior executives made it a priority to lower the cost. Once the migration was finished, the SRE team shut down the legacy platforms as quickly as possible.
In 2018, the SRE team noticed that cost increases were not in line with the growth in the company’s user base and the subsequent revenues the company was earning from its users. Up until this time, the team relied on manual controls to manage AWS spend. But it became obvious now that manual controls were not effective. “In 2018, we started looking seriously at our cloud spend,” said Barry Laffoy, Team Leader of the SRE team. “Up until that point, we were in the phase of grow at all costs—startup then scale-up. It was always a case of making whatever decision was necessary to prioritize the first two objectives of the SRE team, which were uptime and velocity. And that went for pretty much every aspect of the business. But our company had reached a new phase in 2018, and we had to look seriously at our cloud spend, because it had been growing without any restrictions.”
ClearScore needed to control and optimize cloud costs, and to do that, they needed a better way to monitor and manage their cloud environments. So, the team decided to check out Apptio Cloudability. They were immediately impressed with Cloudability’s functionality, specifically its tools for resource optimization analysis and reserved instance portfolio management. Andre van den Heever, Principal Security Engineer at ClearScore, said, “Cloudability is feature rich in the detailed analysis of resources across multiple factors including CPU, Memory, Amazon Elastic Block Store (Amazon EBS), and Instance Types. Cloudability’s Usage Optimization Recommendation provides us with easy to follow tips, andvmore importantly, risk-based assessment and scoring on cost optimizations for our cloud spend.”
Cloudability was purchased in late 2018. And according to Andre, the implementation went smoothly. “It took very little effort to get up and running,” he said.
At ClearScore, AWS is the second highest operating expense in the company. In June 2019, the company realized it was not on track for the year’s budget for AWS spend. To address the issue, the SRE team used Cloudability to identify idling and unused resources and turned them off. “We found a lot of infrastructure that was no longer being used,” said Joe Carpenter, Senior Site Reliability Engineer. “For example, we found projects that had been completed but no one made sure the underlying resources were shut down after the project ended.”
Turning off unused resources lowered costs some, but the SRE team wanted to drive costs even lower. That’s when they turned their attention to how they could best leverage reserved instances and spot instances.
ClearScore’s biggest spend is Amazon Elastic Compute Cloud (Amazon EC2). In the beginning, the company had thousands of instances running on Amazon EC2 on-demand. “As a first pass, we took no up-front payment options and went for full coverage,” said Joe. “We purchased reservations for our entire Amazon EC2 fleet.”
But it wasn’t long before the team realized they could do better purchasing spot instances instead. “Since then, we have been engaged in this cat-and-mouse game of buying and selling reservations on the Amazon EC2 Reserved Instance Marketplace and continually choosing the number of spot instances,” said Joe.
Cloudability’s Reserve Instance (RI) Planning feature has played a big role in helping to make that happen. “We look at dashboards of reservations, utilizations, coverage, spot hours versus on-demand hours, and types of workloads,” said Joe. “Having all that data there…and graphs that light up when you open the page…has been essential to our effort.”
Through better visibility and savvy use of reserve and spot instances, ClearScore has cut their cloud spend nearly in half. “Our costs were on the increase until July 2019,” said Joe. In addition to reducing costs, Cloudability has helped the SRE team improve communications with the rest of the company.
ClearScore is a dynamic business. New needs surface all the time. As a result, new cloud instances and development environments pop up quickly and then go away. To operate the way they do, the engineering teams need a lot of power and autonomy. But to keep cost under control, the SRE team needs to keep tabs on what is going on. By using the containers feature of Cloudability, the SRE team is able to monitor activity and keep the teams in check. “One of the things about Cloudability that keeps saving our bacon is the alerting,” said Joe. “If our engineering teams do something that is going to be expensive, we know it straightaway and can let them know.”
Cloudability has helped improve communication with the finance team, too. “Finance people want numbers,” said Barry. Armed with the financial transparency provided by Cloudability, the team is able to have data-driven discussions with their finance team and make more informed decisions on investments.
Cloudability has enabled ClearScore to lower its cloud costs significantly, but that is only the beginning. Next, the SRE team plans to use Cloudability to help them optimize AWS managed services and other shared resources, so they can showback costs to other parts of the business and further improve the way they manage the business.