How do you reduce the AWS operating cost that swelled 10 times?The back side of the "cost reduction strategy" that the user has rapidly increased (page 1/2)

"NOTE" is a content distribution service that allows you to post sentences and illustrations.Since the Corona evil, the number of users has increased rapidly due to the demand for nesting, and in 2020 the number of monthly active users increased more than the same period of the previous year.However, at the same time, the amount of traffic increased, and the system department of Note, the operator, was forced to respond.In particular, the soaring cost of using cloud services was a major problem.

All systems that support Note services are constructed and operated on AWS (Amazon Web Services) cloud infrastructure, and the cost of traffic has increased to about 10 times.

If the number of traffic increases as it is, the cost can be a bit of a profit.Therefore, the "SRE team", which is responsible for the operation and quality control of the system, will be mainly working on the use of AWS usage costs and reducing them.

"Originally, some representatives checked the cost of use through the AWS management console, but could only get information in a limited range."This is the company's SRE team Subaru Nakamura.He said that he was not very interested in the company.

"So, we decided to build a mechanism that allows all employees to visualize more detailed information and to refer to it."

Using the AWS usage status "AWS Cost USage and Report" to report the usage status of AWS, we have obtained more detailed information on resource usage and cost as CSV data.This is stored in the online storage "Amazon S3" (Amazon Simple Storage Service), and then throws a query through the data analysis PaaS "Amazon Athena", and summarizes and analyzes.A mechanism to display the result on the dashboard tool "Redash" was built.

However, it is difficult to attract interest if you can not easily access the location even if you can see the data.Mr. Nakamura thought so that all employees could refer to this information from Slack.

With the introduction of cost visualization mechanisms, it has gradually seen which service or resources are incurred for a lot of costs.In particular, the cost around the log was outstanding.

"AWS is incurred just by storing log data, so it is necessary to set services to prevent unnecessary logs from occurring, or properly delete and overwrite old logs.However, some services have been dripping for a long time without such considerations. "(Mr. Nakamura)

When the service scale was not so large, these costs were not noticeable, but as the traffic increased, the amount of logs spit by the service increased at a stretch, and the cost of transferring and storing them jumped at once. Oops.

There were many old snapshot data that did not know when it was taken, and it cost quite a bit for storing them.

Therefore, these data was identified and deleted sequentially, and the service settings, which were the factors that were useless cost generated, including logs, were reviewed one by one, and the cost was optimized.

Along with this, a mechanism for analyzing costs with more detailed granularity has been added.The most important thing was to visualize the cost of each team.

According to SRE team Kazuya Kato, all teams have shared a single AWS account.However, this method could not accurately grasp the cost of each team, and it was necessary to recreate resources to separate accounts.

Therefore, when creating AWS resources, we requested the team in the company to always have a tag to identify the team.By setting these tags as "cost distribution tags", the cost of each team can be visualized.

According to Mr. Kato, about 60-70 % of all resources have already been tagged.In the future, in order to further increase the proportion, we are also considering ruling the tagging of resources.

However, some AWS resources cannot be tagged.For this reason, we are also working on giving AWS accounts individually for each team in parallel with the tag.

"In order to separate accounts, the resources that have already been created with the shared account have to be re -created, and it takes a lot of time, but in addition to cost visualization, many benefits in terms of security and data protection.Because it can be enjoyed, we are currently proceeding with account separation in parallel with the tag "(Mr. Kato).

Copyright © ITmedia, Inc.All Rights Reserved.

To read more, you need to agree to the terms of use and register "ITMEDIA NEWS Anchor Desk Magazine".