6 Questions DBAs Need to Consider when Preparing for Major Campaigns (Part II)
In the previous post, we have talked about how to make sure there is no data loss when failures happen and how to ensure system stability during major shopping promotion campaigns in the holiday season.
In this post, Ted Bai, Senior Solution Architect at OceanBase, and Xingxing Xi, Technical Consultant at Yunhe Enmo (Beijing), went on to talk about database performance tuning so that customers have excellent shopping experiences. They also talked about how to optimize costs so that companies can provide excellent shopping experiences with lower TCO (Total Cost of Ownership).
3. How do we optimize the database performance in massive online promotions?
In practice, a simple but reckless approach to the database performance improvement is to pile up hardware, such as adding memory or upgrading the server.
Generally, this approach cannot meet the expectation due to the negative response from the business side in spite of the high server specifications. Again, efficient database maintenance requires solid basic skills of O&M engineers.
As a matter of fact, the optimization of an SQL statement can sometimes improve the performance by several or even tens of thousands of times, while simply increasing the number of servers may not do the trick. When it comes to database optimization, we must get down to the root.
Oracle Database comes with a range of auxiliary tools, such as ASH report, performance data collection, and monitoring tools based on the basic database statistics. These auxiliary tools, in the hand of skilled O&M engineers, can quickly locate the root causes of database errors, allowing targeted database optimization rather than simply increasing server specifications.
For example, before a massive online promotion, we can run a stress test to assess the system performance in special business scenarios based on historical data and identify the system bottlenecks such as insufficient memory space, insufficient CPU capacity, low storage performance, and poorly-written SQL statements.
Therefore, using a database smartly is also very important to achieve satisfactory performance. Regardless of whether a database is commercial, open source, or distributed, we can bring out its full potential if we know its features well.
4. How does OceanBase Database balance performance and costs when faced with a sharp increase in workload?
The capacity of a database must be multiplied to withstand massive online promotions. Before migrating to OceanBase Database, we would spend months in rehearsing resource allocation and scaling. This not only was manpower-consuming but also caused risks due to horizontal or vertical sharding. These problems are solved in OceanBase Database. For example, we can migrate the data from some on-premises IDCs to cloud IDCs in a short time ahead of the promotions and use the resources of the cloud IDCs to cope with the traffic surge.
Combined with stateless routing, the replica balancing process of OceanBase Database is nearly transparent to applications. For example, when you want to switch from IDC A to IDC B for larger capacity, all you need to do as a DBA is to launch the migration task on the data migration platform. You do not need to worry about the consistency verification or incremental data. After the migration is completed, OBProxy, the frontend component of OceanBase Database, gradually routes the traffic to nodes in IDC B. The whole process is performed in the background without interrupting the applications.
OceanBase Database also provides OceanBase Control Platform (OCP) to increase its batch change capability to a new level. A DBA only needs to come up with a scaling strategy and trigger the scaling, and OCP will automatically and gradually scale the target IDC in or out. Thanks to the intelligent design and various tools of OceanBase Database, the O&M of all databases of Ant Group is managed by less than 20 DBAs, even during massive online promotions.
OceanBase Database provides flexible automatic scaling solutions, which allow small and medium-sized enterprises to scale in their databases and keep the costs low after an online promotion.
Enterprises may be concerned about how to improve efficiency and reduce costs after their databases grow to a certain size. Given its massive business territory, Ant Group also faces many challenges in cost reduction.
When it comes to computing resources, improving the resource utilization is a big challenge. Based on its multitenant architecture, OceanBase Database provides resource pools to isolate computing and storage resources of a tenant from those of others in a large cluster.
As for storage resources, OceanBase Database achieves lower storage costs in three ways.
First, OceanBase Database adopts an LSM-tree-based storage engine, which stores data sequentially to ensure that each data block is compact. Every time when the incremental data is merged during a major compaction, the original modified blocks are rewritten all over again, which naturally avoids data voids.
Second, the storage engine of OceanBase uses encoding to compress redundant data in rows and columns. Check the technical details here.
Third, the LSM-tree processes the active incremental data in memory and the baseline data in the disk is not updated by the middleware. Therefore, we can apply algorithms that compress the baseline data to reach a high compression ratio.
These three measures can help compress a 10TB dataset to 3TB or less. In addition, OceanBase Database provides log replicas, which are used only for voting to ensure data consistency when transactions are committed. While maintaining the integrity and validity of a Paxos-based cluster, log replicas can be hosted on affordable servers, which is another way of cost optimization.
The business systems of Ant Group run on an extensive database infrastructure. To keep improving the overall resource utilization, we have made every effort to strengthen the infrastructure, in addition to architecture optimization. For example, we use different models of servers for mixed database deployment in containers based on the multitenancy of OceanBase Database.
Please leave a comment below and let us know what you think can be done to optimize distributed database performance and reduce TCO. There will be one more post coming soon. Stay tuned!
