In recent years, dramatic improvements have been made in computer hardware. In particular, the number of cores on a chip has been growing exponentially, enabling an ever-increasing number of processes to execute in parallel. However, these processors are far from realizing their potential performance when dealing with data-intensive applications such as database management systems (DBMSs). Traditional DBMSs, which are dedicated to improving database performance through I/O optimization, fail to utilize processor resources efficiently, and the increasing processor-memory gap is slowing down the whole system. Moreover, the increasingly powerful concurrent processing ability of multicore platforms is also stressing the DBMSs. An increasing number of concurrent database processes share resources both at the hardware (caches and memory) levels, and any inefficient resource sharing will create new bottlenecks in DBMSs.
Therefore, I analyzed the possibility of optimizing the performance of DBMS on modern multicore platforms and proposed two approaches to break the bottleneck in database engines and to provide cache-efficient query scheduling for concurrent queries for different database applications on modern multicore platforms. All of these proposals are pure middleware, which avoid any modification to existing DBMSs, thereby making them more practical. This is important because the source code for existing DBMSs is large and complex, making it very expensive to modify.
The CARIC-DA middleware can achieve higher performance for DBMSs with OLTP applications on multicore processors by reducing cache misses with a new cache-conscious dispatcher for concurrent OLTP queries. CARIC-DA enables different processor cores to access different subsets by dispatching queries to proper processor cores. In this way, CARIC-DA is expected to achieve better performance via a higher cache hit rate for each core’s private cache. The middleware of PM-DB solved the bottlenecks in the buffer pool manager functions in existing database engines on a modern multicore platform for the mixed workloads as that modeled in the TPC-W benchmark. PM-DB overcomes the bottleneck by introducing a partition-based multi-instance database system on a single multicore platform. Both of the proposals are implemented with unmodified existing Linux and PostgreSQL environments, and evaluated on modern multicore AMD and Intel multicore platforms.
PhD program is a long process of accumulation and the process often takes several years. Therefore, PhD students need a strong will and an optimistic attitude. I believe that all the PhD students in their research process have encountered all kinds of difficulties. For example, the experimental results are not satisfactory, and even got the exact opposite of the expected results. Or suddenly found himself in the wrong method, so that all the work should start over. These are very normal, so do not give up because of these difficulties, don’t be discouraged. Establish a firm target, and maintain an optimistic attitude. A man who has a settled purpose will surely succeed and a willful man always has way. Never give up.