Online Schema Evolution is Almost Free for Snapshot Databases

2025-05-02 0 0 1.56MB 14 页 10玖币

侵权投诉

Online Schema Evolution is (Almost) Free for Snapshot Databases

To appear at VLDB 2023

Tianxun Hu

Simon Fraser University

tha110@sfu.ca

Tianzheng Wang

Simon Fraser University

tzwang@sfu.ca

Qingqing Zhou

Tencent Inc.

hewanzhou@tencent.com

ABSTRACT

Modern database applications often change their schemas to keep

up with the changing requirements. However, support for online

and transactional schema evolution remains challenging in exist-

ing database systems. Specically, prior work often takes ad hoc

approaches to schema evolution with “patches” applied to existing

systems, leading to many corner cases and often incomplete func-

tionality. Applications therefore often have to carefully schedule

downtimes for schema changes, sacricing availability.

This paper presents Tesseract, a new approach to online and

transactional schema evolution without the aforementioned draw-

backs. We design Tesseract based on a key observation: in widely

used multi-versioned database systems, schema evolution can be

modeled as data modication operations that change the entire

table, i.e., data-denition-as-modication (DDaM). This allows us

to support schema almost “for free” by leveraging the concurrency

control protocol. By simple tweaks to existing snapshot isolation

protocols, on a 40-core server we show that under a variety of

workloads, Tesseract is able to provide online, transactional schema

evolution without service downtime, and retain high application

performance when schema evolution is in progress.

1 INTRODUCTION

Multi-versioned concurrency control (MVCC) has been widely

adopted by open-source and commercial systems to provide high

performance for various database applications. The main benet is

that under MVCC, readers may be allowed to proceed even if there

are concurrent and conicting writers [

]. Almost all the main-

stream relational database systems—For example, MySQL [

], Post-

greSQL [

], SQL Server [

] and Oracle [

]—implement MVCC to

oer snapshot isolation (SI) or repeatable read as the default or rec-

ommended isolation level. Many in-memory MVCC protocols have

also been proposed to well leverage the abundant parallelism and

memory available in modern servers [7, 11, 12, 22, 29, 31, 39, 55].

In addition to high performance for forward processing, modern

database applications also require graceful handling of schema evo-

lution to satisfy new requirements, e.g., adding columns, creating

tables from existing data and changing constraints. This is typi-

cally done with data denition language (DDL) statements such as

CREATE TABLE...AS

CREATE INDEX

and

ALTER TABLE

. Internally,

the DBMS handles DDL statements by updating database metadata

to store the new schema, and examining (e.g., against newly added

constraints) and migrating existing table data to conform to the new

schema. Data verication and migration during schema evolution

can incur massive data movement that may block concurrent data

manipulation language (DML) statements. It was common for early

systems to necessitate service downtime or maintenance windows

for schema evolution [

], often during “quiet hours.” With con-

tinuous deployment and integration becoming a norm, however,

schema evolution can happen quite frequently (e.g., several times

a week), requiring reduced or no service downtime and robust er-

ror handling with little DBA intervention [

]. This in turn

requires DDL statements be executed in a way that is (1) online

without blocking concurrent data accesses and (2) transactional

such that in case a DDL statement fails (e.g., attempting to convert

VARCHAR

that contain “illegal” characters to

INT

) the operation can

be safely rolled back to leave the database in a consistent state.

1.1

Ad hoc Schema Evolution in MVCC Systems

There have been some attempts to support online and transactional

schema evolution in single- and multi-versioned systems [

35, 38, 43, 47], but they still fall short in two aspects.

First, existing solutions expose many special, corner cases that

need to be carefully handled by OLTP engine and application de-

velopers. For example, MySQL does not oer transactional DDL,

not all the operations are online/atomic, and concurrent DML is

often forbidden [

]. In case a transaction includes any DDL

statements, the transaction will be silently committed, posing cor-

rectness risks for applications [

]. Some recent work [

] allows

DDL operations to be completed lazily without migrating data right

away (by doing it in the background), but is limited to compati-

ble schema changes; in case the change introduces incompatibility

(e.g., the previous data type conversion case), the user then has to

examine the column content in advance to decide whether the DDL

statement should be issued as once the metadata (schema) change is

done, it is dicult or impossible to be rolled back as a newer version

of the application may have already started to use the new schema,

leaving certain data being dropped or unavailable. Handling these

special cases signicantly complicates system design. Moreover,

the special cases may change as the DBMS itself is continuously

upgraded, further complicating application design.

Second, past solutions often strongly rely on specic DBMS

features, some of which are in essence database applications them-

selves. This may limit the functionality and performance of schema

evolution. For example, some systems rely on views [

] and trig-

gers [

] which present performance bottlenecks as they use table-

level locking that forbids concurrent updates. If a system does not

implement the required features or deviates from the required be-

haviors, the eciency of DDL operations can be negatively aected.

Overall, we observe that the key reason for these issues is DDL

support is often an after-thought instead of a rst-class citizen that

is considered at database engine design time. In other words, they

are ad hoc solutions with “patches” applied gradually to the DBMS.

Also, there is relatively less attention from academia and (in part as

a result) real systems often provide inadequate support, making ap-

plication developers handle DDL operations “in the trenches” (e.g.,

with external third-party tools [

]) and often describe DDL opera-

tions as “dicey” and “dangerous” [

]. Consequently, application

developers often try their best to avoid DDL operations, limiting

application functionality and end-user experience.

1.2 Tesseract: Data-Denition-as-Manipulation

This paper presents Tesseract, a new approach to non-blocking

and transactional schema evolution in MVCC systems without the

aforementioned limitations.

Tesseract provides bake-in support for online and transactional

schema evolution by directly adapting the concurrency control (CC)

protocol in MVCC database engines. This is enabled by leveraging

two very simple but useful observations: (1) Conceptually, transac-

tional DDL operations can be modeled by “normal” transactions of

DML statements that modify the schema and (sometimes option-

ally) entire tables involved. (2) In MVCC systems—thanks to their

built-in versioning support—schemas can be stored as versioned

data records and be used to participate concurrency control in ways

similar to “normal” data accesses for determining table data vis-

ibility. When combined, these observations allow us to devise a

data-denition-as-manipulation (DDaM) approach that supports

online and transactional DDL almost “for free” by slightly tweak-

ing the underlying SI protocol within the database engine, without

having to rely on additional features such as views and triggers.

Like classic catalog management designs [

], Tesseract asso-

ciates each table (or other resources, such as a database) with a

schema record which is stored in a catalog table which in turn

has a predened schema (e.g., table name, constraints, etc.). Us-

ing the system’s built-in versioned storage support, such schema

records are also multi-versioned. This allows us to implement DDL

operations as simple as (1) appending a new schema version via

standard record update routines and (2) nishing data verication

and migration, both in a single transaction that follows the commit

and rollback processes of the underlying CC protocol. To access a

table, the DML transactions simply follows the standard SI proto-

col to read the table’s schema record version that is visible to the

transaction. The schema version record then dictates which data

record versions should be accessed by the transaction.

DDaM is straightforward in concept, but realizing it requires

careful considerations. Specically, DDL transactions can be very

long by accessing entire tables, and so could block the progress

of other transactions, or be aborted due to frequent conicts with

concurrent transactions. It also adds prohibitively high footprint

tracking overhead for writes if we were to follow the SI protocol

naively. Moreover, concurrent DDL and DML operations must be

handled with care to ensure that logically, a DML transaction with

work done based on an older schema version never commits after

a newer schema has been installed (otherwise subsequent reads

would interpret the old content based on a new schema version,

leading to potentially wrong results). To this end, in later sections,

we propose a relaxed DDaM design that further adapts the SI pro-

tocol to allow more concurrency and reduce unnecessary aborts.

Relaxed DDaM is the key for Tesseract to achieve online schema

evolution without sacricing much for DML operations.

Although we focus on MVCC in this paper, single-versioned

systems could adopt Tesseract if extra (but straightforward) steps

are taken to support versioned schema; we discuss possible solu-

tions later. Tesseract can also work with existing lazy migration

approaches [

] to support instant deployment of compatible schema

changes, which we demonstrate in later sections.

We have implemented and integrated Tesseract with ERMIA [

a main-memory MVCC database engine, as its schema management

and evolution solution. Following past work, we use several rep-

resentative schema evolution workloads to evaluate Tesseract. On

a 40-core server, compared to prior approaches, Tesseract allows

the system to evolve database schemas without service downtime

or signicantly impacting application performance. As we show

in detail later, we often observe only up to

∼

10% of drop for DML

operations with DDL operations that involve heavyweight data

copying such as adding a column eagerly.

1.3 Contributions and Paper Organization

This paper makes four contributions.

We make the key obser-

vation that schema evolution can be models as modifying entire

tables, to unify DDL and DML handling and propose a simple but

useful data-denition-as-manipulation (DDaM) approach for trans-

actional and non-blocking DDL operations.

We show how simple

tweaks can be applied to common snapshot isolation protocols to

easily and natively support transactional and non-blocking DDL

without ad hoc “patches.”

Based on DDaM, we build Tesseract

to show Tesseract’s feasibility and address challenges brought by

DDaM.

We compile a comprehensive set of schema evolution

benchmarks to evaluate Tesseract and related work. Tesseract is

open-source at hps://github.com/sfu-dis/tesseract.

Next, we start with the necessary background in Section 2, and

give the basic idea and simple tweaks needed for SI to handle DDL

natively in Sections 3–4. Section 5 then describes Tesseract in detail

and addresses the challenges of DDaM. We cover evaluation in

Section 6 and related work in Section 7, before Section 8 concludes.

2 MVCC BACKGROUND

In this section, we give the necessary background on MVCC and

clarify the assumptions we make throughout the paper.

2.1 Database Model

Following past and recent work on memory-optimized MVCC [

], we adopt the MVCC model described by Adya [

]

where the database consists of a set of records, each of which is

represented by a sequence of totally-ordered versions. An update

then appends a new version to the sequence and delete is modeled as

a special case of update that appends a tombstone version. Similarly,

inserting a record is treated as an update that appends the rst valid

version for the record. To read a record, the transaction picks a

version that is visible to it depending on the isolation level used

(described later). To facilitate this, the system maintains a central

counter usually implemented as a 64-bit integer [

]. Upon

start (or accessing the rst record), the transaction reads the global

counter to obtain a begin timestamp. Upon commit, the transaction

atomically increments the global counter (e.g., using the atomic

fetch-add or compare-and-swap instruction [

]) to obtain a commit

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

OnlineSchemaEvolutionis\50Almost\51FreeforSnapshotDatabasesToappearatVLDB\62\60\62\63TianxunHuSimonFraserUniversitytha\61\61\60@sfu\56caTianzhengWangSimonFraserUniversitytzwang@sfu\56caQingqingZhouTencentInc\56hewanzhou@tencent\56comABSTRACTModerndatabaseapplicationsoftenchangetheirschemastokeepupwi...

展开>> 收起<<

Online Schema Evolution is Almost Free for Snapshot Databases.pdf

共14页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Online Schema Evolution is Almost Free for Snapshot Databases

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: