posted time Created time: 2017-02-06 Last updated time:

Design of Dynamic Partitioning of Distributed Database

Now, 2017/02/05. To make and manage partition table of database tables is essential to deal bigdata. Therefore I have to design the architecture now.

Whether dynamic maintenance of partitioning is possible or not is also up to the records format of storage engine.

Two Layer Partitioning of Tables

Alinous Elastic DB has 2 layer table data partitioning. By using this function, you can manage cold and hot data in different cloud section.

Two Layer Dynamic Partitioning

For example, the main partition key is timestamp of the record created, and sub key is the region. The old data is managed in storages with big capacity, but not so fast. Hot data is manages in storages which are very fast, but not with so big capacity.

Change Partition Key

When we change the key of partition node's maximum value or value of Partition Key.

Single Partition Key

Each distributed table has maximum value. The table storage has records whose value of the key is less than it.

On changing partition node's maximum value, the data is transferred to other node like below.

Dynamic Partition Key Change

The operation to change the partition key is below.

  1. Select Storage Node, Table, and set key values to change
  2. Transfer data
  3. Lock the table
  4. Actually change the key value and release the table lock
  5. Vacuum source data in last node, later

Double Partition Key

Each storage have maximum values of both Partition Key and Sub Partition Key. When we change the Partition Key, the data is transferred into other storage node like below.

Changing Main Partition Key

The red line parts are changed stuff.

Key of Child Partition Node

When changing the key value of the sub partition, it works same with single partition key.

Dynamic Update of Partition Key Value

When we manage database, to manage actual data structure on distributed nodes is necessary, to keep the performance of queries and updates.

Then we do not have to stop the database.

That is because each storage node has maximum and minimum value of the partition key. Therefore, if there are records whose range is out of them, they are ignored on scanning the table.

Go to Top