Targeting the Best of Both Worlds with Next-generation SQL Databases

DB-Part4Conventional relational databases and recently-developed NoSQL databases have led some enterprises to an impasse. They want to scale the systems that are handling their data. However, an RDBMS used to guarantee transaction integrity is difficult to expand. NoSQL systems, although scalable, typically do not offer full transaction integrity via the ACID properties discussed in earlier posts. The latest solution in the SQL/NoSQL saga—next-generation SQL databases—may have the answer.

Where Can Next-generation SQL Make a Difference?

Next-generation SQL systems use both SQL and ACID as design premises. They offer improved ACID-style performance for large numbers of repetitive, short-lived transactions with modest data interactions. Next-generation SQL systems are often designed with a distributed architecture from the start and a middleware layer to provide automatic partitioning over multiple machines. Some next-generation SQL databases outperform their relational and NoSQL peers in terms of elastic scalability and the number of transactions executed per second.

Products and Use Cases Today

Google made its own next-generation SQL system called F1 for its AdWords online advertisement business, promoting it as a "distributed SQL database that scales." Other next-generation SQL database products include MemSQL and VoltDB. MemSQL combines in-memory processing, ACID-compliance, and SQL compatibility to deliver lightning fast transactional performance, while still being a proper SQL database. VoltDB offers a highly-distributed in-memory database to handle rapidly-streaming datasets that require high levels of scalability and transaction processing speed. Next-generation SQL databases have found their way into applications with fast-streaming data and multiple millions of transactions per second used by gaming companies, mobile ad networks, and telecoms operators. They offer advantages to e-commerce companies for order tracking and inventory management.

The Big Data Question Mark

It remains to be seen how next-generation SQL databases will deal with the varying forms of big data, but the general consensus is that they are here to help. Though operational, historical, and analytical data require different handling, the benefits of many next-generation SQL databases can in theory handle it all. In the operational or online transaction processing part (OLTP), the emphasis is on frequent data updating and querying. This means responses to requests should be as fast as possible. Optimization then needs to be done primarily for write operations. By comparison, in the online analytical processing (OLAP) world, the priority is on the optimization of read requests. Data is seldom updated, but needs to be accessed in large quantities for meaningful analysis.

Finally – RDBMS, NoSQL, or Next-generation SQL?

Database technology choices will depend on what you want to do and where you are starting from. If your current IT infrastructure is heavily vested in RDBMS solutions with matching staff, training, and competencies, the choice may already be made for you—for a while, at least. Otherwise, decisions on whether to use RDBMS, NoSQL, or next-generation SQL will depend in the first instance on your need to scale and the stability or changeability of your data model. 

If you haven't already, check out my previous blog in this series, NoSQL - The Great Escape from SQL and Normalization.

big data analysis

Topics: Data Platforms, Analytics, & AI ESG Validation Services