ScyllaDB vs Cassendra the storry from the beginning till the end

 

 

Discord a messages based applications which need to store billions of messages, not only instant messages but audio calls too and many other options they offer.

They moved from Mongo to Cassendra at 2017, as they were looking for a db more scalable, fault tolerant relatively low maintenance.

So they didn't start with relational database such as MySql as they were looking for scalability.

As you know to make the relational database more scalable and respond faster by setup partioning and sharding which basically group of similar data to multiple instances.

However may cause other types of problems especially with transactions, you can implement transactions across shard effectively, are by Design the multiversion concurrency control aspects of it and acidity and all these nice properties or just at the instance level it can't just spread them now I know people will ask me yeah you can two face commands three phase command bla bla there is always away but again not effective at the large scale.

RDBMS can scale horizontally to make it perform better BUT again the data has to exist everywhere, where partioning is actually split the data itself.

So in both cases we have issue, replicas will have all the relational data so it is not effective at the very large scale, ans the transaction concept behind the scene has another layer of cost. 

Sharding simply the data which related to same domain like users split to another instance, and there is a master sharding keys points to the shards.

MongoDB, Cassendra and other nosqls started from the sharding idea and carried on! things are partioned, sharded and fault tolerant and scalable, by default all these options exist.

As much as the data is fewer as much the query will be fast, thi sis the goal always.

 Cassendra read is more costly than write!! thats because the write operation store the data first in memory called memtable and when the memory full it flush to disk as an SSTabl, at the other side Read will look at the data first at the memtable and sometime check the SSTable too and there could be multiple version .. multiple version multiple location then reconcile huh!! that make the read path is costly than write, data replication another factor, Cassendra write to main node and the async sync to other nodes, at read the coordinator node need to fetch data across multiple nodes sometime to return the correct data.

This pull us to talk about log structured merge tree vs b-tree?!

B-trees and Log-Structured Merge-trees (LSM trees) are both data structures used in databases to manage data, but they have different characteristics and are optimized for different use cases.

These are techniques used by the DBs to model, structure and form the data inside their storage, basically store and index.

B-Trees 

Used by Mongo, MySql and Postgres, 

It sort the data in tree structure so if you want 120 go to tree 100 then brancsh 130 - 180 then leef page 110 - 120 

B Trees helps the SQL DBs to setup and structure their indexes to easily finds the data using the primary key, it actually structure the indexes, like book index.

Similaryly B Trees helps the no sql dbs in structure their indexes based on the case, and based on the data type, some DBs key value and some like mongo accept multiple type, however the concept is the same B Tree is away of strucre the DBs indexes to easily reach to the data 

Of course you need to commit the changes to the log , and as you seen in this model everything is in order, even at writing you write to that page (which pointed by the index) then this page then this page and everytime the db engine update the log commit the log

 B Trees has a few challenges writing data to SSDs due to the way SSDs write the data to the volumes

The erase-before-write characteristic of SSDs can cause inefficiencies with certain workloads and data structures, including B-trees. Here's why:

Write Amplification: As mentioned earlier, B-trees often involve writing a large amount of data for small changes due to the need to maintain tree balance and node occupancy. With an SSD, this can lead to many erase-write cycles, which not only reduces the SSD's lifespan but also slows down performance because erasing blocks takes time.

Random Writes: B-tree operations often result in a lot of random write operations, which can be less efficient on SSDs. While SSDs don't suffer from the same performance penalties as HDDs for random access, they still prefer to write data sequentially because of the way data is organized in blocks.

 

Long Structure Merge LSM (Cassendra and ScyllaDB)

Came with another idea of continues write without update or delete, directly to the memtable and once it get full it flush the data to the SSTable

So rather than the complication happening between the B Trees and SSDs due to the update, LSM came with continues write write write write ...etc.

Here Read become a problem! because you will first check the memory then SSTable isn't?

Compaction is one technique to improve where you delete the old data so it became like write write write the new data and delete delete the old data

Note here that read more easier and faster at B Trees as you retrive the exact page one shot rather than looking at memtable then SSTable 

When you write something FAST there are customers wants to read also FAST 

So at Cassendra we ended at read slow write fast!!

Here ScyllaDB came to solve Cassendra JVM Garbage collection problem

ScyllaDB and Apache Cassandra are both highly scalable, distributed NoSQL databases. They have similar data models and APIs, as ScyllaDB is actually a drop-in replacement for Cassandra, designed to be compatible with Cassandra's protocols and data models. However, the two databases have different underlying architectures and performance characteristics.

ScyllaDB was designed to address some of the performance bottlenecks and operational challenges of Cassandra. One of the main differences is how each system handles garbage collection, which is the process of reclaiming memory that is no longer in use by the application.

Cassandra is written in Java, which uses automatic garbage collection. The Java Virtual Machine (JVM) periodically checks the heap memory to find and delete objects that are no longer needed. While automatic garbage collection can make programming easier, it can also cause pauses or "stop-the-world" events when the garbage collector runs, which can lead to unpredictable latency. Cassandra's performance can be significantly affected by how well the JVM's garbage collection is tuned.

ScyllaDB, on the other hand, is written in C++, which does not have automatic garbage collection. Instead, ScyllaDB uses a different approach to managing memory. It uses a framework called Seastar, which is designed for high-performance server applications. Seastar uses a shared-nothing design where each CPU core has its own instance of the database, including its own memory, I/O, and request queues. This architecture avoids many of the pauses and unpredictable latencies that can occur with garbage collection in a JVM-based system like Cassandra.

References

https://discord.com/blog/how-discord-stores-trillions-of-messages

https://www.youtube.com/watch?v=xynXjChKkJc