Database

Ch-ch-changes!

• For source code—as well as natural languages, particularly poetry—elegance is the ratio of intent communicated per line of code. Elegant code is as simple as possible, but no simpler; stripped of all unnecessary steps, yet unmistakably clear.
  
  Any effort invested in making code simple and elegant pays off. It sharpens our understanding and leaves the path clearer for the next reader. Each reader, in turn, can become an editor, refining the code further and compounding that clarity.
   
  • TigerBeetle recently adopted a tournament tree (also known as a loser tree) for its k-way merge iterator; learn more about loser trees and k-way merge algorithms. It is only natural that such an interesting algorithm sparked our curiosity! However, when we tried to rewrite it for better readability, we failed, as the refactored version was twice as slow! Remember, elegance is about both beauty and functionality; we don’t have to choose one over the other! To that end, Tobi took over the refactored code and made it fast again, not only matching the baseline but improving it by 20%.
  • We added and improved various microbenchmarks to ensure we run faster!
  • We simplified the merge iterator interface by removing the stream precedence configuration. Now, when reading a value and a tombstone with the same key, the result depends on the precedence implied by the stream ID.
  • Additionally, we simplified the tracking of snapshots in the LSM tree. Snapshots are tags attached to LSM tables that indicate whether a table is visible to a scan, enabling stable reads even as new data is written (e.g. during compaction). Instead of maintaining a separate snapshot number (and constants such as snapshot_latest), we now rely on the VSR operation number (op number) as the snapshot tag.
  • We tuned the index queue size from 1 to 2 to enable prefetching of index blocks during compaction. Previously, there was a data dependency that required waiting for the last value block to be read before fetching the next index block from storage. More concurrency for the win.
     
• Finally, we made some improvements to the CLI user experience:
   
  • For a more representative workload, account history is now enabled by default during benchmarks, and can be disabled with the --no-history flag. 
  • The --id-order=tbid CLI flag was added to the benchmark to better resemble a real-world application and avoid ID collisions across multiple runs.
  • CLI parsing was tightened to reject signed integers. For example, --cluster=+0 now results in an error.
  • The “slow request” console warning was removed from the logs for clarity.

Community Contributions

• We improved how the Node.js client detects the system ABI when running on Linux distributions with musl libc, making the detection more reliable. Thank you @arravo-github, for pointing this out!
• Sergei Ustimenko contributed patches to improve consistency while logging headers and checksums. Thanks so much, Sergei.
• David Soria Parra contributed fixes to the Rust client; thank you!
• AmirAli Akbari caught various typos in our documentation. Well spotted, and thank you!

"Database television" (DTV)

Last month on IronBeetle, the query engine arc continued with the Stream API! We built a benchmarking harness, explored exhaustive text minimization, and had fun refactoring Tree of Losers—part 1, part 2, and finally it started to work (slowly).

Join us live every week on Twitch or catch up on the TigerTube!

Looking back

Building a Double-Entry Ledger with Elixir and TigerBeetle, Jan 1
“When we first built our ledger on PostgreSQL, it worked well — until it didn’t. As our transaction volume grew, we hit performance bottlenecks and had to implement increasingly complex strategies to maintain consistency.” — part 2 of a series written by Fatih Altuntaş.

Connect to Tigerbeetle using Go, Jan 3
A minimal example of establishing a connection to TigerBeetle, by Mihalis Tsoukalos.

Designing a High-Performance OLTP Database from First Principles, Jan 6
Shubham Rascal took a terrific deep dive into TigerBeetle’s design, inspired by Viewstamped Replication.

One for the Treble, Two for the Time, Jan 14
When we record information, mistakes happen. We thought we knew a fact about the world, but were wrong, or there was something we didn’t know then but know now. Enter the artist known as bitemporality, or modeling information across two timelines at once. A clarifying post, by Lewis Daly!

Looking ahead

The TigerVerse

Thank you!