Hey, the query performance chapters of the TimescaleDB course are finished and the first one about storage management is now released. Analytics applications grow over time to large databases - or even massive ones, if you have a lot of data. And after some time, you're hitting issues you didn't anticipate: Backups take a long time to create and even longer to restore if your machine dies one day. But your costs also increase significantly, as every backup is huge. A couple of cents per backup is a distant dream once you hit hundreds of gigabytes or terabytes on your analytics tables.

However, you probably don't need all the data you saved anymore. Data that is three years old? Probably not needed anymore. Data that is three months old? Depending on your application, that may no longer be needed too. With TimescaleDB's data retention feature, you create a policy that automatically removes data that your application no longer needs. And this is done more efficiently than you could do: Instead of deleting individual rows, TimescaleDB drops entire chunks when all their data is past your cut-of timeframe.

However, the truth is that you probably can't fully delete old data. But you're ok with deleting the original rows if you can keep your continuous aggregates. Because the older the data is, the less you're interested in the raw data - the pre-aggregated results are enough!

TimescaleDB planned for this! You can configure your Continuous Aggregates to retain calculated values even when your raw data is deleted. So you can effectively reduce your storage size while keeping the data in a less granular level that makes sense for your application.

Likewise, you can create retention policies for continuous aggregates too. With hierarchical continuous aggregates, you can gradually delete more detailed data over time while keeping less detailed pre-aggregated results.

That's a rough overview of the course's chapter but it describes the main idea. Learn about data retention and how you can reduce your storage costs with TimescaleDB. If that sounds interesting to you, please take a look at it and let me know what you think!

I hope you learn a lot
Tobias