Data Denormalization In Modern System Design

The myth of normalization and how denormalized views powered by event driven or batch processing architecture can help

What you'll learn

What is a normalized dataset

Where normalization falls short and why is normalization a myth

The concept of denormalized views

How to implement denormalized views with event driven (streaming) workflows

The pitfalls of event driven (streaming) replication to your denormalized view

How to implement denormalized views with batch processing workflows

The pitfalls of batch processing replication to your denormalized view

Tradeoffs of denormalized views

When should you really consider denormalized views

Gain a new perspective in your existing microservice architecture

Requirements

High level understanding of backend system design

High level understanding of databases

High level understanding of microservice architecture and distributed systems

Description

We will dive deep into the cutting-edge world of modern database and software engineering, where traditional data normalization is increasingly giving way to more powerful techniques: denormalized views.Whether you’re struggling with slow queries, high latency, or scaling data for millions of users, this series will teach you how to harness the true potential of denormalized views.We’ll break down complex concepts in simple terms, showing you how modern views replicate and transform data for specific use cases, reduce the need for real-time processing, and improve overall performance. We'll cover how to implement such denormalization process, from using event-driven architecture to using batch processing instead and materialized views. We'll talk about denormalization techniques, and how to map and handle all of the many edge cases they will bring about.In this course, I want to challenge the way we think about databases and system design. In the past, normalization —structuring data into its smallest parts, and forbidding data duplication— was the golden rule for optimizing databases. But nowadays things have changed. When we’re dealing with massive amounts of data, oftentimes normalization can actually slow us down.What if there was another approach? I’ll show how effective denormalization of your data to what I call “denormalized views” can help provide an alternative solution to complete normalization. We’ll demonstrate how replicating your data in a new denormalized format, designed for fast read access for a specific use case, can help you solve a new class of problem you weren’t able to serve before because of performance reasons. Perhaps this is about a specific process in your organisation that is taking increasingly longer and more resources to be scaled and that needs to be rethought as soon as possible. Perhaps a new feature request that your team wasn’t able to prioritize because there was no known solution to the problem at hand. Either way, this approach to system design is potentially one solution to your problems.Whether you’re a backend engineer, system architect, or just someone curious about database performance, this series will arm you with the knowledge to design faster, more scalable, and more reliable systems. Learn how today’s largest platforms — from Netflix to Facebook — use these principles to handle massive data volumes and power seamless user experiences.

Overview

Section 1: Normalization and its tradeoffs

Lecture 1 The Myth of Normalization

Lecture 2 What is Normalization? What are its tradeoffs?

Section 2: What are Denormalized Views?

Lecture 3 The Denormalized View Concept

Lecture 4 Caching Denormalized views

Section 3: Implementing Denormalized Views

Lecture 5 Implementing Denormalized Views Replication Workflows

Lecture 6 The complexities of Batch Processing

Lecture 7 The complexities of Event-Driven Workflows

Section 4: Tradeoffs and conclusion

Lecture 8 Trade-offs and Conclusion

Section 5: Examination

Junior to Senior Software engineers with some professional experience in backend software engineering,Senior+ software engineers looking to challenge their views on distributed systems