Data Transformations in Analytics Engineering (Concepts & Dataform)

In this enlightening installment of our DataOps series, we dive deep into the intricacies of managing and simplifying complex cloud-based data pipelines, especially focusing on in-warehouse transformations. Whether you're a beginner or a seasoned professional, this video is designed to demystify the often intimidating world of data operations and analytics engineering. Video Series Part 1: /watch/UAIHcQ-BwA-BH Video Series Part 2: /watch/UN7ty4Gwp6ewt Demonstration Code and Diagram: https://github.com/nodematiclabs/analytics-engineering-dataform Modeling Tool: https://softwaresim.com This video explores: * The setup of data originating from YouTube analytics via BigQuery Data Transfer. * The structuring and importance of in-warehouse transformations within ELT pipelines using Dataform or DBT. * The journey from raw data to creating derived views and tables, and the architectural principles behind these transformations. * How modularity in data modeling can be a game-changer. * The execution process of data transformations, emphasizing the importance of lineage and dependencues. * The integration and utilization of Git-based methods for workflow automation and management with platforms like GitHub or GitLab. If you are a cloud, DevOps, or software engineer you’ll probably find our wide range of YouTube tutorials, demonstrations, and walkthroughs useful - please consider subscribing to support the channel. 0:00 Conceptual Overview 1:33 BigQuery Datasets 2:15 Dataform Definitions/Transformations 5:28 Lineage and Execution 7:14 Git Integration and GitOps

Category