Systems | Development | Analytics | API | Testing

Why you should use Fivetran history mode for historical analysis over alternative solutions.

Five leading experts share their insights on what’s ahead for the data industry.

It's important to understand the uses and abuses of streaming infrastructure. Apache Kafka is a message broker that has rapidly grown in popularity in the last few years. Message brokers have been around for a long time; they're a type of datastore specialized for "buffering" messages between producer and consumer systems. Kafka has become popular because it's open-source and capable of scaling to very large numbers of messages.

With automated data integration, CaliberMind uncovers data insights for customers. As a Customer Data Platform (CDP), CaliberMind delivers data-driven insights to its customers. To do so, it must connect to its customers’ data sources, extract, process and transform the data, run it through specially designed analytic models, and, finally, present data back to the customer as insights. CaliberMind uses Fivetran to offload the task of ingesting data from its customers’ applications.

With Fivetran and Databricks, Slice reallocates the efforts of three data engineers to mission-critical projects and adds a data science team.

A data pipeline is a series of actions that combine data from multiple sources for analysis or visualization. In today’s business landscape, making smarter decisions faster is a critical competitive advantage. Companies desire their employees to make data-driven decisions, but harnessing timely insights from your company’s data can seem like a headache-inducing challenge.

Thinking of building out an ETL process or refining your current one? Read more to learn about how ETL tools give you time to focus on building data models. ETL stands for extract-transform-load, and is commonly used when referring to the process of data integration. Extract refers to pulling data from a particular data source. Transforms are used to make that data into a processable format. Load is the final step to drop the data into the designated target.

Want to look at how data has changed over time? Simply enable history mode, a Fivetran feature that data analysts can turn on for specific tables to analyze historical data. The feature achieves Type 2 Slowly Changing Dimensions (Type 2 SCD), meaning a new timestamped row is added for every change made to a column. We launched history mode for Salesforce in May and have been delighted with the response.

Bucketing, also known as binning, is useful to find groupings in continuous data (particularly numbers and time stamps). While it’s often used to generate histograms, bucketing can also be used to group rows by business-defined rules. I’ll walk through the simple bucketing various data types as well as custom buckets.

Migrating a data warehouse from a legacy environment requires a massive upfront investment in resources and time. There is a lot to consider before and during migration. You may need to replan your data model, use a separate platform for tasks scheduling, and handle changes in the application’s database driver. Therefore, organizations must take a strategic approach to streamline the process. This article presents a step-by-step approach for migrating a data warehouse to the cloud.