Systems | Development | Analytics | API | Testing

BigQuery is a fast, highly scalable, cost-effective, and fully-managed enterprise data warehouse for analytics at any scale. As BigQuery has grown in popularity, one question that often arises is how to copy tables across locations in an efficient and scalable manner. BigQuery has some limitations for data management, one being that the destination dataset must reside in the same location as the source dataset that contains the table being copied.

When working with tables in data warehouse environments, it is fairly common to come across a situation in which you need to generate surrogate keys. A surrogate key is a system-generated identifier that uniquely identifies a record within a table. Why do we need to use surrogate keys? Quite simply: contrary to natural keys, they persist over time (i.e. they are not tied to any business meaning) and they allow for unlimited values.

Today, Docker or Kubernetes are obvious choices. But, back in 2015, these technologies were just emerging and hoping for massive adoption. How do tech companies make the right open source technology choices early?

One question that I’m often asked is ‘how do I justify funding for a BI project?’ The answer I give is that to win funding you need to stop thinking about BI as a project.

Every now and then you can't beat a bit of Meat Loaf, the singer not the food, as I've not had the pleasure to taste it. I recently found myself recalling a cult classic "Standing on the outside" not because of any failed break up but thinking about the abundance of data available externally that can be used in combination with your internal data. Unfortunately, many are still leaving this data standing on the outside.

REDWOOD CITY, Calif.

Since the release of Talend 7, a major update in our software, users have been given the ability to build a complete integration flow in a CI/CD pipeline which allows to build Docker images. For more on this feature, I invite you to read the blog written by Thibault Gourdel on Going serverless with Talend through CI/CD and Containers.