Systems | Development | Analytics | API | Testing

Enterprises are tapping and leveraging big data to get ahead of the competition. As Peter Sondergaard, ex-Executive Vice President at Gartner said: The problem with the combustion engine is that it does not scale well. As companies grow, the data platforms they previously relied on for analytics start to break apart.

The same way architects design blueprints to plan and support the building of houses, data architects design the data architecture to support building end-to-end data pipelines. In this article, we dive into data architecture, its principles, and how data mesh fits into the story.

Keboola’s Storage UI now comes with a new - slicker - look which will improve the user experience for all Keboola veterans. (New to Keboola? Do not fear. Simply follow along with the guided tour when you sign up for free, and you can unlock all the new features after June 2nd).

Modern enterprises leverage over 400 data sources to stay ahead of the competition. The sheer volume and complexity of data operations raise several challenges for intrepid organizations: How to cut the complexity of data operations? Enterprises turn to data lineage for the answer. Data lineage is the process of recording and visualizing data assets as they flow along your system.

In the age when data is the new oil, more than 80% of IT decision-makers delay their business decisions due to slow data processing. Architecting your ETL pipeline with database replication can speed up your data processes. Database replication creates an analytic database as a separate copy of your production database. This unburdens the transactional database from analytical queries while securing fresh data in the analytical database for faster time-to-insights.

TL;DR: Yes, you can do it. And no, you don’t need a separate tool for it. “Reverse ETL” is a fairly recent addition to the data engineer’s dictionary. While you can read articles upon articles about it (there’s a pretty good ‘primer’ in the Memory Leak blog), it can be summarized as being the art and science of taking data from your data warehouse and sending it somewhere other than BI - generally into other tools and systems where it becomes operational.

We’re happy to announce that we added R to our selection of transformation and workspace backends in our freemium accounts. Access a wide variety of statistical methods offered by R and model data straight from Keboola. No more context switching and working in a few different tools. Start modeling data and running transformations in R. Create a free account and take Keboola for a spin. R transformations complement Python and SQL where computations or other operations are too difficult.

When data is the bloodline of business operations, data breaches, corruptions, inaccessibility due to server downtimes, and accidental deletions can stop business continuity and even drive your business out of business. In this article, we take a look at database replication techniques. That is, how to use data replication to keep data in your database management systems (DBMS) such as PostgreSQL or MySQL accessible and safe.

Companies deploying big data analytics to gain competitive advantage can quickly sour their successes by lacking a big data governance strategy. Which turns their data assets into data liabilities. In this article, we dive into the field of information governance and information management and explore how to set up and automate a big data governance program for success. Big data governance is a set of processes and principles that ensure the high value of data throughout its lifecycle.

With the steep rise of data, smart businesses have started capitalizing on this new oil to build a new type of products and services: data applications. Admittedly, the engineering and business development of data apps overlaps with their cousins - the trusty desktop app and the well-known web app, … But there is a core difference that sets data applications apart: they are first and foremost about the data they use to deliver value.