Systems | Development | Analytics | API | Testing

Architecting Apache Kafka for GDPR compliance

Once upon a time (2017), in an office far far away, you may have been cornered in a conversation with someone from Legal about GDPR. It could have gone something like this: “You there, Data Engineer” “Yep, that’s me” “What PII do we have residing in this Apache Kafka database?” You probably mumbled something about Kafka not being a database. “And who can read/ write the data?

What Is NullReferenceException? Object reference not set to an instance of an object

“Object Reference Not Set to an instance of an object.” Cast the first stone those who never struggled with this error message when they were a beginner C#/.NET programmer. This infamous and dreaded error message happens when you get a NullReferenceException. This exception is thrown when you try to access a member—for instance, a method or a property—on a variable that currently holds a null reference. But what is a null reference?

Going Beyond Observability for Spark Applications & Databricks Environments

Join Chris Santiago, Solutions Engineer Director at Unravel Data, as he takes you through Unravel’s approach to getting better and finer grain visibility with Spark applications and how to tune and optimize them for resource efficiency. An overview of out of the box tools like Ganglia and their overall lack of visibility on Databricks jobs How Unravel helps you gain finer grain visibility, observability, monitoring into Spark data pipelines How Unravel can recommend better configurations and tuning of Spark applications.