Systems | Development | Analytics | API | Testing

With technology innovations raging at incredible speeds over the past few decades, new and exciting platforms for gathering, storing, transforming, and manipulating data are entering the market every day. Apache Hadoop was one of these disrupters when it entered the market in 2006, offering distributed storage and big data processing using a network of many computers.

The Healthcare Insurance Portability and Accountability Act (HIPAA) has been an important federal law in healthcare since 1996. Part of its purpose was to create standards meant to protect sensitive patient information, and it took on even more important once the digitalization of patient health records became widespread. Now it’s required for certain types of businesses to protect patient health information—or face fines that range from $100 to $50,000 per violation.

Organizations have access to massive amounts of data, but they don’t always give enough thought to how they’re going to keep it private and protected. Dozens of data privacy regulations are in effect or in development globally, and the average consumer is learning more about how much of their data gets collected and used by businesses. For this reason, companies need to focus on keeping data safe while it's under their control, but it’s easy to make mistakes.

Data analytics technology helps organizations make sense of an ever-increasing volume of data. As this technology matures, it gets better at delivering actionable insights and helping companies determine outcomes. Prescriptive analytics is a modern solution that builds upon other analytics technology and guides organizations to the right decisions for a particular situation.

Extract, Load, Transform (ELT) technology makes it easy for organizations to pull data from databases, applications, and other sources, and move it into a data lake. But companies pay for this convenience in many ways. ELT solutions can have a negative impact on data privacy, data quality, and data management.

Wherever your business operates, you may have to follow the GDPR or face massive fines. It’s an epic legal document but much of it boils down to some key principles. The best approach is to store and manage your data in a way that makes GDPR-compliance straightforward. Here’s what you need to know.

Picking the right business intelligence (BI) tool is essential to helping you beat your competitors, better serve your customers, and make smarter data-driven decisions. However, there's no one-size-fits-all tool for every enterprise. Not all BI users are created equal, and not all users should have the same level of access to sensitive and confidential data.

Written in Python, Apache Airflow is an open-source workflow manager used to develop, schedule, and monitor workflows. Created by Airbnb, Apache Airflow is now being widely adopted by many large companies, including Google and Slack. Being a workflow management framework, Apache Airflow differs from other frameworks in that it does not require exact parent-child relationships. Instead, you only need to define parents between data flows, automatically organizing them into a DAG (directed acyclic graph).

Personally identifiable information (PII) is some of the most valuable data that organizations can have. It's also some of the most dangerous if you don't follow data security best practices. If you don't treat this data with care, you could end up in the headlines as the victim of the latest data breach, costing you money and damaging your reputation. Of course, you should never leave PII data unprotected. So what is the best way to protect the confidential and sensitive PII that you handle?

Data cleansing is a fundamental aspect of knowing what data you retain and using it most effectively. The process of cleansing reduces the potential for error and enhances data reliability. Here's how to get started.