Systems | Development | Analytics | API | Testing

Apache Kafka has gained traction as one of the most widely adopted technologies for building streaming applications - but introducing it (and scaling it) into your business can be a struggle. The problem isn’t with Kafka itself so much as the different components you need to learn and different tools required to operate it. For those motivated enough, you can invest money, effort and long Friday nights into learning, fixing and streamlining Kafka - and you’ll get there.

Data Engineers are forever flying the flag for open-source technology. But now that we’re safely locked away in our homes - potentially for the rest of the year - a new danger looms: That we get distracted by our new data tools and lose touch with delivering value to the business. Today most Data Engineers around the world are working from home, and at first glance it may seem like this works. After all, a solid internet connection is all we need to carry on doing what we were doing...

Remote working can be a blessing for every parent. In that way, you don’t have to face tough dilemmas like “staying at the office to finish fixing that bug” versus “returning home early to do some karaoke with your children”. Being a working parent used to be all about making tough calls in terms of career or childcare.

Some time ago, the concept of event streaming was not that widespread. In addition to that, the platforms provided were much fewer than today, and required more technical depth to set up and operate. However, as time passed, those platforms matured, community grew, the documentation was improved and the rest of the world started to wake up to the advantages these platforms can bring to address the real-time experiences businesses need. And Apache Pulsar is a great example.

Systems fail from time to time. And there is nothing worse than being unaware a system is down at all. Which is why we wrote the email alerts from Lenses.io tutorial. But what about those evenings where you are enjoying a documentary or just having an email-free evening? After all, system failures rarely happen at a magically convenient time.

Open source is great but sometimes it misses the mark for security at enterprise levels. Take Kafka Connect, I’ve built a few connectors in my time and prior to its introduction to Apache Kafka back in 2017 I used other hand cranked pieces of software and security was always a primary concern. One feature that will quickly put a blocker on your project being successful is not reaching production. There’s a number of reasons for this but high up, usually top, is security.

Events and Conferences are a core element of any growing industry and ours is no different. It’s the easiest way to learn what’s new and to gain a better understanding of any technology. After all, we might love blogs and online courses, but at the end of the day, is there a better way to process any type of information than a passionately delivered speech or a lively discussion?

It feels like only yesterday that we announced Lenses 3.1. The release was focused around helping developers be more productive when developing real-time data applications. Since then, our customers are onboarding new applications and new users onto their data platform at a faster rate than ever. And with more apps and users come stricter and tougher requirements for compliance and governance.

Here at Lenses.io, we’re focused on making data technologies such as Apache Kafka and Kubernetes as accessible to every organization as possible. It’s part of our DataOps vision and company DNA. Lenses is built by developers, for developers. We understand the headaches they live with and the challenges they face seemingly have to learn a new data technology every few months. We believe that’s just not the right model.

If you have worn the Platform or Security Engineer badge, or if you have a Sec/Ops role, you might have experienced something like this at some point in your career. Hopefully not. You receive a call at 3am, it’s your SOC, something’s not right. Oh sh*t! There’s unidentified traffic on the network from an unknown host and it’s communicating with a remote server. Sounds like a Level 3 exfiltration. It’s gonna be a long night.