I spoke at GOTO Chicago last week with Martin Fowler. He gave a keynote on The Many Meanings of Event-Driven Architecture. It wasn’t tied to or specific to any particular technology. In this post, I’m applying some of his points specifically to Kafka and...
There’s a common misconception that says if I just change one thing in Big Data, everything else will be easier. The answer is that there are several different hard problems in Big Data. Changing one problem doesn’t solve the other problems. Sometimes,...
Learning how to ask good questions is an important life skill. This skill doesn’t just help in business or professional life. It helps in your personal life too. When I’m teaching, I encourage questions. I ask them to ask questions all the time and as...
Sometimes I’ll train at a company that’s creating a data engineering team. The team often includes a Data Scientist. I’ll always make a note to talk to the Data Scientist about their experience and interactions with the team before I arrived. These...
My friend, Justus Eapen, interviewed me for his podcast Hacker Practice. The interview is broad and covers everything from creativity to Big Data to my Data Engineering Teams book. This interview goes into deep into many of data engineering concepts and theories. Can...
In my book Data Engineering Teams, I separate out programming as a different skill than distributed systems. The section is the “Skills Needed in a Team” and talks about the various skills that a data engineering team needs. Several people have emailed me...
