Jesse+ by | Jun 19, 2019 | Blog, Data Engineering, Data Engineering is hard |
Note: This post was guest written by John Desmond. My preparation for the course began before I knew about the course, and before I realized that I wanted to specialize in data engineering. When I decided I wanted to learn programming, I hadn’t quite decided if I...
Jesse+ by | Nov 7, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
At DataEngConf Barcelona, I premiered a new talk about the importance of creating a data engineering culture. I share what a data engineering culture is and what management needs to do to be successful with Big Data. You can download the slides from the talk here and...
Jesse+ by | Oct 24, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
There is a common misunderstanding in data engineering that you can do everything you need to create a Big Data data pipeline with SQL. This notion is being promoted by some vendors and companies. They’re wrong and you can’t do all of your data engineering with SQL....
Jesse+ by | Aug 29, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
A common use case for using Kafka and Pulsar is to create work queues. The two technologies offer different implementations for accomplishing this use case. I’ll discuss the ways of implementing work queues in Kafka and Pulsar as well as the relative strengths of...
Jesse+ by | May 1, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
One of the benefits of teaching and consulting is the sheer number of organizations, teams, and people I get to work with. Since I deal with so many different groups, I can see patterns emerge much faster than others. One pattern I saw early on was real-time Big Data....
Jesse+ by | Apr 26, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
Creating real-time data pipelines bring new challenges. There are new concepts and technologies that you’ll need to learn and understand. To help you understand the basic technologies you need in a real-time data pipeline, I break it down into 4 general types. These...
