Jesse+ by | Apr 24, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
The move from batch to real-time Big Data represents change. It will entail using brand new technologies and concepts that you haven’t dealt with before. Batch Big Data Let’s start off by defining batch Big Data. For batch, all data must be there when the processing...
Jesse+ by | Apr 11, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
I wrote a post for the O’Reilly data blog going into my latest thoughts and views on data engineers versus data scientists. I continue on to talk about machine learning engineers.
Jesse+ by | Mar 28, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
There’s an elephant in the room with Big Data. If an organization tries to half-ass their way through a Big Data project, they’re going to fail (usually a 5-10% odds of success). Given this really low success rate, should you even do Big Data? When I worked at a Big...
Jesse+ by | Feb 21, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
As people start with Big Data, they go through the list of necessary skills. One of those crucial skills is to program. The question arises — how good does a person’s programming skills need to be? This is because programming skills are on a wide spectrum. There...
Jesse+ by | Jan 24, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
In my seminal post On Complexity in Big Data I talked about the level of complexity increase with Big Data. The post itself focused on Big Data batch systems. I didn’t really cover real-time complexity increases when dealing with Big Data. In the post, I argue that...
Jesse+ by | Dec 6, 2017 | Blog, Business, Data Engineering, Data Engineering is hard |
Writing your own distributed system shouldn’t be a task you undertake lightly. Too often, I’m seeing teams create their own distributed system. In my experience, this is because they don’t know or think about all of the ramifications of creating their own distributed...
