Jesse+ by | Aug 1, 2018 | Blog, Business, Data Engineering
I’ve been seeing some questions about data pipelines lately. I realized I haven’t written a post that gives the level of detail necessary for a good definition of a data pipeline in the context of data engineering. Instead of just giving my opinion, I’ve brought...
Jesse+ by | Jul 25, 2018 | Blog, Business, Data Engineering |
Note: this is a guest post from Sanjoy Roy who is reviewing my Professional Data Engineering course. Since late 2014, I have been drawn into various analytics projects which required a good mix of skills for both data engineering and data science. There are a lot of...
Jesse+ by | Jul 18, 2018 | Blog, Business, Data Engineering |
There is a common beginner question for engineers starting out with Big Data. An engineer will do a post to a social media site saying “I need to know which Big Data technology to use. I have 3 billion rows in 10,000 files. The whole dataset is 100 GB. Is Big Data...
Jesse+ by | Jun 27, 2018 | Blog, Business, Data Engineering |
There are two different types of data engineering. There are two different types of job types with the title data engineer. This is especially confusing to organizations and individuals who are starting out learning about data engineering. This confusion leads to the...
Jesse+ by | May 1, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
One of the benefits of teaching and consulting is the sheer number of organizations, teams, and people I get to work with. Since I deal with so many different groups, I can see patterns emerge much faster than others. One pattern I saw early on was real-time Big Data....
Jesse+ by | Apr 26, 2018 | Blog, Business, Data Engineering, Data Engineering is hard |
Creating real-time data pipelines bring new challenges. There are new concepts and technologies that you’ll need to learn and understand. To help you understand the basic technologies you need in a real-time data pipeline, I break it down into 4 general types. These...
