There has been quite a bit of writing covering GPT and LLMs from data science and business perspectives. I haven’t seen much from the data engineering side. Let me share my perspective, having been in data and AI for a while and using LLMs before they became popular....
The results and analysis from my 2023 Data Teams Survey left a few open questions. Let’s revisit these questions with some answers. Methodologies and Size of Company Figure 1 – Methodologies Broken Down By Size of Company Using Them We see a few commonalities...
Between January 24, 2023, and February 28, 2023, I ran a survey to get more data for my latest book Data Teams, and to update my previous survey from late 2020. Overall, we had 81 respondents. This survey was designed to get information about how management uses data...
In the beginning, there was Google. Google looked over the expanse of the growing internet and realized they’d need scalable systems. They created MapReduce and GFS in 2004. They published the papers for them in the same year. Doug Cutting took those papers and...
I want to tell you a story about how my life changed. It wasn’t a cult, new religion, or programming language. A million monkeys changed my life. Ten years ago, I randomly recreated every work of Shakespeare. It was quite a project on the technical, news, and...
As a distributed systems person, I’m used to figuring out how to spread a problem out to the most number of computers possible. Spreading out a problem lets me leverage my resources far better and faster. However, we’re failing to apply this optimization...
