- Shane Murray transitioned from studying math and finance in Sydney to leading AI strategy at Monte Carlo Data in New York.
- He pioneered online multivariate experimentation long before A/B testing became mainstream, with notable examples from various industries.
- Shane discusses the digital transformation at The New York Times, emphasizing the shift from print to digital subscriptions and the role of data in this evolution.
- He details the technical journey of building and scaling a data platform on Google Cloud Platform and BigQuery.
- Shane explains the concept of data observability and its importance in monitoring data health across complex stacks like Snowflake and Databricks.
In this episode of Unapologetically Technical, I interview Shane Murray, Field CTO at Monte Carlo Data. Shane shares his compelling journey from studying math and finance in Sydney, Australia, to leading AI strategy at a major data observability company in New York. We explore his early work in choice modeling and pioneering online multivariate experimentation long before A/B testing became mainstream, including fascinating examples from cruise lines, American Express, and even cultural surprises from eBay’s expansion into China.
Shane offers deep insights into the challenges and triumphs of digital transformation at The New York Times, discussing the shift from print to digital subscriptions, the critical role of data in that evolution, and the technical journey of building and scaling their data platform on GCP and BigQuery. He details managing large, cross-functional data teams using a hub-and-spoke model and the importance of fostering high-integrity leadership.He details managing large, cross-functional data teams using a hub-and-spoke model and the importance of fostering high-integrity leadership.
We then dive deep into Monte Carlo Data, defining data observability and the crucial concept of “data downtime” (TTD + TTR). Shane diagrams Monte Carlo’s architecture, explaining how it uses agents, metadata, and query logs to provide lineage and monitor data health across complex stacks (Snowflake, Databricks, etc.). He addresses the “build vs. buy” dilemma for observability tooling and shares his perspective on solving solved problems.
Finally, Shane outlines how observability is crucial for emerging AI/ML workflows like RAG pipelines, discussing the monitoring of vector databases (like Pinecone), unstructured data, and the entire AI system lifecycle, concluding with a look at Monte Carlo’s exciting roadmap, including AI-powered troubleshooting agents.
Join us for a technical discussion covering experimentation, data platform architecture, observability, the practicalities of scaling data teams, and the future of reliable AI systems.
Don’t forget to subscribe to my YouTube channel to get the latest on Unapologetically Technical!
Interested in exploring how to ensure the accuracy and reliability of your data? You might find the resources and information available on the Monte Carlo Data website valuable.
Discover solutions for data observability and learn how organizations are tackling data downtime to build trust in their data pipelines.
Discover solutions for data observability and learn how organizations are tackling data downtime to build trust in their data pipelines.
Feel free to check it out here:
Frequently Asked Questions (AI FAQ by Summarizes)What was Shane Murray's career transition?
Shane Murray transitioned from studying math and finance in Sydney to leading AI strategy at Monte Carlo Data in New York.
What innovative approach did Shane pioneer in the field of experimentation?
Shane pioneered online multivariate experimentation long before A/B testing became mainstream, with notable examples from various industries.
How did The New York Times adapt to digital transformation according to Shane?
Shane discusses the digital transformation at The New York Times, emphasizing the shift from print to digital subscriptions and the role of data in this evolution.
What is data observability and why is it important?
Shane explains the concept of data observability and its importance in monitoring data health across complex stacks like Snowflake and Databricks.
What dilemma does Shane address regarding observability tooling?
He addresses the 'build vs. buy' dilemma for observability tooling and shares insights on solving existing problems.
How does observability relate to AI/ML workflows?
Shane outlines how observability is crucial for emerging AI/ML workflows, including RAG pipelines and monitoring vector databases.
What does Shane emphasize about leadership in data teams?
Shane emphasizes the importance of fostering high-integrity leadership within large, cross-functional data teams.
