Sitting in a café in Brisbane, finally meeting my manager in person for the first time over coffee, he lays out the obstacles preventing my team from publishing analytics-driven actionable business value. I take a sip of my dirty chai latte – lukewarm, even though I had asked for extra hot (Every African knows a hot brew must scorch the tongue to be enjoyable!). I resolve to shelve this “distribution problem” until after the Easter holidays – four whole weeks away. Then the bombshell drops, my manager casually mentions that this issue needs to be addressed for the upcoming strategy session – in just two days!
Snapping back to reality, I realise the issue is not about analytics or algorithms. What he is describing is, at its core, a data management problem. I have been here too many times before, where the real work of data science is not glamorous algorithms and meaningful dashboards but, is the unglamorous foundations of managing data well. It hits me: all these years, I thought I was doing data science. In reality, I have been doing data management.
What is Data Management?
Data science is often romanticised as machine learning algorithms, experimentation, and storytelling/insightful dashboards to drive action. While these activities play a critical role in deriving insights, the fundamental backdrop of data science is data management. Without proper data handling, even the most sophisticated initiatives and models fail to produce reliable results.
Effective data management encompasses activities such as data governance, architecting, structuring, security, data storage and operations to ensure data is useful and meaningful. In many ways, data science is not just about finding patterns but about establishing a solid data foundation that supports advanced analytics. For our organisation’s LLM use-case, drafting a data science/advanced analytics strategic framework means confronting these gritty realities head-on.
Key components of Data Management for Data Science
At its core, data management is about empowering organisations to leverage data effectively -in alignment with regulations and policies – to unlock its greatest potential and value. I have had this experience across several organisations when trying to deliver business value and am sure many data science practitioners and leaders have faced the same. Often, the challenges organisations face are not about exploration, experimentation, or data science techniques. They have often been about how data is managed within the organisation i.e., the available tools and architecture, data governance and quality, and increasingly, security protocols.
Below are four key data management components which organisations should get right before expecting data science/analytics teams to deliver business impact – and the same ones my manager unknowingly flagged over that lukewarm latté:
a. Data collection and acquisition
This step is about ensuring that all required and relevant data is collected, and it is available and accessible when needed in the environments and formats necessary to support meaningful analysis. Poor acquisition leads to “garbage in, garbage out”- like my lukewarm dirty chai, incomplete data leaves stakeholders unsatisfied. Data scientists and analysts waste weeks hunting for datasets instead of deriving insights.
b. Data storage and organisation
This builds on the first point to include the structuring of data in databases, data lakes, or cloud-based storage systems to ensure accessibility. Good storage practices support efficient querying, reduce redundancy, and make collaboration across teams easier. My team’s “distribution problem” stemmed from data being there but not accessible by analytics environments and, sadly, not usable.
c. Data governance
Governance is about enforcing compliance, ethics, security, and quality control. For implementing LLMs, this meant defining access controls, audit trails and bias mitigation. Without consistent and transparent governance, insights lack credibility – like serving lukewarm latté and calling it “premium.”
d. Data operations (DataOps)
Manual pipelines delay insights. DataOps establishes processes for continuous data integration, monitoring, and automation to streamline analytics workflows. It supports experimentation and makes for more efficient collaboration and utilisation between data engineering, analytics, data science and business teams. By implementing DataOps, organisations can improve the speed and accuracy of their data-driven decision-making processes, ultimately making data science more effective. It is the barista who ensures your latté is extra hot and delivered fast.
Why Data Management is essential for business success
My manager’s challenge is not unique. Businesses often chase data science and AI hype without fixing their data foundations. Poor management leads to inaccurate insights, faulty predictions, and misguided strategic decisions. Companies that invest in strong data management practices gain a competitive edge by:
• Making informed decisions based on high-quality data.
• Reducing errors and inefficiencies in data processing.
• Accelerating the deployment of data-driven solutions.
• Ensuring compliance with data privacy regulations.
• Enhancing collaboration between teams by providing consistent, well-managed data.
Conclusion
Data science may be powered by machine learning and artificial intelligence, but it is built on data management. Without a solid foundation (extra hot milk, fresh spices, efficient customer service), even the best ingredients fall flat. Similarly, without effective collection, storage and organisation, governance, and DataOps, data science becomes an exercise in frustration.
As I leave the café, I realise the urgency of my manager’s ask is not a crisis- it is a wake-up call. Data management is not a “backstage” task. It is the main act!
Organisations that recognise the importance of data management will not only survive the next strategy session, but they will also unlock the full potential of their data science efforts and thrive in a data-driven future.
Lawrence is an expert analytics and data leader with a proven track-record in developing and automating solutions for complex analytics and data problems across numerous industries. He is passionate about a people-centric approach to people management that develops into life-long professional relationships. Outside of work, Lawrence is an avid sports follower, tennis umpire and plays acoustic guitar with a lot of love and very little talent.
See Lawrence’s profile here
Collaborative Accountability of Human and AI within Australia
