Fundamentals of Metadata Management Uncover the Meta Grid and Unlock IT, Data, Information, and Knowledge Management (Ole Olesen-Bagneux) (z-library.sk, 1lib.sk, z-lib.sk)

Fundamentals of Metadata Management Uncover the Meta Grid and Unlock IT, Data, Information, and Knowledge Management Ole Olesen-Bagneux Foreword by Joe Reis

9 7 8 1 0 9 8 1 6 2 8 2 5 5 7 9 9 9 ISBN: 978-1-098-16282-5 US $79.99 CAN $99.99 DATA Whether it’s to adhere to regulations, access markets by meeting specific standards, or devise data analytics and AI strategies, companies today are busy implementing metadata repositories—metadata tools about the IT, data, information, and knowledge in your company. Until now, most of these repositories have been implemented in isolation from one another, but that practice lies at the core of problems with data management in many companies today. Author Ole Olesen-Bagneux, chief evangelist at Actian, shows you how to masterfully manage your metadata repositories by properly coordinating them. That requires a data discovery team to increase insights for all key players in enterprise data management, from the CIO and CDO to enterprise and data architects. Coordinating these repositories will help you and your organization democratize data and excel at data management. This book shows you how. • Learn what metadata repositories are and what they do • Explore which data to represent in these repositories • Set up a data discovery team to make data searchable • Learn how to manage and coordinate repositories in a meta grid • Increase innovation by setting up a functional data marketplace • Make information security and data protection more robust • Gain a deeper understanding of your company IT landscape • Activate real enterprise architecture based on evidence Fundamentals of Metadata Management “Fundamentals of Metadata Management presents a design and a new way of treating and managing metadata for resilience and practical management of metadata systems.” Jessica Talisman, senior information architect at Adobe “Metadata management must evolve beyond data to encompass information and knowledge coordinated by a data discovery team and unif ied via a meta grid architecture. This book of fers a practical blueprint.” Kalyan Kumar (KK), chief product officer, HCLSoftware “This is the missing link that will unify and elevate modern data management practices. A must-read for any data management professional.” Piethein Strengholt, author of Data Management at Scale and Building Medallion Architectures Ole Olesen-Bagneux is chief evangelist at Actian. He holds a PhD in information science from the University of Copenhagen and has worked within the field of data management and governance as a leader, architect, and practitioner for over a decade.

Praise for Fundamentals of Metadata Management Fundamentals of Metadata Management by Ole Olesen-Bagneux provides a vital perspective for anyone grappling with complex IT ecosystems. Metadata management must evolve beyond data to encompass information and knowledge coordinated by a data discovery team and unified via a meta grid architecture. This book offers a practical blueprint. It acknowledges the reality of fragmented metadata repositories and proposes a coordinated approach, rather than an unachievable single source. This strategic meta grid framework holds immense value for enhancing operational efficiency, ensuring compliance, and fostering innovation. By embracing the meta grid and empowering a data discovery team, we can move toward a more holistic and actionable understanding of our IT landscape, leading to better decision making and optimized resource use across the organization. Ole’s unique perspective is in drawing our attention to and emphasizing the prominence of the often overlooked element in any enterprise: the metadata. This adds considerable value to the entire industry as it forms the basis of how we look at data management, the meta grid architecture, and overcome becoming single-view monoliths of the IT landscape. —Kalyan Kumar (KK), chief product officer, HCLSoftware Ole Olesen-Bagneux has written a book that, at first glance, addresses classical issues in the context of data in companies, and that suggests how to distill and combine the best of existing approaches. Additionally, the book recommends new, sustainable organizational practices for continuous improvement. But what Ole really does is get us thinking about what we are missing out on, due to managing our data and IT landscape in a less-than-ideal manner. Prepare to be inspired! —Sabrina Schiele, data professional and enthusiast

If programs are the skeleton of the system, metadata is the brain. A sign of the maturation of the IT industry is the increasing awareness of the role and importance of metadata. I highly recommend this book as a starting point in your journey. —Bill Inmon Fundamentals of Metadata Management presents a design and a new way of treating and managing metadata for resilience and practical management of metadata systems. The meta grid is pragmatic and proposes an alternate methodology and framework for overcoming dysfunction in metadata systems architecture. The book presents ways to handle overspending with SLAs, vendor and contractor dependence, for more efficient metadata systems. Companies now have a roadmap for overcoming metadata weaknesses, as the book proposes a data discovery team to facilitate the implementation of the meta grid, and to navigate the tasks associated with the meta grid framework. —Jessica Talisman, senior information architect at Adobe Fundamentals of Metadata Management introduces the groundbreaking concept of the meta grid—a transformative architecture poised to reshape data decentralization, much like the data mesh before it. This is the missing link that will unify and elevate modern data management practices. A must-read for any data management professional. —Piethein Strengholt, author of Data Management at Scale and Building Medallion Architectures (O’Reilly) Ole is a spearhead in the quest for unifying business, information, data, and IT: platforms based on symbols, linguistics, rules, cognition, and formal logic must be understood together. —Thomas Frisendal, business development, data architecture, graph modeler, author, and ISO IQL graph standard contributor After delivering the data industry must-read The Enterprise Data Catalog, Ole doesn’t just evolve his concepts around data discovery and metadata management in Fundamentals of Metadata Management—he introduces the genuinely groundbreaking acknowledgement of the meta grid. It’s both a lens for seeing the “bigger picture” of metadata and a practical framework for tackling the messiest metadata management challenges we all face. Another engaging must-read for anyone brave enough to wrestle with metadata. —Tiankai Feng, author of Humanizing Data Strategy (Technics Publications)

Fundamentals of Metadata Management is the missing piece we’ve been waiting for—a gift to the entire data and AI community. Ole makes it unique by delivering the foundational knowledge for handling metadata and by showing how to push the possibilities further with his forward-thinking approach. This book leads the way, explaining in detail what the third wave of decentralization looks like and what it can unlock. —Yoann Benoit, cofounder and head of data at Hymaïa It is inspiring to see Ole Olesen-Bagneux articulate and describe so well what we’ve been building for years. Combining data mesh with integrated metadata and a shared vocabulary is invaluable for any large organization. This book is essential reading for anyone seeking practical insights into building a truly data-driven organization. —Gregor Wobbe, head of data architecture, UBS If we just build better toasters, nothing will change. As Ole Olesen-Bagneux portrays it, the meta grid has always been there, but we’ve not seen it like this before, so our eyes hurt. Long live the meta grid. —Säde Haveri, data management professional and entrepreneur Reading this book is like switching on the lights in a dark room—you suddenly see how fragmented your metadata has been all along. And the meta grid weaves these fragments into a coherent whole for clarity and control. —Dr. Simon Harrer, cofounder and CEO, Entropy Data Ole Olesen-Bagneux doesn’t offer yet another metadata framework built from scratch—he asks us to open our eyes to what’s already there. The meta grid concept he introduces reveals a foundational truth: we all need metadata to unlock the value of our data and AI solutions, but metadata repositories already exist across a variety of tools and teams. The real challenge is not invention—it’s coordination. Starting from a blank slate is never the case in real organizations. This book provides professionals with a methodology—not a standard—for making sense of fragmented metadata landscapes and transforming them into a coherent, strategic asset. A must-read for anyone serious about governing data in the real world, not just in theory. —Nino Letteriello, award-winning data and project entrepreneur, FIT Group

This book is a refreshingly honest and much-needed take on the real-world state of metadata in organizations. First, it expands our understanding of metadata—not just as documentation for data, but as a key enabler across IT, data, and knowledge management. Second, it highlights a crucial reason why so many technological solutions have failed: they proposed centralized architectures that ended up as unsustainable monoliths. The meta grid offers a compelling alternative—a “third wave of decentralization” that acknowledges and connects what already exists. As a lecturer and educator in the field, I see this book as a great source of methodological insight and a practical guide to follow for metadata management today. —Michele Valentini, data management practitioner, lecturer, and educator, FIT Academy Metadata quietly connects everything in a data-driven world. It’s time we recognize it as the meta grid. —Olga Maydanchik, data management practitioner and educator Ole Olesen-Bagneux’s Fundamentals of Metadata Management offers a compelling framework for streamlining metadata management, enhancing compliance, and reducing IT inefficiencies. A must-read for data governance professionals. —Bjarte Tolleshaug, senior consultant, certified data management professional (CDMP) The book effectively provokes thought on catalogs, data management, and, particularly, metadata management, highlighting what we have yet to uncover and what continually evolves within an organization—critical for AI initiatives. Effective metadata management is essential for building generative AI models, RAG systems, and RL frameworks. Organizing metadata throughout its lifecycle is crucial. —Gaurav Grigo, senior director, DDIT-R&ED, Novo Nordisk Ole Olesen-Bagneux did it again: he wrote a book to shape the industry! Metadata is more than “data about data”; it’s a way to understand the reality of your organization from IT systems to data, from information to knowledge. Metadata can be the connector. At the same time, metadata management has been done differently by professionals with varying purposes in siloed repositories. This reality has been overlooked until now. Ole has given us a novel perspective on what (and where) metadata is and how we can manage it. This book is a tie-breaker: a way to change our view, accept reality, and finally be able to use metadata in organizations. —Winfried A. Etzel, data governance professional

This is a true “fundamentals” book that will be relevant for a long time. Ole’s “meta grid” is deeply original and timely, providing the architectural clarity we need to zoom out and unite the management of inherently decentralized metadata. Framed as the third wave of data decentralization, it offers an implementable approach and a way of thinking when designing systems to make data discoverable for AI and humans without causing IT landscape disruptions or unnecessary lock-ins. I also applaud the concept of the “data discovery team” as a versatile way to embed metadata management into mainstream enterprise org charts, ensuring the organization can achieve a heightened state of data management. I highly recommend this book to technical people in data and AI and to management alike. —Karl Ivo Sokolov, managing partner, SPG Data Ole Olesen-Bagneux offers a welcome and thought-provoking alternative to traditional, technical approaches that often overlook the organizational and functional divides in metadata management. He introduces the meta grid—a simple yet powerful architecture that embraces the fragmented enterprise reality and provides a tangible, scalable framework for coordinating siloed tools, functions, and practices. At the heart of this approach is the concept of a data discovery team—designed to bridge gaps, enable alignment, facilitate discovery, and offer sparring across all levels of the organization. —Nikolaj A. Sabinsky, principal consultant, data program manager In Sufi metaphysics, the Lataif are subtle faculties of perception; layers through which hidden reality becomes discernible, not by force, but by refinement. Ole Olesen-Bagneux’s meta grid belongs to this lineage of thought. It does not impose structure; it reveals it. Like a constellation only visible to the attentive, it allows metadata, architecture, and governance to cohere without centralization. This is a work of deep clarity about metadata repositories but more than that, about how we come to sense what connects them. —Nagim Ashufta, founder and CEO, DRIVA GmbH

Ole Olesen-Bagneux Foreword by Joe Reis Fundamentals of Metadata Management Uncover the Meta Grid and Unlock IT, Data, Information, and Knowledge Management

978-1-098-16282-5 [LSI] Fundamentals of Metadata Management by Ole Olesen-Bagneux Copyright © 2025 Ole Olesen-Bagneux. All rights reserved. Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Aaron Black Development Editor: Angela Rufino Production Editor: Katherine Tozer Copyeditor: Shannon Turlington Proofreader: Vanessa Moore Indexer: Krsta Technology Solutions Interior Designer: David Futato Cover Designer and Illustrator: Karen Montgomery Interior Illustrator: Kate Dullea August 2025: First Edition Revision History for the First Edition 2025-08-04: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098162825 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Fundamentals of Metadata Manage‐ ment, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly, Actian, and HCL. See our statement of editorial independence.

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new “features.” —M. D. McIlroy, E. N. Pinson, and B. A. Tague, “UNIX Time-Sharing System: Forward,” Bell System Technical Journal 57, no. 6 (1978): 1899–1904. The truth was a mirror in the hands of God. It fell, and broke into pieces. Everybody took a piece of it, and they looked at it and thought they had the truth. —Rumi, Fihi Ma Fihi (It Is What It Is)

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix 1. Toward Holistic Metadata Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Metadata Management Happens in Many Places 1 The Data Discovery Team 3 The Meta Grid: The Third Wave of Data Decentralization 6 Microservices 7 Data Mesh 9 Meta Grid 10 Summary 13 Part I. Metadata Repositories for IT, Data, Information, and Knowledge Management 2. Metadata Repositories for the IT Landscape. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 What Is Metadata? 17 Types of Metadata in IT Landscapes 20 What Is a Metadata Repository? 21 Driver: The Many Waves of Metadata Repositories 23 Purpose: Core Capability 27 Place: Metadata Repositories at Various Levels 28 Structure: The Metamodel in Metadata Repositories 29 Summary 32 xi

3. IT Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Endpoint Management System 35 Integration Repository 37 Asset Management System 41 Configuration Management Database 43 IT Service Management System 46 Enterprise Architecture Management Tool 47 Metadata Repositories for IT Management 50 Summary 51 4. Data Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Data Catalog 53 Database Model Management 55 Other Metadata Repositories for Data Management 57 Data Warehouse 58 Data Lake 58 Data Lakehouse 59 Data Pipeline Tools 59 Data Quality Tools 60 Identity and Access Management 60 Rebundling of Data Management Technologies 60 Metadata Repositories for Data Management 61 Summary 62 5. Information Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Records and Information Management System 63 Record-Retention Scenarios 64 The Organizational Aspect of RIMS 65 RIMS As a Metadata Repository 66 Information Security Management System 67 The Organizational Aspect of ISMS 68 ISMS As a Metadata Repository 69 Data Protection Repository 70 The Organizational Aspect of a DPR 72 DPR As a Metadata Repository 73 Business Process Management System 74 Metadata Repositories for Information Management 76 Summary 78 6. Knowledge Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Content Management System 81 Knowledge Management System 82 xii | Table of Contents

Learning Management System 84 Quality Management System 85 Collection Management System 86 Metadata Repositories for Knowledge Management 87 Summary 89 7. Why We Have Been Doing Metadata Management Wrong. . . . . . . . . . . . . . . . . . . . . . . 91 Different Practices Have Different Metamodels 92 Dark Metadata 96 Other Applications and Domains 97 A Possibility: The Coming Together of Teams and Technologies 98 Money 99 Structured Versus Unstructured 100 Summary 101 Part II. Metadata Repositories Must Be Coordinated by a Data Discovery Team 8. The Good, the Bad, and the Ugly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Setting the Stage 106 The Good 107 The Bad 109 The Ugly 112 Summary 113 9. The Data Discovery Team. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Our Problem: Managing Multiple Truths Across Teams 117 Conway’s Law 120 The Metadata Monolith 121 The Solution: The Data Discovery Team 122 What Is the Data Discovery Team? 124 On the Political and Technological Mess of Companies 126 Embracing the Multiple Truths and Providing a Way Forward 126 How the Data Discovery Team Collaborates 127 Collaborating with Enterprise Architects 127 Collaborating with the Data Protection Officer 128 Collaborating with the Chief Information Security Officer 130 Collaborating with Records and Information Management 131 Collaborating with Data Science Teams 132 Summary 133 Table of Contents | xiii

Part III. Metadata Repositories Should Be Connected in a Meta Grid 10. What Is the Meta Grid?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 The Meta Grid Manifesto 137 The Meta Grid Is the Third Wave of Data Decentralization 138 The Meta Grid Unlocks Single-View-of-the-World Monoliths 138 The Meta Grid Is Never Finalized 138 The Meta Grid Is Simple, Small, and Slow 139 What the Meta Grid Is and Is Not 139 Documenting the Meta Grid 141 Examples of the Meta Grid 144 Data Types 144 Applications 146 Data Models 147 Integrations 148 Data Lineage 149 Servers 150 Organization 151 Processes 153 A Real-World Meta Grid Architecture 154 Questions 154 Answers 154 Summary 157 11. The Meta Grid Contextualized. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 You Don’t Build the Meta Grid—You Uncover It 160 Uncovering Unconscious Meta Grid Architectures 161 Data Driven (Ambition) 162 FinOps 164 Intake Funnel 165 The Meta Grid Is a Nuclear Architecture 166 Energy 168 Expansion 168 Explosion 172 Microservices, Data Mesh, and Meta Grid 173 Microservices in the Meta Grid 173 Data Mesh in the Meta Grid 175 Meta Grid Must Not Turn into Data Mesh or Microservices 176 Technologies That Support the Meta Grid 178 Summary 180 xiv | Table of Contents

12. The Benefits of the Meta Grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 The Meta Grid Is Not a Technology 183 Better Overview of the IT Landscape 184 Smoothly Implemented Metadata Repositories 184 Empowered Owners of Metadata Repositories 185 More Secure Data Governance for Both Risk and Privacy 187 A Stronger Possibility of Data-Driven Innovation 187 Reduced Cost of the IT Landscape and Consultancy Support 187 Examples of Cost Reductions 188 A Greener IT Landscape 189 The Meta Grid Is a Technology 190 Create a Knowledge Graph of Metadata Across Metadata Repositories 190 Search the Meta Grid Conversationally with Generative AI 193 Perform the Meta Grid Automatically with Agentic AI 199 Summary 199 13. The Data Discovery Team and Meta Grid As a Team Topology. . . . . . . . . . . . . . . . . . . 201 Afterword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Table of Contents | xv

Foreword I first met Ole several years ago, shortly after he published his book, The Enterprise Data Catalog (O’Reilly). Sometimes, you meet someone who has an uncommon clarity of thought and a gift for making the complex feel obvious. When I invited him onto my podcast, he struck me as a deeply thoughtful and cerebral person. We’ve since become good friends, and I was honored when he asked me to write this foreword. Ole starts this book with a story that will resonate with anyone who has spent time in the trenches of enterprise technology. He recounts meeting a chemist at a large com‐ pany who showed him a seemingly endless list of IT systems, the “ITSO.” When asked what all these systems did, the chemist grinned and admitted that nobody knew. A significant portion were likely doing nothing, but turning them off was too risky. What if a factory shut down? The cost of the unknown was greater than the cost of maintaining the bloat. This Kafka-meets-Dilbert anecdote is not an outlier. Sadly, it is the default state of affairs in most organizations. We are constantly grappling with foundational ques‐ tions: What applications do we have? What data do they hold? How are they connec‐ ted? The uncomfortable truth is that, often, no one can provide a complete and trustworthy answer. The problem isn’t a lack of information but a surplus of it scat‐ tered across dozens of uncoordinated, siloed systems. Fundamentals of Metadata Management brilliantly dissects this vicious cycle, but its actual value lies in the pragmatic path it offers us to escape. The book argues that for too long, we have been doing metadata management wrong. We’ve viewed it through the narrow lens of data management alone, leading to the proliferation of what Ole calls “single-view-of-the-world monoliths.” Each management discipline, whether IT, data, information, or knowledge management, has independently mapped the same landscape, creating a cacophony of conflicting truths. This isn’t just a technical failure but also a human one, driven by the complex sociology of employees, consultants, and vendors, each with their incentives that often perpetuate the chaos. xvii

When the AI revolution hit a few years ago, I could see the wheels turning in Ole’s mind. The result is the meta grid. Ole positions the meta grid as the “third wave of data decentralization,” a natural evolution following microservices and data mesh. Its power is not in high-speed data exchange but in logical cohesion. Ole’s most critical insight here is that you don’t build the meta grid—you uncover it. It already exists, unconsciously, in the fragmented relationships between your existing repositories. This book provides the methodology to make that grid conscious, transforming it from a source of chaos into a “nuclear architecture,” small yet dense with the energy to power a more rational and cost-effective enterprise. What makes this book so essential is its grounding in reality. It does not preach an idealistic future that requires ripping and replacing everything. Instead, it provides a methodology for understanding, contextualizing, and coordinating the metadata repositories you already have in place. The meta grid has been hiding in plain sight this whole time! This book serves as a guide for leaders struggling to rationalize their application portfolios, map their asset inventory, and update their architectures, par‐ ticularly in the context of a rapidly evolving AI landscape. The work detailed in these pages provides a blueprint for moving from a state of expensive ignorance to one of informed, strategic control. The benefits are tangible: reduced costs; enhanced security; and a more adaptable, greener IT landscape. But the ultimate payoff may be the most timely. By creating a robust, coordinated, and logical map of your enterprise, the meta grid provides the perfect, high-quality con‐ text needed to unlock the true potential of AI. This is very much needed today. The meta grid is the foundation upon which effective conversational and agentic AI can be built. This book gives you the tools to finally answer the fundamental questions and, perhaps, to confidently start turning off the systems that no one knows anything about. — Joe Reis Best-selling author and global educator July 2025 xviii | Foreword