Driven-by-Data, Constrained-by-Data
The challenges of changing legacy architecture
Many CDOs are tasked with the ambitious goal of transforming the status quo to a data-driven business, a business where everyone’s decisions can be supported by easy, timely access to relevant, high quality data. The starting point for this journey is often a difficult one, in terms of understanding where you are before even planning where to go. And whilst technology is only one part of achieving this goal, it is an enabling foundation for building a culture of data ownership, collaboration and innovation.
Existing data architecture, usually full of a myriad of data tools and operational data stores, is a major constraint in making progress. The issues experienced are common and compound upon each other: siloed data stores; the data spaghetti of point-to-point integration; no shared definitions of data; no understanding of the flows of data; inconsistent data quality; and fundamentally weak data governance. Even within the field of data management, the tools for workloads such as metadata management, data integration, data quality, data mastering and data collaboration are themselves siloed, complex and difficult to orchestrate. Ever-more cost and time is incurred in integrating and maintaining these data management functions to cover the whole lifecycle of data.
A different approach is needed, one where using internet-scale technology at an enterprise level enables data, its rules and transformations to move out of the silos and into a shared, collaborative set of services that can be used by whole organisation to source, create and analyse high quality data. This is the technology foundation that Datasynthesis have developed in their platform to enable the CDO and the Data Office to actively take ownership of data, its quality and its management.
Actively Governing Data
Data governance as a document or as a driver
Top-down data governance is rarely successful when it comes down to the level of data consumers. Heavy documentation of policies can seem difficult to access and far removed from the day-to-day activities of gathering, organising and analysing data.
Metadata management tools can help in sharing understanding of what data means, what restrictions apply to it and where to find it. However, these tools typically present an out-of-date snapshot of a still complex data architecture, often excluding unstructured data from the picture all together. Whilst they are more bottom-up and supportive of data consumers, they remain a passive tool that has limited ability to actively drive change.
The approach taken in the Datasynthesis Platform is build further on the role of metadata in data quality management and indeed to take it to its logical conclusion of using metadata to drive data integration. So instead of simply assisting in the understanding of data, metadata is used to enable data experts to define data, its rules and transformations but also its flows around the enterprise, all without the need for technology knowledge or coding. This no-code approach combined with full utilisation of metadata enables data governance to move from its passive roots in documentation to turning data policy directly into data operations.
Pathways to a Modern Data Architecture
Incremental improvement and scale
Migration to more efficient ways of managing data quality is a difficult problem in the context of legacy systems in existing data architectures. A big-bang approach to modernising data architecture would be almost impossible to implement and more importantly, almost certain to fail.
The Datasynthesis Platform takes a much more flexible, non-invasive approach where it can be incrementally take on certain workloads whilst sitting alongside your existing set of tools. As more data management workloads are migrated across, no new software needs to be installed and the services managed by the platform automatically scale to meet the new processing, storage and concurrency needs. Gradually the complexity and costs of your existing data architecture can be drastically reduced, whilst at the same time data quality and enterprise consistency are improved.
Achieving Data Standards
Improved understanding of data
Whilst good data quality local to any system is to be encouraged, the inconsistency of data between systems can result in enterprise data quality being compromised. This inconsistency and lack of shared understanding of data is at best confusing but is often the root cause of resources being expended on costly integration and reconciliation efforts.
The Datasynthesis Platform uses a Common Data Model (alternatively known as a Canonical Data Model) to represent all data entities and relationships in their simplest possible form. For each operational system only one transformation is needed from Canonical form to that of the system’s local data model, allowing enterprise consistency to be maintained through this common understanding used by all systems. Combine this enterprise level understanding with industry standards such as FIBO and you have a platform for increased data collaboration and reduced reconciliation.
Real-Time Enterprise Data Quality
Pro-active prevention of the spread of data issues
Many errors are only identified downstream in the destination systems that need the data. Without the propagation of the correction back to the data source, then at best this results in multiple data teams identifying and duplicating corrective action. At worst, data errors are corrected locally in just one system, leaving the other users and systems unaware of the issue. This siloed approach to data quality, often with multiple data quality tools used across multiple departments, leads to inefficiency, expense and inconsistent data quality at an enterprise level.
The cloud-native scalability of the Datasynthesis Platform enables a much more pro-active approach to be taken to enterprise data quality. Rather that manage data quality downstream on a local value-by-value exception basis, data quality can be managed upstream as an enterprise process. Hence no waiting needed for batch processes to complete, data quality for the entire enterprise can be monitored in real-time on data quality dashboards and issues mitigated before data errors have spread to downstream systems and users.
Data Quality by Data Experts
Decouple IT and data to the benefit all
For many data experts, it is frustrating that many organizations use data quality tools that need programming skills to monitor and improve data quality. And for IT experts, it is frustrating to receive seemingly ad-hoc requests from data professionals to correct some data issue, particularly when these requests compete for time with the project deliverables you have been tasked with.
Adopting a self-service approach, the Datasynthesis platform offers no-code data governance and data quality management. This frees the data expert from the dependency on IT time and resources needed to address data issues and increases the productivity of the IT expert through leaving data issues to the people who know the data best. Combining this approach with the scalability of the platform enables data experts to ask questions such as “Tell me about any data issues right now” and receive enterprise-scope responses in real-time, delivering a new level of transparency for enterprise data quality.
Data Collaboration for All
Self-service data access and analysis
Business users, data experts and IT staff all want easy access to high quality data. The Datasynthesis Platform offers everyone the data they need, where they need it and when they it. A single set of services to find data, to improve data quality in tools such as Slack and Microsoft Teams, to access data in all major programming environments and to analyse data in BI tools like Microsoft Power BI and Tableau. The Datasynthesis Platform is the technology foundation for encouraging data collaboration, a foundation that can help the CDO take control, reduce complexity, reduce costs and encourage innovation with data.
Next steps
Click here to find our more about how data governance, data integration and data quality can be synchronised and fully integrated.
Please contact us if you would like to discuss how your data policies can automatically and directly define data operations.