The growing demand for data is just beginning as we sit at the doorstep of that age of AI. Attention to data governance and getting the strategy right will enable successful organizations to balance governance with access and succeed in the next era of innovation. A federated model for modern data governance is beginning to gain traction. This strategy represents a shift from governing all data through a central authority to incorporating a cross-section of professionals into overseeing data governance.
A Federated approach to data governance is a collaboration between centralized IT and domain-level teams that produce data within their domain. A domain could be a line of business, business function, or geography. Domains are defined by Bounded Context, which essentially means that each domain has a different role and rely on unique terms and rules to accomplish its objectives as efficiently as possible. These rules and terms and objectives define the boarder of a domain. For example, Otis Elevator Company sells lifts in the UK domain and elevators in the US domain, and the definition of a customer in the sales domain may have a slightly different meaning than a customer in a support context.
The IT and domain-level teams have specific roles in a federated governance approach. Central IT manages high-level governance policies applicable across domains and enforces standards to ensure interoperability. Domain teams focus on policies specific to the data they collect and manage within their domain.
Essentially, IT creates frameworks flexible enough for domains to manage their data so it works best for them but stays within the broader governance framework.
This approach is the cornerstone of a modern data mesh architecture that enables greater agility and access to data across the organization.
There are several benefits of pursuing a federated governance approach to data management.
Federated data governance enables organizations to implement more distributed systems that are more agile, innovative, efficient, and secure.
The typical data governance strategy consolidates all the responsibility of ensuring data is secure, private, accurate, available, and usable in the hands of IT. It is incredibly challenging to manage this complexity across domains, each with different policies and terminology. Implementing one-size-fits-all controls that default to the most restrictive rules is the simplest way to manage this complexity and ensure data integrity but blocks access to data.
With IT departments typically risk-averse, policies tend to be stricter than they need to be, restricting access. Without the capacity to understand the nuances of the data they protect, rules that apply to one data set may not be relevant to others. Policies covering one part of a data set are typically applied to an entire data assset. This approach often limits access to legitimate users who can utilize it to drive real value.
With the access to trustworthy data throughout organizations so imperative, those that limit access will be at a competitive disadvantage.
Over-restrictive data governance and rules that are not applicable or sensible for specific data sets can actually lead to security vulnerabilities. When policies do not meet the needs of data users, they tend to find ways around them. This approach can lead to risky, unknown vulnerabilities.
Typically, those who collect data have a much better knowledge of what it represents and the nuances around its sensitivity and quality. This fact puts them in a much better position to manage the governance rules around it. Different business functions and geographies also have different policies and regulations. GDPR is only required in the European Union, and different businesses have varying data quality thresholds. For example, it is much more critical that transactional data is accurate compared to marketing estimates.
Even within data sets, some columns may contain PII data, but others may not. Access restrictions applied uniformly across the data set limits access to valuable information. Those closer to the data have a better handle on what parts of a data set are more sensitive and which are not and are much better informed to create more relevant governance policies. For example, instead of barring sales and marketing access to insurance adjusters' claims data, authorities in the insurance adjustment domain can use fine-grained access controls to mask sensitive columns within a table to enable sales and marketing to access the table without exposing sensitive data. Marketers can now use this data for more accurate market segmentations and sale targeting.
In financial domains, specific rules surround how sensitive financial data is handled. Applying these restrictions to less sensitive data is unnecessary and limits access to valuable data.
Domain teams also know where their data may have weaknesses or quality issues and, more importantly, why data sets have specific problems. With greater insight into how the data is collected, they better understand its quality and can control how certain professionals use it. Or they can simply limit access to the specific columns causing the issues.
When business domains are free to make decisions within their area of expertise, processes can be much more agile. Operators don’t need permission to make changes within their domain and can react to changing needs. For example, sales and marketing teams can respond quicker to new opportunities because they don’t have to get permission from central IT to access data that can help them. R&D teams can explore new data sets without waiting to get authorization from central IT but can ask the more accessable domain team for access. In most organizations, IT is typically overextended, so delegating decision-making to domains can also reduce the burden on these teams. Greater autonomy also empowers individuals, which drives greater participation and engagement.
While federated data governance can deliver a wide variety of benefits, implementing it can be tricky, requiring a well-defined strategy.
The first challenge is defining the roles and responsibilities of each domain. To avoid domain overlaps conflict and reduce redundancies, central IT must determine which domains are responsible for what data sets.
Central IT also has an incredibly essential role to enusre federated data governance succees. They must maintain the right balance between limited central governance and the autonomy of domains. Most importantly, central governance must enforce standards that ensure interoperability across domains. If groups implement different technology stacks that are not interoperable new data silos will emerge in place of the ones that were torn down.
There must also be a corporate-wide data quality standard to ensure shared data can be trusted. These requirements must be defined and enforced by the central IT department.
A central technology framework and governance rules need to be established and enforced.
As organizations plot their journey toward federated data governance, they can take concrete steps to smooth the transition.
As discussed earlier; frameworks are key to federated data governance. They must work for domains and central IT, and be smart. Federated governance can’t evolve into different departments fighting for more control but needs to focus on figuring out who is in the best position and most knowledge to most effectively implement rules. As technology and environments change, frameworks also need to be able to adapt to remain efficient.
As control and authority are distributed across the organization, disputes will inevitably ensue. What happens when multiple domains assert authority over a particular data set? Before embarking on data federation, there must be a predefined system for working out issues similar to how the US Constitution established the court system to resolve legal matters.
With a federated system, authority is distributed across a wide range of individuals. To be successful, these individuals need to understand what their responsibilities are. By documenting roles and responsibilities, confusion around who does what can be reduced.
There is no one-size-fits-all federated data governance strategy, so it must be allowed to evolve. A center of excellence can capture what is working and what needs improvement. A central depository for best practices helps everyone improve their processes.
A central data mesh platform can be beneficial in supporting a federated data governance strategy. A central platform can be the technological foundation that sets the standard that ensures interoperability. Managing governance controls independently but using similar technology and controls helps support a federated framework. A platform can be the mechanism that defines domains.