The Data-Centric Future

A paradigm shift in information systems is underway. Paradigm is a big word, and often misused. (Drama for advertising purposes.) But this one is real. In the coming years there’s going to be a fundamental change in how people think about information systems, how they design them, and what the systems can do. It will transform how the human service sector manages its information. This second stage of the Human Service Informatics blog will explore how.

I’ll start with a quick recap of the first stage of the blog. (A one-year blog followed by a five-year break deserves some explanation.)

I posted Human Service Informatics from 2013 to 2014 to lift up the sector’s chronic problems with managing information, and to look at emerging solutions. The main problems I saw were four: isolated agency silos, unsuccessful information system projects, barriers to producing performance measures, and uncoordinated demands on services providers.

All of those are partly the results of fragmentation. There are many voices at the human service table speaking many different languages:

Who has a stake in human service information? There are different roles. There are agency executives and program managers and front-line service providers. There are measurers of performance and program evaluators and social scientists. There are funders. And there are technologists who design and implement information systems. They are seated in different system locations: government agencies, nonprofit service providers, philanthropic foundations, professional associations, consulting firms and software development companies. And they have different areas of substantive focus: child welfare, homelessness, substance abuse, domestic violence, job training… the list goes on.

In the face of that fragmentation, there were encouraging trends toward sharing information. Data interoperability, common performance measures, and open civic data have the potential to wire together autonomous organizations and help the sector act as a responsive whole.

Yet it seemed to me that a key ingredient was in short supply: systems thinking about data on clients and programs. Stakeholders commonly wish for more holistic information about the population of clients in a single program, about individual clients served by multiple programs, and about whole populations at the intersection of programs. That kind of information makes it possible to understand population health trends, the effectiveness of referral networks, and collective impact. Producing higher quality data would require rethinking the boundaries of human service systems and learning to design data that can serve many different purposes.

That would also require a different way of thinking about information systems. It would mean acknowledging stakeholders’s real—though unstated—desire: to have a good interrogable model of the organization’s work and environment. That sets a much higher bar for what an information system must do. Its data must be as meaningful as possible, designed to be useful for open-ended inquiry and as-yet-unanticipated purposes.

And that, in turn, would call for a different approach to software development. The traditional emphasis on functional requirements and use cases would need to gently be put in the back seat. Instead, system designers would need to focus on understanding meaning first, so as to create more accurate and more holistic models to represent human service programs. (One part of building good models is recognizing and avoiding the funny-looking taxonomies embedded in so many systems.)

But then I hit a wall. I knew that this approach works. It’s been a minority view within software development thinking for decades. I’d learned it from the writings of a few methodological gurus. I’d applied it to our sector in articles on how to design better human service data, the importance of flexibility in human service system design and why information systems often fail to support performance measurement. I’d met others who were working along similar lines. But we, who focused on data and meaning—and who looked for information systems to support the coherence of a larger Whole—were clearly few and far between. Overall, the software development industry was (and still is) oriented toward simply delivering functionality.

I wondered why. I took a break from blogging. I studied the history of software development methods and systems thinking. I taught a course about it at NYU’s Wagner School. And then—to my astonished pleasure—I noticed that the world has been changing. Our minority has been growing and becoming more vocal. More and more people are focusing on data and meaning first.

The term being used to describe this is data-centric. In an article this month, Nick Lazzaro of the Forbes Technology Council wrote about a generational shift occurring: Being data-centric fundamentally means putting the data first, and not your applications.

Dave McComb of Semantic Arts has published an influential series of articles and books about this approach, most recently The Data-Centric Revolution. Over six hundred technologists have signed his Data-Centric Manifesto.

Another, from Peter Aiken and Todd Harbour, lays out a new set of values:

The Data Doctrine

Data Programmes Preceding Software Projects
Stable Data Structures Preceding Stable Code
Shared Data Preceding Completed Software
Reusable Data Preceding Reusable Code

That is, while there is value in the items on the right,
we value the items on the left more.

This is not a flash in the pan. It points to a real paradigm shift. I’ve recently written about how it emerged and basic principles that can support it.

Upcoming posts will explore the implications for our sector.

—Derek Coursen