Any and every education reform design is going to fail for two reasons. The first is that the problem is not one that is solvable by “design” in the traditional engineering sense — the education system, including all its human elements, is too complex for that. The second is that the system as currently built contains feedback loops that damp out change.
At the Gov 2.0 Summit, Deputy Director for Policy for the White House Office of Science and Technology Policy,Thomas Kalil, referred to the challenge of educational software that improves the more students use it.
What would it mean to talk about a whole school system that improves the more students use it? I’ve heard the Department of Education’s Steve Midgely refer to school as a service and education as a platform — why not apply this kind of systems thinking to the Gordian knot of our education system, using the Internet as a lens and a platform model?
Applying the Web 2.0 model to education
The Web 2.0 model can be applied to education on two levels. The first is at the level of software that provides or supports educational experiences. The second is at the level of the human systems and workflows that are entailed by the day-to-day work of educators and students and citizens.
As a veteran software organization leader and management geek, I see deep parallels between the architecture of software platforms and that of human organizations. For example, in both cases, the question of quality has moved from “does it work as designed” (which is now considered table-stakes) to “is it designed to do the right thing?”
A software program that works exactly as intended but isn’t useful to its end users is a waste, as is a team that is executing to the letter of a Statement of Work at the cost of the real needs of its multiple and diverse stakeholders. So is an educational system that mass-produces graduates who are prepared for an agrarian/industrial economy when they will have to live and work in an economy powered by knowledge, collaboration and creativity.
One possible criterion for education as a platform is whether the underlying services can be mashed up to serve different sets of goals. As Clayton Christensen, Curtis Johnson, and Michael Horn outline in “Disrupting Class,” our educational system was originally created to prepare the elite to rule, then to prepare all people (or at least white men) for informed citizenship, then to provide all students with an equal opportunity to prepare for college or careers, and now to prepare every child for higher education. These changing goals have strained the current school architecture to the point where any number of responsibilities that schools have traditionally considered important, such as vocational training and the arts, are now being dropped in favor of “the basics”. There are many who argue credibly that the goals of school ought to go beyond the basics to include modern work-force preparation (learning how to learn, collaborate, invent, communicate, etc.) and global citizenship.
By this criterion, the inability of our education system to accommodate these multiple and shifting goals suggests that it does not have the architecture of a platform (at least not a well-designed one). “Disrupting Class” suggests that, much as the web disrupted many traditional business models, technology in the form of online learning will disrupt school. The argument is that computer-aided education and online learning will be welcomed into schools as a low-cost way to provide services they otherwise couldn’t afford. These new offerings may be paid for by parents or schools and may give students school credit. Although the technologies will initially be very poor substitutes for classroom teaching, they will be much better than nothing and the schools will welcome them as they lose the resources to provide these services for students in traditional ways. Over time the technologies will continue to improve until they revolutionize what teaching and learning look like.
This disruptive mechanism has intriguing implications for education as a platform. First, critically, given the assumption that technologies are going to improve, there is an implicit feedback loop driven by criteria for what “better” looks like and that allows publishers and providers of educational services to continually adapt and improve — more on that later.
Second, there is an implicit design approach that eschews traditional top-down, requirements-based engineering and assumes that the complex solutions that will someday serve education begin with very simple systems that then evolve in the market, much in the way the Internet evolved from the simple TCP/IP stack rather than the complex, traditionally designed OSI stack.
Third, I infer that innovation demands multiple, varied providers of computer-aided and online learning, which in turn calls for a layer of standardized interfaces, API’s, and delivery mechanisms that make it easy for great education innovations to spread virally in the way popular Internet innovations do.
Defining “better”
These implications for the content and software infrastructure of Education 2.0 translate in interesting ways to the parallel pedagogical and human infrastructures. First, and critically, in order to establish positive feedback loops for the work of students and educators there needs to be a set of criteria for what “better” looks like. Metrics in this area are the absolute crux of education reform and poor metrics are a major contributor to how innovations are damped out in the education system — again, more on that later.
Second, my own experience in leading large software organizations suggests that traditional hierarchical, silo’d human systems are inefficient and resistant to change, but that they improve dramatically when restructured to support self-organizing teams. Self-organizing teams are the organizational equivalent of “simple systems” — they evolve to support complex work with extraordinary capacity and adaptability. One key to self-organizing teams is to set goals (or in the language of emergence, “simple rules”) that drive positive feedback loops — such as “be the school everyone wants to transfer to”. Another is to remove constraints from the systems and allow for greater autonomy and innovation on the part of each member of the team. In industry, a classic example is to reward people on the basis of outcomes rather than seat time (great code doesn’t care if it was written in a suit from 9-5 or in pajamas at midnight.) What if students could advance based on outcomes rather than seat time in the classroom, and teachers were rewarded for many dimensions of student achievement and professional contribution?
Third, I am of the opinion that the distinction between formal learning (school) and informal learning (museums, Internet, community classes, affinity groups, etc.) is one that is both artificial and obsolete. In Education 2.0 there should be multiple providers of educational experiences, and standard discovery mechanisms that allow great experiences to spread virally as well as standard ways to give students credit for what they know and can do rather than for what classes they’ve sat through.
Student achievement beyond a test
There is broad agreement that testing in our educational system is fundamentally broken. Today, our educational feedback loops are driven by “the test” — state-wide standardized tests in the basics that are used to assess whether every student is able to perform to a certain lowest common denominator. Collectively, the results of these tests are used to grade schools and determine whether they will continue to receive funding. Indirectly, attempts to reward teachers for performance are often based, in part or whole, on student performance on these tests.
The unintended consequence of using student achievement on “the test” to define what “better” looks like is that the feedback loops in the educational system will damp out work and innovations that aren’t measured at the lowest common denominator. In some cases, teachers will spend up to half the school year in “test preparation”, which doesn’t leave time for any learning above and beyond the basics. There are horror stories of mathematically rich curricula developed at great expense and with deep thoughtfulness under National Science Foundationfunding that aren’t adopted because much of the mathematical depth that students gain from it is not reflected by “the test”. There is a disservice done to teachers who are faced with extraordinary obstacles, such as teaching students who come to school hungry or fearing for their safety, or teachers who create extraordinary learning environments where students learn to create and collaborate in ways that are not reflected by “the test.”
Education as a platform must support vibrant innovation in the area of metrics. States, assessment publishers, web start-ups, researchers, parents and teachers must be able to experiment with different ways to measure student achievement, and, indeed, with what things are important to measure. In a world of assessment innovation, a student portfolio might contain a combination of completed projects in addition to state test results, richer third-party assessment results, and innovative assessments of non-traditional skills such as collaboration and creativity. Colleges and employers might value this multi-dimensional view of a student more than just grades and standardized test results when evaluating applicants. Parents and students might take ownership of enriching their portfolio of assessments according to their own values. Publishers of curriculum and educational experiences might be able to improve their offerings based on a broad set of assessments of student outcomes — driving innovation in educational content. Administrators and states might be able to reward teachers for many different kinds of critical achievements.
What kinds of services would comprise an education platform that encourages innovation around metrics? Certainly data science is key — instrumenting how students and teachers interact with technology and digital content and capturing that data according to open standards; storing and cleaning that data with appropriate privacy considerations; performing data mining, analytics, and analysis of the data; and creating meaningful visualizations of the data are all areas that can be both services in support of education as a platform as well as areas for innovation in their own right. The data itself will be a sort of national treasure — the key to understanding what works and what doesn’t and the fuel for innovative services and applications.
Also key are mechanisms for discovering quality assessments — Web 2.0 mechanisms of crowd-sourcing, peer review, and rating are commonly invoked for sorting out the best of the best when there is an explosion of innovative products available. But to date, much education-related crowd-sourcing has assumed that teachers will voluntarily rate and recommend educational products. With the burden that teachers are already carrying, that additional volunteerism is not a realistic expectation. Fortunately, there are two communities that might well provide data on products either because they are motivated or as a side effect of using them: education researchers and families. How will education-as-a-platform leverage these communities? Are these the same mechanisms required for discovering exceptional educational experiences?
These kinds of services from the technology infrastructure also serve the human infrastructure. With educators, administrators, and schools using a much richer metric of “better” than state test results, the same feedback loops that currently damp out change can instead drive improvement. Schools (and informal learning channels) can differentiate themselves by offering educational experiences that support the learning goals of each student — as reflected by the unique portfolio that student is building. Schools themselves can actually “get better the more people use them.”
But how can these hypothetical innovative approaches to metrics (or for that matter curricula, pedagogy, digital textbooks and games or other innovations) spring up in a cumbersome, over-constrained system like our schools? Probably not through top-down design or national mandate. Perhaps, like online learning, an assessment ecosystem will have to evolve outside those constraints — where there is currently no or little competition. Perhaps parents and students looking to have a more complete picture of a child’s skills and strengths and gaps will be the initial market for innovative assessments. What parent doesn’t want to understand their kids’ achievements in a deeper way than letters on a report card? If a wealth of affordable multi-dimensional assessments were only a click away, how many families would use them and in so doing help make them better? Perhaps, like online learning, innovation in assessments will, in the best way possible, disrupt school and class?
This post was originally published by Marie Bjerede onO’Reilly Radar 28 September 2010
