What is the measure of success?

The stock market is at record-setting highs. Graduation rates are soaring. The unemployment rate is low. Advancing technology makes it easy for us to have a face-to-face conversation with someone across the globe, ask Alexa what the capitol of Vermont is, or to order a pizza without talking to anyone. So, why don’t we feel successful?

Maybe it’s because the traditional indicators for success don’t capture the full picture.

A high school diploma doesn’t necessarily indicate that a student is career- or college-ready. The unemployment rate doesn’t consider those that are underemployed or have given up on finding full-time work altogether.

The old adage, “You get what you measure,” may be true, but only if we’re not taking for granted what we’re measuring, just because we always have. That’s because there might be better ways to support individuals and our communities, by fully understanding what we are measuring, as well as by taking advantage of advances in data collection and technology to better measure our success in critical areas like education and the economy.

You might think we’d be satisfied if our children get good grades, graduate from high school, and ace their college entrance exams. These are all traditional markers of progress for young people in the United States. And, they are important indicators of how well our schools and students are doing.

While these accepted measures continue to be the pistons driving the educational machine, there is increasing evidence that they don’t deliver a full picture of our children’s readiness for career and life. That’s because too much reliance on some accepted measures has created an incomplete assessment of how well our schools are performing, and so may be hindering their ability to ensure that students are ready to face a rapidly changing world when they graduate.

The good news is that some schools and organizations are actively experimenting with promising alternative models.

Imagined and narrated by Stacy Caldwell, executive director of the Mastery Transcript Consortium. A tale of education where there are no “grades,” class is work and work is class, and learning and success thrives in a wonderland of systematic disruption.

The Report on Grades

We are all familiar with the report card. Grades become standard in good part because of compulsory education laws and the reality of dealing with rising numbers of students (K – 12 enrollment almost tripled between 1870-1910), making it more difficult for teachers to provide in-depth feedback. For efficiency’s sake, more than as a pedagogical tool, grades must have made sense. In today’s system, it has shown that the grades students receive in high school are a good predictor of college performance. In fact, not only do grades measure up better than tests as a means of predicting college success, they are also an indicator of future earnings of high school students.

I have no special talent. I am only passionately curious.

Albert Einstein

“Among researchers, it is now widely accepted that grades reflect multiple factors valued by teachers, and it is this multidimensional quality that makes grades good predictors of important outcomes. In addition to student achievement, grades may reflect such qualities as behavior, attitude, willingness to attend class and turn in assignments, and other indicators of effort,” says a 2017 University of Chicago report.

But as multidimensional a measure as grades are, they don’t capture all the human qualities needed to be successful. Grades favor students who learn to work within the conventions of the system. Unconventional thinkers haven’t always thrived in school: Albert Einstein famously disliked rote learning, and billionaire entrepreneurs Steve Jobs and Richard Branson didn’t make great grades either, for example.

However, this only further highlights the need to look more holistically at factors that are related to student achievement but not fully captured by a GPA, such as grit, persistence, and growth mindset.

Use a No. 2 Pencil

Standardized tests were also initially developed as a means of efficiently comparing the performance of students. The earliest known multiple-choice test was published in 1914, created by Frederick J. Kelly, a Kansas school director, to reduce “time and effort” in administration and scoring.

But their use really took off with the requirement that all sates implement test-based school accountability systems. Going back to the Elementary and Secondary Education Act of 1965, the federal government has tried to figure out a better approach to supporting schools. Each administration attempted to make improvements to the accountability, measurements, and rewards systems, with the three most recent being No Child Left Behind (2001), Race to the Top (2009), and the Every Student Succeeds Act (2015). These programs were designed to address concerns about the slipping performance of American schools compared to those in other countries, as well as the growing achievement gaps for students of color, English Language Learners (ELLs), and special education students.

Education must not simply teach work – it must teach Life.

W.E.B. Du Bois

With the federal government stepping in to encourage transparency and accountability, it was believed school districts would be compelled to improve. In time, these programs became politicized and continue to lead to arguments that they overemphasize testing at the expense of other important values and skills that we want schools to impart. Somewhere along the line, the original intent of standardized testing – to ensure the educational system works for all students – has gotten lost in the rhetoric.

This rhetoric has led to a perception problem for testing. In a recent survey conducted on behalf of the Kauffman Foundation, 60 percent of Kansas City area parents indicated they did not think statewide standardized tests are an effective way to measure high school accountability. And, in other parts of the country standardized testing has led to parent “opt-out” movements.

With school and classroom performance being looked at more closely than in the past, many teachers feel burdened by testing requirements imposed from their states and districts, and believe students spend too much time taking tests. With more of their time being devoted to preparing for and administering tests, teachers are increasingly growing disillusioned with their profession.

But, while an anti-testing sentiment may persist amongst some educators and parents, there is a counter argument that the data being produced makes testing time well spent. As with many fields, good teaching may require some balance of art and science. For example, while old-school baseball coaches insisted on operating from “gut,” the small-market Oakland A’s showed that data can make a difference in decision-making, a breakthrough that revolutionized that sport and many others.

Inherent behind the well-used phrase “teaching to the test” is the idea that standardized tests and the data they produce don’t provide value. While the Finnish education system is held up as an example of what happens when teachers are highly regarded professionals, given autonomy to run their classrooms, it is less known that behind the reform in that country was a “tightly scripted, mandatory national curriculum,” that was in place for several decades until success came about. And, while testing is perceived to put minorities at a disadvantage, most civil rights groups support standardized tests as a way to help ensure greater fairness.

Is Graduation Enough?

From the moment children enter kindergarten, they are engrained with the goal of graduating from high school. In fact, school graduation rates have steadily increased during the past 10 years, hitting 84 percent of on-time graduations in 2016. And this is great news, because the difference between completing high school and dropping out has a significant effect on later life outcomes, such as employment, earnings, incarceration, and a host of others.

However, there is a very real concern that when young people cross the stage for high school graduation, they may not be fully prepared for college and careers. A recent study by the Alliance for Excellent Education found that “U.S. schools hand out 98 different kinds of high school diplomas, and 51 of them fail to prepare students adequately for college or careers. A disproportionate share of those weaker diplomas go to students of color and students from low-income families.”

The study goes on to say: “High school graduation rates are an important but incomplete indicator of success. In addition to measuring whether students receive diplomas, it also is critical to gauge the value of the diploma itself. Allowing students to walk across the stage at graduation with paper-thin diplomas – that do not signify readiness for postsecondary education – is a disservice both to students and to the economic potential of the United States.”

Do standard tests (like I.Q.) lie about how smart you really are? Do they show what you can really achieve? In this paradigm-shifting talk, UPenn psychologist Scott Barry Kaufman presents a new theory of human intelligence. | TEDx Talks

The Saturday Morning Rite of Passage

High stakes testing for college admissions has consistently received flak on a number of fronts: for perceived cultural and racial bias, for giving an advantage to those students who can afford expensive prep services and tests, and for not accurately reflecting a student’s preparedness for higher education.

While admission tests have been a Saturday morning rite of passage for most college-bound students, a number of U.S. colleges and universities have gone “test-optional,” not requiring prospective students to submit ACT or SAT scores as part of the mix for admissions considerations. Why? Some institutions believe that by not requiring test scores, they will benefit from admitting a more diverse group of students than they might have otherwise – a belief that has yet to be backed by data.

That said, these exams can provide important, if incomplete, information about student preparedness for college. Two University of Minnesota professors, Nathan Kuncel and Paul Sackett, recently laid out arguments in defense of the SAT and ACT. They believe there are many myths about tests – including that the tests are a measure of social class, prevent diversity, and fail as a predictor of success in the real world – that can be refuted by research.

In Search of a Better Way

These indicators are all flawed in various ways, but each can also tell us something important about the pipeline of students moving through the education system. However, with recent advances in data collection and assessment technology, we have the ability to supplement these new indicators and create a better-rounded picture of the extent to which students are being prepared for work and life after high school.

There are a number of promising alternative approaches that have emerged.

One organization, the Mastery Transcript Consortium (MTC), is exploring ways to disrupt the education model by taking a new approach to evaluating student learning. The MTC model, which is still in development, would show a student’s progress as credits are earned within a holistic transcript containing segments such as “Adaptability, Initiative, and Risk-taking”; “Global Perspective”; and, “Leadership and Teamwork.” The transcript would represent a portfolio of work completed by the student and the feedback received, rather than a set of grades.

Stacy Caldwell, executive director of MTC, says the organization wants to change the way students prepare for college and career.

“I think that we all understand that much of the educational model that was developed 100 years ago may not be the best way for students to be educated today… [when] they’re going to change careers five to 10 or 20 times over their lifetime, or they’re going to need to be continuous learners… [and] constantly evaluate new technologies and new sources of information,” Caldwell says. “We need deeper authentic learning that is geared around the specific interests and skills of the student.”

Schools reward their students for a combination of intelligence, perseverance and hard work – in the classroom and on the playing fields. But these metrics don’t help kids understand that great grades are not a pass for a great life.

Steve Blank

The inherent assumption behind changing the measurement model is that educators would have the freedom to try different approaches to assessing student understanding or mastery. Students can work at their own pace, going deeper into subjects of particular interest and collaborating with others to complete projects that span multiple subjects. All of which could benefit from advances in both what and how we assess student progress.

In addition to the MTC model, we can look to the CORE Districts in California, a collaborative of districts from Fresno to Santa Ana, which took advantage of a waiver from the federal government to experiment with alternative indicators to measure student and school success. Through this collaboration, the organization developed an accountability system that incorporates social-emotional skills and the schools’ cultural climate. By providing a more comprehensive view to teachers and leaders, the CORE Districts can start to address some of the root causes to disengagement and college and career readiness.

But, there’s a long way to go and many questions that need answering as we face a future that is shifting under the prevalence of new technology, which is creating change faster than any other time in history. At the recent Rethink Ed convening in Kansas City, Missouri, a cross-cut of community members, business leaders, educators, students, and parents came together to address a key question: How do we transform the high school diploma to reflect the skills, abilities, and mindset needed for all students to succeed?

“The current system isn’t working for many, many young people,” said Tony Simmons, executive director of the High School for Recording Arts in St. Paul, who was a Rethink Ed Firestarter. “We have to reimagine the next iteration of education. If we really want young people to be prepared for this new economy, we need to break out from the old mold.”

Learning together, with all stakeholders at the table, is a start.

If we need additional ways to evaluate how well our youth are prepared for the future, then we can also improve how we measure if the economy will work for them as adults.

More specifically: Is the U.S. economy working in a way that benefits all of its citizens? Whether you believe it is or isn’t, conventional ways of measuring the economy – unemployment, the stock market and GDP – are increasingly being viewed as incomplete indicators, that don’t tell the full economic story. And, this in turn, has made room for discussion of more inclusive and holistic economic measures.

How We Measure Now

Let’s take unemployment – a number reported widely in the media each month as a barometer of our economic health. Unemployment numbers have been released each month by the Bureau of Labor Statistics (BLS) since 1940, and recently went below 4 percent – the lowest rate since 2000. Recently, it peaked at 10 percent in 2009 and has fallen steadily since then. However, this number excludes people who are underemployed or have just given up looking for a job. Looking at “U-6” data, also provided by the BLS, shows “total unemployed, plus all marginally attached workers, plus total employed part time for economic reasons”; as of May 2018, this number is a more sobering 7.8 percent.

These days, it may be trendy to talk about the gig economy: the ranks of Uber drivers, TaskRabbit handymen and other freelancers who, out of choice or necessity, pick up work where they can. Certainly, technology platforms have enabled people to work independently in a way that is unprecedented in history. However, these workers are often not fully employed, but are also not showing up on the unemployment reports. And, signs are that this segment of workers is growing. Intuit estimates more than a third of the U.S. workforce is on demand, a figure that may hit 40 percent by 2020.

The stock market is not a proxy for the economy of America. I’m deeply concerned about the millions of Americans who are not participating in the economy.

Howard Schultz, executive chairman of Starbucks

And, despite the tight labor market indicated by the unemployment rate, wages are not rising as much as might be expected. So, while we may have been conditioned to think that falling unemployment numbers are always good news for the American worker, they are an incomplete indicator: Below the surface there may be a large segment of people who can’t find meaningful, full-time work.

Then there’s the stock market. When the Dow Jones Industrial Average (DJIA) hits new highs, it is bound to get a mention in the nightly news. And if the market makes a big drop, we might have a moment of worry about our retirement savings account. But, there’s actually little evidence that the performance of the stock market correlates to the health of the economy at large. While the DJIA made significant gains from 2011-2015, for example, other indicators like GDP and job growth were sluggish. Gains in the stock market also have an uneven benefit to the U.S. economy as a whole, as the top 10% of the wealthiest U.S. households own 84% of all stocks.

The Gross Domestic Product (GDP), a measure of all the goods and services produced by the people and companies within a particular country, certainly sounds important. It continues to be touted as a means of comparing the economies of different countries. The trouble is, GDP isn’t a very nuanced indicator, just measuring the “heat” of economic activity – for better or worse – and could potentially send the wrong signals to decision-makers:

“… in particular, it grows rapidly when wealth is concentrated in a very few hands, when natural resources are squandered, and when society becomes less safe, overcrowded, polluted, and prone to disasters,” according to the Center for Sustainable Economy.

The New Economics Foundation, a UK-based organization that seeks to democratize decisions about the economy, saying that the traditional corridors of power aren’t getting the job done, because “[m]illions of people feel they have lost control over their lives and are now being left behind by changes in the economy, technology, and climate, even while being promised a parody of control that threatens to make matters worse.”

What do gender equality in Morocco and anti-corruption watchdogs in Poland have in common? | TED Institute

What’s a Better Measure of Progress?

To set economic priorities that benefit everyone, the New Economics Foundation suggests the following five factors should be considered:

Stable jobs that provide a fair standard of living
A sense of well-being
A healthy environment
Income equality: a reasonable gap between the highest and lowest earners
Quality healthcare available to all

Closer to home, a number of states have explored a similar approach, known as the Genuine Progress Indicator (GPI). While the concept for a GPI has been around for several decades, and enjoyed a resurgence of interest earlier this decade, efforts have waned as of late. Essentially GPI attempts to quantify economic well-being at a national, state and local level by totaling societal benefits (such as good infrastructure, quality schools, preservation of environment), as well as costs (such as pollution, homelessness, debt) that are overlooked by the GDP. States like Maryland have experimented with the GPI because it is “… designed to measure sustainable economic welfare rather than just economic activity.”

Factor in the Job Creators

While entrepreneurship adds vitality to economies at all scales, it is overlooked by even these more holistic measures. Startups are sources of job creation and innovation, and ultimately do more for economic development than do attempts to lure existing companies. Even so, how entrepreneurship drives local economic growth in communities around the country is a topic we still know relatively little about. Historically, there has been very limited timely and granular data on entrepreneurial ecosystems.

That’s why the Kauffman Foundation’s new “Uncommon Methods and Metrics” portfolio is funding the work of researchers looking at a number of factors impacting the success of entrepreneurial communities, including access to capital and talent; the presence of entrepreneurial support organizations; educational institutions and businesses that embrace entrepreneurship; and policies influencing entrepreneurship. It is our goal that this research and data ultimately help community leaders make decisions that create more entrepreneurial environments for new and growing companies, which in turn are the engines responsible for nearly all net new job creation.

Why Aren’t We Happy?

But, learning and economics metrics like graduation rates and unemployment have difficulty measuring one question: Are we happy?

A Danish author claims that social interaction – “hygge” – is more important than prosperity (though admittedly Denmark is economically stable) when it comes to the happiness of its citizens. Denmark trades off with other Scandinavian countries for the top spot in the annual World Happiness Report. And though the argument is perhaps self-serving, the author does point out that despite lower taxes, a rising GDP and low unemployment rates in the U.S., our citizens are actually getting unhappier – the U.S. ranked third among the 23 Organization for Economic Co-operation and Development (OECD) countries in the 2007 survey, but fell to 19th out of the 34 OECD countries in the 2016 survey.

When it comes down to it, it shouldn’t seem like too much to ask of ourselves: a system that creates educated citizens, fueled by lifelong learning, able to find meaningful work that provides a comfortable way of life.

How do we effectively prepare our students for life after high school and an ever-changing economy? What does a thriving economy look like?