Thinking Differently About Assessment
Editor’s note: The author of this feature article, David Grant, will be leading a session on “Setting goals, assessing outcomes, and achieving success” at NCFP’s 2015 National Forum on Family Philanthropy along with two family foundations. This session will provide insight on how to work with board and staff members to set goals and assess outcomes in their work, while also supporting core grantees to do the same. David also served as a moderator at NCFP’s 2014 National Forum on Family Philanthropy on the session, “100 years of family philanthropy: The Andrus family and Surdna Foundation.”
When I became president and CEO of the Geraldine R. Dodge Foundation in Morristown, New Jersey in 1998, I did not suspect that the topic of assessment would feature so prominently in my twelve years there, or in my post-Dodge life as a consultant to mission-driven organizations.
My first week at the foundation coincided with a deadline for reports back from grantees from the previous year. It seemed that these were the key documents in our efforts to assess the impact of our philanthropy, but as I read through them, I was struck by the fact that Dodge hadn’t made a single bad grant! Every single organization had fulfilled its obligations, achieved success in a complex environment, and was ready for more funding.
Having been a teacher for most of my life up to that point, I saw assessment as a fundamental part of learning, so I wrote our grantees and said, in effect, “I am not seeing an assessment system in the relationship between us that helps you get better at your work. Would you like to invent one together?”
That inquiry began a series of meetings, which grew into a foundation-funded technical assistance initiative, with assessment as a core component. That initiative in turn morphed into an ongoing, annual Board Leadership Training Series of eight workshops stretching over eight months. Assessment is the topic of the opening workshop, and I am grateful to my successor and my former colleagues that I am still invited back to teach it.
Freed from my day job, I have taken the last few years to write a book based on what I learned during the Dodge years about how assessment works – or might work – in the social sector. The book was published this spring by Chelsea Green Publishing and its title, The Social Profit Handbook, emphasizes the purpose behind putting effort into creating a good assessment system; it’s not about the assessment itself – it’s about the social benefits such a system can help bring about.
Key themes of the book include:
Assessment should come before the work, not after it.
This may seem counter-intuitive if you think the purpose of assessment is to judge work that has already happened. But what if its primary purpose was to shape and improve work that hasn’t happened yet? That’s what formative assessment does. When I am explaining this concept in workshops, I ask people to think about where they got the idea that assessment happens at the end. Most exclaim, “School!” Indeed, we should not underestimate the influence of experiencing literally thousands of quizzes, tests, and exams coming at the end of chapters, units, books, and courses. We are very familiar with being “graded” for past work. But school also provided us with at least two examples of adults with a different approach to assessment – coaches and teachers of performing arts. Their efforts were focused on upcoming performances with clear criteria for success, and they gave timely feedback to help us succeed. We can take our lead from them if we consider ourselves performers in pursuit of impact in our communities. It remains for us to be clear about what success will look like so we can get and give the feedback we need along the way.
Assessment should be qualitative as well as quantitative.
Well-chosen metrics can be very helpful in tracking progress, but most nonprofit leaders I know say there is something missing if everything must be expressed in numbers. And I agree with Michael Fullan when he writes in his book Change Leader, “Statistics are a wonderful servant and an appalling master.” What we need is a tool that allows us to use words – as many as we want – to describe the levels of success we aspire to in relation to the criteria for success that matter most to us. Then we can plan backwards from these shared visions. For me, the best tool is the qualitative assessment rubric, a simple matrix with criteria for success on one axis (picture them running down the left-hand column) and levels of success on the other (picture them running along the top row). We now have a format for describing together what we are aiming for, in the areas that matter most to us.
The best assessment tools are designed locally.
Many foundations try to create an assessment tool to measure and compare social benefits across a portfolio of grants. I think this well-intended effort does not help us very much in the end. It reduces us to common measures such as “numbers served,” without asking how those people were served, and what difference it made in their lives. Qualitative assessment, as noted above, is about excellence, and excellence is best described by the people closest to the work and to its context. A locally-designed rubric can literally hold, as in contain, an ongoing conversation about good work between nonprofit staffs and their boards, between nonprofits and the people they exist to serve, and between nonprofits and their funders.
Assessment needs “mission time.”
We don’t take enough time to bring people together around shared visions of success. And we don’t take enough time to monitor together how we are doing along the road to those visions, to ask how the world is changing around us, and to make necessary adjustments. Paradoxically, the time it takes to design a formative assessment system, with qualitative assessment tools – time we don’t think we have – comes back to save us much more time later. And more importantly, it improves our work. In this regard, organizations that make time for ongoing assessment work are like individuals who make time to exercise. They have taken an important activity that too often gets bumped because it is not urgent, and they have made it happen. And they are healthier as a result.
Applying the assessment rubric: GreenFaith
Let me illustrate how these themes work together through the story of GreenFaith, a faith-based environmental organization in New Jersey that embraced the principles of formative assessment in the early 2000’s and a decade later has established itself as a national leader in its field. When the leaders of GreenFaith first came to a Dodge assessment workshop in 2003, they had recently launched an initiative to install solar panels on religious institutions around the state. The idea of formative assessment and the blank rubric invited them to envision the next level of success – and to use words to convey it. This resulted in a remarkable new initiative, Sustainable Sanctuaries, guided by a remarkable rubric that allowed them to described what successful “greening” looked like in seven areas of institutional life, not just in facilities.
These seven areas – picture them in the left-hand column of their rubric – included: Worship and Spirituality; Religious Education; Influence on Members, Denomination, Community; and Financial Management. Across the top of the rubric, they had a little fun with Christian imagery as they identified levels of success from low to high: Out of the Garden; In the Wilderness; Building a New Ark; Eden Restored.
With the matrix thus defined, they could describe together what it looked like to integrate environmental care into these aspects of their institutional life and also what it looked like when it wasn’t happening. Thus at the low end of the rubric, “Out of the Garden,” we see, for example:
- Plants and animals are never integrated into worship services,
- Children are taught that God is found only through sacred texts,
- No thought is given to the environmental impact of food-consumption or fellowship-related supplies, and
- The investment portfolio had only one criteria for success: make money.
At the top end, “Eden Restored,” we see:
- Occasional meaningful outdoor services and a mature eco-theology,
- Young people are introduced to the natural world as a revelatory gift,
- Locally-sourced food is purchased for community events, and
- The board oversees an environmentally-screened investment portfolio.
The staff at GreenFaith took the time they needed – mission time – to draft the rubric, then they invited others in on its further creation.
The rubric eventually helped all the churches, synagogues, and mosques that GreenFaith worked with become “greener,” but the story does not stop there. GreenFaith’s approach to comprehensive “greening” led them to create a national certification program that has supported houses of worship in 22 states become environmental leaders in their regions. Moreover, internally, when the GreenFaith board saw the positive impact of the Sustainable Sanctuary rubric, they decided to write one for themselves. Executive Director Fletcher Harper describes the board rubric as:
… a collectively affirmed ethos about how the board will function. This creates pride, a sense of identity and strength. In addition, the presence of the rubric, and the fact that we treat our rubrics like evolving documents, has given board members a sense that they have a say, some control over the organization’s developing vision for its board. It’s very important.
I love the fact that GreenFaith used formative assessment internally as well as externally. This approach can be applied to the performances of senior staff, to internal communications systems, to staff meetings – any aspect of the workings of an organization that matters and can be improved. But let’s follow GreenFaith’s example and take a formative assessment approach to governance. It’s an area where funders and grantees have the same goal – they all want highly-effective boards.
Board assessment: Working on mission time
A typical (ineffective) approach to assessing the effectiveness of a board is to design an instrument that asks, “How have we been doing?” in a number of areas: attending meetings; fund-raising (if applicable); financial oversight; etc. The task is usually relegated to someone on the governance committee, or a committee on trustees. It often involves a survey, followed by a short report, and not much changes as a result. But what if we began instead with some questions:
What kind of board do we want to be?
How would we judge ourselves successful a year from now?
What would we have accomplished, and how would we have gone about it?
The answers will vary depending on the organization’s history and stage of development, its needs, and its opportunities. The organization – and again this includes foundations as well as the nonprofits they support – may need more strategic thinking from its board; it may need the board to act more systematically as advocates and ambassadors of the group’s mission; it may need them to support the executive leader through a better assessment system; it may need them to anticipate a turnover in long-standing executive leadership a few years before it happens.
The key is it will take some mission time to talk through and prioritize criteria for success. Then, collectively, the board can be as specific as possible about what they are committed to do together and plan backwards from their vision. There are many benefits from this exercise: clarity about roles and responsibilities; customized opportunities for board members to serve based on their individual talents, experience, and interests; a sense of knowing each other better; a renewed focus on why they are together in the first place; an aspirational vision for both the board and the organization.
Nothing like this can happen without being intentional about taking assessment into our own hands. In short, I believe we have to stop thinking about assessment as something others do to us, reflecting on past work, and start thinking about it as something we do with our colleagues and for our organizations in support of our best efforts and highest aspirations. I know it is within our power to create assessment systems such that if someone tried to take them away, we’d fight to get them back. Outside forces will always judge our work – there is no escaping that. But let’s give them excellent work to judge.
In short, I urge nonprofits who are swimming in a sea of metrics required by a range of funders to take assessment into their own hands. And I urge foundations not only to allow them to do so, but also to support them and, wherever possible, join them in this formative approach.