Professionals involved in program evaluation projects occasionally find that their understanding of how the work will be used and adapted by the organization does not necessarily parallel their clients’ perception of the project.
The DIY Method, a unique tool I developed, proves to make a vast difference in both collaborative work and the process of organizational performance, changes of policy, and program evaluation capabilities. The DIY Method begins at buildingcapacity. We aim to include all managers and stakeholders in the process of program evaluation to better find common ground and agree on the initial need for the evaluations. Secondly, we seek to clarify the research question and discover the gaps in information about the program or initiative. It is vital to recognize what is unknown and requires further investigation. Thirdly, assuming that our research question is well defined, we then focus on constructing research tools for program managers or officers to use for data collection. These tools contain basic statistics and cross-tabulations to assist in day-to-day management. They can either be simple Excel based software, or built into the CRM or preexisting organizational database. It is decided by the clients which method works best and allows for smooth and beneficial program evaluation. This stage may take some time as there is a need to adjust, apply, and modify the tools to contribute with managerial decisions and facilitate formative program evaluation. Our professionals are there to help create a beneficial environment and assist with program evaluation tool implementation.
When all is working and in place, our consultants analyze the data collected for the quarter or year, and articulate a research report to summarize the results, data, and trends. A significant portion of our time is dedicated to writing practical recommendations, supplying a beneficial use for evaluations, and assisting with continuous work to facilitate a smart and successful atmosphere for decision making and operations.
If this method of work interests you and you are curious to learn more, contact me today for a FREE initial consultation!
Many organizations are doing a great job. They know exactly how to engage their target audience, they know to deliver the service on the best way possible, well.. they also know to fundraise money from the local community.
However, they do often underestimate the tremendous power community relations have in order to help them scale and grow. Community relations is not just having someone to actively address and answer phone calls, and provide with information; it is about being proactive and initiative, let people know who you are and what you do. There are so many good organizations out there, why you and not others?!
Of course, you are special and unique, and you know it with all of your heart; nevertheless, you should let decision makers and community leaders know about you, your organization, and your achievements. I know you have, I know you did a lot. Now make this information accessible to everyone – let them read about you in the newspaper, let them watch you on a Facebook video, let them know good things about your activities. Never forget to crunch numbers, to provide accurate data, to show a real picture of your impact and change of community’s life.
You will be surprised to learn how many out there did not know your organization even exists! You will meet many prospect donors and clients who wish to donate their time or money to help you, and grow the social impact of what you are doing. Make sure to involve politicians, from all levels and parties, let them speak for you, bring your voice higher, assist you to achieve social goals. It is working to a great extent, but the first step is to DO IT.
Last time I wrote about the possibilities collaborations may open your organization to, and I focused on the very fact that there are people who need a boost or encouragement in order to internalize this basic element of organization’s healthy life. I eventually summarized that most of the managers are luckily capable of this, and may collaborate with other organizations, at least for the sake of their own performances.
Today, I would like to focus on another essential fact of collaboration which is the benefit of the society. I would call it Synergy… when your organization collaborates with another organization, you create a new opportunity, which in turn benefits your organization, the other organization, and your and theirs clients.
However, the most important point, in my view, is that you have created a much larger benefit – you have made a new opportunity of positive influence and connection, that can lead the world to be a better place to live in. Think about it, you did not just “do good” to your own performance at work, and so not your collaborative partner, you did not just create a good service for your clients, you have also made the world a positive environment to be in.
So, next time when you are successful in collaborative efforts, bear in mind – you have just did something good for the whole society.
Ahhmmm…. got to tell you I am excited.
In the past weeks, I have had the opportunity to meet many professionals virtually, especially thanks to our lovely brother LinkedIn. I found myself in a process of learning, that I missed so much! well, enough with the introductions, let’s start thinking (-:
I would like to take it one step ahead, and correct me if I am wrong, the operations-minded people are usually dealing with something we can touch, or at least can see impact simply just by looking what the organization does – water, agriculture, farms, vaccines, you name it. The impact measurement in this type of organizations is a short and sweet ROI analysis. Efficiency, effectiveness, benchmark, and performance measurement are relatively easy to conduct, as well as goals and objectives setting.
Another important suggestion was to collaborate and cooperate, and even amalgamate organizations that do the same/similar work. I agree with this way of thinking but would like to ask you managers – will you work on your ego and let another organization to work with you or instead of you? Think about it. If it works – there may be great, extremely successful models of business supply chain management, which employ this attitude – more for less – by a chain of organizations. Phrase it like this: The bottom-line is the VALUE profit. As long as there are organizations which may do the same, or even better than your organization – the real impact will be achieved by working together, and collaborate. It also saves you money, and reduces costs!
I know it is the hard part of managers’ ego, but probably one of the realistic ways to save the sector alive in terms of impact.
In short, the question to ask yourself is this – What is your organization’s unique selling proposition? What is your competitive advantage on others? What is the special value your organization creates? You supposed to have a very good answer to this question.
An additional note in this regard, is Collective Impact. Someone referred me to the collective impact website. I researched it, and browsed the web, and I must compliment them for doing the first operations step – creating coalitions and collaborations in order to increase the impact. However, this is only the first necessary step. The sector is getting shrunk, and will continue this trend, therefore there is a vital need to amalgamate or eliminate ineffective programs, not just collaborate.
The Social Value Impact Aspect
The problem of impact comes to life again when we deal with the social hot potato, and you know what?! I am dealing with it!
There is certainly a broad agreement on the need, although I must admit that I am still shocked to see huge foundations refuse to measure themselves (ego and power issues??)… But let’s put it aside. Just another small reference – social enterprises are a relatively small part of the nonprofit sector. I will never call them the “forth sector”, because they are not. I will never agree that they act differently than the third sector, they are value-driven, and this is the crucial aspect. They DO NOT care about money more than value, and DO NOT care about profit and value the same way. Therefore they DO NOT have double bottom-line, but one, and the last is very similar to the pure nonprofits. I can count few real social enterprises, but they will be again the ones we can touch – bakery, restaurant, agriculture, cafe, and the like.
Show me one social services enterprise… it ain’t exist, because it is impossible, and here comes the social impact measurement to help us.
I did not define Standardization last time, so here it is: in my opinion is needs to be simple. The metrics should include up to 4 core elements, which will be relevant across the sector. By this, there will be an option to compare between one organization to another. There is an option to add as many as indicators you like and want, and it won’t harm the metrics, but will give your organization the specific information you are looking for.
So, this is my review, happened somehow to be very small and narrowed…
I rank tools in 1-3 scale. 1=low, 2=medium, 3=high. Hence, the highest total score is 9.
I try to keep it as simple as possible, so do not rank 1-10 or something like that.
An important note! If your organization does not have a work plan which included vision, goals, and objectives, you cannot employ social impact measurement at this time. You MUST define the above in advance. Do not know how to do it? Drop in my post on setting goals, and keep up the good work!
SROI I love this measure, however and in short, this does not apply in many social services and education nonprofits. If you are dealing with employment or any other outcomes which involve money, this may be the measure for you. Usefulness (1): Applies to a narrow type of organizations (usually employment services, financial assistance, micro finance, and similar) Friendliness (1) If you do not learn it thoroughly, and gain lots of knowledge – you probably won’t be able to conduct a reliable SROI analysis Standardization (2) This part gets high score, because lots of research has been done, however there is no option to apply it broadly enough. Total score: 4/9, 44%/100%
GRI / IRIS I like the business attitude. This metrics will not save your life, but definitely will give you a way to benchmark your organization. This tool is used by many for-profits in order to monitor their performance, so I would rank it as the following: Usefulness (1): it gets a low score here, because I am not sure how it is going to help many organizations in their day-to-day management in terms of measuring social outcomes. However some organizations might fall under the suggested social objectives, so I suggest to check it out. Friendliness (2): the tool seems to be highly recommended and highly used by a variety of organizations. It does not gain the 3 points, because it does not fit every organization. Standardization (2): you win the entire pot here. The tool is absolutely standardized, and you may feel free to compare your organizational performance to similar organizations in the industry. It is a huge advantage. I ranked it 2, because it does not apply in every field. Total score: 5/9, 55%/100%
Social Impact Bonds (SIB)
I like the idea of social finance, because it makes much sense. It really builds a reputation for impact investing. In short, the system is designed to invest money in social projects, in order to PREVENT problems from reoccurring in the future (such as second-generation issues, recidivism, unemployment in specific sectors, etc.). The model briefly works by this: funding is given > intervention is made > evaluation of outcomes is conducted > in case of success (i.e. less recidivism, more employment) the government returns money to the investors. Even though I like the idea, I have no clue regarding the metrics and indicators they are using in order to evaluate social programs… it does not seem standardized or friendly, but it is just my outsider opinion.
Moreover, in my opinion, the social impact bonds model seems to fit to a narrow type of outcomes, kinda similar to SROI. Total score: Unknown!
My list is much shorter than expected. I reviewed over 10 tools and methodologies, but did not like them at all, so why to mention them?
With that said, when it comes to social services and education, and other soft, hard-to-touch outcomes, the measures and indicators become useless. No usefulness, no friendliness, no standardization. Nothing helps. Therefore we must agree that there is another way to measure, and you know what?! it is not the kind, the gentle one… it is about achieving your objectives, measuring your VALUE. It is that simple.
Your objectives include “improvement of students’ grades”? Show the improvement, between the beginning of the year and the end. Your objectives include “women’s empowerment”? Define what empowerment is, let’s say, they will be more responsible for their day-to-day tasks. Show they have changed their behavior/attitudes.
Do not use excuses like “they are happy”, “their self esteem is higher”, “I feel it”… these are NOT your objectives, and therefore not the social value you wanted to create.
My post that dealt with the lack of measurement in success terms has led some enlightening comments from my colleagues and past managers, so I have decided to dedicate my coming posts to research this field and dig a bit more. It should be noted that I have done some research in the past, and sort of consider myself as someone who gained some knowledge re SROI or other impact measurement efforts, however insufficient and this is why I find it imperative to research now.
Another note before we dive-in… I plan to write a series of posts in this topic. Firstly, I would like to cover some methodology, i.e. set some conditions for my research. Secondly – review existing solutions in respect to the methodology. Finally, I hope to come to conclusion with the most relevant tools or suggestions for the future.
Ready to think?! Let’s do it!
The first and above all is the question WHY do we need to measure impact in social projects? It is indeed a vital and important question, and… I have a very good answer in my pocket. A very smart senior executive and philanthropist, emailed me the following statement, based on his extensive experience with foundations: about 85% of the funding in social projects is going lost without achieving its goal. These are insane numbers my friends. A simple math will reveal a bare truth. In the US itself ~333.5 billion dollars were donated in 2013. However, based on the above, we can cut out about $300 billion. I stop here, because it hurts to think in global terms (not to mention the lovely governmental “match”). I am sure most of you already know that there is a critical issue with funding-impact ratio, and this statement is just the straw to break the camel’s back. It was for me, anyway, and as a consequence I have decided to write a wake-up and start thinking post.
Are you ready to think with me?! I suggest you to comment here or by email, and I promise to integrate your thoughts and credit you in my next posts. I honestly believe that we can work together in order to achieve this goal, as we all have one mission – to find a decent solution. But! we cannot accept every solution. I have developed several criteria in order to consider a route to be a solution, and you are welcome to add more or suggest adjustments:
1. Usefulness To what extent can we use the measures in day-to-day management? and by this I talk about the dual role of metrics. I hold this opinion for years, and every time I state it, people are looking at me if I had fallen from the moon. But I actually did not, as long as I recall I was born on earth (-:
So if I get back to the point – the dual role is enabling the use of metrics/measures by both sides, the foundation/funding body and the charity/organization alike. No more measurement for THEM, no more shortening of evaluation time and tools. You want it for your organization because YOU deserve to know what the hell is going on!
The issue of usefulness hit me like crazy back in 2007, when I was working with senior managers in social services, whom their project was founded by a very large north-American federation. They did not want to measure, nor to learn – they already knew the true, thanks, but no need to measure at all. I asked them why they despise it so much, and they simply said – it is too much work for our overloaded staff, we have no spare time for collecting useless data… they added that they already know the ins and outs so well, so no evaluation or measurement will enlighten anything.
It hit me again when I read Jed Emerson’s post last week, especially here: “I recall a breakout session presentation by one of the world’s best known impact investment organizations—one that appears on everyone’s list of favorite impact funds—listening to a nuanced and well-considered presentation by their head of impact performance. Following the formal PowerPoint show that included impressive definitions, charts and data, the presenter was asked, “How do these impact metrics inform your work?” to which the presenter responded, “Oh—we don’t actually use these metrics in our work. We just need them to give our funders!” After everyone had a good laugh, he said, “No, no—I’m serious—we don’t use them at all!” Peals of continued laughter echoed…”
Things are about to get worse, my dear readers, not a long ago I met a very senior level manager who works for a wealthy foundation. It happened to be with some connection to one of the projects in which I have been involved in the past, so I was more than curious to learn how they used the data and metrics we sent then for review. Sorry to disappoint you – they did not. They donated the money and forgot about it, and the real hell is yet to come – they never use metrics for themselves, as a foundation… and you know what?! they are absolutely not alone. The same shock made me shake, back in 2011 when I realized that a billion dollar foundation NEVER measures impact, never tried to develop something useful, and moreover when I was trying to educate them I got the same feeling of Mars and Venus, Earth and Moon, whatever you name it, I bet you understand me.
If so, I conclude: we need something useful – something that people will want to use, need to use, and feel it’s helpful. And please twice, one for the giving side, one for the taking one. I would say that in this aspect we have to talk business, and learn from business cases how to measure ONCE and EFFICIENTLY for more than one stakeholder.
To what extent normative managers can use the measures in day-to-day management? and by this I talk about managers who do not have extensive research background.
It reminds me one of my nonprofit job interviews (-: funny story actually, especially if you consider my very limited knowledge about this sector back then, and specifically the funding issue. The position was highly customer-faced, and dealt with measurement and evaluation in education and social services. The interviewers, there were two of them, asked me if I think that every social service manager or every school principal should use SPSS (yes, the statistical package). I was sure they are kidding, but they did not, so I answered “of course NOT, they can use Excel which is much easy to adapt, use and learn”, and got the job.
I tell you this story, because I will never agree that managers need to be researchers. They certainly do not! they need to do their job, to manage! and metrics is just another great tool to facilitate decision making and performance measurement. Yes, it is an essential tool, one to be top prioritize for every manager in the nonprofit sector. Yet, I wonder how this vital tool will become friendly and suit every normative manager? I think we can again keep an eye on good business practices. We also may bear in mind that we want something simple, Excel based, and easy to collect data and interpret.
I have to warn you, I got together with enough organization that invested tons of money in IT solutions for their database, and sadly I can barely count the organizations that really USE this information. They often tend to forget about it, and when they tried to retrieve some data, it was always such a burden, and poor quality.
I would also like to add one more thing, I believe in short things. What do I mean? I never like the idea of tons of questions or gathering endless information, we are not conducting an academic study (NO we are not!), we have a mission to measure the impact. In order to do that we have to keep in mind that there are busy people who realistically cannot dedicate themselves to information gathering as their life mission, and therefore it has to be short and useful, not just short, not just useful. BOTH.
If I summarize my impressions and thoughts in this respect, and would say that there is a vital need for a FRIENDLY and RELEVANT tool to measure nonprofits performance. I do not want to sound as criticizing some existing measures, but I feel they are too complicated, and may not be a good answer to address the friendliness need.
3. Standardization: to what extent we can expect to use the same measurement for a wide range of projects? I will leave this measure open for your comments, because I already, kind of, formatted this so firmly in my mind, and I feel too strict…
Hope you enjoyed the reading. Please comment, share and subscribe to my newsletter, I promise to make you think (-:
P.S. – A super-talented friend of mine who works in the industry and does a great job with impact an so on, read this post before published, and was quite amazed that I still insist to make efforts to spread these ideas. He said: “You have got to forget SROI and this, no one wants to learn, no one wants to know, they know everything. Even when I try to spoon feed them they won’t listen”. Well, I dedicate couple of songs to you, keep calm and continue doing your great job. Good times are just about to come (-:
Click here if you want to listen the songs. None of them fully expresses my purpose, but absolutely set the tone.