Category Archives: Program Evaluation

Not-for-profit But yes-for-what continues… and today I will start to review the solution!

My post that dealt with the lack of measurement in success terms has led some enlightening comments from my colleagues and past managers, so I have decided to dedicate my coming posts to research this field and dig a bit more. It should be noted that I have done some research in the past, and sort of consider myself as someone who gained some knowledge re SROI or other impact measurement efforts, however insufficient and this is why I find it imperative to research now.
Another note before we dive-in… I plan to write a series of posts in this topic. Firstly, I would like to cover some methodology, i.e. set some conditions for my research. Secondly – review existing solutions in respect to the methodology. Finally, I hope to come to conclusion with the most relevant tools or suggestions for the future.
Ready to think?! Let’s do it!

The first and above all is the question WHY do we need to measure impact in social projects? It is indeed a vital and important question, and… I have a very good answer in my pocket. A very smart senior executive and philanthropist, emailed me the following statement, based on his extensive experience with foundations: about 85% of the funding in social projects is going lost without achieving its goal. These are insane numbers my friends. A simple math will reveal a bare truth. In the US itself ~333.5 billion dollars were donated in 2013. However, based on the above, we can cut out about $300 billion. I stop here, because it hurts to think in global terms (not to mention the lovely governmental “match”).
I am sure most of you already know that there is a critical issue with funding-impact ratio, and this statement is just the straw to break the camel’s back. It was for me, anyway, and as a consequence I have decided to write a wake-up and start thinking post. 

Are you ready to think with me?! I suggest you to comment here or by email, and I promise to integrate your thoughts and credit you in my next posts. I honestly believe that we can work together in order to achieve this goal, as we all have one mission – to find a decent solution. But! we cannot accept every solution. I have developed several criteria in order to consider a route to be a solution, and you are welcome to add more or suggest adjustments:

1. Usefulness
To what extent can we use the measures in day-to-day management? and by this I talk about  the dual role of metrics. I hold this opinion for years, and every time I state it, people are looking at me if I had fallen from the moon. But I actually did not, as long as I recall I was born on earth (-:
So if I get back to the point – the dual role is enabling the use of metrics/measures by both sides, the foundation/funding body and the charity/organization alike. No more measurement for THEM, no more shortening of evaluation time and tools. You want it for your organization because YOU deserve to know what the hell is going on!
2014-09-28 11.22.05
The issue of usefulness hit me like crazy back in 2007, when I was working with senior managers in social services, whom their project was founded by a very large north-American federation. They did not want to measure, nor to learn – they already knew the true, thanks, but no need to measure at all. I asked them why they despise it so much, and they simply said – it is too much work for our overloaded staff, we have no spare time for collecting useless data… they added that they already know the ins and outs so well, so no evaluation or measurement will enlighten anything.
It hit me again when I read Jed Emerson’s post last week, especially here: “I recall a breakout session presentation by one of the world’s best known impact investment organizations—one that appears on everyone’s list of favorite impact funds—listening to a nuanced and well-considered presentation by their head of impact performance. Following the formal PowerPoint show that included impressive definitions, charts and data, the presenter was asked, “How do these impact metrics inform your work?” to which the presenter responded, “Oh—we don’t actually use these metrics in our work. We just need them to give our funders!” After everyone had a good laugh, he said, “No, no—I’m serious—we don’t use them at all!” Peals of continued laughter echoed…”
Things are about to get worse, my dear readers, not a long ago I met a very senior level manager who works for a wealthy foundation. It happened to be with some connection to one of the projects in which I have been involved in the past, so I was more than curious to learn how they used the data and metrics we sent then for review. Sorry to disappoint you – they did not. They donated the money and forgot about it, and the real hell is yet to come – they never use metrics for themselves, as a foundation… and you know what?! they are absolutely not alone. The same shock made me shake, back in 2011 when I realized that a billion dollar foundation NEVER measures impact, never tried to develop something useful, and moreover when I was trying to educate them I got the same feeling of Mars and Venus, Earth and Moon, whatever you name it, I bet you understand me.

If so, I conclude: we need something useful – something that people will want to use, need to use, and feel it’s helpful. And please twice, one for the giving side, one for the taking one. I would say that in this aspect we have to talk business, and learn from business cases how to measure ONCE and EFFICIENTLY for more than one stakeholder.

Ready To Think

2. Friendliness:
To what extent normative managers can use the measures in day-to-day management?
and by this I talk about managers who do not have extensive research background.
It reminds me one of my nonprofit job interviews (-: funny story actually, especially if you consider my very limited knowledge about this sector back then, and specifically the funding issue. The position was highly customer-faced, and dealt with measurement and evaluation in education and social services. The interviewers, there were two of them, asked me if I think that every social service manager or every school principal should use SPSS (yes, the statistical package). I was sure  they are kidding, but they did not, so I answered “of course NOT, they can use Excel which is much easy to adapt, use and learn”, and got the job.
I tell you this story, because I will never agree that managers need to be researchers. They certainly do not! they need to do their job, to manage! and metrics is just another great tool to facilitate decision making and performance measurement. Yes, it is an essential tool, one to be top prioritize for every manager in the nonprofit sector. Yet, I wonder how this vital tool will become friendly and suit every normative manager? I think we can again keep an eye on good business practices. We also may bear in mind that we want something simple, Excel based, and easy to collect data and interpret.
I have to warn you, I got together with enough organization that invested tons of money in IT solutions for their database, and sadly I can barely count the organizations that really USE this information. They often tend to forget about it, and when they tried to retrieve some data, it was always such a burden, and poor quality.

I would also like to add one more thing, I believe in short things. What do I mean? I never like the idea of tons of questions or gathering endless information, we are not conducting an academic study (NO we are not!), we have a mission to measure the impact. In order to do that we have to keep in mind that there are busy people who realistically cannot dedicate themselves to information gathering as their life mission, and therefore it has to be short and useful, not just short, not just useful. BOTH.

If I summarize my impressions and thoughts in this respect,  and would say that there is a vital need for a FRIENDLY and RELEVANT tool to measure nonprofits performance. I do not want to sound as criticizing some existing measures, but I feel they are too complicated, and may not be a good answer to address the friendliness need.

3. Standardization:  to what extent we can expect to use the same measurement for a wide range of projects?
I will leave this measure open for your comments, because I already, kind of, formatted this so firmly in my mind, and I feel too strict…

Hope you enjoyed the reading. Please comment, share and subscribe to my newsletter, I promise to make you think (-:

 Charity defense councilSource: Charity Defense Council

P.S. – A super-talented friend of mine who works in the industry and does a great job with impact an so on, read this post before published, and was quite amazed that I still insist to make efforts to spread these ideas. He said: “You have got to forget SROI and this, no one wants to learn, no one wants to know, they know everything. Even when I try to spoon feed them they won’t listen”. Well, I dedicate couple of songs to you, keep calm and continue doing your great job. Good times are just about to come (-:
Click here if you want to listen the songs. None of them fully expresses my purpose, but absolutely set the tone.

 

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Not-for-profit BUT yes-for-what?

I am in love with the not-for-profit sector, I switched around 2006 and almost never look back. I love the idea of helping people, doing good to others, support the world, and make my living as a by-product.
However, I must say that I feel a bit worried. While many great things are happening around, the nonprofit sector seems to stay aside and wait. Wait for what?! you do a great job, however you must understand – there is an essential need to measure your performance. Do not get me the wrong way – YOU need it. not your boss, certainly not the funding organization, not your manager, and not your board of directors. You need it, and you deserve to know what works and what not.

2014-10
I do believe that not-for-profit is not for profit, but if we are so confident in that – let’s ask: what are you there for? yes, I get you. You are there to help, to support, to facilitate, to give a hope. I agree, I am absolutely with you. However… how do you know that you really help, support, facilitate, etc.?
Let’s get straight to the point – you need to measure. You have to measure your impact, to benchmark between projects, to justify budgets. You cannot just do the “on the surface” stuff like counting and have nice graphs. Because the question: “how many people used our service in the last 12 months?” is a very good question, but what about the impact? what about the help, support, facilitation and so on? The question: “how many people were satisfied from our service?” is a neat question, however does not provide with any insight of your communal impact.

The questions, mistakenly perceived as unimportant, irrelevant, or annoying, are dealing with your impact. This impact can be evidently measured and give you a very clear picture of what works and what not.

Ask yourself – What is the difference between projects? Why project X and not project Y? Why project X must continue? What project holds the highest SROI? What project has got significant impact on your client’s life?

If you do not have answers to these questions – you better start to work on your impact measurement very soon.

The process of impact measurement will be described in my future posts, so stay tuned and feel free to subscribe to the newsletter.

Second article in this series

Third article in this series

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Everyone uses SMART goals, and you?!

I know – this is almost boring. Actually it is boring, why do I try to embellish reality?!
You have heard over thousands times that goals need to be S.M.A.R.T., and you are tired of it. You are tired because you need a reason to believe that it is going to help.
I used to feel frustrated when meeting with managers who do not like the idea of a work-plan, or do not find the time to do that, or whatever… But you know what?! that is why I started this blog, I wanted to write my thoughts and insights so I can express my enjoyment and belief that these processes really work and benefit organizations. Writing a work-plan may be a real fun, as long as you understand why you do it, and HOW to do it. I promise to do my best and explain the basics as well as the advanced, so if you are ready to think – we go on it, together, right now (-:

2014-09-28 11.21.45

Let’s firstly start with two vital facts:

As a consequence, you must be able to connect these two, and be able to measure, define, criticize, and change your goals and plans.
So, how do you set goals and objectives? Pretty simple actually. We need to clearly split three elementary parts of a work-plan:

  • The vision
  • The goals
  • The objectives

The vision is the wish, the large picture, the general direction, the utopia. The vision is usually articulated by senior executives, mainly board of directors members, and from my experience visions are usually good, and often very impressive. The vision typically includes one to two very meaningful sentences, where every word worth its weight in gold, or at least supposed to. I certainly agree that vision phrasing may take a while, however I am not that convinced it should take forever, and completely disagree with “no vision yet” or a “dead-end” circumstances and excuses. I am very sharp at this point because I see no logic in doing your job when you do not know where to go, and what is the purpose; or even worse – when your employees are lost. (let me put something here – no, you do not work just to earn your salary or remain employed). So, in short – message #1 is: have your vision handy.

Once we are done (if you are not – I promise to write soon on how to write a vision effectively and efficiently, so stay tuned…), and we have our vision – we need to break the vision to small pieces, the goals. Not too small, let’s say up to seven, and the most safe is something between 3-5. I know that there are managers who like the details, or feel that every word needs to get strong attention, or any other persuasive explanation – I suggest you to take it as an exercise – try to focus on 3-5 most important messages from the vision. Why? because you plainly want to do it, and hold it in front of you. Another helpful tip is to extract 3-5 single words that describe the vision the most.

In order to keep yourself in truck – bear in mind, this should take no more than 3-6 meetings with your relevant team, as long as everyone takes notes  independently, and do their homework. (however large organization may stretch it to a process, but still no more than 10 meetings with all of the relevant people are needed).
I suggest the following structure: meeting #1: overall view and open discussion regarding the most important messages from the vision; take notes, do your homework and conference back in meeting #2: the most important messages analysis – you should have a list to narrow. The final product is the 3-5 “raw” goals. This stage may take more than one meeting, but in my opinion and based on my experience – the efficient organizations will benefit the most… In meeting #3 the focus is on phrasing the goals. As I said at the beginning – this is not the SMART stage, we will have it later on. For now you need to just phrase 3-5 full, but short, sentences that express the vision and call for action. In short – message #2: write your goals shortly and clearly, in light of the vision.

2014-09-29 11.44.22

I was trying to think of example that will facilitate my explanation above in a friendly way, and that is why I chose a corny one – Coca-Cola. This document shows Coca-Cola’s 2020 vision, and I recommend you to take a glance especially at the left side – 2020 mission (Vision here), our vision (Words here) , our goals (Goals here). I am not 100% fan of their structure, but I think it gives a clear idea for managers who want to pursue planned processes.

In short:
> Vision: the big picture, usually contains a very small number of meaningful sentences.
> (Words/Raw goals: the fractions of the vision, the words that describe the vision)
> Goals: the actions needed to be taken in order to practice the theory

Part #2 of this article will be covering the connection between goals and objectives, an improved SMART model, and a wrap up of my arguments re vision-goals-objectives.

P.S. I am more than sorry for not entertaining you today with a must watch movie… I could not trace a relevant one in my mind – but will be happy to get your ideas, please share them in comments or email. In the meantime live happily.

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

He was coherently focusing on his goal, and you?!

Firstly, I would like to thank you my readers! I am more than delighted to see the stats growing every day, and as you remember I am numbers fan… so imagine how happy I am (-:
Secondly, since I have promised last time that I will be talking today re sticking on your goal, be sure – I will do it here and now. Ready to think? Let’s start.

Before I start shooting my arguments, please have a peek at Liam Neeson (I recommend you to watch the whole movie if you haven’t done this yet). The story, in short, deals with “a retired CIA agent travels across Europe and relies on his old skills to save his estranged daughter, who has been kidnapped while on a trip to Paris”. I would like to extract and focus on his techniques: He analysed the situation, targeted the goal, stuck to it, paid costs in order to reach it, and finally kept his promise.

Getting back to our management situation, the beginning seems easy to most of us – just think what you want to do, and you are all set; another way to do it is to write something – yes, work on it, conference relevant people, write a draft and then articulate a final work plan. However, please be aware, at this very early point things are starting to struggle, because you (and your team and/or managers) are deeply convinced that the goal is well known and absorbed, and you may all go rest on your laurels.

Please pay attention to mistake #1: you forget what you have decided, because you did not document it, or because you absently stored it in an abandoned storage site.
A worse version is to actually see it in front of you every day, but let it gather some dust on the shelf, because it mistakenly perceived easy to think freshly every day… The worst option, which you will be surprised to know that there are many people who love to choose, is to see it, remember it, but intentionally ignore it. When I ask managers why they do it, they simply say – I do not believe in plans; I know my goals; I don’t need reminders; I have significantly better ideas; I have no time for reading it, and let’s not forget the best excuse: I am focusing on the “doing”…

Well, my friends, this is mistake #2: you definitely do not remember your goals and plans. No one does. That is why you wrote it. You have a great tool, the plan or the paper with your goal written on it, use it. Read it every time you feel unconfident or search for routes. Promise you – it helps, a lot! Bear in mind, a written plan is very often better than a random idea, as great as it is. Not to mention the time consuming aspect… you go back to your written plan – you also save your expensive time!
I can assure you, there are people who will not like your strong-minded attitude of being planned, but sooner or later they will agree with your paradigm, because the results will be there load and clear.

In order to write your goal in a usable way for future examination and development – you should phrase it correctly, what do I mean by that?! Please follow my next post.

In short: keep calm and trust your written plan.

P.S. If you wish to continue reading in this topic, please consider reviewing this article on how to set goals in work plan.

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page