Tag Archives: management consulting

DIY: The Crafting Job of Program Evaluation

Professionals involved in program evaluation projects occasionally find that their understanding of how the work will be used and adapted by the organization does not necessarily parallel their clients’ perception of the project.
The DIY Method, a unique tool I developed, proves to make a vast difference in both collaborative work and the process of organizational performance, changes of policy, and program evaluation capabilities. The DIY Method begins at building capacity. We aim to include all managers and stakeholders in the process of program evaluation to better find common ground and agree on the initial need for the evaluations. Secondly, we seek to clarify the research question and discover the gaps in information about the program or initiative. It is vital to recognize what is unknown and requires further investigation. Thirdly, assuming that our research question is well defined, we then focus on constructing research tools for program managers or officers to use for data collection. These tools contain basic statistics and cross-tabulations to assist in day-to-day management. They can either be simple Excel based software, or built into the CRM or preexisting organizational database. It is decided by the clients which method works best and allows for smooth and beneficial program evaluation. This stage may take some time as there is a need to adjust, apply, and modify the tools to contribute with managerial decisions and facilitate formative program evaluation. Our professionals are there to help create a beneficial environment and assist with program evaluation tool implementation.

 

Program Evaluation
Program Evaluation

When all is working and in place, our consultants analyze the data collected for the quarter or year, and articulate a research report to summarize the results, data, and trends. A significant portion of our time is dedicated to writing practical recommendations, supplying a beneficial use for evaluations, and assisting with continuous work to facilitate a smart and successful atmosphere for decision making and operations.

 

If this method of work interests you and you are curious to learn more, contact me today for a FREE initial consultation!

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Introduction to systems thinking & social theory and practice – post #1

In the following  series of posts, I will introduce, review, discuss, explain, and examine the wicked topic of systems, by integrating several disciplines of knowledge (such as physics and sociology).

I will talk about actual consequences and implementations related to the social element in our life. By “social” I refer to the knowledge, perception, and practice about the human society. The materials I use are taken from academic books, lectures, YouTube videos, peer reviewed journals and so on.

 

Systems Thinking

 

The basic of systems thinking is that there is interrelationships between everything to everything. In simple words, what goes around comes around. This is not random this expression is using the round shape. It is circular. Think for a moment about earth, rounding around itself, rounding around the sun, other stars rounding around on the same system, the milky way within the galaxy, the galaxy within universe, and so forth. Step back, and see the same pattern for humanity. A child was born, circled by family, circled by broader family, circled by neighborhood, city, country, human beings, etc. now, add more components (let’s call them “agents”) to the system. Wild life, weather, forests, energy. Another frequent example is to imagine cloudy sky. We automatically know it’s going to rain, and then the sky will be blue again.

Russel Ackoff (2000) explains what is a system: “A system is whole which cannot be divided to parts, the system is dependent on how the parts interact, not how parts act alone. An example: life, our body; part of cars”.

We live in a never-ending system. In effect this is an infinite system, which is interrelated, and every agent is affecting other agents, and those relationships cause a dynamic change of the system. In fact, we are part of this complex dynamic system, and what we do is undoubtedly affecting other agents, in so many ways, shapes, and variations, but we tend not to see it, because we are not used to it, we think linearly.  We see straight lines. Cause and effect. A straight arrow from A to B. A led to B. we are having a hard time to internalize the obvious fact that it is a circle.

Systems Thinking

This may sound weird to you, but before getting familiar with Physics (especially quantum theory and information theory), and systems thinking in general, I was doubtful and considered myself a woman of facts and strong reality, with a special affection to multiple linear regression.

Therefore, repeatedly, Peter Senge introduced this topic of systems thinking in his book “the fifth discipline”, and in other lectures on (2014), with the very clear statement that Gut and heart are fundamental for every process of effective learning and action; and leader are the key. We firstly grasp it from the heart, then we translate it to thinking.

Ricardo Valerdi (2011) is convinced that system thinking is not a natural act. He explains that interruptions distract us from what we are doing; our brain is limited to boundaries of complexity and dynamics. As an example, he mentions Dan Ariely – “predictably irrational” book – on how people tend to wrong decisions because of abilities limitations.

In summary, systems thinking is not natural to us, we are not used to it, our school systems educate us to think linearly, however once we start seeing the patterns, interrelationships, grasp the complexity as a life fact, it is a matter of time, until we view the world utterly differently. Moreover, with enough practice, we can leverage our potential to achieve much better results, and avoid repeating problems.

As Senge says in his book “the fifth discipline” (1994): most of the problems we face as a humanity, reflect our inability to grasp and internalize complex problems.

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Regression to the Mean and its Implications on Program Evaluation

Background and Problem Identification

Most of the social / educational program are evaluated this way or another, and on this post I would like to focus on repeated measures of the same group of participants or individuals, as opposed to different groups comparisons or tests.

In many occasions we want to learn what is the impact of an intervention on attitudes, perceptions, and behavior; and by this we want to isolate the impact of the specific intervention, hence the program, and see how it changed the  attitudes, perceptions,  behavior, or a specific situation; in order to infer whether the intervention was effective or not.

Many of us will conduct t-test or repeated measures test. Another common way to  investigate those questions is using a linear regression model; and by this try to predict the change on our dependent variable by a series of controlled variables. However, here comes the “catch” –

Regression to the mean (RTM) is a statistical phenomenon that can make natural variation in repeated data look like real change. It happens when unusually large or small measurements tend to be followed by measurements that are closer to the mean.”
( Barnett et al., 2005)

The problem (RTM) may occur whether we measure an individual or a group, due to the random error (within-subject variance and between-subject variance).

A similar problem is identified as “a standard error of measurement (SEM), which refers to the standard deviation of an individual’s observed scores from repeated administrations of a test (or parallel forms of a test) under identical conditions”
(Koizumi et al., 2015)

The problem: variations in data sometimes DO NOT reflect a real change, but a correction of a previous random error.

 

In other words, we jump too fast to define a correlation as a causation, without checking carefully it really is!

Indeed, research conducted to investigate these measurement errors in social implications shows that many changes are accounted for RTM or SEM, and do not reflect a real change (Marsden amd Torgerson, 2012; Koizumi et al., 2015).

Ready to Think Regression to the Mean

Solutions and Food for Thoughts:

Be careful when you aim to predict something. Do not assume a vacuum. On the contrary, plan the study cautiously and take into account alternative explanations, and different routes for interpretation. In fact, there is some good advise on how to reduce the chance your study’s results will be affected by natural errors such as RTM.

Research Design:

  • assign participant randomly for all groups
  • make sure groups are the same size
  • always include a control group
  • control for alternative variables
  • use tools with high reliability
  • control for background variables and context

Data Collection and Analysis:

  • conduct more than one pretest
  • collect two or more baseline data
  • control for baseline average / st. dev. by adding the group mean to the equation (either on regression or Ancova)

(Koizumi et al., 2015; Bonate, 2000; Marsden and Torgerson, 2012)

 

Implications on Program Evaluation

Many social and education program seek to change an attitude or perception, and assist participants in gaining knowledge of certain areas (such as financial literacy or second language).

Evaluation for these program usually focuses on perception measurement using a  before-after design. Most of the time, RTS is not taken into account, and therefore interpretation of  program impact may be wrong. Needless to say, designs without a “before” measurement worth NOTHING in terms of explaining program impact or change. In addition, there is a second aspect to emphasize which is the presence of a control group. Very often it is very difficult to compose a group of participants just for the sake of evaluation; however you should take into account that if you do not do it, you will never be able to correctly assess neither a baseline nor a change in your group of study.

In short: be cautious, plan and conduct evaluation carefully, when bearing in mind that a change in attitudes, perception, behaviour or knowledge, can be explained by a variety of explanations, that may be slightly different than the intervention you evaluate.

 

***

Subscribe if you liked (:

…and feel free to contact me regarding program evaluation consulting projects

***

References:

Barnett, A.G., Van der Pols, L., Dobson, A. (2005). Regression to the mean: what it is and how to deal with it. Int. J. Epidemiol. , 34 (1):215-220.

Bonate, P. L. (2000). Analysis of pretest–posttest designs. Boca Raton, FL.

Guisasola, J., Solbes, J., José-Ignacio, B., Maite, M., Antonio, M. (2009). Students’ Understanding of the Special Theory of Relativity and Design for a Guided Visit to a Science Museum. In: International Journal of Science Education 31(15), 2085-2104

Koizumi, R., In’nami, Y., Azuma, J., Asano, K., Agawa, T., Eberl, D. (2015). Assessing L2 proficiency growth: Considering regression to the mean and the standard error of difference. Shiken, 19(1).

Marsden, E., Torgerson, C. J. (2012). Single group, pre- and post-test research designs: Some methodological concerns. Oxford Review of Education, 38, 583–616.

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Why Community Relations Matter?

Many organizations are doing a great job. They know exactly how to engage their target audience, they know to deliver the service on the best way possible, well.. they also know to fundraise money from the local community.

However, they do often underestimate the tremendous power community relations have in order to help them scale and grow. Community relations is not just having someone to actively address  and answer phone calls, and provide with information; it is about being proactive and initiative, let people know who you are and what you do. There are so many good organizations out there, why you and not others?!

Marketing for nonprofits

Of course, you are special and unique, and you know it with all of your heart; nevertheless, you should let decision makers and community leaders know about you, your organization, and your achievements. I know you have, I know you did a lot. Now make this information accessible to everyone – let them read about you in the newspaper, let them watch you on a Facebook video, let them know good things about your activities. Never forget to crunch numbers, to provide accurate data, to show a real picture of your impact and change of community’s life.

You will be surprised to learn how many out there did not know your organization even exists! You will meet many prospect donors and clients who wish to donate their time or money to help you, and grow the social impact of what you are doing. Make sure to involve politicians, from all levels and parties, let them speak for you, bring your voice higher, assist you to achieve social goals. It is working to a great extent, but the first step is to DO IT.

Good luck my friends!

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Good Managers Make Decisions

I have not been in a writing mood for a while, and every time I was thinking of getting to write in my blog, I have decided that there was not a good enough story to share. I met only good managers, only ones who work hard to achieve organizational goals… (:
However, in the last days, the very simple yet extremely important idea of how to make decisions was striking me again and again, so here we go…

Every one of us, has been in a situation in which we were expected to make a decision, which will influence the future. Most of the time we do it without hesitation; however how many of us have had to make a critical decision that will impact the life of many? isn’t it the time when we start shaking, revisit, revise, analyze the data from a different aspect, consult again, and rethink if we are going to make the right decision?! I am not sure what about you, but I certainly was part of this group…

DSC05907

..and here is the important lesson I have learned… you make a decision, and THIS IS IT. Given that you already searched, researched, discussed the topic on management meetings, and you have a good understanding of what may happen in every option, you just need to jump the last step – DECIDE. EXECUTE. Do it.

The lesson above in very strong not only in terms of good and efficient management, but also in terms of our managerial ability to get improved. Once we learn to make decision efficiently, the organization can scale and grow; and our managerial skills are higher and better.

 

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

3 Nonprofit Issues to Reflect and Ponder on

Nowadays I am involved in some very exciting projects, and would like to share with you some thoughts. By reflecting on the following issues, I have no intentions to convince you to adapt my view, and have no meaning to advocate for any system or line of operation; I rather wish to give you some food for thought. Something to reflect on when you have free time, some topics to discuss with colleagues, co-workers, directors, stakeholders, clients, etc. Ready To Think?! I hope you will also enjoy (:

1. Politics:
Well, this is quite a sensitive issue, but since we all have some sort of relation to politics – I would like to raise the question:
How much time do we invest in politics, and how much time is really required for making good out of it?
By asking this question, think, and ponder on, I am trying to count, quantify, and evaluate the output of the time I invest into the issues such as:
* What belongs to whom, and why?
* Why X obeys to Y?
* Is a specific type of relationship between stakeholders really necessary?
* What structure brings the most powerful results?
* How much does crony capitalism (of any sort…) affect the organization?
The last question I ask myself is – what questions are relevant to my organization, and what is the benefit from discussing it?

Not for profit management

2. Collaboration and collaborative action
This is another sensitive issue, but collaboration is a vital part of every organization, even if you do not realize it at first. You always want to leave doors open, you always want to collaborate, you can always share, and you do not have to always spy for sharks. They are out there, no doubt, but you can always find the benefit for the organization to grow from EVERY collaboration. Quit thinking in ego terms, and forget the matching part. Yes, do not forget if someone shuts the door on your face; BUT if they reopen it – go in and present yourself. Collaborate with similar organizations to learn and absorb; collaborate with complementary organizations to provide a better service to your clients; and collaborate with different organizations to amplify your impact and open your organizations to new directions. Forget, get read of competition terms, and find the benefit in collaborations. Believe me it works BIG time, even if not immediately.
The question is how and why do we invest on this connections, and how do we value them? 

3. The board role
The fact that businessman basically manage the nonprofit world is pretty precise. But given this fact, there is a bunch of benefits and opportunities for every nonprofit organization. You can advocate for your organization and get support, you can learn some business methodologies, you can learn business terminology, you get to understand business way of thinking. Until here I counted what YOU can learn. But there is another side, not less exciting – you can teach new methods, you can advocate for compassion, you may teach some beneficial terminology, you may convince to think in a social profit terms, you can gain more support on your side. Yes you can. Use your board, and make them work for the organization.
I would ask myself, over and over, what can I do more in order to engage relevant individuals in helping the organization out? How can I engage relevant individuals on pursuing some issues? Trust me, when you start interview and engage board members out of the board room, aiming to address a specific need or a topic, you get to discover a treasure of opportunities.

***

In summary, I wrote about three issues only, whereas there are hundreds of them. However, my goal is to suggest a different way of reflection and analysis to managerial issues in nonprofits, and I trust my reader to interpret it their own way (:

Thank you again for reading and thinking.

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

4 Questions to Answer Yourself Before You Measure Outcomes

Hello friends, I am now writing for over 6 months, and every time I publish something I feel that I somehow helped the world (: So thank you for reading my comments.

Today I’d like to focus on a very sensitive issue – outcomes measurement. Yeah! you need to measure. If you make money out of your activity – you may want to know what was the impact in order to maintain and retain clients; If you do nonprofit you may want to know what was the impact of your services, and how you can grow.

I will make my points simple and clear:

1. Feelings are not true. They are misleading. Never trust your senses or impressions. Measurement does not work this way, and this is why you want to employ it. Facts are usually different than what we tend to think… a smart women quoted me a very strong message 10 years ago:”without data you are just another person with an opinion”. This is still so true.

2. Your boss/es are not interested in statistics and data, they just want the work to be impactful. Really?! how are you going to know it? Write down your goals, translate into objectives, and have these objectives measured. Be the one who brings rational and data to the table, be the one to help the organization grow.

Nonprofits consulting - ready to think

3. You must have some sort of education with regard to measurement and evaluation. If you do not have, you are having very high possibilities to be mistaken. The most problematic trajectories are: questionnaires design and data analysis. I have seen thousands of questionnaires in my life so far, almost each one of them contained a critical mistake, which turns the whole business to be useless. So, bear in mind, neither your MBA nor MSW qualified you to write questionnaires and interpret data. Use professional advice.

4. What are you planning to do with the data? make sure there is a good reason to collect data and evaluate. Once you have the data, make the best out of it. Translate it into strategic steps and apply it. Then, recheck your performance and adjust accordingly.

Thank you for thinking and reading! On the next post I will share an example for a measurable impact questionnaire I created, for free use. Please stay tuned and subscribe (:

 

 

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

4 Questions to Ask Yourself Before You Donate to Nonprofit Organizations

The world is complicated, there are so many causes to donate money to, and your spare time and / or money to donate are limited. The questions of how to give, to whom, and why, are getting an increasing attention due to the growing needs on the one hand, and a shrinking amount of donated money and time on the other hand.

Nevertheless, the issue of giving has not got enough thinking and discussion so far, in my opinion, although it is only reasonable to assume that ones will want to know what profit their money can generate. Even in terms of the not for profit sector, it is still true and plausible to expect high efficiency, effectiveness, and return on investment.

Therefore, I have developed a simple Four Questions Test in order to assist donors in decision making regarding donating their time or money.

Question # 1:
What does the organization do?
A clear answer is required here. Neither vision nor mission statement are needed here.
The very simple answer is what is done by the organization.
Want three clear examples? Here You Go.

Question # 2:
Is it the most  important cause you can donate to?
The answer to this question is a bit tricky, and requires framing an opinion by asking yourself what is your definition of the best cause(s) to give to. The basic assumption is that you want to help as many as you can, which means you want to make the best out of your investment. Think about it and develop your view. You may find This Inspiring Book helpful. Although the author claims that poverty alleviation should be number one priority for everyone of us, I still find this book very helpful in shaping your mind in regard to finding the cause that close to your preferences of helping others.

 

Philanthropy

Question # 3:
What are the organization’s overhead costs?
Not a fancy question to ask, but you deserve an answer. As a general rule, you may expect no more than 15% overhead costs. More than this percentage, or worse – no information in this regard raise a question mark. A big one. Bear in mind, your money is limited, and you want to generate the higher profit possible. Assuming that – it is your duty to do your research and find out if the overhead costs are within the range. (And a personal note, I do not believe in crazy fundraising costs either, whether they are conveniently calculated out of the overhead costs or not! 20% or more of fundraising costs seem unacceptable to me, because it basically says that you cut 20 cents out of each dollar you give. You can do better with you money for sure.)

Question # 4:
What is the Return on Investment?
This term has countless definitions, but only one bottom line – what is the percentage of the generated profit in the project? Clear answer is required here. How much your dollars worth within the project/organization?! The ratio between investment and its return is very easy to catch. You should ask and get this information from the organization/s you wish to support. Any ratio above 1:1.5 is considered good profit in my opinion, because it is basically reflects 150% profit, and this is a clear indicator of doing good with your money, and making it worth more. The same process you do with your for profit investments applies here as well. (And another personal note here… I do not buy excuses in this issue. Every organization has the duty to measure, evaluate, and research itself on an ongoing basis, and part of it, is getting to know the organization’s services/activities’ impact in terms of dollars.)

Now, take a pen and paper and answer the questions.
If you do not know the answers – research. Do it diligently, because you want the best profit out of your money.
If an organization you consider to support does not provide with the information – ask to get it.
If an organization does not have the information – reconsider your intentions. You will not know where the money goes, what is done, and what is your return, so better to think twice, and research better options within your fields of interest.

Ready to think?! Go ahead and do the best you can.

****
Some clarification notes (in response to many emails and comments I have received):

1. Every donor follow their heart. I am not trying to convince anyone to choose any cause which is not their focus of giving. Do what you think is right.

2. Overhead costs is not a synonym of salaries. It includes rent and other costs… and yes in many cases there are too high costs of fundraising. It seems unreasonable to me to spend only 60-50% of donors money on the mission (assuming this mission really happens). However, there are many cases such as a small organization, starting-up a new project, or increasing of fundraising effort in which higher costs are acceptable. I give you food for thoughts, and you do the math.

(….And by the way, let’s phrase it a bit differently please… assuming that it is impossible to have 15% overhead costs and only 30% or so is a relevant expectation, plus every organization that tells you they have less than 30% overhead simply “plays with the numbers”… just makes the problem worse!!! No one wants to show high overhead costs, right?! so if they say they have 40% fundraising costs (or overhead), you may derive the inevitable conclusion – they spend much more than this amount. So, where does donors money go?!)

3. Salaries – I am a big believer of decent salaries, and will never support anyone who thinks that the sector’s workers need to sacrifice themselves because they help others. BUT if you chose to help others do not find excuses why you deserve the largest car and a crazily high 6 figures salary. It is not fair for your donors, and it is not fair for your employees; and it is bad to our world to have people who think they are the only talented people to bless earth, and the world owe them back.

4. Reading the comments I have received proved me something I already new. I am a millennial.  This is it (:

5. Cyber-bullying, even with a fake cover of legitimate discussion, is disgusting and very harmful. Think twice before you shoot someone just because you disagree with them. (By the way the impression about your style of talking to others does not do good to you either… even if my ESL is not perfect, I bet you understand perfectly everything I write. I am also kinda sure my first language level is far better than the first language of many educated and smart commenters).

6. Very nice examples of organizations that use donor’s money efficiently (and there are many more, Find them and support them):
UNICEF
Plan Canada
United Way

7. A great article you may consider to read, and may change your way of thinking.

8. Do not forget why you are here – do good, do good, the your best to do good!

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Collaboration is Essential

Last time I wrote about the possibilities collaborations may open your organization to, and I focused on the very fact that there are people who need a boost or encouragement in order to internalize this basic element of organization’s healthy life. I eventually summarized that most of the managers are luckily capable of this, and may collaborate with other organizations, at least for the sake of their own performances.

Today, I would like to focus on another essential fact of collaboration which is the benefit of the society. I would call it Synergy… when your organization collaborates with another organization, you create a new opportunity, which in turn benefits your organization, the other organization, and your and theirs clients.

nonprofit management Ready to think

However, the most important point, in my view, is that you have created a much larger benefit – you have made a new opportunity of positive influence and connection, that can lead the world to be a better place to live in. Think about it, you did not just “do good” to your own performance at work, and so not your collaborative partner, you did not just create a good service for your clients, you have also made the world a positive environment to be in.
So, next time when you are successful in collaborative efforts, bear in mind – you have just did something good for the whole society.

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page

Collaboration IS Possible!

Are you ready to think?! Let’s start (:
Today’s’ lesson is about a great Executive Director, who I met in networking recces at a professional conference.  The conference happened to be a great place for me to grow the network for an organization I work with, and I planned to do it the best I could.

Just a short background – I work with a nonprofit which provides services to other nonprofits’ clients. Without a large network they cannot make a real impact, so it is essential to grow the network proactively. That ED was contacted in the past but never wanted to really collaborate, although the very fact that both organizations may highly benefit from having their clients satisfied and living decent life…

Ready To Think Management
I have decided to go talk to the ED no matter what the result will be. The conversation started with big No No, “we will not collaborate”. Exact words! You could have feel the energy of saying “No” so sharply… but I have decided something, right?!, so stuck to my point I said: “But you know your organizations’ clients will be incredibly benefited from our services, and you know no other organization in the area has this type of great capacity and operations”. As an answer I got a business card with email address on it… later that night I received a reply “Yes, we will do it”.

And I say GO FOR IT my friends. Once you have a goal to do good – do it and convince others to collaborate with you. It is a universal true – we all want to do good and help others, so if you happen to be in a position that allows you to do good – make an effort and make it happen, and never forget – collaboration is absolutely possible.

Share the knowledge...Tweet about this on TwitterShare on Google+Share on LinkedInShare on FacebookEmail this to someonePrint this page