ESG has come under fire in recent months, with questions swirling about whether ESG efforts are truly making an impact or simply lining investors pockets. Its not the first time the industry has faced backlash. Models for measuring impact are complex and difficult to standardize.
Social impact has been particularly maligned, with calls to skip measuring it altogether. Our industry has known for a long time how frustratingly difficult it is to come up with a measurement method for social impact that is both comparable across companies and not overly onerous on the business.
Often, the quest for comparability has devolved into a checklist about company intent. This superficial approach makes no attempt to listen to the people who are experiencing the social benefit. The alternativeactually speaking with those beneficiariesis seen as prohibitively expensive and resulting in data that sits in a silo and cannot be compared across companies.
Impact is a journey, not a switch. Typically, people do not grow out of poverty in 10 yearsits more like a generation. We need to see impact measurement in the same way: it happens over time, in fits and starts, and it can even change direction over time as people, products, and the world change.
For years, I used a financial diary method to conduct in-depth research with vulnerable populations all over the world and provide deep insights into the financial practices of these families.
When COVID made face-to-face interviews difficult, we began using technology-enabled methods in our research, including Natural Language Processing (NLP). Weve now realized that these methods are better at tracking social outcomes, thanks to the ability to pick up nuanced responses cheaper and faster.
We work with companies all over the globe, tracking their social impact on vulnerable people. Each of these companies does dramatically different things and has a dramatically different social impact.
However, we discovered that companies were having impactand it often went beyond what they intended or expected. A great example is our work with an Indian food manufacturer that employs an impact model designed to change social norms by hiring women from conservative communities. They not only hire, train, and pay these women, but also actively work to change the gender perceptions of these women, who then go home and pass on their empowered views to their families and community.
By asking open-ended questions of employees, we found that the companys impact on their employees was significant across a wide range of indicators. But the most important result was that tenure at the company matteredthe longer the dose of gender empowerment employees received, the more they were able to advocate in the way that was the most difficult for many women in these communities: asking their families to help with the housework.
This discovery is now leading the company to renew its efforts in holding workshops to allow the women to exchange experiences and ideas about how to engage their family members to help alleviate their double burden of work both inside and outside the home.
Because we analyze open-ended responses from those who are impacted, were able to uncover outcomes that would have been lost in a more standardized survey. By listening between the lines, companies can improve their social impact by ensuring that their programs and initiatives have the intended outcome.
Daryl Collins, Ph.D., is the founder and CEO of social research firm Decodis, author of the acclaimed Portfolios of the Poor and a social research pioneer.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
