3 simple lessons when relying on reports

The Kids Co demise has been well documented and some of the lessons for procurement have already been drawn out by Peter Smith, for example, the need for contract management and sufficient reserves. However, today we have learnt that two separate  'independent' reports were cited as evidence of the good work of Kids Co which we can also learn from.

The reports were cited by Kids Co and probably swayed some funders decision making. I would suggest that perhaps funders may well have been naive and that those who based decisions on 'evaluations' need to look much more deeply before being influenced.

The first question worth asking was 'who funded the report'? Well one of the reports was funded by Kids Co and the other appears to have had Kids Co covering the costs. It's not unreasonable to see the potential conflict of interests.

Next, 'what was the purpose of the report'? Neither of these reports appear to have been outcome evaluation reports or indeed value for money reports. One was written by a Professor and leading psychologist who benefited from studying "the language of love". The second focused on the children who used the services of Kids Co.  The reports may well have been fit for purpose but that purpose was not one of commercial due diligence.

Thirdly, 'what was the approach to ensuring rigour'? I could be wrong, but neither of these reports seem to have been subjected to peer review which would have considered the research approach, reliability and validity.

Those three simple tests may have shone the spotlight on the weaknesses of relying on the reports. While it is easy to see the weaknesses we also need to reflect on those same points within a procurement environment, for example, when commissioning reports, when drawing on reports as part of a bid evaluation process, when taking the advice of 'independent' experts particularly in preparation of specifications. Caveat emptor as they say.