Is testing and learning a methodology or a core skill?
Today I was chatting with Irina and Trilly about implementing the skills framework for product managers at NHS England.
Irina had a great idea: she pointed out that the framework didn’t emphasise experimentation enough. There’s a lot of talk about using insights, delivering value and achieving outcomes, but the skills framework we use to shape our roles and assess our performance doesn’t focus on testing and learning.
As currently written, the framework doesn’t give talk about exploring ideas, gathering evidence or learning before proceeding. It’s sort of implicit…but is that enough?
The framework suggests that everything is predetermined or that you can quickly move from A to B. You frame problems, prioritise towards the ‘right’ outcomes, and make evidence-based decisions. All good things. But it assumes the answers exist and your job is just to deliver it well.
It made me wonder if we could completely overhaul the skills framework, to allow experimentation and uncertainty to be more present.
What if it emphasised experimentation more than delivery – or at least acknowledged that experimentation is a core discovery mechanism?
Experimentation and adaption is only mentioned in the Agile and Lean section, and only in the context of ways of working. But testing & learning is a mindset you need to embed – it influences how we think about outcomes, what value is, how user insights inform both of those, and more.
Often your job as a product manager is to design the flow and feedback loops that generate the evidence – either in the team or in the product. It’s a holistic approach that applies to every phase and aspect of the work, not only a delivery method.
As you become more strategic, you need to apply methods of learning as well as methods of delivery. I’m not sure the current framework espouses learning enough.
A good team focuses on epistemology – methods of gaining and sharing knowledge – as much as operations.
The uncertainty that teams often face and need to work through is missing too. That’s a gap you’ll need to be able to bridge on day one as a mid-level product manager – and more often as you move up the career ladder.
If Test & Learn is going to work at the policy level, it needs to be reflected in the delivery level too. Theory should be linked to practice; framing theory and practice through shared language creates joins.
Note: A lot of good work has been done to improve the skills framework over the last few years, particularly around creating value for money, managing product outcomes and applying user-centred insights. The adaptation proposed above should build on that work – but may question some realities that informed it.