L0 Governance — Do you know the rules of the evidence game?

→ Proof Points

Something new starts today.

For the past few months this newsletter has spent a lot of its time reacting to the news. Which has been fun. But I want to give you something more useful.

So over the next nine issues, I'm going to walk you through ProofStack's Evidence blueprint for every MedTech or HealthTech company we partner with.

This is the diagram I put in front of clients. It has been resonating rather better than anything else I've built, so it seems daft to keep it to pitch meetings.

The way it works in a meeting is simple. Everything starts grey… meaning you own all of it, and you're carrying all of it alone. Then we go layer by layer and work out what you'd keep, what we'd share, and what we'd take off your hands entirely.

What surprises most companies is seeing the whole thing laid out at once (in a dashing green color coding scheme).

We start at the bottom. Level zero. Governance.

It is the least glamorous layer on the stack and the one almost everybody skips.

It is also the one I get called about at 11pm….

Deep diveDo You Know the Rules of the Evidence Game?

Here is something odd about scientific training.

There is excellent guidance on how to use a pipette. There is excellent guidance on how to take consent. There is excellent guidance on statistical analysis.

There is almost nothing on how… to get published.

That part is “folklore”. You pick it up in journal clubs. It gets handed down by a professor who may or may not be right. You learn it on the go, badly, and then you pass on your version of it to someone else.

Which is survivable in academia, where everyone around you learned roughly the same folklore.

It is not survivable in digital health.

The tribe problem

Your team has a statistician, a physician, a psychologist, a UX researcher. Possibly a linguist. Every one of them arrived with a different set of unwritten rules about how evidence gets made, shared, reviewed, and published.

Genetics: findings get discussed in closed rooms. You hint carefully at the region of the gene you're circling, enough to advance the field, not enough for a rival to beat you to it.

Computer science: preprints and conference proceedings, shared early, often without peer review at all.

Medical journals: nothing leaves the building until peer review is complete, which means results emerge two years after the work was done.

Now put those three tribes on the same team and ask them to publish something together.

Nobody is wrong. Everybody is operating on defaults they've never had to articulate. And the defaults are incompatible.

Governance is just writing the rules down before that becomes a problem.

What actually gets decided

The three questions that cause the most damage are often left unspoken (dangerously so):

1. Who decides what we work on: The default state at a startup is that someone met an academic at a conference, said it'd be great to collaborate, and a few Slack messages went round. Somebody replied "yes, we should do this sometime." That is not a commitment. That is a vibe. Most early-stage companies have a hundred ideas circulating and measurable progress on none of them.

2. Who decides when it's due: When the CEO says it’s perfect", which we'd advise rewriting to "when it's ready for peer review". That's a difficult habit to unlearn but it's crucial for what comes next.

3. Who decides to help: Because you cannot do this alone. You need an engineer to pull the data, a designer who understands the patient buying psychology, a medical writer to hold it together, a physician to sanity-check your clinical assumptions.

So we write a standard operating procedure specific to the company. It names a decider—often the Chief Medical Officer. Her job is to sort the hundred ideas into A, B, C, and the bin. And crucially, to be the one who lets the intern legally know that we own the data (see the full story in Three Real Case Studies below).

Authorship is where the real trouble lives.

If I have an idea and hand it to a colleague who spends a thousand hours on it… who is the first author? The colleague thinks she is. I think I am. The founder who raised twenty million has views too.

We are not in the industry to get academic tenure. We are here to build a body of proof. So if there are no rules in place, the rule I set is that the first author is whoever did the most work and carried responsibility for getting it done, including synthesising everyone else's input.

Then, we must get clear on the first and possibly the first second author (yes, that exists) and the second and so on. Somebody is always mildly annoyed. That is fine, so long as they are annoyed at me rather than each other.

Then there's intellectual property, which reads differently in industry than academia. You may want to publish replicable science and protect a trade secret at the same time. In a world where AI can clone a method very effectively, that tension is getting sharper every day.

The nightmare scenario

You spend two years on a trial. You spend another year writing it up. It is accepted at a genuinely excellent journal.

Then author seventeen, who was laid off and is bitter, writes to the editor.

What can follow is an Expression of Concern: a warning notice attached to your paper. From that day on, nobody reads your work first. They read the warning first.

Your best scientific asset, permanently caveated. Not because the science was wrong. Because nobody wrote down what happens when someone leaves.

Three Real Case Studies

The intern who left with the data. A student did a placement, wasn't offered a job, got upset, and took the dataset back to the university. She was the only person who'd ever held it. It lived on her hard drive. The company got nothing, and could never publish the work. The research output belongs to the organisation and that governance was never defined.

The first author who quit. The relationship ended badly and the founder wanted the name removed from the manuscript entirely. You cannot do that. The person had done substantive work, and the ICMJE criteria are not negotiable on the basis of a personal falling-out. What you can do is use joint first authorship, add "*these authors contributed equally" so the person who rewrote the paper gets first billing and the person who left keeps the credit they earned. Nobody was delighted. Nobody was furious either. Worth noting: because the founder was the CEO, nobody internally could say this to them. That is often what I'm actually for.

The collaboration that went dark. A client sent a data file to a professor at a prestigious university and heard nothing for two years. In the interim the professor handed it to a PhD student who wasn't yet qualified to analyse it, who didn't check anything with the company, and who published it. The company found out by reading the journal. No authorship. Their product was described badly and their trademark was misused. 

These cases are more common than you'd like to believe. The fix here is a written contractual agreement that your people do enough of the work to merit authorship, so you retain visibility.

The Proof Point

Governance is not paperwork. It is the layer that stops a good paper turning into an expensive one.

Here are three questions to take to your next leadership meeting:

Do we have a process for making decisions quickly?
Do we have written guidance on who is an author?
Do we have a process for collaborating with external third parties?

These are the three items people most often answer "no" to on our Evidence Scorecard. They are also, almost without exception, the reason someone eventually DMs me on a Sunday.

— Paul

Next issue: L1 — Data. Where claims are grounded.

From our deskWe've Grown!

We’re thrilled to announce that Eliane Boucher, PhD, has joined ProofStack as an evidence consultant, coming from Happify and Equip. She's leading our behavioural and mental health work, an area I think is about to matter a great deal more than it currently does.

She'll be at Behavioral Health Tech in Nashville September 22-24. I won't, since I'll be coming straight off a week in Boston.

If you're going, reply and I'll introduce you!

More on the team over the next few issues.

Upcoming eventsTea Party, anyone?

I'll be stateside in Boston, MA on 13 September to see a couple of clients I'm proud to work with. Will you be in the area? If yes, reply to me here!

Thanks for reading,

Paul Wicks, PhD
Founder & CEO, ProofStack Health
Move Fast. Prove Things.

P.S. Here are 3 ways I can help you: 

1. Take the Evidence Scorecard Quiz. Answer 15 questions and we’ll send you a personalised report with feedback tailored to your specific needs.

2. Follow or connect with me on LinkedIn. I publish top resources and in-depth insights related to building your evidence stack.

3. Book a strategy session. Uncover the gaps in your evidence and marketing in your Digital Health/MedTech startup.