The wicked problem of paediatric sepsis

Why it's near impossible to improve finding the elusive needle in the haystack

One condition strikes more fear into the heart of paediatricians than any other, and that is sepsis. For something which is so universally feared, you would think we would have no problem defining what it is. You would be wrong.

A simple description of sepsis would be a catastrophic, aberrant immune response to a severe infection leading to organ failure. Being able to define precisely what that looks like in an actual child is very difficult, even more so than in adults (which is controversial in and of itself). Work is ongoing to define sepsis in children, but this difficulty speaks to how vague the clinical presentation can be.

We often focus instead on “invasive bacterial infections” (IBI), which are much more easily defined as a bacterial infection in the blood (bacteraemia) or brain (meningitis or encephalitis). The diagnosis of IBI can be made relatively cleanly by identifying a bacteria in the blood or fluid around the brain/spine by modern microbiological methods. For this article I will use IBI and sepsis more-or-less interchangeably; they are certainly NOT the same thing, but for the purposes of research and clinical care we treat them very similarly (the assumption being if you have an IBI, you are treated as though you will develop sepsis at some point, if you have not already).

What is the problem?

Paediatricians fear sepsis for 2 reasons:

1. It is dangerous and potentially lethal

2. In it’s early stages it can be utterly indistinguishable from normal, benign childhood infections

The problem with point 2, is that we see A LOT of children with normal, benign childhood infections. Febrile illnesses are the single most common reason for children to present to acute healthcare (for more about childhood fever you can read here).

This is the haystack.

Sepsis on the other hand is extraordinarily rare - particularly so in the modern era of vaccination which have reduced Hib and Meningococcus to rates so low that many junior paediatricians will never have seen a case in their careers. A whole department may only see a handful of cases per year.

This is the needle.

All paediatricians are aware that whilst wading through the haystack, at some point they may pick up a needle and discard it, because at the time it looked identical to a straw of hay.

The child who presents to healthcare with overt signs of sepsis (the massive, obvious needle, which more resembles a javelin) is not so much a problem. We are well prepared to manage this situation, and with rapid treatment the outcomes are very good.

But the tiny, hay-like needle is a big worry. Once the child has been sent home, they may rapidly deteriorate and re-present to healthcare too late for effective treatments to be implemented. This thought it what keeps us awake at night.

What can we do?

You might think if a problem keeps paediatricians up at night, that we would invest a significant amount of resources into trying to improve the situation, and you would be right! However, this is a much more thorny issue than it is given credit for, and here we will explore why it is an almost impossible problem.

Screening

One of the most common methods of trying to improve identification of sepsis in children is using “sepsis screening tools”, which are usually some sort of scoring system based on physiological parameters (such as heart rate and respiratory rate) implemented at first assessment which are supposed to alert nursing or medical staff that the child might have sepsis. These tools suffer from many issues, but I will highlight the 2 most important:

1. They perform extremely badly

As discussed previously, you will notice that we lack a consensus definition on what actually constitutes sepsis in children. Children have a very pronounced physiological response to fever, which increases their heart rate and breathing rate, makes them feel a bit cold and clammy, and can make their skin look mottled. These are all considered signs of sepsis - but they are also completely normal.

What happens when all the signs of sepsis overlap with the signs of normal fever? The tool tells you almost every child has sepsis.

What happens when a tool tells you every child has sepsis? One of two things:

1. Everyone ignores the tool

2. Every child gets treated for sepsis

As the primary goal of initial treatment for sepsis is IV antibiotics, route 2 means a lot of broad spectrum IV antibiotics to a lot of children - not good for the children, not good for the health system, and certainly not good for the ongoing problem of antibiotic resistance.

2. The group of children who will significantly benefit is small

It is helpful to consider children coming to hospital with sepsis as being in 4 different groups:

1. Obviously have sepsis, would be recognised and treated with or without a tool (outcome unchanged)

2. Do not obviously have sepsis, but would be recognised without a tool eventually and get treatment in time anyway (outcome unchanged)

3. Do not obviously have sepsis and would not be recognised without the tool, would otherwise not get treatment and become severely ill or die (outcome changed)

4. Do not obviously have sepsis, but doesn’t get recognised even with the tool (outcome unchanged)

Of these 4 groups, there is only one where using a sepsis tool actually makes any difference- group 3. For all the others, either the tool was unnecessary or didn’t work. In order to reach this one group, we need to deploy our screening tool on every child presenting to hospital with an infection or a fever.

Sepsis is rare anyway - now imagine how rare the children in category 3 are among the children with sepsis. Now imagine how many children need to be screened and how many will get unnecessary treatment based on the screening tool.

I’m sure you can now see why this is a serious problem.

Better tests

Much work is being done to develop blood tests which can more accurately tell us which children have an IBI (test’s can’t tell us who has sepsis because we can’t define sepsis) using various ‘omics and big data techniques through machine learning etc. Some of them look promising! Hopefully this would overcome the issue with sepsis screening tools of poor performance (although there is only opportunity to change the outcome for a small number of children). But it also brings new problems.

Who do you test?

We cannot do blood tests on every child who comes into hospital. For children who look unwell enough that we consider doing blood tests on them, it is exquisitely rare that an IBI is missed by both the clinical team and the blood tests. This doesn’t address the children who we are most worried about - the children with sepsis who on initial presentation do not look different enough to the other children to cause concern.

The base rate

Even if we were able to test every child and the tests performed well, there new issue - separating test performance from test predictive value.

The issue here is complex, but important. Say a test was highly sensitive (it correctly caught lots of cases of IBI) and highly specific (it very rarely incorrectly said a child had an IBI when they didn’t) - in a setting where IBIs are so rare, we would still end up unnecessarily treating large numbers of children.

Let’s do some maths:

Say our test is 90% sensitive and 95% specific (this would be very good!).

Figures from a recent study showed that of >5000 febrile children presenting to ED, 6 had an IBI. We therefore estimate the rate to be ~0.1% (or 1 in 1000).

If we have 10,000 febrile children, ~10 will have an IBI. Our test will catch 9/10.

Of the 9,990 who don’t have an IBI, the test will be negative in 95% of them, but positive for 5% of them - this is ~500 children.

So our test will be positive for 509 children, of whom only 9 actually have an IBI.

So our positive predictive value (PPV - the probability a child who tests positive for IBI actually has an IBI) is 9/509 - or just under 2%.

Even if we limit our testing just to children with abnormal vital signs (41% of the cohort) and assume this includes all 10 cases of IBI, we get a PPV of 4%.

The problem here is the base rate. So few children actually have sepsis, that it is almost impossible to get a test that performs well enough that it can cut down the number of false positives to an acceptable number, without solving the issue of “who to test”. We have a very large denominator (big haystack), and a very small numerator (very few needles).

People might look retrospectively at cases which were missed and say, “well if only they had this test then things might have been different”. This could be true, but what is not acknowledged is that for the missed case to have even ben, we would have to be testing almost every child and would end up wrongly treating 25 - 50 times more children as a result of false positive tests.

We have not yet even addressed the issue of the child who presents and at that point doesn’t have an IBI yet, but will go on to develop one later. How can we test for something the child doesn’t even have?

What is the answer?

This is a wicked problem. There are a tragic, small number of cases each year of a child who presents to a hospital with sepsis late in the course of their illness, or is missed on initial presentation and goes on to become severely ill and die. No one wants this to be the reality, and it is a problem worth trying to solve. We will not get closer to solving it without getting to grips with the very real problems which exist, and we need to be realistic about how many of these tragedies are truly preventable. Some certainly are, but some are not.

For now, our best bet remains education and safety netting for parents. On discharge, it is not goodbye forever. Things can change, and informing parents about the signs to look out for to return to hospital could be the difference between arriving in time for treatment, or not.

Equally, we can make the haystack smaller. If parents feel more comfortable managing simple febrile illnesses at home, we can reduce the size of the haystack (children coming to hospital with fever) making it easier to find, or test for, the needles.

Of course, we can hope that advances in vaccination simply make the needles disappear.