Lateral flow testing at CDP: A surprising result
When we modelled the use of lateral flow testing at CDP we discovered something surprising: there was a high chance we could test everyone every day without preventing a single transmission of COVID-19. This application of mathematical modelling and bioscience gave us powerful evidence on which to base our response, allowing us to direct our efforts where they will have maximum impact: improved ventilation and new air filtration installations in our offices, labs, and workshops.
In common with all businesses, CDP has been closely watching developments in practises, and technology to keep our people and community safe from COVID-19 infections, while maintaining business operations. In the UK, lateral flow tests (LFTs) have been rolled out in a variety of settings over the last few months. These tests have major benefits in that they are low cost, give a result in half an hour, and require no medical expertise to administer. When the use of these LFTs became a possibility here at CDP, our COVID-19 team began drawing up plans for the roll out.
The two key questions were “who to test” and “how often to test”
As a multidisciplinary business with diverse capabilities and specialisms, our people work in a variety of locations and patterns. Most of these working patterns and risk profiles don’t match those of the early adopters of these tests, such as those in clinical and educational settings. As a result, we built a team to analyze the available data and tailor our use of the tests for maximum impact in our particular case. The team was led by myself, a simulation scientist, and my colleague Richard Owen, our Senior Consultant Bioscientist. The team identified the latest bioscience data available on the parameters of COVID-19 and the LFTs, then developed a bespoke Monte Carlo model – used to predict the probability of different outcomes – to model potential infections across the business. We used the popular Anaconda python platform for scientific computing.
What are the key inputs?
COVID-19 infection timeline
A viral infection typically progresses through several stages: when a person first catches the infection, the virus multiplies until they become infectious and often continues to multiply causing symptoms before the immune system is able to fight back and eliminate the virus. LFTs can provide an “early warning” when virus levels start to increase, but before symptoms start.
‘Effective R’ within CDP
We’ve changed our working environment in a variety of ways to reduce transmission potential. If this was 100% effective then LFTs wouldn’t offer any benefit, but we all understand that the measures are instead designed to reduce the risk to the lowest reasonable level. While we have no evidence for transmission on-site, we’re aware of some cases, unfortunately, brought in from the outside community and so we applied a “reasonable worst case” estimate of transmission.
Background population case rate
Clearly more cases of COVID-19 circulating outside CDP would result in more infected people coming onto our site and identification of each one could potentially prevent further infection. We recognized that this value has changed rapidly so we investigated the benefit of LFTs in a variety of scenarios.
Sensitivity: if a person with COVID-19 takes an LFT, what is the chance that it will give an accurate, positive result?
This property of the LFTs on the market is very important. While they can be more than 90% sensitive for symptomatic people, those people should already have isolated and obtained a “gold-standard” PCR (lab) test. When used in asymptomatic populations with well-functioning immune systems, the sensitivity can be as low as 3%. Considering the population demographic in this study compared to our own, our model took a less pessimistic view of LFT performance and erred on the side of higher sensitivity.
Specificity: if a person without COVID-19 takes a lateral flow test, what is the chance that it will give an accurate, negative result?
The LFTs on the market are thought to have a specificity of around 99.5%. While 0.5% might sound low, current estimates are that only 0.1% of the population has COVID-19; therefore the 0.5% false positives actually make up significantly more people than the number that are really infected. This is the source of some controversy as it can cause unnecessary isolation when the case rate is low; however, this risk was not considered a significant problem for us at CDP as we took a “better safe than sorry” approach.
What did we learn from the model?
There are multiple measures for the success of a testing program. In our analysis we simply looked at the number of people becoming infected, and how much this could be reduced by a variety of regimes. We ran the model many times with differing input values to evaluate the impact of testing regimes and understand the sensitivity of our results to the various inputs, which are either uncertain estimates or subject to change over time. In a result that surprised us all, we discovered that in our specific situation the benefit of LFTs is actually very small. Of course, the keywords here are “our specific situation” – by tailoring our model to CDP we gained maximum value for our own decision. However, this model is inherently not a generalized result and is not a valid evidence base for decisions in other contexts. There was a high chance that we could test everyone, every day (totaling thousands of tests) without preventing a single transmission of COVID-19.
What was the outcome?
We both verified that each small “cog in the machine” was behaving as expected and validated that the results of the whole model matched reality (we already had a historic dataset for what COVID-19 transmission looked like without lateral flow testing). We also further explored uncertainty in the driving factors, to assure ourselves that the remaining uncertainly in the inputs would not substantially change the outputs. Following this process, the non-intuitive result allowed us to confidently redeploy our efforts onto alternative COVID-fighting initiatives. Following the evolving scientific knowledge, we’ve improved the ventilation of our offices, labs and workshops and installed air filtration to reduce the potential for airborne viruses to move between people.
We inevitably enter investigations with preconceptions, but by applying science to the big decisions we’re able to confidently manage our choices and prioritize our resources to keep ourselves and others safe in this weird world. By combining our expertise in both mathematical modelling and bioscience, we created a team that is more powerful than the sum of its parts, demonstrating the power of mathematical modelling in making decisions.
Consultant Simulation Scientist