COVID-19: TALKING ABOUT THE NEW NHS COVID-19 APP

This morning I was interviewed by David Prever on BBC Radio Oxford, talking about why I won't be downloading the new NHS COVID-19 app.  This was raising some of the key points from this blogpost about the app.

The whole feature started 3hr6:15 and I was the second speaker after 3hr10.

As I said on air, "I am supportive of the app in principle" hoping the app had been well written, and be a very useful contribution to the fight against COVID-19.  That is because people are infectious before they develop symptoms, and the app could help identify individuals who ought to self-isolate before showing symptoms, thereby reducing transmission of the virus.

But having tested many far more sophisticated systems over the years, the app does not appear to be fit for purpose.  There is one aspect of the design that is apparently a 'showstopper', plus a number of othe significant issues.  Unlike the Bake Off creations, this type of 'showstopper' is a reason for the app not to have been released.

On further investigation, there is a subtle difference between how alerts to self-isolate are issued to users depending whether the user has 'encountered' (my term) someone subsequently proven infected:

• Automatically using the proximity algorithm and Bluetooth:
  
• This is an instruction to self-isolate for 14 days
  
• For venues check-ins, manually triggered by the local health protection team:
• "it will not usually tell you to self-isolate, just to be aware and to get a test if you develop symptoms". The latter part is no different from what you would do anyway, so adds little if any value
• There is also the possibility that the alert will be a request to self-isolate 
  

In that context, would I download the app? 

Update 3/10/20: There are a couple of reasons why I might download it, discussed here.  Mainly as there is less chance of incurring a £1000 fine by using the QR code scanning in the app rather than providing my details separately to the venue.

THE CONSEQUENCES OF SELF-ISOLATION

Many people on receiving a self-isolation instruction will have major problems with self-isolation, for example:

• Those on zero-hour contracts will lose earnings, and possibly lose the job completely
• Self-employed will potentially lose income, depending whether they can work at home, and again risk losing a contract
• Single parents would have difficulty with child care
• Those caring for older relatives will not be able to do so.  So who will provide the care?
• Everyone will be inconvenienced, unless already shielding
  

There is also the prospect of a £1000 fine for breaking a self-isolation request. Although it has been confirmed that alerts issued on the app mean a fine is not enforceable, presumably because the de-centralised architecture of the system means the authorities have no knowledge of alerts, nor sufficient proof to issue fines.

Nonetheless, people will want to 'do the right thing'.  It is therefore vital that the vast majority of alerts are correct, with 'false alerts' ('false positives') being negligible.

 

THE POTENTIAL SHOWSTOPPER

The potential showstopper is that the app lets you check-in to venues, but doesn't let you check out.  

Let's say you leave the venue at 2pm and go to work rather than another venue, and someone arrives at the venue at 6pm who subsequently proves positive for COVID-19.

The system will assume you were there at the same time, and could result in you receiving an alert.  As described above, that may be just a warning, or could ask you to self-isolate for 14 days.  Given the de-centralised architecture of the system, it is unlikely you would be provided with sufficient information to know whether that alert was a 'false alert' or was correct.

Experience of the beta test of the first version of the app in the Isle of Wight was people started deleting the app when they got fed up with it.  Thereby defeating the purpose.  

I'm prepared to accept alerts and self-isolate if they are far more likely to be correct than false.  But this design is inherently likely to produce more false alerts than correct ones.  The question is whether those alerts would be merely a warning, or an instruction to self-isolate.  Either way, I'd soon get fed up!

So maybe not as bad as I first was led to believe.  But in any case no thank you.  Until there is much more clarity and experience, I won't be putting myself in the position of receiving false self-isolation alerts by downloading and using the app.

 

BUT WHAT ABOUT MY CIVIC DUTY?

Primary purpose

David raised two very important points:

• Have you got any evidence of false alerts ("false positives")?
  
• Isn't it socially irresponsible not to download and use the app?

I said that it was too early to have evidence of false alerts, as it was too early for people to tell whether they have received false alerts.  This is because the app gives the user little or no information as to where or when the contact with the positive person was thought by the system to have occurred.  It can't, because the key information is only held on individual mobile phones. It is only after some time of multiple alerts, especially when they don't result in developing symptoms, that people will twig that alerts are unreliable.  

The central system appears to have no way of knowing whether alerts are correct either.

As to being socially irresponsible by not using the app, I said that it was socially irresponsible to worry people unnecessarily, especially if that was an instruction to self-isolate.  That is because the design on venues looks as if it could create more false alerts than correct ones.

Update 4/10/20: Here is a real life example of the distress caused by an unclear and potentially false alert, where this man's wife is high risk and panicing, n hsi step-daughter is having to live elsewhere as a key worker.  As he says, echoing my view, "...damn it, this is basic beta stuff that should be ironed out".  He is no longer self-isolating, defeating the object.

WHAT IF THE VENUES ISSUE WAS FIXED?

Let's say the app developers fix the venues issue by allowing a checkout.  Would I then download and use the app?

Firstly I'd have to remember to manually do the checkout.  Unlikely to be 100%. But putting that aside, we then have to consider the second league of deficiencies identified.  Are they serious enough not to use the app?:

• The other cause of 'false alerts', proximity, where two people are actually either side of a party wall or ceiling.  The FAQs state that if someone is 2 metres apart for 15 minutes, an alert to self-isolate would be sent to one person if the other person is proven positive.  Two examples:
• If chairs are positioned either side of the party wall, then two people will be within proximity.  
• Then there's public transport, where everyone should be wearing a mask.  The app cannot distinguish whether a mask is being worn or not
• The app is not cognisant as to whether the contact is outdoors or indoors, two completely different risk scenarios
• The Police force has been told not to download the app to work phones, and to ignore self-isolation alerts, or preferably switch off the app.  Presumably because of concerns about false alerts.
  
• Not being able to record test results when carried out by NHS labs.  That probably won't affect me, and is being fixed
  
• Battery drain, given my phone can die without switching Bluetooth on
  

This leaves me with three problems:

1. My concern is that the venues issue and the NHS labs issue suggests bad design and inadequate testing.  What else have they overlooked in rushing the app out?  Here's a critique of the QR code coding, which is another issue which doesn't inspire confidence in the app's development
   
2. Is the proximity issue either side of a party wall a major problem?  Or use on public transport?  I just don't know.  I have no faith in the testing.
3. I don't want to drain my phone battery
   

I really wanted to download the app and use it.  But after testing many systems over the years, this is coming across as not being fit for purpose as it is.  Fixing the most significant issue would still not give me sufficient confidence in its design and validity.

What a crying shame!

So sorry, David.  I won't be downloading it until my confidence in it is significantly improved.   That's going to require some significant improvements.

Update 3/10/20: I strongly suggest that people do not use the Bluetooth proximity function, but the app might be worth using for some limited purposes.

My concern is people will in any case lose trust in the app over the next few weeks.  That will undermine the value not only of this app, but also any subsequent improved iteration.