Using NIHSS to select large artey occulsion stroke patients (most suitable for thrombectomy)

Clinical assessment of stroke has been proposed as a method of selecting patients for transfer directly to comprehensive stroke centres which provide both thrombectomy and thrombolysis. Use of NIHSS has been shown to have the best performance of all clinical tests for prediction of large arety occlusion [1], though NIHSS is not easily or quickly performed pre-hospital, so simpler tests have been proposed, with RACE having the best performance [1], and being based on, and having been shown to be well correlated with, NIHSS [2,3].

In this experiement we look at how NIHSS predicts use of of thrombolysis, alongside publsihed results for NIHSS predicting the presence of large artery occlusion [4].

[1] Duvekot MHC, Venema E, Rozeman AD, et al. Comparison of eight prehospital stroke scales to detect intracranial large-vessel occlusion in suspected stroke (PRESTO): a prospective observational study. The Lancet Neurology 2021;20:213–21. doi:10.1016/S1474-4422(20)30439-7

[2] de la Ossa Herrero N, Carrera D, Gorchs M, et al. Design and Validation of a Prehospital Stroke Scale to Predict Large Arterial Occlusion The Rapid Arterial Occlusion Evaluation Scale. Stroke; a journal of cerebral circulation 2013;45. doi:10.1161/STROKEAHA.113.003071

[3] http://racescale.org/2015/08/03/scale-validation/

[4] Turc G, Maïer B, Naggara O, et al. Clinical Scales Do Not Reliably Identify Acute Ischemic Stroke Patients With Large-Artery Occlusion. Stroke 2016;47:1466–72. doi:10.1161/STROKEAHA.116.013144

# import libraries
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# import data
raw_data = pd.read_csv(
    './../data/2019-11-04-HQIP303-Exeter_MA.csv', low_memory=False)

Restrict original data to hospitals with at least 300 admissions + 10 thrombolysis patients

keep = []

groups = raw_data.groupby('StrokeTeam') # creates a new object of groups of data

for index, group_df in groups: # each group has an index and a dataframe of data
    
    # Skip if total admissions less than 300 or total thrombolysis < 10
    admissions = group_df.shape[0]
    thrombolysis_received = group_df['S2Thrombolysis'] == 'Yes' 
    if (admissions < 300) or (thrombolysis_received.sum() < 10):
        continue
    
    else: 
        keep.append(group_df)

# Concatenate output
data = pd.DataFrame()
data = pd.concat(keep)

print (data.shape)

(239505, 62)

Restrict to known NIHSS

# Restrict data to determined NIHSS
mask = data['S2NihssArrival'] >= 0
data = data[mask]

Restrict to known onset within 6 hours

mask = (data['S1OnsetToArrival_min'] <= 360) & (data['S1OnsetToArrival_min'] >= 0)
print (f'Proportion with known onset of up to 6 hours: {mask.mean():0.3f}')
data = data[mask]

Proportion with known onset of up to 6 hours: 0.434

Iterate through NIHSS cutoff

results = []
for nihss in range(0, 16):
    mask = data['S2NihssArrival'] >= nihss
    selected_data = data[mask]
    row_results = dict()
    row_results['Min NIHSS'] = nihss
    row_results['Proportion'] = selected_data.shape[0] / data.shape[0]
    row_results['Arrive in 4 hours'] = np.mean(selected_data['S1OnsetToArrival_min'] <= 240)
    row_results['Infarction'] = np.mean(selected_data['S2StrokeType'] == 'Infarction')
    row_results['Haemorrhage'] = np.mean(selected_data['S2StrokeType'] == 'Primary Intracerebral Haemorrhage')
    row_results['Thrombolysis'] = np.mean(selected_data['S2Thrombolysis'] == 'Yes')
    row_results['Prop of all thrombolysis'] = (
        np.sum(selected_data['S2Thrombolysis'] == 'Yes') / np.sum(data['S2Thrombolysis'] == 'Yes'))
        
    results.append(row_results)
    
results = pd.DataFrame(results)
results

	Min NIHSS	Proportion	Arrive in 4 hours	Infarction	Haemorrhage	Thrombolysis	Prop of all thrombolysis
0	0	1.000000	0.865353	0.863379	0.135339	0.263754	1.000000
1	1	0.949110	0.867161	0.861880	0.136977	0.276948	0.996591
2	2	0.873663	0.871033	0.856163	0.142784	0.298120	0.987500
3	3	0.772523	0.877520	0.846576	0.152381	0.328134	0.961089
4	4	0.681749	0.884194	0.836926	0.162104	0.355088	0.917829
5	5	0.603316	0.890638	0.827512	0.171495	0.370004	0.846356
6	6	0.536375	0.896123	0.817681	0.181298	0.376989	0.766654
7	7	0.479221	0.901052	0.808963	0.189980	0.380866	0.692006
8	8	0.431297	0.904100	0.800268	0.198629	0.381683	0.624138
9	9	0.390545	0.906716	0.792871	0.205938	0.380174	0.562931
10	10	0.356698	0.908933	0.786602	0.212123	0.378930	0.512461
11	11	0.326188	0.910237	0.780710	0.217896	0.377776	0.467202
12	12	0.298500	0.911917	0.775500	0.223115	0.374420	0.423746
13	13	0.273355	0.913683	0.770766	0.227835	0.372490	0.386050
14	14	0.250452	0.914703	0.765939	0.232575	0.369950	0.351293
15	15	0.228345	0.915000	0.760478	0.237938	0.365076	0.316066

Appendix - performance of RACE

The detailed performance of RACE test is described at http://racescale.org/2015/08/03/scale-validation/

Analysis of patient counts

Note: Quotes sensiyivity and specificity in paper exclude some of these patients from the analysis of sensitivity and specifict presented above (n=320 vs n=357 here).

Patient counts by RACE Score:

RACE	LVO	nLVO	Haemorrhagic	Mimic
0	1	26	2	8
1	2	26	2	12
2	5	27	2	5
3	7	26	3	9
4	9	17	9	5
5	12	12	8	2
6	19	14	13	3
7	19	9	6	0
8	20	2	5	1
9	5	2	2	0

Patient counts by cutoff.

RACE	LVO	nLVO	Haemorrhagic	Mimic	Sens	Spec
>= 0	99	161	52	45	1.000	0.000
>= 1	98	135	50	37	0.990	0.102
>= 2	96	109	48	25	0.970	0.245
>= 3	91	82	46	20	0.919	0.317
>= 4	84	56	43	11	0.848	0.440
>= 5	75	39	34	6	0.758	0.584
>= 6	63	27	26	4	0.636	0.687
>= 7	44	13	13	1	0.444	0.853
>= 8	25	4	7	1	0.253	0.916
>= 9	5	2	2	0	0.051	0.979

Patient counts by cutoff (excluding mimics)

RACE	LVO	nLVO	Haemorrhagic	Sens	Spec
>= 0	99	161	52	1.000	0.000
>= 1	98	135	50	0.990	0.038
>= 2	96	109	48	0.970	0.076
>= 3	91	82	46	0.919	0.115
>= 4	84	56	43	0.848	0.173
>= 5	75	39	34	0.758	0.344
>= 6	63	27	26	0.636	0.497
>= 7	44	13	13	0.444	0.746
>= 8	25	4	7	0.253	0.863
>= 9	5	2	2	0.051	0.961

Converting RACE performance results to NIHSS

From RACECAT published results, NIHSS = 2.34 RACE.

# Set up figure
fig = plt.figure(figsize=(12,6))
ax1 = fig.add_subplot(121)
x = results['Min NIHSS']
y1 = results['Proportion']
y2 = results['Prop of all thrombolysis']
y3 = y2 - y1

ax1.plot(x, y1, label = 'Proportion of all 6 hour arrivals')
ax1.plot(x, y2, label = 'Proportion of all thrombolysis given')
ax1.plot(x, y3, label = 'Difference')

ax1.set_xlabel('NIHSS cut-off (minimum NIHSS included)')
ax1.set_ylabel('Proportion of patients\n(of those arriving in 6 hours of known onset')

ax1.set_ylim(0, 1.02)
ax1.set_xlim(0, 15)
ax1.set_xticks(np.arange(0, 16))
ax1.set_yticks(np.arange(0, 1.01, 0.1))
ax1.legend()
ax1.grid()
ax1.set_title('Relationship between NIHSS and thrombolysis use')

# LAO performance
ax2 = fig.add_subplot(122)
x = [4,5,6,10,11,14]
sensitivity = [93, 90, 87, 76, 73, 61]
specificity = [46, 54, 60, 80, 83, 88]
ax2.plot(x, sensitivity, label = 'Sensitivity')
ax2.plot(x, specificity, label = 'Specificity')

ax2.set_xlabel('NIHSS cut-off (minimum NIHSS included)')
ax2.set_ylabel('Sensitivity and specificity for LAO')
ax2.set_xticks(np.arange(0, 16))
ax2.set_yticks(np.arange(0, 101, 10))
ax2.legend(loc='lower right')
ax2.grid()
ax2.set_title('Sensitivity and specificity for\nlarge artery occlusion (LAO)')

plt.tight_layout(pad=2)

plt.savefig('output/nihss_cutoff.jpg', dpi=300)

plt.show()