A comparison of the Linear Array and the Automated INNO-LiPA HPV Genotyping Systems on Pathological Specimens
Article Information
Beatrix Bicskei, PhD1, Pia Moltu, BSc1, Melinda Lillesand, MSc1, Emma Rewcastle, MSc1, Irene Kraus Christiansen, PhD2, Einar G Gudlaugsson, MD, PhD1, Jannicke M Berland, MD1, Emiel AM Janssen, PhD1,3, Irene T Øvestad, PhD1*
1Department of Pathology, Stavanger University Hospital, Stavanger, Norway
2Department of Microbiology and Infection Control, Akershus University Hospital, Lørenskog, Norway
3Department of Mathematics and Natural Science, University of Stavanger, Stavanger, Norway
*Corresponding Author: Dr. Irene T Øvestad, Department of Pathology, Stavanger University Hospital, Stavanger, Norway
Received: 13 July 2020; Accepted: 03 August 2020; Published: 29 August 2020
Citation: Beatrix Bicskei, Pia Moltu, Melinda Lillesand, Emma Rewcastle, Irene Kraus Christiansen, Einar G Gudlaugsson, Jannicke M Berland, Emiel AM Janssen, Irene T Øvestad. A comparison of the Linear Array and the Automated INNO-LiPA HPV Genotyping Systems on Pathological Specimens. Journal of Cancer Science and Clinical Therapeutics 4 (2020): 349-364.
View / Download Pdf Share at FacebookAbstract
The causative role of HPV infection in a wide range of diseases is well established, highlighting the importance of accurate HPV genotyping techniques applicable to various tissue types and conservation methods. This study compared the performance of the automated INNO-LiPA HPV Genotyping Extra II and the manual Linear Array HPV genotyping assays in diagnostically relevant specimens. Samples with discrepant results were also tested with Anyplex II HPV28 detection assay. DNA from 120 samples, primarily from anogenital and head-neck FFPE tissues and cervical cytology specimens, were analysed. Interclass correlation efficiencies between the assays and percentage and kappa agreements for individual genotypes were calculated. Overall, a high agreement was found between the two genotyping methods (>0.95), however, INNO-LiPA was more likely to identify genotypes in samples indicating low viral load and/or originating from FFPE tissues. Specifically, INNO-LiPA detected more genotypes in FFPE material harbouring multiple HPV infections; in particular HPV51 and 52 were the most affected high-risk genotypes. In contrast, Linear Array identified more low-risk HPV54 infections in both cytology and FFPE specimens. The discrepant genotypes detected by Anyplex resembled more the results of Linear Array than that of INNO-LiPA. We conclude that the assays are highly comparable, but differences may arise in a context dependent manner.
Keywords
HPV genotyping; Linear Array; INNO-LiPA; Head and neck squamous cell carcinoma diagnostics and cervical cancer screening
HPV genotyping articles, Linear Array articles, INNO-LiPA articles, Head and neck squamous cell carcinoma diagnostics and cervical cancer screening articles
HPV genotyping articles HPV genotyping Research articles HPV genotyping review articles HPV genotyping PubMed articles HPV genotyping PubMed Central articles HPV genotyping 2023 articles HPV genotyping 2024 articles HPV genotyping Scopus articles HPV genotyping impact factor journals HPV genotyping Scopus journals HPV genotyping PubMed journals HPV genotyping medical journals HPV genotyping free journals HPV genotyping best journals HPV genotyping top journals HPV genotyping free medical journals HPV genotyping famous journals HPV genotyping Google Scholar indexed journals Linear Array articles Linear Array Research articles Linear Array review articles Linear Array PubMed articles Linear Array PubMed Central articles Linear Array 2023 articles Linear Array 2024 articles Linear Array Scopus articles Linear Array impact factor journals Linear Array Scopus journals Linear Array PubMed journals Linear Array medical journals Linear Array free journals Linear Array best journals Linear Array top journals Linear Array free medical journals Linear Array famous journals Linear Array Google Scholar indexed journals INNO-LiPA articles INNO-LiPA Research articles INNO-LiPA review articles INNO-LiPA PubMed articles INNO-LiPA PubMed Central articles INNO-LiPA 2023 articles INNO-LiPA 2024 articles INNO-LiPA Scopus articles INNO-LiPA impact factor journals INNO-LiPA Scopus journals INNO-LiPA PubMed journals INNO-LiPA medical journals INNO-LiPA free journals INNO-LiPA best journals INNO-LiPA top journals INNO-LiPA free medical journals INNO-LiPA famous journals INNO-LiPA Google Scholar indexed journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening Research articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening review articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening PubMed articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening PubMed Central articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening 2023 articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening 2024 articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening Scopus articles Head and neck squamous cell carcinoma diagnostics and cervical cancer screening impact factor journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening Scopus journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening PubMed journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening medical journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening free journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening best journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening top journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening free medical journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening famous journals Head and neck squamous cell carcinoma diagnostics and cervical cancer screening Google Scholar indexed journals Human papillomaviruses articles Human papillomaviruses Research articles Human papillomaviruses review articles Human papillomaviruses PubMed articles Human papillomaviruses PubMed Central articles Human papillomaviruses 2023 articles Human papillomaviruses 2024 articles Human papillomaviruses Scopus articles Human papillomaviruses impact factor journals Human papillomaviruses Scopus journals Human papillomaviruses PubMed journals Human papillomaviruses medical journals Human papillomaviruses free journals Human papillomaviruses best journals Human papillomaviruses top journals Human papillomaviruses free medical journals Human papillomaviruses famous journals Human papillomaviruses Google Scholar indexed journals Papillomaviridae family articles Papillomaviridae family Research articles Papillomaviridae family review articles Papillomaviridae family PubMed articles Papillomaviridae family PubMed Central articles Papillomaviridae family 2023 articles Papillomaviridae family 2024 articles Papillomaviridae family Scopus articles Papillomaviridae family impact factor journals Papillomaviridae family Scopus journals Papillomaviridae family PubMed journals Papillomaviridae family medical journals Papillomaviridae family free journals Papillomaviridae family best journals Papillomaviridae family top journals Papillomaviridae family free medical journals Papillomaviridae family famous journals Papillomaviridae family Google Scholar indexed journals epithelial cells articles epithelial cells Research articles epithelial cells review articles epithelial cells PubMed articles epithelial cells PubMed Central articles epithelial cells 2023 articles epithelial cells 2024 articles epithelial cells Scopus articles epithelial cells impact factor journals epithelial cells Scopus journals epithelial cells PubMed journals epithelial cells medical journals epithelial cells free journals epithelial cells best journals epithelial cells top journals epithelial cells free medical journals epithelial cells famous journals epithelial cells Google Scholar indexed journals oncogenic potential articles oncogenic potential Research articles oncogenic potential review articles oncogenic potential PubMed articles oncogenic potential PubMed Central articles oncogenic potential 2023 articles oncogenic potential 2024 articles oncogenic potential Scopus articles oncogenic potential impact factor journals oncogenic potential Scopus journals oncogenic potential PubMed journals oncogenic potential medical journals oncogenic potential free journals oncogenic potential best journals oncogenic potential top journals oncogenic potential free medical journals oncogenic potential famous journals oncogenic potential Google Scholar indexed journals anogenital cancers articles anogenital cancers Research articles anogenital cancers review articles anogenital cancers PubMed articles anogenital cancers PubMed Central articles anogenital cancers 2023 articles anogenital cancers 2024 articles anogenital cancers Scopus articles anogenital cancers impact factor journals anogenital cancers Scopus journals anogenital cancers PubMed journals anogenital cancers medical journals anogenital cancers free journals anogenital cancers best journals anogenital cancers top journals anogenital cancers free medical journals anogenital cancers famous journals anogenital cancers Google Scholar indexed journals colorectal cancer articles colorectal cancer Research articles colorectal cancer review articles colorectal cancer PubMed articles colorectal cancer PubMed Central articles colorectal cancer 2023 articles colorectal cancer 2024 articles colorectal cancer Scopus articles colorectal cancer impact factor journals colorectal cancer Scopus journals colorectal cancer PubMed journals colorectal cancer medical journals colorectal cancer free journals colorectal cancer best journals colorectal cancer top journals colorectal cancer free medical journals colorectal cancer famous journals colorectal cancer Google Scholar indexed journals
Article Details
1. Introduction
Human papillomaviruses (HPVs) form a large and diverse group of small, non-enveloped, circular, double-stranded DNA viruses belonging to the Papillomaviridae family. Currently, there are 202 HPV types in 49 species and five genera that are recognised by the International Human Papillomavirus Reference Center, with more being identified [1]. HPVs infect basal epithelial cells in the epidermis and mucosae and depending on their oncogenic potential can cause a wide range of diseases from benign lesions to invasive tumours. Specifically, infection with high-risk (HR) HPVs is the major cause of cervical cancer [2], and also a causative factor in other anogenital cancers such as anal, vulvar, penile, vaginal, as well as head and neck cancers, [3] while HPV`s contribution to colorectal cancer is more debated [4-6]. Systematic reviews estimated the prevalence of HR HPV infections to be 100% in cervical, 88% in anal, 70-78% in vaginal, 50-51% in penile, 25-43% in vulvar, 26-31% in oropharyngeal, 4% in oral and 5% in laryngeal cancers [7, 8].
The high prevalence of HPV infections in cancer cases highlights the importance of HPV screening and its increasing value to pathology as a molecular diagnostic technique. The link between cervical cancer and HPV has been long established and its role in other anogenital and head and neck cancers is also recognised. As such, HPV screening forms the foundation of HPV primary screening of cytology specimens for cervical cancer prevention. However, clinically validated techniques suitable for HPV detection in formalin fixed paraffin embedded (FFPE) tissues available for biopsy specimens are also required. Validation of assays has become particular important as direct HPV testing is recommended in the revised WHO/IARC classification (2017) [9] to assess a correct diagnosis of the Head and Neck Squamous Cell Carcinoma (HNSCC), specifically HPV positive Oropharyngeal Squamous Cell Carcinoma (OPSCC), who have a better prognosis with a distinct epidemiological profile. HPV genotyping is appropriate to determine whether a lesion or a carcinoma is HPV associated. For example, the prognoses can be stratified based on molecular profiling of HPV positive HNSCC [10] . It is noteworthy, that recent studies have investigated the effectiveness of combined HPV genotyping and p16 immunohistochemistry (a surrogate marker for an active HPV infection) in HNSCC [11, 12]. Differences in survival between HPV positive OPSCC patients compared to HPV negative, imply that combination testing should be taken into consideration regarding risk stratification and future treatment regimens of this patient group [13]. While the results indicate higher sensitivity for combination testing, our HNSCC samples were retrieved prior to this knowledge being implemented in hospital protocols. The assigned clinician or pathologist may also request HPV genotyping in cases where the presence of a particular HPV genotype is relevant to decide the origin of a detected metastasis.
A recent study also reported the use of fine needle aspirate from metastatic lymph nodes, for direct HPV testing to confirm primary tumour localisation in the oropharynx [14]. On rare occasions, the result of routine HPV testing of cervical cytology specimens do not explain the cell morphology of the specimen and additional HPV genotyping is requested for quality assurance purposes. It is noteworthy that although studies of the clinical sensitivity of some HPV genotyping methods exist for LBC specimens [15-19], this type of research is largely lacking for FFPE material. One could argue that high clinical sensitivity is more important for methods employed as screening techniques, such as the cervical cancer screening of LBC specimens, whereas high analytical sensitivity is a priority in contexts where the presence or absence of a particular HPV genotype is a contributing factor of the follow up procedure. The better survival of these patients and a better response to chemotherapy and radiation treatment, will most likely change future therapeutic regimen for these patients and recent meta-analysis highlights the need for using the most sensitive analysis with combined HPV/p16 testing [9, 11, 13, 20]. A number of molecular tools are available for the detection of HPV, such as nucleic acid hybridization assays, signal amplification assays and nucleic acid amplification assays. Nucleic acid amplification assays are relatively quick to perform, highly sensitive, require small amount of DNA, allow multiplexing, yet can provide information regarding individual genotypes [21]. Two commercially available PCR-based HPV genotyping tests are the Linear Array (LA) (Roche Molecular Systems, Branchburg, NJ, USA) and the INNO-LiPA HPV Genotyping Extra II (IL) (Fujirebio Europe N.V., Gent, Belgium) detection systems. They are both able to detect the same 13 HR and 6 probable HR (pHR) HPV genotypes. In addition, LA and IL have the ability to identify a further 18 and 13 LR (low-risk)/not yet classified genotypes, respectively. Although IL lacks more LR/not yet classified risk HPV probes, than LA, it unambiguously assigns all the HPV genotypes it features to a single probe. This is not the case of the LA, where HR HPV 52 is only represented by a cross-reactive probe shared with three other HPV genotypes. Both assays target the same region of the HPV genome, but differ in the length of the DNA fragment they require for identification. Although LA has been developed for cytology specimens, it has been repeatedly demonstrated to be suitable for genotyping FFPE material [22-26]. Thus, the aim of the current study was to compare the performance of the automated INNO-LiPA Extra II assay to the well documented and widely used Linear Array assay in a routine setting and in a diverse, diagnostically relevant set of samples. Inconsistent results were subjected to a third HPV genotyping method; the semi quantitative Anyplex II HPV28 Detection assay (Seegene, Seoul, Korea) hereafter AP.
2. Methods
2.1 Clinical material
The study cohort consisted of a total of 120 samples originating from various tissues. Ninety five specimens (79%) were FFPE and 25 (21%) were conserved in PreservCyt Solution (Hologic, Marlborough, MA, USA). Of the FFPE tissues, 44 originated from the cervix, 40 Cervical Intraepithelial Neoplasia (CIN) grade 3, two CIN2 and two invasive cervical cancers, 22 from other anogenital areas, 25 Head and neck squamous cell carcinoma (HNSCC) of which 20/25 (80%) from oropharyngeal sites (OPSCC) and 5/25 (20%) from other head and neck areas, three were lung tissue and one was a lymph node biopsy from the lung area. The lung and lymph node biopsies are termed as “other” tissue. Fifteen of the samples that were preserved in PreservCyt Solution were liquid based cytology (LBC) samples from the Norwegian cervical cancer screening program, 5 normal cases, 5 low-grade intraepithelial lesions (LSIL), 2 atypical squamous cells of unspecific significance (ASC-US) and three atypical squamous cells of unspecific significance suspect for high-grade (ASC-H). Ten were PreservCyt solution specimens containing cell preparations from Hela, Caski, CC10b and Siha cultures provided by the Quality Control for Molecular Diagnostics (QCMD) (www.qcmd.org) in 2016. All LBC, and the majority of FFPE specimens had undergone HPV genotyping according to routine practise at the Stavanger University Hospital. All cervical biopsies were collected during routine practice, however; the majority of these were genotyped for research purposes. Approximately half of the samples were first genotyped on the LA assay before being selected for the study and genotyped a second time by IL and vice versa for the second half of the samples. Since the aim of the study was to calculate agreement between the assays, in terms of relative analytical sensitivity, a selection was performed to maximise the number of HPV genotypes included in the analysis, while keeping the number of negative and invalid experimental samples to a minimum. As per best practise, negative control samples were included in every run to avoid sample-to-sample cross-contamination and consequent false-positive results. The study was approved by the Regional Committees for Medical and Health Research Ethics (REC) - West Norway (2017/1515/REK vest). Informed opt-out consent was obtained from all participants and all methods were carried out in accordance with approved guidelines.
2.2 DNA extraction
DNA was isolated from FFPE samples using the E.Z.N.A® Tissue DNA Kit (Omega Bio-tek Inc., Norcross, USA) following the manufacturer’s instructions; Proteinase K digestion proceeded overnight and deparaffinization was performed using Deparaffinization Solution (QIAGEN, Hilden, Germany). LBC samples were processed on the automated Cobas 4800 System (Roche Molecular Systems, Branchburg, NJ, USA) following manufacturer’s instructions.
2.3 HPV genotyping
The same LBC and FFPE sample extracts were tested with both Linear Array HPV Genotyping Test and using the INNO-LiPA Extra II kit on the TENDIGO machine; although for the LA method DNA was normalized to 5-8 ng/µl concentration. Both assays are IVD approved and were performed according to manufacturer’s protocol. Briefly, DNA is amplified in a multiplex PCR using pooled biotinylated HPV and human DNA control primers. The amplicons are chemically denatured and the sample is hybridised to strips coated with HPV probes representative of the genotypes detected by each method in addition to DNA control probe lines. Following washing steps, the biotin-labelled amplicons that have hybridised to the complementary oligonucleotide probes on the strip are visualized in the presence of conjugate and substrate solutions. Hybridization and subsequent washing and colour development steps were fully automated in the IL protocol. In contrast, for the LA assay, reagents were added and removed manually using an automated pipette and vacuum aspirator. Two independent readers interpreted the results; any discrepancies were referred to a third reader.
The L1 region of the HPV genome is used for the formal classification of HPV and both genotyping methods target this region. LA uses the PGMY09/11 primers allowing the detection of 37 HPV genotypes (6, 11, 16, 18, 26, 31, 33, 35, 39, 40, 42, 45, 51, 52, 53, 54, 55, 56, 58, 59, 61, 62, 64, 66, 67, 68, 69, 70, 71, 72, 73, 81, 82, 83, 84, 89 and IS39 (subtype of 82)), including 13 HR types on approximately 450 base pair (bp) amplicons, whereas the amplicon for the β-globin gene, used to control for the DNA quality, is 268 bp. In contrast, the IL employs the SPF10 primer set, amplifying 32 HPV genotypes (16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 68, 26, 53, 66, 70, 73, 82, 6, 11, 40, 42, 43, 44, 54, 61, 62, 67, 81, 83, 89) , including 13 HR types on 65 bp fragments. Amplification of the 80bp HLA-DPB1 gene is used as DNA control. Samples that showed discrepancies between the IL and LA assays were subject to a third genotyping method. DNA specimens were sent to the Norwegian national HPV reference laboratory at Akershus University Hospital, where genotyping was performed using the AP assay. AP detects and genotypes 28 HPV types; 14 high-risk and 14 low-risk types, in two reactions, A (HPV 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, 68) and B (HPV 6, 11, 26, 40, 42, 43, 44, 53, 54, 61, 69, 70, 73, 82), respectively. According to manufacturer’s instructions, 5μl DNA was added to both A and B mixtures and reactions were analysed with the CFX96 real-time detection system (Bio-Rad Laboratories). Positive samples were semi-quantitative, indicated by results at 30 (+++) cycles, 40 (++) cycles, or 50 (+) cycles corresponding to high, medium and low viral load respectively. The amplicons of this assay range between 100-200bp [27].
2.4 Statistical analyses
Only genotypes present on both assays were included in the analysis and obvious cross-hybridisation bands as reported by manufacturers were ignored. As such, since HPV55 and 84 are not present on IL they were excluded from the comparison. HPV52 is known to cross-hybridize to probe 31 on the IL assay, thus any faint band on position 31 was ignored where HPV52 infection was identified. Discordant negative samples were analysed under the non-negative genotype. Definitions of the categories used in the study are detailed in Table 1.
Samples were grouped three ways for data analysis purposes. First, a global analysis was performed combining all genotypes studied. Next, a global analysis was applied to probable HR or HR and HR genotypes only. Finally, genotypes that were inconsistently identified by the two assays were further analysed on an individual basis. To compare the two HPV genotyping methods, inter-rater reliability (IRR) was assessed as a measure of agreement [28]. For the global analyses intraclass correlation coefficient (ICC) (two-way random effects, absolute agreement, single score), F-test and confidence intervals were calculated, whereas for the genotype-wise comparison percentage agreement and unweighted Cohen’s kappa were computed. All calculations were performed in the R software version 3.4.1 [29] using the agree, kappa2 and icc functions from the Irr package version 0.84 [30]. κ values were interpreted as follows [31]. Poor if κ < 0.00; Slight if 0.00 ≤ κ ≤ 0.20; Fair if 0.21 ≤ κ ≤ 0.40; Moderate if 0.41 ≤ κ ≤ 0.60; Substantial if 0.61 ≤ κ ≤ 0.80; Almost perfect if κ > 0.80.
Category |
Sub-type |
Definition |
Results |
Concordant |
Absolute agreement between assays |
Compatible IL+ |
Common genotype(s) and additional genotype(s) on IL |
|
Compatible LA+ |
Common genotype(s) and additional genotype(s) on LA |
|
Discordant |
No similarity between assays |
|
Invalid |
Negative DNA control |
|
Specimen type |
All PreservCyt |
LBC or cultured cell preparations |
LBC |
PreservCyt material from the cervix |
|
Cultured cells |
Human cell line with/without integrated HPV DNA |
|
All FFPE |
FFPE material from various tissue biopsies |
|
Cervical biopsy |
FFPE material from the cervix |
|
Other anogenital |
Various non-cervical FFPE material of the anogenital area |
|
Head and neck |
FFPE material of the head and neck area |
|
Other |
Uncategorized FFPE material from the lung area |
|
HPV genotype |
Multiple |
More than one HPV genotype was detected by at least one assay |
Single |
No more than one HPV genotype was detected by both assays |
|
None |
No HPV genotypes was detected by either of the assays |
Table 1: Definitions of the categories applied to the study samples.
3. Results
3.1 Overview of study samples
A total of 120 samples were genotyped on both assays. Three FFPE samples were excluded due to invalidity on LA. Of the remaining 117 samples, 114 had at least one genotype that was present on both assays and these were included in the data analysis. Please note that in order to reduce resources, the number of invalid and negative experimental samples were deliberately kept at a minimum and thus their numbers are not representative of a random population of samples. A summary of the composition of the 114 samples are presented in Table 2. Overall, 95% of the samples showed concordance or compatibility between assays. Although concordance was relatively low (50%) for multiple infections, discordance remained low (6%). Moreover, IL identified nearly four times as many HPV genotypes when multiple infections were present, than LA did. All of these samples were from FFPE tissues and/or showed a faint HPV type specific band.
There are 30 possible genotyping outcomes (29 HPV genotypes and no infection) that are shared between the two assays. Apart from two HR (HPV35 and 56) and two LR/unknown-risk (HPV 81 and 83) genotypes, all other common genotypes are represented in the analysis. For details see Table 3.
Category |
Sub-type |
Concordant |
Comp. LA+* |
Comp. IL+* |
Discordant |
Total |
Con/m** |
Total |
|
94 (82 %) |
3 (3 %) |
11 (10 %) |
6 (5 %) |
114 |
95 % |
All PreservCyt |
20 (80 %) |
2 (8 %) |
1 (4 %) |
2 (8 %) |
25 |
92 % |
|
LBC |
12 (80 %) |
2 (13 %) |
0 (0 %) |
1 (7 %) |
15 |
93 % |
|
Cultured cells |
8 (80 %) |
0 (0 %) |
1 (10 %) |
1 (10 %) |
10 |
90 % |
|
All FFPE |
|
74 (83 %) |
1 (1 %) |
10 (11 %) |
4 (4 %) |
89 |
96 % |
Cervical biopsy |
33 (79 %) |
1 (2 %) |
7 (17 %) |
1 (2 %) |
42 |
98 % |
|
Other anogenital |
16 (76 %) |
0 (0 %) |
3 (14 %) |
2 (10 %) |
21 |
90 % |
|
Head and neck |
21 (95 %) |
0 (0 %) |
0 (0 %) |
1 (5 %) |
22 |
95 % |
|
Other |
4 (100 %) |
0 (0 %) |
0 (0 %) |
0 (0 %) |
4 |
100 % |
|
HPV genotype |
Multiple |
16 (50 %) |
3 (9 %) |
11 (34 %) |
2 (6 %) |
32 |
94 % |
Single |
69 (95 %) |
0 (0 %) |
0 (0 %) |
4 (5 %) |
73 |
95 % |
|
None |
9 (100 %) |
0 (0 %) |
0 (0 %) |
0 (0 %) |
9 |
100 % |
*comparable with LA/IL identifying additional genotypes, **concordant or comparable.
Table 2: Overview of the composition of the study samples and the type of agreement between IL and LA.
Table 3: The number of samples in each of the possible genotyping outcomes as identified by IL and LA.
3.2 Overall performance of the assays
For all 30 possible genotyping outcomes, as well as for HR and pHR genotypes and HR genotypes only (Table 4), intraclass correlation coefficients (ICC) showed a high degree of agreement (>0.95) between the two assays.
Genotypes |
Total |
ICC |
p-value |
All* |
30 |
0.96 |
0.00000 |
pHR and HR** |
19 |
0.97 |
0.00000 |
HR only*** |
13 |
0.97 |
0.00000 |
*All HPV genotypes, **probable high-risk and high-risk genotypes, and ***high-risk genotypes only.
Table 4: Interclass correlation coefficients of HPV genotypes compared.
3.3 Assay-wise comparison of individual genotypes
The two assays showed inconsistencies in 15 out of 29 genotypes. The percentage and kappa agreements for these genotypes are presented in Table 5. Percentage agreements for all HPV genotypes were >95%, apart from HPV 52 that was 94%.This was due to IL identifying nearly twice as many HPV 52 infections as LA (15 vs 8). Apart from HPV 51 and 54, that showed fair and moderate agreement respectively, all genotypes had substantial to almost perfect kappa agreement between assays. All inconsistently identified HPV 51 and 54 genotypes were detected in samples with multiple HPV infections. IL identified more HPV 51, whereas LA detected more HPV 54 genotypes. All of the inconsistencies found among the samples harbouring HPV 51, 52 and 54 infections also had additional HPV genotypes identified by both methods (Table 6).
Out of the 114 samples compared, 20 showed inconsistent results between the IL and LA assays. Two samples were inter-laboratory control samples supplied by QCMD (QCMD2 and 9) with known genotype information, two samples did not have sufficient material left for further analysis, thus the remaining 16 samples were genotyped by a third method, the AP assay. Inconsistencies were more common among HPVs that were present in low copy number (Table 6). In fact, compared to the results of AP, high and medium copy number HPVs were only missed by either of the compared assays on two and four occasions respectively, all involving samples harbouring multiple infections. Three samples that were HPV negative according to LA, but not by IL were also found to be negative on AP. However, a single low viral load sample provided by QCMD was correctly identified as HPV16 positive on IL, but was found negative on LA. Superior performance of IL on samples harbouring HPV51 and/or 52 infections was not supported by AP; however LA missed a HPV52 genotype on an educational sample provided by QCMD. Two HPV31 genotypes detected on IL were excluded from the analysis based on possible non-specific reactivity from HPV52. However, these two HPV31 infections were also found by AP, whereas the corresponding HPV52 infections were not detected.
Genotype |
Risk type |
IL+ |
LA+ |
% agreement |
kappa |
k agreement |
Z |
p-value |
HPV 6 |
LR |
16 |
14 |
98 % |
0.92 |
Almost perfect |
9.89 |
0.00000 |
HPV 11 |
LR |
8 |
6 |
98 % |
0.85 |
Almost perfect |
9.16 |
0.00000 |
HPV 16 |
HR |
39 |
37 |
98 % |
0.96 |
Almost perfect |
10.30 |
0.00000 |
HPV 33 |
HR |
9 |
7 |
98 % |
0.87 |
Almost perfect |
9.33 |
0.00000 |
HPV 42 |
LR |
2 |
3 |
99 % |
0.80 |
Substantial |
8.68 |
0.00000 |
HPV 45 |
HR |
6 |
5 |
99 % |
0.91 |
Almost perfect |
9.70 |
0.00000 |
HPV 51 |
HR |
6 |
1 |
96 % |
0.28 |
Fair |
4.26 |
0.00002 |
HPV 52 |
HR |
15 |
8 |
94 % |
0.67 |
Substantial |
7.54 |
0.00000 |
HPV 54 |
LR |
1 |
3 |
98 % |
0.49 |
Moderate |
6.11 |
0.00000 |
HPV 58 |
HR |
4 |
3 |
99 % |
0.85 |
Almost perfect |
9.20 |
0.00000 |
HPV 59 |
HR |
1 |
2 |
99 % |
0.66 |
Substantial |
7.52 |
0.00000 |
HPV 62 |
Unknown |
1 |
2 |
99 % |
0.66 |
Substantial |
7.52 |
0.00000 |
HPV 68 |
HR |
2 |
1 |
99 % |
0.66 |
Substantial |
7.52 |
0.00000 |
HPV 73 |
pHR |
3 |
4 |
99 % |
0.85 |
Almost perfect |
9.20 |
0.00000 |
HPV 82 |
pHR |
4 |
2 |
98 % |
0.66 |
Substantial |
7.48 |
0.00000 |
Table 5: Percentage and kappa agreements of HPV genotypes that were inconsistent between IL and LA.
Identifier |
IL |
LA |
AP/QCMD |
Viral load |
Sample type |
Preservation |
1 |
6, 11 |
6 |
6 |
High |
Other anogenital |
FFPE |
2 |
6, 11, 42, 51 |
42 |
11, 42 |
Low, High |
Other anogenital |
FFPE |
3 |
(31), 45, 52 |
45 |
31, 45 |
Low, Medium |
Cervical biopsy |
FFPE |
4 |
45, 82 |
45 |
45, 82 |
Low, Low |
Cervical biopsy |
FFPE |
5 |
16, 51 |
16 |
16 |
Low |
Cervical biopsy |
FFPE |
6 |
33, 51 |
33 |
33 |
Medium |
Cervical biopsy |
FFPE |
7 |
45, 52, 68, 82 |
59 |
NA |
NA |
Cervical biopsy |
FFPE |
8 |
16 |
16, 54 |
16, 54 |
Medium, Low |
Cervical biopsy |
FFPE |
9 |
16, 52 |
16 |
16, 42 |
Medium, Medium |
Cervical biopsy |
FFPE |
10 |
(31), 33, 45, 51, 52 |
45 |
31, 33 |
Low, Low |
Cervical biopsy |
FFPE |
11 |
73, 51, 52 |
73 |
NA |
NA |
Cervical biopsy |
FFPE |
12 |
70, 82 |
55, 70, 73, 82 |
70, 73, 82 |
High, High, High |
LBC |
PreservCyt |
13 |
16 |
None |
None |
NA |
Head and neck |
FFPE |
14 |
6 |
None |
None |
NA |
Other anogenital |
FFPE |
15 |
33 |
None |
None |
NA |
Other anogenital |
FFPE |
16 |
40 |
40, 62 |
40, 54 |
High, Low |
LBC |
PreservCyt |
17 |
44, 52 |
42, 54 |
42, 44, 54 |
High, Medium, Medium |
LBC |
PreservCyt |
18 |
6, 58 |
6 |
6, 58 |
Medium, Medium |
Other anogenital |
FFPE |
QCMD2 |
16 |
None |
16 |
Low |
Cultured cells |
PreservCyt |
QCMD9 |
51, 52 |
51 |
51, 52 |
NA |
Cultured cells |
PreservCyt |
Table 6: All HPVs detected by either of the three genotyping methods or provided by QCMD. Note, that HPV 44, 55 and 62 are not detectable by LA, IL and AP, respectively. Moreover, faint HPV31 bands were not scored in the presence of a HPV52 infection, as indicated by brackets.
4. Discussion
Although comparisons between the manual Inno-Lipa HPV Genotyping Extra (IL Extra) and LA assays exist, this is a report of the automated version of the Inno-Lipa HPV Genotyping Extra II assay (IL Extra II), evaluated in diagnostically relevant pathological specimens. The latter version of the assay was launched in 2015 and according to the manufacturer; the sensitivity to detect HPV 59 and 68 is improved and the IL Extra II does not use shared probes as the former IL Extra. A comparison of LA and IL Extra II in cervical cytology specimens has been reported [32], but not for DNA from FFPE tissue. WHO`s new classification of OPSCC is based on HPV testing [9] and entail the need for evaluation of available assays to perform HPV genotyping on FFPE, as well as liquid based material. Using reliable assays could reduce considerable time and effort to obtain the correct diagnosis and initiate treatment of the patient.
Based on the findings of our study, we conclude a good overall agreement between the LA and the automated IL assays and a perfect or almost perfect agreement for the five HPV genotypes most frequently detected in cervical cancer (HPV 16, 18, 31, 33 and 45) [33] and also highly prevalent in other anogenital [34, 35], as well as head and neck cancers [36, 37]. An important difference between the assays was that in contrast to LA, all samples genotyped by IL returned a valid result. Due to the primer design, IL is more sensitive than LA, especially for testing degraded DNA from FFPE specimens [24]. Firstly, false negative results may occur in FFPE tissues tested on LA, due to the smaller amplicon size of the DNA quality control gene, compared to the HPV genotype specific fragment; 268 versus 450 bp, respectively. Secondly, IL is most probably less sensitive to DNA degradation as it detects a nearly seven times smaller HPV DNA fragment (65 bp) than LA does (450 bp). For the same reason, LA has been shown to be more sensitive to the choice of DNA extraction method (38) and was developed by the manufacturer for cytology specimens only. Indeed, the only category where LA identified marginally more genotypes than IL did was on cytology samples extracted on the Cobas 4800 platform, resulting in 93% concordance for LBC samples. This is in line with a recent study showing overall good concordance (93%) between IL Extra II and LA to detect high-risk HPV types in cervical cytology specimens [32], while substantial differences was found between the assays when comparing individual genotypes. In addition to the literature, false negative results of LA are also, to some extent, supported by the QCMD scheme, but not by results of the AP assay. Specifically, a QCMD sample with low HPV16 viral load was only correctly identified by IL but was found negative on LA. In contrast, three FFPE samples that were found to be positive by IL, but not by LA, were also negative according to AP. Both LA and AP methods have been developed for cervical swabs and LBC specimens and require a longer fragment size than IL, potentially affecting the genotypes retrieved from FFPE. Although, under optimal archival and amplification conditions, up to 250bp amplicons can be successfully amplified from FFPE blocks [39], exceeding the minimum requirements for both IL and AP. The differences between the results of AP and IL could have arisen due to detection level differences between the assays. This is supported by a recent study on FFPE samples from HNSSC where IL detected HPV DNA in 10% of samples found negative by AP and discrepancies were associated with low viral load [40].
Concordance between the assays was particularly high for single infections, but considerably lower when multiple infections were present. Subsequently, inter assay agreement was higher for head and neck tissues typically presenting a single infection, and lower for anogenital specimens that tend to harbour multiple HPVs. In contrast, discordance remained comparably low for both multiple and single infections. This is because the discrepancies observed in samples infected with multiple HPVs stemmed from the assays identifying additional genotypes probably due to primer competition and/or as a result of differences in detection levels of individual genotypes. It has been suggested that multiple infections in FFPE samples are particularly difficult to detect due to preferential amplification of the HPV genotype that is present in a molar excess and thus less affected by DNA degradation [24]. The LA and IL assays did not contribute equally to the detection of additional genotypes. Although not fully supported by AP, IL identified nearly four times as many genotypes as LA did. Since our data was derived largely from FFPE material, this could be a result of both the smaller DNA fragment size requirement of IL and differences regarding virus detection level for the successful amplification of HPV genotypes. Higher sensitivity of IL compared to LA in detecting HPV genotypes in FFPE has also been reported for vulvar and cervical tissue [24, 41]. Although on cultured cell preparations provided by QCMD, also conserved in PreservCyt solution and extracted on the Cobas platform, IL showed a better sensitivity identifying genotypes that were present in low copy number in single as well as in multiple infection samples.
Regarding correlation of individual genotypes, the numbers of some rare genotypes are low and the correlation may not be exact from a scientific view, but interesting to discuss from a technical aspect. HPV51, 52 and 54 showed the lowest agreement between assays (Table 5). Samples causing the discrepancies were largely FFPE material and all harboured multiple infections. HPV52 was detected in nearly twice as many samples by IL, than by LA. HPV52 is represented by a unique probe on IL, but it features a shared probe design on LA with no reported inclusivity level. Despite its shared probe design, LA has demonstrated high specificity for detecting HPV52 in LBC specimens [42, 43], however this may not be the case for FFPE material. The performance of the LA has been evaluated on paired cytology and FFPE tissues [24] and although a good overall concordance (Cohen’s k = 0.85, SE = 0.082, p = 0.000) was found, HPV52 was detected in three cytology samples vs. a single FFPE tissue. While this discrepancy can also arise from differences in the sampling time and area, it can also result from differences in DNA preservation methods. If the interaction between probe design and method of preservation leads to decreased sensitivity of the LA then it is plausible that IL with its smaller HPV amplicon size and single probe design is more successful in amplifying HPV52 in FFPE tissue. Unlike in the case of HPV52, the underperformance of LA in detecting HPV51 cannot be explained by presuming a low inclusivity level, since it is below that of HPV16, suggesting a very high sensitivity (according to product insert). The ability of some L1 consensus primer designs to accurately identify multiple HPV genotypes has previously been questioned [44]. In particular, the authors report a decreased sensitivity of the MY09/11 design employed by LA to detect HPV51 and 52. Similarly, studies conducted on cervical FFPE tissue specimens [45] and on cervical cytology samples [32, 46] also found lower sensitivity of LA compared to IL to detect HPV51 and 52.
Despite the support from the literature and to some extent from the QCMD control samples, the detection patterns of HPV51 and 52 by AP of the current study resembled the results of LA. It is noteworthy, that although we chose to ignore HPV31 genotypes where HPV52 was present on the IL assay as possible cross hybridisation due to sequence homologies is indicated by the manufacturer (product insert), we speculate that this might be a two way process. The manufacturer states that one of the limitations of the IL procedure is that “If HPV52 is present, a weak non-specific reactivity on probe-line HPV31 might be observed due to sequence homologies”. In addition to this, we observed on two occasions where although both HPV31 and 52 were present on the IL assay, only HPV31 was detected by AP. This study does not have sufficient data to determine whether this was indeed caused by HPV31 hybridizing to HPV52 probes. For identification of individual genotypes in cervical cytology specimens, the study by Xu et al report that IL Extra II identified 12 times more HPV52 and marginally more HPV 54 as compared to LA, while LA amplified nearly twice as many low-risk HPV42 [32]. On the other hand increased sensitivity of LA in detecting HPV54 was confirmed by AP at both low and medium viral DNA copy numbers, indicating a lower detection level for this genotype of IL compared to both LA and AP. Although, HPV54 and 42 are low-risk genotypes, while HPV 51 and 52 are considered oncogenic and thus accurate detection of these genotypes is more relevant regarding diagnostics.
5. Conclusion
We conclude that the report demonstrates a high overall agreement between the two assays; however, IL showed a somewhat superior performance in detecting HPV genotypes in FFPE materials than LA did.
Ethics Approval and Consent to Participate
The study was approved by the Regional Committees for Medical and Health Research Ethics (REC) - West Norway (2017/1515/REK vest) and informed opt-out consent was obtained from all participants.
Acknowledgements
The authors have received financial support from the Folke Hermansen Fund and research funding from the Stavanger University Hospital. Bianca van Diermen Hidle, (BSc) contributed with HPV genotyping performed for the manuscript. She was an employee at the Department of Pathology, Stavanger University Hospital, Stavanger, Norway. Sadly she passed away in 2018.
Author Contributions Statement
BB has contributed to the experimental design, DNA extraction, HPV genotyping, performed the data analysis and drafted the manuscript. PM ITØ and IKC have contributed the HPV genotyping, ML and ER have performed the majority of the DNA isolations. EGG and JMB have assigned diagnoses to the pathological specimens. EAMJ and ITØ have contributed to the experimental design and EAMJ, ITØ and IKC have critically reviewed the manuscript. All authors have read and approved the manuscript.
Data Availability
Authors confirm that all relevant data are included in the article. Any further information is available upon request.
Competing Interests
The authors declare no competing interest.
References
- Davit Bzhalava, Carina Eklund, Joakim Dillner. International standardization and classification of human papillomavirus types. Virology 476 (2015): 341-344.
- Walboomers J M, Jacobs M V, Manos M M, et al. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol 189 (1999): 12-29.
- Bruni L, Albero G, Serrano B, et al. Human Papillomavirus and Related Diseases in the World. Summary Report. ICO Information Centre on HPV and Cancer (HPV Information Centre) (2019).
- Louise Baandrup, Louise T Thomsen, Tina Bech Olesen, et al. The prevalence of human papillomavirus in colorectal adenomas and adenocarcinomas: a systematic review and meta-analysis. Eur J Cancer 50 (2014): 1446-1461.
- Raul D Bernabe-Dones, Maria Gonzalez-Pons, Alejandro Villar-Prados, et al. High prevalence of human papillomavirus in colorectal cancer in hispanics: a case-control study. Gastroenterology research and practice 2016 (2016): 1-8.
- Lucine Vuitton, Carine Jaillet, Elise Jacquin, et al. Human papillomaviruses in colorectal cancers: A case-control study in western patients. Dig Liver Dis 49 (2017): 446-450.
- Catherine de Martel, Jacques Ferlay, Silvia Franceschi, et al. Global burden of cancers attributable to infections in 2008: a review and synthetic analysis. The Lancet Oncology 13 (2012): 607-615.
- Martyn Plummer, Catherine de Martel, Jerome Vignat, et al. Global burden of cancers attributable to infections in 2012: a synthetic analysis. The Lancet Global Health 4 (2016): e609-e616.
- William H Westra, James S Lewis Jr. Update from the 4th Edition of the World Health Organization Classification of Head and Neck Tumours: Oropharynx. Head and neck pathology 11 (2017): 41-47.
- Laura D Locati, Mara S Serafini, Maria F Iannò, et al. Mining of Self-Organizing Map Gene-Expression Portraits Reveals Prognostic Stratification of HPV-Positive Head and Neck Squamous Cell Carcinoma. Cancers (Basel) 11 (2019): 1057.
- Albers Andreas E, Qian Xu, Kaufmann Andreas M, et al. Meta analysis: HPV and p16 pattern determines survival in patients with HNSCC and identifies potential new biologic subtype. Scientific reports 7 (2017): 16715.
- Marc T Goodman, Mona Saraiya, Trevor D Thompson, et al. Human papillomavirus genotype and oropharynx cancer survival in the United States of America. Eur J Cancer 51 (2015): 2759-2767.
- Annekatrin Coordes, Klaus Lenz, Xu Qian, et al. Meta-analysis of survival in patients with HNSCC discriminates risk depending on combined HPV and p16 status. Eur Arch Otorhinolaryngol (2016): 2157-2169.
- Rollo F, Dona’ MG, Pellini R, et al. Cytology and direct human papillomavirus testing on fine needle aspirates from cervical lymph node metastases of patients with oropharyngeal squamous cell carcinoma or occult primaryCytopathology 29 (2018): 449-454.
- Poljak M, Ostrbenk A, Seme K, et al. Comparison of clinical and analytical performance of the Abbott Realtime High Risk HPV test to the performance of hybrid capture 2 in population-based cervical cancer screening. J Clin Microbiol 49 (2011): 1721-1729.
- Maria Benevolo, Amina Vocaturo, Donatella Caraceni, et al. Sensitivity, specificity, and clinical value of human papillomavirus (HPV) E6/E7 mRNA assay as a triage test for cervical cytology and HPV DNA test. J Clin Microbiol 49 (2011): 2643-2650.
- Tabrizi SN, Stevens MP, Khan ZA, et al. Comparison of PapType to Digene Hybrid Capture, Roche Linear Array and Amplicor for detection of high risk human papillomavirus genotypes in women with previous abnormal pap smears. J Clin Microbiol 50 (2012): 2796-2798.
- Rao A, Young S, Erlich H, et al. Development and characterization of the cobas human papillomavirus test. J Clin Microbiol 51 (2013): 1478-1484.
- Hesselink A T, Sahli R, Berkhof J, et al. Clinical validation of Anyplex™ II HPV HR Detection according to the guidelines for HPV test requirements for cervical cancer screening. J Clin Virol 76 (2016): 36-39.
- Michelle M Rietbergen, Peter JF Snijders, Derakshan Beekzada, et al. Molecular characterization of p16-immunopositive but HPV DNA-negative oropharyngeal carcinomas. Int J Cancer 134 (2014): 2366-2372.
- André LP Abreu, Raquel P Souza, Fabrícia Gimenes, et al. A review of methods for detect human Papillomavirus infection. Virology journal 9 (2012): 262.
- Siriaunkgul S, Suwiwat S, Settakorn J, et al. HPV genotyping in cervical cancer in Northern Thailand: adapting the linear array HPV assay for use on paraffin-embedded tissue. Gynecol Oncol 108 (2008): 555-560.
- Candice C Black, Heather A Bentley, Thomas H Davis, et al. Use of a linear array for the detection of human papillomavirus genotypes in head and neck cancer. Archives of Pathology & Laboratory Medicine 134 (2010): 1813-1817.
- Dona MG, Ronchetti L, Giuliani M, et al. Performance of the linear array HPV genotyping test on paired cytological and formalin-fixed, paraffin-embedded cervical samples. J Mol Diagn 15 (2013): 373-379.
- Irene T Ovestad, Undis Vennestrøm, Liv Andersen, et al. Comparison of different commercial methods for HPV detection in follow-up cytology after ASCUS/LSIL, prediction of CIN2-3 in follow up biopsies and spontaneous regression of CIN2-3. Gynecol Oncol 123 (2011): 278-283.
- Ane Cecilie Munk, Einar Gudlaugsson, Anais Malpica, et al. Consistent Condom Use Increases the Regression Rate of Cervical Intraepithelial Neoplasia 2–3. PLoS One (2012): e45114.
- Lillsunde Larsson G, Carlsson J, Karlsson MG. Evaluation of HPV Genotyping Assays for Archival Clinical Samples. The Journal of molecular diagnostics: JMD 17 (2015): 293-301.
- Gwet, Kilem Li. Inter-Rater Reliability Using SAS: A Practical Guide for Nominal, Ordinal, and Interval Data. Advanced Analytics, LLC (2010).
- R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2017).
- Matthias Gamer, Jim Lemon, Ian Fellows, et al. Various Coefficients of Interrater Reliability and Agreement. R Package version 0.84 ed (2012).
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 33 (1977): 159-174.
- Lan Xu, Elizaveta Padalko, Anja Oštrbenk, et al. Clinical Evaluation of INNO-LiPA HPV Genotyping EXTRA II Assay Using the VALGENT Framework. Int J Mol Sci 19 (2018): 2704.
- Clifford G, Franceschi S, Diaz M, et al. Chapter 3: HPV type-distribution in women with and without cervical neoplastic diseases. Vaccine 24 (2006): S3/26-S3/34.
- Miralles-Guri C, Bruni L, Cubilla A L, et al. Human papillomavirus prevalence and type distribution in penile carcinoma. J Clin Pathol 62 (2009): 870-878.
- Hugo De Vuyst, Gary M Clifford, Maria Claudia Nascimento, et al. Prevalence and type distribution of human papillomavirus in carcinoma and intraepithelial neoplasia of the vulva, vagina and anus: a meta-analysis. Int J Cancer 124 (2009): 1626-1636.
- Kreimer AR, Clifford GM, Boyle P, et al. Human papillomavirus types in head and neck squamous cell carcinomas worldwide: a systematic review. Cancer Epidemiology Biomarkers & Prevention 14 (2005): 467-475.
- Farshid Dayyani, Carol J Etzel, Mei Liu, et al. Meta-analysis of the impact of human papillomavirus (HPV) on cancer risk and overall survival in head and neck squamous cell carcinomas (HNSCC). Head & Neck Oncology 2 (2010): 15
- Dona MG, Benevolo M, Pimpinelli F, et al. Comparative evaluation of different DNA extraction methods for HPV genotyping by linear array and INNOLiPA. J Med Virol 83 (2011): 1042-1047.
- Pratiksha Dedhia, Shivraj Tarale, Gargi Dhongde, et al. Evaluation of DNA extraction methods and real time PCR optimization on formalin-fixed paraffin-embedded tissues. Asian Pac J Cancer Prev 8 (2007): 55-59.
- Veyer D, Wack M, Grard O, et al. HPV detection and genotyping of head and neck cancer biopsies by molecular testing with regard to the new oropharyngeal squamous cell carcinoma classification based on HPV status. Pathology 51 (2019): 421-425.
- Sarah E Tan, Suzanne M Garland, Alice R Rumbold, et al. Human papillomavirus genotyping using archival vulval dysplastic or neoplastic biopsy tissues: comparison between the INNO-LiPA and linear array assays. J Clin Microbiol 48 (2010): 1458-1460.
- Wentzensen N, Gravitt PE, Long R, et al. Human papillomavirus load measured by Linear Array correlates with quantitative PCR in cervical cytology specimens. J Clin Microbiol 50 (2012): 1564-1570.
- Oštrbenk A, Kocjan BJ, Poljak M. Specificity of the Linear Array HPV Genotyping Test for detecting human papillomavirus genotype 52 (HPV-52). Acta Dermatovenerologica Alpina Pannonica et Adriatica 23 (2014): 53-56
- Mori S, Nakao S, Kukimoto I, et al. Biased amplification of human papillomavirus DNA in specimens containing multiple human papillomavirus types by PCR with consensus primers. Cancer Sci 102 (2011): 1223-1227.
- Felipe A Castro, Jill Koshiol, Wim Quint, et al. Detection of HPV DNA in paraffin-embedded cervical samples: a comparison of four genotyping methods. BMC Infect Dis 15 (2015): 544.
- Michel P Bihl, Luigi Tornillo, André B Kind, et al. Human Papillomavirus (HPV) Detection in Cytologic Specimens: Similarities and Differences of Available Methodology. Appl Immunohistochem Molecul Morphol 25 (2017): 184-189.