High-impact clinical studies in anesthesia: a review
In a previous study,1 we analyzed important new developments in anesthesiology over the past 50 years. The topics for such developments were selected using an objective indicator – the degree to which the number of academic articles on a topic increased. More than 20 such topics were identified. The aim of this study was to determine for each of these topics the most influential initial clinical studies, also using an objective indicator – the number of citations generated by the study’s publication.
In science, priority rule is the credit given to the person who first published a new finding or proposed a theory.2,3 Determining priority of discovery is a two-step process that includes its disclosure and validation.4 The validation reflects the scientific community’s response to a disclosure centered on two questions: Is a new finding correct, and is it of sufficient interest to merit attention and further development? In practice, this acknowledgement usually comes in the form of citations in papers by other scientists, which can accumulate over time. Unlike disclosure (eg, publication), which is an event with a definite time stamp, validation can take a variable amount of time.4
The chain of accomplishments leading to any important new development has many links. Among the main ones are disclosure(s) of a new idea (eg, patents), definitive bench and/or animal studies, and human studies leading to the formal approval of a drug or technique for use in clinical practice. The aim of the current study is to assess only one of these links – initial clinical studies that prominently contributed to the acceptance of a new development by the scientific community. We sought to assess the contribution of initial clinical studies to the success of a new development by combining two factors: the time of the study publication and its acknowledgement by the scientific community in the form of total citation impact. For this, we identified initial studies that proved the effectiveness of a drug (or technique) in achieving a specific aim in anesthesia, and were the first to have an impressive citation impact.
On Topics of New Developments
The selection of the topics of new developments was based on the results of the previous study, in which the degree of increase in the number of academic articles on a topic was used for selection.1 The main tool for selection was popularity index (PI),5 the percentage of articles on the topic among all articles related to the PubMed (MeSH) term “Anesthesia” over the same 5-year period. A topic was selected if it had reached a PI =/> 0.5 during any 5-year period from 1966 to 2015, or the total number of articles on the topic reached at least 500 articles.1 There were 27 such topics, 19 of them were drug-related (ketamine, isoflurane, enflurane, etomidate, propofol, midazolam in anesthesia, vecuronium, alfentanil, atracurium, sevoflurane, sufentanil, mivacurium, rocuronium, desflurane, ropivacaine, remifentanil, dexmedetomidine in anesthesia, levobupivacaine, and sugammadex) and 9 technique-related (ultrasound-guided peripheral nerve block, capnography in anesthesia, target-controlled intravenous anesthesia, pulse oximetry in anesthesia, total intravenous anesthesia, transesophageal echocardiography in anesthesia, combined spinal-epidural anesthesia, bispectral index, and transversus abdominis plane block). In the present study, we excluded from the list of drugs three agents that were discontinued in the USA in 2020 – enflurane, mivacurium, and levobupivacaine; the list of technique-related topics was not modified. Basic search details on these topics are presented in Table 1.
Table 1 Search Details on Initial High-Impact Clinical Studies
Requirements for Selection of Initial High-Impact Studies
The selection of initial high-impact studies was based on the following general requirements: 1) the principal aim of the study should be the assessment of the effectiveness of a new agent (or technique) in achieving a specific aim in anesthesia; 2) the study should be published as an original research article; 3) the study should be one of the initial clinical studies on a topic; 4) the study should have an impressive citation impact (the Thomson Reuters Web of Science Database),6 preferably more than 100, but not fewer than 20 citations, most importantly, it should be published before other clinical studies on the topic with citation records =/> 20. If the study had a citation record =/> 20, but not =/> 100, an additional study with =/> 100 citations was also selected, on condition that it was published the next on the timeline after the first study. As a result, there were two types of initial studies with =/> 100 citations: the first high-impact studies with =/> 100 citations and the additional high-impact studies with =/> 100 citations. The choice of the number of citations thresholds (20 and 100) was a voluntary one.
Specific requirements for the selection of studies with 20-citation threshold: If several studies with =/> 20 citations were published almost simultaneously (within six-month period), all of them were identified, unless they had <30% of the number of citations of the study with the highest citation rate.
Specific requirements for the selection of additional studies with 100-citation threshold: 1) If the =/> 100 citations study is an additional one, its date of publication should be closest to that of the first study with <100 citations (but >20 citations). 2) If several studies with the =/>100 citation count were published during the same six-month period, all were listed according to their publication dates. Any of these studies with fewer than 30% of the citations of the first study were excluded.
Table 2 presents data on studies according to the number of citations they elicited as well as their publication dates (year and month), which are the principal criteria for the assessment of their priority. The table indicates the first authors of the articles, all other authors (along with the article titles, journal names, and other publication details). The review’s list of selected studies as systemized in Table 2 can be viewed in Supplement. In Table 2 studies are divided into two categories: those which crossed only =/> 20 citation-threshold and those which crossed a higher threshold, =/> 100 citations. The selected studies were prioritized based on publication date, but only if their citation count was =/> 20. Subject to a number of specifications (presented in the Methods), one study for each topic was designated as the first high-impact study. If the first high-impact study did not reach the 100 citation-threshold, the study next to it on the timeline was also selected; it was designated as the additional high-impact study – published at a later date, but having a higher citation count. As a result, there were two types of initial studies with =/>100 citations: the first studies with =/> 100 citations and additional studies with =/>100 citations; both of them are presented in Table 2. They are listed separately in Table 3.
Table 2 Initial High-Impact Clinical Studies in Major Anesthesia Developments Since 1965
Table 3 Number of Citations Generated by Initial High-Impact Clinical Studies
Overall, 25 studies were designated as first high-impact studies, 16 of which are drug-related and 9 technique-related. Only half of the first high-impact studies had a citation count of =/>100: 8 of 16 with drug-related topics, from 106 to 555 citations; and 4 of 9 with technique-related topics, from 102 to 421 citations. The other half of the first high-impact studies did not reach the 100-citation threshold: for drug-related topics their counts were from 41 to 97, and for technique-related topics – 46 to 65. There were no citation counts below 40 with any topic, although citation rates from 20 to 40 were searched for. Given the principles for selection, all additional high-impact studies had =/>100 citations (Table 3, right column): with eight drug-related topics the citations count varied from 100 to 344, and with four technique-related topics – from 133 to 219.
The selection of the first high-impact study (or the additional high-impact study) was often made more difficult by the presence of several studies with very close dates of publication and similar citation counts. The greatest difficulties in this respect were with five drug-related topics: midazolam, vecuronium, sevoflurane, ropivacaine, and sugammadex. This can be seen when dates of publication and citation counts of several similar high-impact studies (Table 2) are compared with those of a single study presented as the final choice (Table 3). For example, there were three studies on the use of midazolam for anesthesia published in 1978: Conner13 in January–February, Reves14 in May, and Fragen15 in July; the citation counts for all three studies varied only from 106 to 128. The choice of Conner’s study (Table 3) was based on several months earlier publication. Another example is the three studies on ropivacaine published in 1990: Katz28 in January, Concepcion29 in January, and Brown30 in April. In this case, two studies (Katz and Concepcion) were published in the same issue of Anesthesia & Analgesia and had similar citation counts – 70 and 63, respectively (Table 2). Both studies are placed in Table 3 as the first high-impact study, as an exception to the rule of one study per topic.
In this review, we identified the initial high-impact clinical studies which fomented new developments in anesthesiology over the past 50 years. These studies were assessed by combining two factors: the study’s publication date and the degree of its acknowledgement by the scientific community in the form of citations. As a result, twenty-five studies were designated the first high-impact studies (one for each topic); 16 are drug-related and 9 are technique-related. Half of the first high-impact studies had a citation count of =/>100, (range: 100 to 555). The citation count of the other half of high-impact studies did not reach the 100-citation threshold (range: 41 to 97). If a selected first high-impact study had a citation count <100, a next-on-timeline, additional study with citation count =/>100 was also selected (range: 100 to 344).
These studies represent 25 topics of the most meaningful developments in anesthesiology. What is a meaningful development? According to Helmer,52 one of the most important conditions for meaningful research is a practically useful impact on society. In this review, due to the retrospective view of developments whose usefulness was confirmed during many decades of clinical practice, research studies with impact on these developments were historically shown to be practically useful. They include the following topics which are well known to any anesthesiologist. Sixteen drug-related topics: ketamine, isoflurane, etomidate, propofol, midazolam in anesthesia, vecuronium, alfentanil, atracurium, sevoflurane, sufentanil, rocuronium, desflurane, ropivacaine, remifentanil, dexmedetomidine in anesthesia, and sugammadex; and also nine technique-related topics: ultrasound-guided peripheral nerve block, capnography in anesthesia, target-controlled intravenous anesthesia, pulse oximetry in anesthesia, total intravenous anesthesia, transesophageal echocardiography in anesthesia, combined spinal-epidural anesthesia, and bispectral index.
It is important to underline that all selected studies had two specific features. First, only original research articles, in which the principle goal was the assessment of possible use of a new agent or technique for achieving a specific aim in anesthesia, were selected. Studies beyond the initial question of the basic advantage of a new drug (or technique) for anesthesia, such as drug’s formulation, the general characteristic of drug’s pharmacokinetics, or possible additional clinical indications, were not selected as the initial high-impact studies.
Second, only initial clinical studies were selected. Their impact on the opinion of the scientific community was sufficient to convince it that a new agent or technique is of interest for further development, indicating that the previous chain of events, from conceiving an idea (eg, patent) to bench or/and animal experiments, and finally to clinical trial, had real promise. The high citation impact of the initial clinical study elevates a new agent, following multiple study-steps on various aspects of the agent’s action, to its destination – formal approval for use in clinical practice. In the long life of an introduced drug or technique, significant new research findings may change the drug’s role in clinical practice. This assessment concentrated only on initial clinical studies that gave birth to new developments.
Among limitations of this review is, described in the methods, voluntary choice of the number of citations thresholds for selecting the studies. The other limitation is associated with the preselected aspects of the review, which requires the following explanation. The presented analysis of the studies has two aspects: 1) Which stage of development to analyze, and 2) What method to use. With the first aspect, each new development has a number of stages, such as the generation of a novel idea, its disclosure in a patent, definitive bench or/and animal studies, initial clinical studies, and advanced clinical studies leading to formal approval for general use in clinical practice. The aim of present assessment was to analyze only initial clinical studies. The second aspect, related to the methods of comparative assessment of the studies on a new development, could be based on a list established by academic leaders or through committee vote;53 another more objective one is based on various methods derived from raw citation data;54 the most important factor in discovery assessment is associated with the date of disclosure.3 In the present assessment, a combination of two factors was used: the study’s priority in terms of the time of its publication and the degree of its acknowledgement by the academic community in the form of citations. With different approaches to these two aspects of analysis – what stage to analyze and what method to apply for the study selection – various outcomes are possible. This may be regarded as a potential limitation of the current review. For example, our approach, based on initial clinical studies, is not directed toward the most important element of any new development – the generation of a new idea including its disclosure (eg, patent). At the same time, it was centered on the most critical consideration in the development of a new idea – is it clinically useful? Thus, with different approaches the importance of different studies could be assessed quite differently; therefore, our results should be regarded only as an initial attempt in the selection of high-impact studies that fomented new developments in anesthesia.
In conclusion, the present results show that an initial high-impact clinical study on a new development in anesthesiology can be determined and that related citations usually vary from one hundred to five hundred. The review’s List of Selected Studies. These initial clinical studies fomented the most important development in anesthesiology over the past 50 years.
The author reports no conflicts of interest in this work.
1. Kissin I. Academic journals assessed as springboards for new developments: a study of leading anesthesia journals over past 50 years. J Anesth Hist. 2019;5:7–12. doi:10.1016/j.janh.2018.08.005
2. Merton RK. Priority in scientific discovery: a chapter in the sociology of science. Am Sociol Rev. 1957;22:635–659. doi:10.2307/2089193
3. Stevens M. The role of the priority rule in science. J Philosophy. 2003;100:55–79. doi:10.5840/jphil.2003.100224
4. Vale RD, Hyman AA. Priority of discovery in the life sciences. eLife. 2016;5:e16931. doi:10.7554/elife.16931
5. Vlassakov KV, Kissin I. Scientometrics of anesthethic drugs and their techniques of administration, 1984–2013. Drug Des Devel Ther. 2014;8:2463–2473. doi:10.2147/dddt.s73862.ecollection2014
6. Web of science core collection results accessed via apps. 2020. Available from: webofknowledge.com.ezp-prod1.hul.harvard.edu.
7. Domino EF, Chodoff P, Corssen G. Pharmacological effects of Cl-581, a new dissociative anesthetic, in man. Clin Pharmocol Ther. 1965;6:279–291. doi:10.1002/cpt196563279
8. Dobkin AB, Byles PH, Ghanooni S, Valbuena DA. Clinical and laboratory evaluation of a new inhalation anesthetic: forane (compound 469) CHF2-O-CHCICF3 (1-chloro-2,2,2-trifluoroethyl difluoromethylether). Can Anaesth Soc J. 1971;18(3):264–271. doi:10.1007/BF03025461
9. Stevens WC, Cromwell TH, Halsey MJ, et al. The cardiovascular effects of a new inhalation anesthetic, Forane, in human volunteers at constant arterial carbon dioxide tension. Anesthesiology. 1971;35(1):8–16. doi:10.1097/00000542-197107000-00005
10. Doenicke A, Wagner E, Beetz KH. Blutgasanalysen (arteriell) nach dreikurzwirkenden i. v. Hypnotica (Propanidid, Etomidate und Methohexital [Arterial bloodgas analyses following administration of three short-acting i.v. hypnotics (propanidid, etomidate, methohexitone)(author’s translation)]. Anaesthesist. 1973;22(8):353–356. [German].
11. Holdcraft A, Morgan M, Whitwam JG, Lumley J. Effect of dose and premedication on induction complications with etomidate. Br J Anaesth. 1976;48(3):199–205. doi:10.1093/bja/48.3.199
12. Kay B, Rolly G. I.C.I. 35868, a new intravenous induction agent. Acta Anaesthesiol Belg. 1977;28(4):303–316.
13. Conner JT, Katz RL, Pagano RR, Graham CW. RO 21-3981 for intravenous surgical premedication and induction of anesthesia. Anesth Analg. 1978;57(1):1–5. doi:10.1213/00000539-197801000-00001
14. Reves JG, Corssen G, Holcomb C. Comparison of two benzodiazepines for anaesthesia induction: midazolam and diazepam. Can Anaesth Soc J. 1978;25(3):211–214. doi:10.1007/BF03004881
15. Fragen RJ, Gahl F, Caldwell N. A water-soluble benzodiazepine, RO21-3981, for induction of anesthesia. Anesthesiology. 1978;49(1):41–43. doi:10.1097/00000542-197807000-00012
16. Agoston S, Salt P, Newton D, Bencini A, Boomsma P, Erdmann W. The neuromuscular blocking action of ORG NC 45, a new pancuronium derivative, in anaestheti zed patient s. A pilot study. Br J Anaesth. 1980;52(Suppl 1):53S–59. doi:10.1093/bja/52.suppl_1.53S
17. Krieg N, Crul JF, Booij LH. Relative potency of ORG NC 45, pancuronium, alcuronium and tubocurarine in anaesthetized man. Br J Anaesth. 1980;52(8):783–788. doi:10.1093/bja/52.8.783
18. Kay B, Stephenson DK. Alfentanil (R39209): initial clinical experience with a new narcotic analgesic. Anaesthesia. 1980;35(12):1197–1201. doi:10.1111/j.1365-2044.1980.tb05078.x
19. Ausems ME, Hug CC, Stanski DR, Burm AG. Plasma concentrations of alfentanil required to supplement nitrous oxide anesthesia for general surgery. Anesthesiology. 1986;65(4):362–373. doi:10.1097/00000542-198610000-00004
20. Payne JP, Hughes R. Evaluation of atracurium in anesthetized man. Br J Anaesth. 1981;53(1):45–54. doi:10.1093/bja/53.l.45
21. Holaday DA, Smith FR. Clinical characteristics and biotransformation of sevoflurane in healthy human volunteers. Anesthesiology. 1981;54(2):100–106. doi:10.1097/00000542-198102000-00002
22. Katoh T, Ikeda K. The minimum alveolar concentration (MAC) of sevoflurane in humans. Anesthesiology. 1987;66(3):301–303. doi:10.1097/00000542-198703000-00006
23. Bovill JG, Sebel PS, Wauquier A, Rog P. Electroencephalographic effects of sufentanilanaesthesia in man. Br J Anaesth. 1982;54(1):45–52. doi:10.1093/bja/54.l.45
24. de Lange S, Boscoe MJ, Stanley TH, Pace N. Comparison of sufentanil–O2 and fentanyl –O2 for coronary artery surgery. Anesthesiology. 1982;56(2):12–118. doi:10.1097/00000542-198202000-00006
25. Weirda JM, de Wit AP, Kuizenga K, Agoston S. Clinical observations on the neuromuscular blocking action of Org 9426, a new steroidal non-depolarizing agent. Br J Anaesth. 1990;64(4):521–523. doi:10.1093/bja/64.4.521
26. Jones RM, Cashman JN, Eger E 2nd, Damask MC, Johnson BH. Kinetics and potency of desflurane (1–653) in volunteers. Anesth Analg. 1990;70(1):3–7. doi:10.1213/00000539-199001000-00002
27. Rampil IJ, Lockhart SH, Eger EI 2nd, Yasuda N, Weiskopf RB, Cahalan MK. The electroencephalographic effects of desflurane in humans. Anesthesiology. 1991;74(3):434–439. doi:10.1097/00000542-199103000-00008
28. Katz JA, Bridenbaugh PO, Knarr DC, Helton SH, Denson DD. Pharmacodynamics and pharmacokinetics of epidural ropivacaine in humans. Anesth Analg. 1990;70(1):16–21. doi:10.1213/00000539-199001000-00004
29. Concepcion M, Arthur GR, Steele SM, Bader AM, Covino BG. A new local anesthetic, ropivacaine. Its epidural effects in humans. Anesth Analg. 1990;70(1):80–85. doi:10.1213/00000539-199001000-00014
30. Brown DL, Carpenter RL, Thompson GE. Comparison of 0.5% ropivacaine and 0.5% bupivacaine for epidural anesthesia in patients undergoing lower-extremity surgery. Anesthesiology. 1990;72(4):633636. doi:10.1097/00000542-199004000-00010
31. Brockway MS, Bannister J, McClure JH, McKeown D, Wildsmith MD. Comparison of extradural ropivacaine and bupivacaine. Br J Anaesth. 1991;66(1):31–37. doi:10.1093/bja/66.1.31
32. Dershwitz M, Randel GI, Rosow CE, et al. Initial clinical experience with remifentanil, a new opioid metabolized by esterases. Anesth Analg. 1995;81(3):619–623. doi:10.1097/00000539-199509000-00035
33. Hogue CW, Bowdle TA, O’Leary C, et al. A multicenter evaluation of total intravenous anesthesia with remifentanil and propofol for elective inpatient surgery. Anesth Analg. 1996;83(2):279–285. doi:10.1097/00000539-199608000-00014
34. Lang E, Kapila A, Shlugman D, Hoke JF, Sebel PS, Glass PS. Reduction of isoflurane minimal alveolar concentration by remifentanil. Anesthesiology. 1996;85(4):721–728. doi:10.1097/00000542-199610000-00006
35. Aantaa RE, Kanto JH, Scheinin M, Kallio AM, Scheinin H. Dexmedetomidine premedication for minor gynecologic surgery. Anesth Analg. 1990;70(4):407–413. doi:10.1213/00000539-199004000-00011
36. Aantaa R, Kanto J, Scheinin M, Kallio A, Scheinin H. Dexmedetomidine, an alpha 2-adrenoceptor agonist, reduces anesthetic requirements for patients undergoing minor gynecologic surgery. Anesthesiology. 1990;73(2):230–235. doi:10.1097/00000542-199008000-00007
37. Gijsenbergh F, Ramael S, Houwing N, van Iersel T. First human exposure of Org 25969, a novel agent to reverse the action of rocuronium bromide. Anesthesiology. 2005;103(4):695–703. doi:10.1097/00000542-200510000-00007
38. Shields M, Giovannelli M, Mirakhur RK, Moppett I, Adams J, Hermens Y. Org 25969 (sugammadex), a selective relaxant binding agent for antagonism of prolonged rocuronium-induced neuromuscular block. Br J Anaesth. 2006;96(1):36–43. doi:10.1093/bja/aei314
39. la Grange P, Foster PA, Pretorius LK. Application of the Doppler ultrasound blood flow detector in supraclavicular brachia!! plexus block. Br J Anaesth. 1978;50(9):965–967. doi:10.1093/bja/50.9.965
40. Kapral S, Krafft P, Eibenberger K, Fitzgerald R, Gosch M, Weinstabl C. Ultrasound-guided supraclavicular approach for regional anesthesia of the brachiaI! plexus. Anesth Analg. 1994;78(3):507–513. doi:10.1213/00000539-199403000-00016
41. Linko K, Paloheimo M, Tammisto T. Capnography for detection of accidental esophageal intubation. Acta Anaesthesiol Scand. 1983;27(3):199–202. doi:10.1111/j.1399-6576.1983.tb01934.x
42. Schuttler J, Schwilden H, StoekeI H. Pharmacokinetics as applied to total intravenous anaesthesia. Practical implications. Anaesthesia. 1983;38:53–56. doi:10.1111/j.1365-2044.1983.tbl5179.x
43. Schwilden H, Schuttler J, Stoekel H. Closed-loop feedback control of methohexital anesthesia by quantitative EEG analysis in humans. Anesthesiology. 1987;67(3):341–347. doi:10.1097/00000542-198709000-00011
44. Yelderman M, New W Jr. Evaluation of pulse oximetry. Anesthesiology. 1983;59(4):349–352. doi:10.1097/00000542-198310000-00015
45. de Grood PM, Ruys AH, van Egmond J, Booij LH, Crul JF. Propofol (‘Diprivan’) emulsion for total intravenous anaesthesia. Postgrad Med J. 1985;61(Suppl 3):65–69.
46. Roizen MF, Beaupre PN, Alpert RA, et al. Monitoring with two-dimensional transesophageal echocardiography. Comparison of myocardial function in patients undergoing supraceliac, suprarenal infraceliac, or infrarenal aortic occlusion. J Vasc Surg. 1984;1(2):300–305. doi:10.1016/0741-5214(84)90061-2
47. Rawal N, Schellin J, Wesstrom G. Epidural versus combined spinal epidural block for cesarean section. Acta Anaesthesiol Scand. 1988;32(1):61–66.
48. Kearse LA, Manberg P, DeBros F, Chamoun N, Sinai V. Bispectral analysis of the electroencephalogram during induction of anesthesia may predict hemodynamic responses to laryngoscopy and intubation. Electroencephalogr Clin Neurophysiol. 1994;9(3):194–200.
49. Kearse LA, Manberg P, Chamoun N, deBros F, Zaslavsky A. Bispectral analysis of the electroencephalogram correlates with patient movement to skin incision during propofol/nitrous oxide anesthesia. Anesthesiology. 1994;81(6):1365–1370. doi:10.1097/00000542-199412000-00010
50. Vernon JM, Lang E, Sebel PS, Manberg P. Prediction of movement using bispectral electroencephalographic analysis is during propofol/alfentanil or isoflurane/alfentanil anesthesia. Anesth Analg. 1995;80(4):780–785. doi:10.1097/00000539-199504000-00023
51. McDonnell JG, O’Donnell B, Curley G, Heffernan A, Power C, Laffey JG. The analgesic efficacy of transversus abdominis plane block after abdominal surgery: a prospective randomized controlled trial. Anesth Analg. 2007;104(1):193–197. doi:10.1213/01.ane.0000250223.49963.0f
52. Helmer S. What is meaningful research and how should we measure it? Scientometrics. 2020;125:153–167.
53. Bergstrom CT, West JD. Assessing citations with the EigenfactorTM metrics. Neurology. 2008;71:1850–1857. doi:10.1212/01.wnl.0000338904.37585.66
54. Bollen J, Van de Sompel H, Hagberg A, Chute R. A principal component analysis of 39 scientific impact measures. PLoS One. 2009;4:e6022. doi:10.1371/journal.pone.0006022