Citation Analysis

by user








Citation Analysis
Citation Analysis
If I have seen farther, it is by standing on the shoulders of giants.
-Isaac Newton’
AN ESSENTIAL PART of research papers, particularly in the sciences, is the
list of references pointing to prior publications. As Ziman observes, “a
scientific paper does not stand alone; it is embeddedin the ‘literature’of
the subject.”’ A reference is the acknowledgment that one document
giues to another; a citation is the acknowledgment that one document
receives from a n ~ t h e r In
. ~ general, a citation implies a relationship
between a part or the whole of the cited document and a part or the
whole of the citing d o ~ u m e n tCitation
analysis is that area of bibliometrics which deals with the study of these relationships.
There are many published studies exploring citation analysis and
its applications. Some reviews of this literature have already a ~ p e a r e d , ~
and Hjerppe‘ has compiled a bibliography of more than ZOO0 entries
including many studies in citation analysis. Eugene Garfield’swritings
are a rich source of information on this subject, particularly his book on
citation indexing’ and many of his “Current Comments” columns
reprinted from Current Contents.* The present paper does not attempt
to review this extensive literature in detail. Instead, it focuses on the
development of citation analysis as a research method, uses and abuses
of this method, and prospects for the future.
Linda C. Smith is Assistant Professor, Graduate School of Library and Information
Science, University of Illinois at Urbana-Champaign.
As noted above, a citation represents a relationship between the
cited and citing documents. The nature of this relationship is somewhat
difficult to characterize, however, due to the many reasons authors cite,
such as the fifteen enumerated by Garfield:
1. Paying homage to pioneers
2. Giving credit for related work (homage to peers)
3. Identifying methodology, equipment, etc.
4. Providing background reading
5. Correcting one’s own work
6. Correcting the work of others
7. Criticizing previous work
8. Substantiating claims
9. Alerting to forthcoming work
10. Providing leads to poorly disseminated, poorly indexed, or uncited
11. Authenticating data and classes of fact-physical constants, etc.
12. Identifying original publications in which an idea or concept was discussed 13. Identifying original publications or other work describing an epo- nymic concept or term... 14. Disclaiming work or ideas of others (negative claims)
15. Disputing priority claims of others (negative h ~ r n a g e ) . ~
Bavelas suggests that “the two extremesof this array of reasons might be
true scholarly impact at the one end (e.g., significant use of the cited
author’s theory, paradigm, or method) and less-than-noble purposes at
the other (e.g., citing the journal editor’s work or plugging a friend’s
publications).”’0 Furthermore, it is possible that norms for citing vary
from discipline to discipline.
Just as there are a number of reasons why citations exist, there may
be a number of reasons why a citing author has not provided a link to
certain other documents. Although the most obvious reason is that a
prior document is not relevant to the present work, i t may also be due to
the fact that the author was not aware of the document, or could not
obtain it, or could not read the language in which it was published. As
Kochen observes: “it is not surprising that there is a great deal of
arbitrariness in the way authors select references €or their bibliographies. Undoubtedly, many documents which should have been citedare
missed; and many documents which the author does cite are only
slightly relevant.”11
In spite of the uncertainties associated with the nature of the
citation relationship, citations are attractive subjects of study because
they are both unobtrusive and readily available. Unlike data obtained by
interview and questionnaire, citations are unobtrusive measures that do
Citation Analysis
not require the cooperation of a respondent and that do not themselves
contaminate the response (i.e., they are nonreactive).12Citations are
signposts left behind after information has been utilized and as such
provide data by which one may build pictures of user behavior without
ever confronting the user himself. Any set of documents containing
reference lists can provide the raw material for citation analysis, and
citation counts based on a given set of documents are precise and
Development of Citation Analysis
The development of citation analysis has been marked by the
invention of new techniques and measures, the exploitation of new
tools, and the study of different units of analysis. These trends have led
to a rapid growth in both the number and types of studies using citation
The easiest technique to use is a citation count, determining how
many citations have been received by a given document or set of documents over a period of time from a particular set of citing documents.
When this count is applied to articlesappearing in a particular journal,
it can be refined by calculating the impact factor, the average number of
citations received by articles published in a journal during a specified
time period. This measure allows one to compare the “impact” of
journals which publish different numbers of articles. Pinski and Narin
have developed further refinements of citation counts which take into
account the length of papers, the prestige of the citing journal, and the
different referencing characteristics of different segments of the
li tera ture.13
Two techniques have been devised to identify documents likely to
be closely related: bibliographic coupling’‘ and cocitation ana1y~is.l~
Two documents are bibliographically coupled if their reference lists
share one or more of the same cited documents. T w o documents are
cocited when they are jointly cited in one or more subsequently published documents. Thus in cocitation earlier documents become linked
because they are later cited together; in bibliographic coupling later
documents become linked because they cite the same earlier documents.
The difference is that bibliographic coupling is an association intrinsic
to the documents (static), while cocitation is a linkage extrinsic to the
documents, and one that is valid only so long as they continue to be
cocited (dynamic).16The theory and practical applications of bibliographic coupling and cocitation analysis have been reviewed by Weinberg and Fkllardo, re~pective1y.l~
Citation counts and bibliographic
coupling were the characteristic citation analysis techniques in the
1960s, but in the 1970s cocitation analysis became the focus of much
research activity. Cocitation analysis is of particular interest as a means
for mapping scientific specialties.18
Use of new techniques in citation analysis has been made possible
by the availability of new tools. Early citation studies frequently were
based on lists of references found in articles appearing in a small
number of journals. Citations had to be transcribed and manipulated by
hand. Because of the tediousness of this process, most studies were
necessarily quite limited in scope. The availability of the computer has
significantly improved this situation in two ways: through the production of printed indexes which contain citation data from thousands of
document^,'^ and through the analysis of citation data available in
machine-readable form. Products of the Institute for Scientific Information (1%) now provide a wealth of data for citation analysis. Subject
coverage has been expanded from the initial Science Citation Index
(SCI) to include the Social Sciences Citation Index (SSCI)and the Arts
and Humanities Citation Index (AkHCI) as well. And with each passing year the time coverage becomes more extensive-SCI dates from
1961, SSCI from 1966, and A&HCI from 1976. In 1973, IS1 introduced
the Journal Citation Reports (JCR), a companion volume to the citation index which includes rankings of journals by citations and by
impact factor, as well as two ranked lists for each journal covered: those
journals which cite a given journal most heavily, and those journals
which a given journal most frequently cites.2oAt present, JCR volumes
are available for both SCI and SSCI.
Although discussion thus far has suggested counting citations only
for individual articles or journals, in fact various levels of aggregation
are possible. The units of analysis can be individual articles or books,
journals, authors, industrial organizations?1 academic departments,
universities, cities, states, nations, and even telescopes.22If one assumes
that citations are indicators of importance, then one can use such
analyses to determine the most important scholars, publications,
departments, etc., in a particular discipline or subdiscipline. This
assumption is just one of several which deserves closer scrutiny if the
results of citation analyses are to be understood.
Critique of Citation Analysis
Critics have questioned both the assumptions and methods of
many studies found in the citation analysis literature. The strongest
Citation Analysis
advocates of citation analysis recognize its limitations and exercise care
in its applications.23 Unfortunately, other investigators seem to be
Unaware of these limitations and misinterpret the results of theiranalyses. This section of the paper will enumerate both the assumptions
underlying citation analysis and the limitations of citation data, setting
the stage for the discussion of applications which follows.
Assumptions frequently underlying citation analysis are described
below, together with supporting evidence and/or counter-examples.
1. Citation of a document implies use of that document by the
citing author. This assumption actually has two parts: (1) the author
refers to all, or at least to the most important, documents used in the
preparation of his work; and (2) all documents listed were indeed used,
i.e., the author refers to a document only if that document has contributed to his work. Failure to meet these two conditions leads to “sins of
omission and c o m m i s ~ i o n ”certain
: ~ ~ documents are underrated because
not all items used were cited, and other documents are overrated because
not all items cited were used. With respect to underrating, it should be
evident to anyone who has written a paper that citation does not
necessarily fullyand faithfullyreflect usage. Often whatiscitedisonlya
small percentage of what is read; not all that is read and found useful is
cited. Although the author usually does not provide any evidence of
omissions, there are exceptions. Consider a paper by Bottle which has as
its reference 29: “Reference omitted to avoid embarrassing its author”!25
With respect to overrating, Davies offers a “fundamental law of reference giving”: it is quite unnecessary to have read or even seen the
reference yourself before quoting it.26Without looking at the text of
both the citing and cited documents, i t may not be possible to make a
judgment as to whethera particularcitation doesindeed represent useof
material in the cited document.
2 . Citation of a document (author, journal, etc.) reflects the merit
(quality, significance, impact) of that document (author, journal, etc.).
The underlying assumption in the use of citation counts as quality
indicators is that thereis a high positivecorrelation between the number
of citations which a particular document (author, journal, etc.) receives
and the quality of that document (author, journal, e t ~ . )The
. ~ ~use of
citation analyses for evaluative purposes is the issue that has generated
the most discussion. While Bayer and Folger note that measures derived
from citation counts have high face validity,% Thorne argues that
citation counts have spurious validity because documents can be cited
for reasons irrelevant to their merit.29Nevertheless, this assumption has
been tested and has found support in a number of studies, including
studies of scientific papers, journals and scholars.30In each case some
nonbibliometric measure(s) of quality must be compared with bibliometric measures based on citation counts. The difficulty is that quality
is a complex attribute, and there generally is no single widely accepted
nonbibliometric measure. Furthermore, one cannot autorilatically
assume that an infrequently cited document (author, journal, etc.) i s
without merit. In the case of journals, for example, the usefulness of
citations as a measure of the journal’s quality varies according to the
function of the journal; news journals may be of high quality but
infrequently cited. Until more is understood about the reasons for
citing, citation counts can at best be viewed as a rough indicator of
quality. Small differencesin citation counts should not be interpreted as
significant, but large differences may be interpreted as reflections of
differences in quality and impact. Results of citation counts should be
compared with alternative quality indicators to look for correlations.
The validity of the measure is most fragile in citation counts for individual documents and authors. One can have more confidence in comparisons of counts based on larger units, such as journals.
3. Citations are made to the best possible works. One can better
understand the nature of citations if one knows the population from
which they are selected. If one assumes that citations are made to the best
possible works, then one must imagine that authors sift through all of
the possible documents that could be cited and carefully select those
judged best. But studies of science information use have suggested that
accessibility may be as important a factor as quality in the selection of an
information source. Soper conducted a study to investigate the effect of
physical accessibility upon the selection and use of reference^.^' She
found that the largest proportion of documents cited in authors’ recent
papers was located in personal collections, a smaller proportion was
located in libraries in departments and institutions to which respondents belonged, and the smallest proportion was located in libraries in
other cities and countries. Thus a paper might well have been cited
because i t happened to be on the citer’s desk rather than because it was
the ideal paper to cite. Accessibility of a document may be a function of
its form, place of origin, age, and language. If a journal article, its
accessibility may be determined by the journal’s circulation, reprint
policies, and coverage by indexing and abstracting services. Just as a
document may be more or less accessible, a researcher may be more or
less visible. An author is likely to be most aware of the work of his
colleagues. Other scientists’ work may come to the author’s attention as
a result of their discoveries, their leadership in the scientificcommunity,
Citation Analysis
or their activities in the world of politics and contr~versy.~’
As with
documents, researchers cited therefore do not necessarily represent the
most outstanding in a particular field. It may be that anything which
enhances a researcher’s visibility is likely to increase his citation rate,
irrespective of the intrinsic quality of his work.
4. A cited document is related in content to the citing document; if
two documents are bibliographically coupled, they are related in content; and if two documents are cocited, they are related in content. To the
extent that citation indexes can be used to retrieve relevant citing documents given a cited document, one has support for the first part of this
assumption. Additional support is found in the results of an experiment
conducted by Barlup in which authors were asked to assess the degree of
relatedness of citations to their own
The authors judged 72
percent to be definitely related, and only 5 percent to be definitely not
related. The difficulty with the second and third parts of the assumption
becomes evident when one considers an early statement by Garfield
regarding citation indexes: “If one considers the book as the macro unit
of thought and the periodical article the microunit of thought, then the
citation index in some respects deals in the submicro or molecular unit
of t h ~ u g h t . ’Given
’ ~ ~ this observation, Martyn contends that a bibliographic coupling is not a valid unit of measurement because one does
not know that two documents citing a third are citing the identical unit
of information in it.%Thus, bibliographic coupling is merely an indication of the existence of the probability (possibly zero)of a relationship
in the content of the two documents. The same applies to cocitation as
well; the fact that two papers are cocited does not guarantee a relationship between their contents.
5. All citations are equal. This paper began with a discussion of the
problematical nature of the relationship between cited and citingdocuments. Yet studies using citation counts generally assume that all citations (with the possible exception of self-citations) can be weighted
equally. In recent years many investigators have sought ways to refine
citation analysis which would not necessarily treat all citations to the
same article (author, journal, etc.) as equivalent. These can be subdivided into two types of refinements: mechanical v . intellectual. Mechanical refinements require no judgment or inference; intellectual
refinements require (at least at present) human analysis.
Mechanical refinements look at easily definable properties of a
citation, such as multiple Occurrence or location in a document. The
hope is that knowing this property will allow one to predict something
about the relationship between citing and cited documents. Bertram
investigated whether the level (or amount) 01material actually cited by
citing articles in science journals would vary significantly with the
section of the source article in which the citation occurs.36She identified
three levels [whole, part, word(s)]and three sections (title/introduction,
results/discussion, experimental), and found that indeed the title/introduction tended to cite whole articles, results/discussion tended to cite
only a part, and experimental tended to cite words. Thus, at least for the
articles in Bertram’s study, a significant relationship doesexist between
citation level and the section of the citing article in which a citation
occurs. A study reported by Herlach tested and accepted the hypothesis
that the mention of a given reference more than once within the same
research paper indicates a close and useful relationship of citing tocited
paper?’ She further noted that use of multiple mention as a retrieval
criterion would yield good precision but low recall. Voos and Dagaev
agree that location and multiple mention can be used to distinguish
citations of particular value.%Self-citations are also readily identifiable
as a special class. Tagliacozzo completed a study todetermine theextent
to which authors of scientific articles cite their previous publications
and to find the principal distinguishing features of this particular type
of citation.39She found that self-citations were more recent than references to other authors. This suggests that conclusions about time distributions of citations would vary depending on whether or not
self-citations were included.
In contrast to mechanical refinements, intellectual refinements rely
on content analysis. As Small observes, “in the last few years sociologists
of science have begun to explore the fine structure of citation practice by
examining the contexts in which citations occur-specifically the text
surrounding the footnote number.”40 Many of these studies have
attempted to develop and apply classification schemes. An early classification scheme was that of Lipetz, who devised a set of indicators to
characterize the citing article as well as the kind of relationships of the
citing to the cited article.41Several other classification schemes have
been developed in the last few years.42Categories suggested by these
schemes include confirmative/negational-to distinguish material
judged to be g o d from material judged to be bad-and organic/perfunctory-to distinguish necessary citations from dispensable
ones. All these attempts at classification are useful supplements to
simple citation counts.
Rather than trying to create exhaustive classification schemes, a
more recent development is the interpretation of cited documents as
concept symbols. As Small observes, the interpretation of citations in
Citation Analysis
this way is more closely related to the way citations are used by authors
in scientific ~ a p e r s . He
4 ~ notes that most citations are the author’s own
private symbols for certain ideas he uses. Where documents are frequently cited, their use as concept symbols may be shared by a group of
scientists. Small has recently extended this approach through the development of cocitation context analysis.44Statements characterizing the
structure of a cocitation map are obtained from an analysis of the
contexts or passages in which documents are cocited.
The difficulty with such intellectual refinements is the time
required to apply them. Human judgment is needed to analyze citation
contexts and make inferences, so studies employing intellectual refinements are likely to be limited in scope. Nevertheless, both mechanical
and intellectual refinements offer alternatives to treating citations as
masses of undifferentiated units. Although for some applications it is
sufficient to treat citations equally, for others it is appropriate to investigate “the fine structure of citation practice.”
Given the difficulties with the assumptions which underly many
citation analyses, one must also be aware of the problems which can
exist in sources of citation data. Some of these problems are characteristic of all sources of citation data, while others only pose difficulties in
the use of secondary sources, the citation indexes. Cole and Cole discuss
many of these problems and ways of handling them in statistical analys ~ s Problems
. ~ ~
1. Multiple authorship. Cited articles listed in the citation indexes include only the first-named authors. To find all citations to publications of a given author, including those in which he is not firstauthor, one needs a bibliography of his works so that all articles can be
checked in the citation index. Errors can be introduced unless such
complete counts are made.& There is also the problem of allocating
credit in multiauthored works.47Should such works be treated the
same as single-authored works in citation counts or should credit be
divided proportionally? Should one consider the sequence of author
names in allocating credit, as this sequence often is an indication of
the contribution of each author to the work reported?
2. Self-citations. If self-citations are to be eliminated from citation
counts, this is easily done for papers written by a single author.
Again, multiauthored papers may require further checking. An even
more difficult problem is to eliminate group self-citations, i.e., references from any member(s)of a research group toany other member(s)
of that research group. In this case one would have to find a source
identifying all members of the research group.
3 . Homographs. Many scientists with the same nameand initialscould
be publishing in the same field. To differentiate among them, additional information such as institutional affiliation is needed. Otherwise citations could be attributed incorrectly to an author, particularly if he has a common name.
4. Synonyms. Citations will be scattered unless a standard form for the
author name can be established. Examples of “synonyms” in the
context of citation indexes include an author’s name with a variable
number of initials (e.g., Licklider, J.; Licklider, J.C.; Licklider,
J.C.R.), a woman’s maiden and married names, different treatments
of foreign names, and misspellings. Although ISI’s editing programs
manage to reconcile many of the differences introduced by citing
authors, variations still occur.48Journal names may also create synonym problems when the task is to identify citations of articles
appearing in a particular journal. In addition to variations in the
abbreviated form for a given title, journals merge, split into new
journals, change titles, and appear in translation. There is a need to
establish which forms are equivalent for the purposes of .citation
5. Types of sources. The type(s)of sources used in a citation analysis can
influence the results, as demonstrated in a study by Line in the social
science^.^' Analyses of references drawn from journals and monographs showed differences, some of them large, in date distributions,
forms of material cited, subject self-citation and citations beyond the
social sciences, and countries of publication cited. Line concludes
that any citation analyses that are based on only a limited number
and type of sources without specific justification must be regarded
with suspicion. Oromaner notes that authors of any typeof literature
are advised to keep their audience in mind when writing, so materials
for different types of audiences may have differingcitation patterns.50
Citation data found in the citation indexes are drawn from many
journals and selected monographs which are international in scope
and from a variety of disciplines. Although the citation indexes do
not seriously suffer from limitations in number of sources, they are
limited in type. This is not a hindrance where journals within a field
give a complete and accurate reflection of all important aspects of
scholarship. Brittain and Line describe advantages and disadvantages of various sources of citations for analysis
Choice of
types and numbers of sources should depend on the purpose of the
6. Implicit Citations.Most citation analyses consider only ex.plicitcitations, and these are what generally is made available in citation
Citation Analysis
indexes as well. An exception is the A&HCI, which includes implicit
citations when an article refers to and substantially discusses a work
but fails to include an explicit ~ i t a t i o n . ~But
' implicit citations are
also frequently found in the form of eponyms in the scientific literature. Furthermore, papers containing important ideas will not necessarily continue to be highly cited. Once an idea is sufficiently widely
known, citing the original version is unnecessary. If one were using
citation analysis to measure the impact of an individual author, such
implicit citations would fail to be included.
7 . Fluctuations with time. There may be large variations in citation
counts from one year to another, so citation data should not be too restricted in time.
8. Field variations. Citation rates (citations per publication) vary greatly
in different fields, leading to difficulties in cross-discipline comparisons. Bates has proposed the criterion rate as a refinement of citation
rate, because citation counts as a measure of the quality of a
researcher's work are influenced not only by the inherent value of
that work, but also by the size of the pool of available citers in a given
field.%A researcher's work can be evaluated in relation to a criterion
rate of citation, the citation rate of the top researchers in that field.
9. Errors. Of course, citation analyses, including those based on citation
indexes, can be no more accurate than the raw material used.
Although processing of citations for inclusion in citation indexes
may introduce some errors while eliminating others, many errors due
to citing authors remain. These can include errors in cited author
names, journal title, page, volume, and year. The incorrect citing of
sources is unfortunately far from uncommon. Two studies found the
percentage of error for citations from various journals to range from
10.7 to 50 percent.54
This section has considered two types of limitations which can
affect citation analyses: the assumptions made may not be true, and the
data collected may have inadequacies. Invalid conclusions will be made
unless these limitations are taken into account in the design of a study
and in the interpretation of results. The most reliable results may be
expected when citation abuses and errors appear as noise under conditions of high signal to noise ratio, i.e., the noise represents only a
relatively small number of the citations analyzed.55The limitations of
citation analysis do not negate its value as a research method when used
with care. There are, in fact, several application areas where citation
analysis has been used successfully.
The applications described in this section reflect two major
themes-use of citations as tools for the librarian and use of citations as
tools to analyze research activity. Citations and cocitations are part of
the range of empirical data available to historians and sociologists of
science, as well as to librarians. For each application area, representative
studies are mentioned to illustrate the types of questions which have
been investigated through citation analysis. In addition, weaknesses of
the method are identified, reflecting points made in the critique above.
1. “Literature of” studies. In this case one looks at citations in a
particular subject area to describe patterns of citation. The sources of
citation data may be as limited as a single journal in the field (e.g.,
#en‘s study of references in articles appearing in the Bulletin of the
Medical Library Association56),or they may encompass many sources,
including types of material in addition to journals. Characteristics of
cited materials frequently examined include types, age, highly cited
authors and journals, languages and countries of origin, and subject
d i s t r i b ~ t i o n s This
. ~ ~ type of study may also look for changes, in these
characteristics over time. A major problem with these studies is their
lack of compatibility which makes comparisons and synthesis difficult.
One application which has been suggested for this type of study is the
definition of appropriate secondary service coverage and scope of retrospective bibliographies in a given subject area.= By studying the range
of subjects, countries, languages, and document forms referred to by a
group of known core sources, one can begin to establish the boundaries
of a subject literature, with the limitation thatcitationsdonot reflect all
literature use. The value of this method in the determination of current
policies is a function of the extent to which these data can be projected
forward in time. Bibliographic coupling and cocitation have been used
to create mappings of the micro- and macrostructures and relationships
of discipline^.^^ Small, for example, has used cocitation analysis to
explore the relationship of information science to the social sciences.60
2.“Type of literature” studies. Citation analysis can be used to
gauge the dissemination of results reported in certain types of literature,
such as government documents, dissertations, or the exchange literature
of regional scientific societies.61The source of citations used for analysis
clearly can determine the generality of one’s conclusions in this type of
study. Nelson, in a study of citations to art collection catalogs, remarks
that one must recognize the potential usefulness of what she terms
“self-styled” citation methodsa2In her case, citation analysis of the fine
arts nonserial literature was the appropriate approach. Such studies can
involve content analysis, documenting not only where but also how
certain types of literature have been used.
Citation Analysis
3 . User studies. Although studies in this category are descriptive,
they have implications for collection development and design of services. One approach is the analysis of reference lists in works written by
library users, e.g., term papers, theses/dissertations or technical reports,
in order to determine types of materials, age of materials, subject,
language, and whether locally owned.63An alternative approach is to
test a specific hypothesis about information use, e.g., scientific literature is little used by engineers, or academic researchers use different
information sources than practiti0ne1-s.~~
It should be noted that citation analysis can be used to compare user behavior today with user
behavior several years ago, with the understanding that citations donot
strictly parallel use.
4. Historical studies. Historical research using citation analysis is
based on a literary model of the scientific process.65 In this model
scientific work is represented by papers written and published to report
it, and relationships between discrete pieces of work are represented by
references in papers. Citations can be used to trace the chronology of
events, relationships among them, and their relative importance. Missing and implicit citations obviously pose problems for such an analysis. The subject of study may range from the influence of a single idea
(e.g., Smith’s investigation of the influence of Vannevar Bush’s memex
on subsequent research and development in information retrieval) to an
individual’s entire scientific career (e.g., Ruff’s study of Istvan
Kovacs).66Patent citation networks offer a novel technique for displaying the history of a technical
The changes in patterns of
cocitation from year to year can reveal something about the history of
ideas in a given specialty.mPatterns found through such an analysiscan
be validated through interviews with specialists and questionnaire surveys, as in Small’s longitudinal study of collagen re~earch.~’
cocitation context analysis has been proposed
the structure of paradigms, the consensual structure of concepts in a
5 . Communication fiatterns. Citations can be thought of as plausible indicators of scientific communication patterns. Although citation
linkages do not necessarily reflect social contacts, it is probable that
there is a certain amount of congruence between documental and social
structures. Of particular interest is the analysis of these patterns to
identify problem areas in communication. These could include linguistic isolation, limited dissemination of new ideas, and barriers between
basic and applied science or between specialists and the public at large.
Shepherd and Goode, for example, sought to determine whether
research workers quoted in newspapers were really representative of
their respective fields.71 They examined whether authors quoted in
newspapers were also highly cited by their peers.
6. Eualuative bibliometrics. In these studies, citation analysis is
defined as the evaluation and interpretation of the citations received by
articles, scientists, universities, countries, and other aggregates of scientific activity, used as a measure of scientific influence and p r o d ~ c t i v i t y . ~ ~
Although there is much about the meaning of citation rates that is not
yet known (e.g., factors affecting rates, variation from field to field),
citation analysis is being used with increasing frequency as an evaluative tool by science administrator^.^^
7. Information retrieval. Use of citation relations has perhaps had
the greatest impact in information retrieval where citations have been
used to augment more traditional approaches to literature searching.
Experiments by Salton have confirmed that citations are useful supplements to keywords in identifying relevant documents.74Citation relations have been used in developing document representations, in
automatic classification, and in various retrieval algorithms which
make use of the ability to find “like” documents in the file independent
of words and language.75 Citations as a retrieval tool have the advantages that they are unaffected by changing terminology, they provide
access to interdisciplinary literature, and they reveal papers relevant to a
subject not found by using conventional indexes. Extensive use of
citations in computer-based retrieval has been hindered by a lack of
systems tailored specifically for citation manipulation. This may not
prove to be a barrier in the future, however. Yermish describes an
interactive information retrieval system which he developed to manipulate citation relations existing among bibliographic records effi~ i e n t l yEach
. ~ ~ document record has an associated REFLIST (list of all
documents that have been cited by a given document) and CITELIST
(list of all subsequent documents that cite a given document). These
allow one to use direct citation and citation coupling search modes in
addition to the more conventional keyword search. Two recent papers
describe the use of cocitation as a search strategy to retrieve documents
relevant to a given topic using commercially available search systems
and the citation index data bases.77Both cocited author and cocited
document searches are possible. Garfield has announced the pilot testing of BIOMED SEARCH, a retrieval system based on research front
specialties defined through cocitation clustering7’ Finally, O’Connor
has investigated procedures for the computer identification of citing
statements found in documents for which the full text is available in
machine-readable form, so thata retrieved set couldinclude not only the
identification of citing documents but also the citing statements them-
Citation Analysis
selves. As citation relations are more actively exploited for literature
search purposes, it should be possible to develop a better understanding
of the reasons for success and failure in this application area.
8. Collection development. It is appropriate tobegin the discussion
of citation analysis as a tool for collection development with Cayless’s
observation that “the main purpose of quantitative measures is to
provide information on which to base qualitative judgments, not to
replace them.”s0 Citation analysis has been applied primarily to the
development of journal collections, where decisions to be made include:
to acquire or not acquire a particular title, to continue or discontinue a
subscription, to weed or not to weed a backset. Beginning with a study
by Gross and Gross published in 1927 which used citation frequency asa
measure of journal significance, citation analysis has been advocated as
a tool in journal evaluation.” This application has not been without
critics. Brodman was perhaps the first to test the assumptions which
underly the method: (1) the value of a periodical to a professional
worker is in direct proportion to the number of times i t is cited in the
professional literature; (2) the journal(s) used as a source of citations
is(are)representative of the entire field; and (3)if more than one journal
is used as a source of citation data, all can be weighted equally?’ She did
not find support for these assumptions, and concluded that results of the
method should be used with caution. Others question journal rankings
by citation counts because such rankings may bear little relation to the
frequency of journal use in a particular library, as citation analysis and
use analysis measure different activities.m The difference in results of
use studies in different libraries suggests the limited value of a generalized technique such as citation analysis. In addition, there is the problem of noncited journals, such as trade and technical journals and
professional magazines.&9Line and Sandison discourage the use of
citation counts, instead advocating journal uses per unit of expenditure
(purchase, processing, binding, storage) as a basis for selection and
journal uses per unit of shelf space occupied as a basis for discarding.=
In spite of these criticisms, there is still a place for citation analysis
as a tool in collection development. Even though he disapproves of the
use of citation analyses in general, Line does acknowledge three uses to
which ranked lists derived from citation counts can be put: (1) highly
ranked journals not available locally and within subject scope are worth
examining in more detail; (2)low-ranked journals that are taken locally
should likewise be examined; and (3) lists based on source journals in a
particular subject can indicate journals outside of that subject which
may not yet have been acquired but may be valuable for local users.86In
his review of the applications of citation analysis to library collection
building, Broadus concludes that in the absence of highly expert subject
specialists on a library staff, citation studies can be of considerable value
in choosing serials and even mon~graphs.’~
Given the uncertainties
involved in using citation counts in isolation, i t is appropriate to
consider their use in combination with other measures, as in the model
for journal selection which gives highest priority to journals found to be
highly cited, abstracted and used.B8Although a tool like JCR gives
citation rankings based on a large body of literature, librarians may also
analyze citations found in their users’ publications, as described above
under “user studies.” Kriz, for example, analyzed reference lists in
engineering theses.’’ Finding books to be more frequently used than
journals, he shifted funds from journal subscriptions to purchase more
books. Citations are indicators of use, but there is probably a need for
multiple indicators, as demand does not strictly parallel citation. Many
materials are borrowed and read but not cited; authors who cite are only
a subset of the total reading public. Other measures of use such as
in-house use, circulation and interlibrary loan can be used’to supplement citation analysis in developing a more comprehensive view of user
needs as a basis for collection development.
Future Developments
Thus far this paper has described the uses, as well as abuses, of
citation analysis. Given the increasing availability of raw material for
citation analysis (asA&HCIjoins SSCI and SCI)and the development of
computer systems with which to manipulate these data easily, it is safe
to predict that citation analysis will continue to be a commonly used
technique. But the large number of studies using citation indexes has
led one critic to remark that uses of citation indexes other than for
literature searching seem to be examples of Kaplan’s law of the instrument: “Give a small boy a hammer, and he will find that everything he
encounters needs pounding. ”90 Superficially, citation analysis appears
to be a simple technique to apply, and there is a danger that it will fall
into disrepute through uncritical or overenthusiastic use. As with any
methodology, citation analysis produces results whose validity is highly
sensitive to the skill with which it is applied.
The critique of citation analysis in this paper outlined theassumptions often made and the problems which arise in data collection. In
order to better understand the possibilities and limitations of citation
analysis, more studies which test the assumptions and explore the
problem areas are needed. Another way to strengthen studies using
Citation Annlysis
citation analysis is to apply multiple methods in the study of a phenomenon, as in the coupling of citation analysis and contentanalysis. As no
research method is without bias, citation analysis should be supplemented by methods testing the same variables but having different
methodological weaknesses. For example, to investigate communication patterns among scientists, one could supplement citation data with
those obtained via interview or questionnaire.
Not enough is known about the “citation behavior” of authorswhy the author makes citations, why he makes his particular citations,
and how they reflect or do not reflect his actual research and use of the
literature. When more is learned about the actual norms and practices
involved, we will be in a better position to know whether (and it what
ways) it makes sense to use citation analysis in various application
areas.91 It would also be interesting to study in more detail the characteristics of documents which do not cite and/or are not cited, and to
identify characteristics of documents which can be used to predict
Advances in theory and practice have marked the development of
citation analysis, and researchers are likely to continue contributing in
both these areas. Gilbert, for example, has proposed a theory of citing
which views referencing as persuasion.93 In practice, simple citation
counts have been supplemented by bibliographic coupling, cocitation
analysis, evaluative bibliometrics, and cocitation context analysis. Garfield recently noted that one of the major methodological changes in his
studies in the near future will be to shift from counting citations to
counting “authors influenced by.”91
To conclude this paper, two questions affecting the future of citation analysis will be posed. Is i t possible that increased use of citation
analysis will cause a change in citation behavior? How will citation
behavior be affected by the increasedbse of electronic media for generation, storage and dissemination of information? Although both questions have already received some attention in the literature, the
responses to them are necessarily somewhat speculative.
It has been suggested that the very existence of citation indexes and
the growing abundance of citation analyses will likely have various
feedback influences on the writing and citing habits of future authors.%
Just as authors may title their papers more carefully to ensure their
retrievability through keyword indexes, authors could be motivated to
acknowledge their intellectual debts to prior documents accurately, lest
their papers go undetected by the user of a citation index. Thus this
paper is titled “Citation Analysis” rather than the more metaphorical
“Standing on the Shoulders of Giants,” and care has been taken to
reference accurately works by Garfield, Small and other key researchers
in citation analysis, as well as to include one self-citation. In an article
on the ethics of scientific publication, Price asserts that now that citations to previous work have become a valuable tool for literature
indexing, referees and editors should summarily reject bibliographies
that are either insufficient or padded.% Fears have been expressed
regarding the possibilities for abuse: “[Ilt might create a bandwagon
effect whereby authors who wish their document to be used will cite, and
try to get cited by, the most popular documents. This would be an
aberration, a disease of the information ~ystem.”~’
Whether or not such feedback influences are felt, other changes are
likely tocome with the increased use of electronic media for information
handling. The first question which arises is the form of bibliographic
references for material available in machine-readable form. Proposals
have already been put forward for both data files and computer conference comments.98 Questions of quality control, accessibility and
author’s permission must be addressed before the latter can be handled
as conventional publications. Whether the technological chhnges available to the next generation of researchers will undermine the role of the
paper in the process of scholarship remains to be seen. What is already
available are information facilities for electronic publishing and document handling such as the Xanadu Hypertext System.* The basic unit
of this service is the windowing document. With the full text of documents available in machine-readable form, a reader may either explore a
document or step through the window to explore the next document,
such as one referred to in a footnote. After exploring a further document,
the reader may return to the one that showed him to it, or proceed on
tangents that become available. Thus the links which citations represent are converted to electronic form, and new possibilities for citation
analysis arise. One can also imagine the use of graphics devices for the
display of citation networks and cluster maps.
This paper began with a quotation from Newton, the image of
science advancing by “standing on the shoulders of giants.” In fact: “the
process by which the boundaries of knowledge are advanced, and the
structure of organized science is built, is a complex process
indeed....[T]he whole effort is highly unorganized. There are no direct
orders from architect or quarrymaster. Individuals and small bands
proceed about their businesses unimpeded and uncontrolled, digging
where they will, working over their material, and tucking i t into place
in the edifice.”’00Perhaps the greatestpotential contributionof citation
analysis lies in the new insights which it can offer into this process. It is
Citation Analysis
a process which concerns not only scientists and sociologists of science,
but also those who work with the literature of science.
1 . Isaac Newton. Quoted in Robert K. Merton. On the Shoulders of Giants: A
Shandean Postscript. New York: Free Press, 1965.
2. Ziman, John M. Public Knowledge: A n Essay Concerning the Social Dimension
of Science. Cambridge: Cambridge LJniversity Press, 1968, p. 58.
3. Narin, Francis. et al. Evaluative Bibliometrics: T h e Use of Publication and Citation Analysis in the Evaluation of Scientific Activity. Chemy Hill, N. J.: Computer
Horizons, Inc., 1976, pp. 334, 337. (PB 252 339)
4. Malin, Morton V. “The Science Citation Index :A New Concept in Indexing.”
Library Trends 16(Jan. 1968):376.
5. Gupta, B.M., and Nagpal. M.P.K. “Citation Analysisand Its Applications: A Review.” Herald of Library Science 18(Jan.-April 1979):86-93:Hall, Angela M. T h e Useand
Value of Citations: A State-of-the-Art Report. London: Information Service in Physics,
Electrotrchnology and Control, 1970 (R70/4); Hjerppe, Roland. “An Outline of Bibliometrics and Citation Analysis” (TRITA-LIB-6014). Stockholm: Royal Institute of Technology Library, 1978. (ED 167 077); Martyn, John. “Citation Analysis.” Journal of
Documentation 31(Dec. 1975):290-97; Miller, Elizabeth, and Truesdell, Eugenia. “Citation Indexing: History and Applications.” Drexel Library Quarterly 8(April 1972):15972; and Mitra, A.C. “The Bibliographical Reference: A Review of Its Role.” Annals of
Library Science and Documentation 17(.Sept.-Dec. 1970):117-23.
6. Hjerppe, Roland. “A Bibliography of Bibliornetrics and Citation Indexing &
Analysis” (TRITA-LIB-2013). Stockholm: Royal Institute of Technology Library, 1980.
7. Garfield, Eugene. Citation Indexing-Its Theory and Application in Science,
Technology, and Humanities. New York: Wiley, 1979.
. Essays of a n Znformation Scientist, 3 vols. Philadelphia: Institute
for Scientific Information Press, 1977, 1980.
. “Can Citation Indexing Be Automated?” In Statistical Association
Methods for Mechanized Documentation (NBS Misc. Pub. 269),edited by Mary E. Stevens,
et al., p. 189. Washington, D.C.: National Bureau of Standards, 1965.
10. Bavelas, Janet B. “The Social Psychology of Citations.” Canadian Psychological
Review lS(Apri1 1978):lO.
11. Kochen, Manfred. Principles of Information Retrieval. L o s Angeles: Melville,
1974, p. 74.
12. Webb, Eugene J.. et al. Unobtrusive Measures: Nonreactive Research in the
Soczal Sciences. Chicago: Rand McNally, 1966.
13. Pinski, Gabriel, and Narin, Francis. “Citation Influence for Journal Aggregates
of Scientific Publications.” Information Processing and Management 12(1976):297-312.
14. Kessler, M.M. “An Experimental Study of Bibliographic Coupling Between
Technical Papers.” ZEEE Transactions on Znformation Theory IT-9(Jan. 1963):49-51;
. “Bibliographic Coupling Between Scientific Papers.” American Docuand
mentation 14(Jan. 1963):lO-25.
15. Marshakova, I.V. “ASystemof Document LinksConstmctedon the Basisof Citations (According to the ‘Science Citation Index’).’’ Automatic Documentation and
Mathematical Linguistics 7( 1973):49-57. (English translation of article in Nauchno
Tekhnicheskaya Informatsiya Seriya 2, no. 6 , pp. 3-8, 1973); and Small, Henry. “CoCitation in the Scientific Literature: A New Measure of the Relationship Between Two
Documents.” Journal of the ASZS 24(July-Aug. 1973):265-69.
16. Garfield, Eugene, et al. “Citation Data as Science Indicators.” In Toward a
Metric of Science: The Advent of Science Indicators, edited by Yehuda Elkana, et al., p.
185. New York: Wiley, 1978.
17. Weinberg, Bella H. “Bibliographic Coupling: A Review.” Znformation Storage
and Retrieval 10(May-June 1974):189-96;and Bellardo, Trudi. “The Use of Co-Citations
to Study Science.” Library Research 2(Fall 1980):231-37.
18. Small, Henry, and Griffith, Belver C. “The Structure of Scientific Literatures. I:
Identifying and Graphing Specialties.” Science Studies 4(Jan 1974):17-40.
19. Weinstock, Melvin. “Citation Indexes.” In Encyclopedia of Library and Information Science, vol. 5, edited by Allen Kent, et al., pp. 16-40. New York: Marcel Dekker,
20. Garfield, Eugene. “The New IS1 Journal Citation Reports Should Significantly
Affect the Future Course of Scientific Publication.” In
, Essays, vol. 1, pp.
21. Small, Henry, and Greenlee, Edwin. A Citation and Publication Analysisof U S .
Zndustrial Organimtions. Philadelphia: Institute for Scientific Information, 1979.
22. Abt, Helmut A. “The Cost-Eftectivenessin Terms of Publications and Citations
of Various Optical Telescopesat the Kitt Peak National Observatory.”Publicationsof the
Astronomical Society of the Pacific 92(June 1980):249-54.
23. Griffith, Belver C., et al. “On the Useof Citations in Studying Scientific Achievements and Communication.” Society for Social Studies of Science Newsletter P(Summer
1977):9-13; and Garfield, Citation Indexing, pp. 240-52.
24. Foskett, Anthony C. The Subject Approach to Information. 3d ed. Hamden,
Conn.: Linnet Books, 1977, p. 52.
25. Bottle, R.T. “Information Obtainable from Analyses of Scientific Bibliographies.” Library Trends 22(July 1973):71.
26. Davies, David. “Citation Idiosyncrasies,” letter to the editor. Nature 228(26 Dec.
27. Edwards, Shirley A., and McCamey, Michael W. “Measuring the Performance of
Researchers.” Research Management 16(Jan. 1973):34-41.
28. Bayer, Alan E.,and Folger, John. “Some Correlatesof a Citation Measureof Productivity in Science.” Sociology of Education 39(Fall 1966):381.
29. Thorne, Frederick C. “The Citation Index: Another Case of Spurious Validity.”
Journal of Clinical Psychology 33(0ct. 1977):1157-61.
30. Virgo, Julie A. “A Statistical Procedure for Evaluating the Importance of Scientific Papers.” Library Quarterly 47(0ct. 1977):415-30;McAllister, Paul R., et al. “Comparison of Peer and Citation Assessment of the Influence of Scientific Journals.” Journal of
the ASIS 31(May 1980):147-52;and Smith, Richard, and Fiedler, Fred E. “The Measwement of Scholarly Work: A Critical Review of the Literature.” Educational Record
52(Summer 1971):225-32.
31. Soper, Mary E. “Characteristics and Use of Personal Collections.” Library Quarterly 46(Oct 1976):397-415.
32. Coodell, Rae. The Visible Scientists. Boston: Little, Brown, 1977, p. 4.
33. Barlup, Janet. “Relevancy of Cited Articles in Citation Indexing.”Bulletin of the
Medical Library Association 57(July 1969):260-63.
34. Garfield, Eugene. “Citation Indexes for Science.” Science 122(15 July 1955):108.
35. Martyn, John. “Bibliogmphic Coupling.” Journal of Documentation 20(Dec.
36. Bertram, Sheila J.K. “The Relationship Between Intra-Document Citation Location and Citation Level.” Ph.D. diss., University of Illinois at Urbana-Champaign. 1970.
37. Herlach, Geruud. “Can Retrieval of Information From Citation Indexes Be Simplified?” Journul of fhe ASZS 29(Nov. 1978):308-10.
38. Voos, Henry,and Dagaev, Katherine S. “Are All Citations Equal? Or, Did We Op.
Cit. Your Zdcm?” Journal of Academic Librarianship l(Jan. 1976):19-21.
39. Tagliacouo, Renata. “Self-Citations in Scientific Literature.’’ Journal of Documentation 33(Dec. 1977):251-65.
Citation Analysis
40. Small, Henry G. “Cited Documents as Concept Symbols.” Social Studies of
Science 8(Aug. 1978):?27.
41. Lipetz, B e n - h i . “Improvement of the Selectivity of Citation Indexes to Science
Literature Through Inclusion of Citation Relationship Indicators.” American Documentation 16(April 1965):81-90.
42. Chubin, Daryl E., and Moitra, Soumyo D. “Content Analysis of References: Adjunct or Alternative to Citation Counting?” SocialStudiesofScience5(Nov.
Frost, Carolyn 0.“The Use of Citations in Literary Research A Preliminary Classification of Citation Functions.”Library Quarterly 49(0ct. 1979):399-414;Moravcsik, Michael
J.: and Murugesan, Poovanalingam. “Some Results on the Function and Quality of
Citations.” Social Studies of Science 5(Feb. 1975):86-92;Murugesan, Poovanalingam, and
Moravcsik, Michael J. “Variations of the Nature of Citation Measures with Journals and
Scientific Specialties.” Journal of the ASIS 29(May 1978):141-47;Oppenheim, Charles,
and Renn, Susan P. “Highly Cited Old Papers and the Reasons Why They Continue to he
Cited.” Journal of the ASIS 29(Sept. 1978):225-31;and Spiegel-Rosing, Ina. “Science
Studies: Bibliometric and Content Analysis.” Social Studies of Science 7(Feb. 1977):97113.
43. Small, “Cited Documents,” p. 328.
44. Small, Henry G. “Co-Citation Context Analysis.” Proceedings of the ASIS Annual Meeting 16(1979):270-75.
45. Cole, Jonathan, and Cole, Stephen. “Measuring the Quality of Sociological Research: Problems in the LJseof the Science Citation Index.” American Sociologist 6(Feh.
1971 ):23-29.
46. Long, J. Scott, c t al. “The Problcm of Junior-Authored Papers in Constructing
Citation Counts.” Social Studies of Science 10(May 1980):127-43.
47. Lindsey, Duncan. “Production and Citation Measures in the Sociology of
Science: The Problem of Multiple Authorship.” Social Studies of Science 10(May
48. Garfield, Eugene. “What’sin a Surname?” Current Contents 13(16Feb. 1981):5-9.
49. Line, Maurice B. “The Influence of the Type of Sources Used on the Results of
Citation Analyses.” Journal of Documentation 35(Dec. 1979):265-84.
50. Oromaner, Mark J . “The Audienre as a Determinant of the Most Irnportant
Sociologists.” American Sociologist 4(Nov. 1969):332-35.
51. Brittain, J. Michael, and Line, Maurice B. “Sources of Citations and References
for Analysis Purposes: A Comparative Assessment.” Journal ofhcumentation 29(March
52. Garfield, Eugene. “Will ISI’s Arts Q Humanities Citation Index Revolutionize
, ESSUYS,~ 0 13. , pp. 204-08.
Scholarship?” In
53. Bates, Marcia J. “A Criterion Citation Rate for Information Scientists.”Proceedings of the ASZS Annual Meeting 17(1980):276-78.
54. Boyce, Bert R., and Banning, Carolyn S. “Data Accuracy in Citation Studies.”
R Q 18(Summer 1979):349-50;and Goodrich, June E., and Roland, Charles G. “Accuracy
of Published Medical Reference Citations.” Journal of Technical Writing and Communication 7(1977):15-19.
55. Cawkell, A.E. “Citations as Sociological and Scientific Indicators-A Review.”
In EURIM 11: A European Conference on the Application of Research in Information
Seruices and Libraries, edited by W.E. Batten, pp. 31-39. London: Aslib, 1977.
56. Chen, Ching-Chih. ”A Citation Analysis of the Bulletin of the Medical Library
Association.” Bulletin of the Medical Library Association 65(April 1977):BO-92.
57. Friis, Th. “The Use of Citation Analysis as a Research Technique and Its Implications for Libraries.” South African Libraries 23(July 1955):12-15.
58. Nicholas, David, and Ritchie, Maureen. Literature and Biblzometrics. Hamden,
Conn.: Linnet Books, 1978.
59. Griffith, k l v e r C., et al. “The Structure of Scientific Literatures. 11: Toward a
Macro- and Microstructure for Science.” Science Studies 4(0ct. 1974):339-65.
60. Small, Henry. “The Relationship of Information Science to the Social Sciences:
A &-Citation Analysis.” Information Processing and Management 17(1981):39-50.
61. Gwhlert, Robert. “A Citation Analysis of International Organization: The Lise
of Government Documents.” Government Publications Review 6( 1979):185-93;O’Connor, Mary A. “Dissemination and Use of Library Science Dissertations in the Periodicals
Indexed in the Social Sciences Citation Index.”Ph.D. diss., Florida State [Jniversity, 1978;
and Gibson, Sarah S. “Some Characteristics of the Exchange Literature of Regional
Scientific Societies.” Library Research 2(Spring 1980-81):75-81.
62. Nelson, Diane M. “Methods of Citation Analysis in the Fine Arts.” Special Libraries 68(Nov. 1977):[email protected]
63. Mancall, Jacqueline C., and Drott, M. Carl. “Materials LJsed by High School
Students Preparing Independent Study Projects: A Bibliometric Approach.” Library
Research 1(Fall 1979):223-36;Popovich, Charles J. “The Characteristics of a Collection
for Research in BusinessIManagement.” College & Research Libraries 39(March
1978):llO-17; and Hockings, E.F. “Selection of Scientific Periodicals in a n Industrial
Research Library.” Journal of the ASIS 25(March-April 1974):131-32.
64. Waldhart, Thomas J. “Utility of Scientific Research: The Engineer’s Use of the
Products of Science.” IEEE Transacfions on Professional Communicafion PC-17(June
1974):33-35;and Culnan, Mary J. “An Analysis of the Information Usage Patterns of
Academics and Practitioners in the Computer Field.” Information Processing and Management 14(1978):395-404.
65. GarfieId, Citation Indexing, p. 81.
66. Smith, Linda C. “‘Memex’ as an Image of Potentiality in Information Retrieval
Research and Development.” In Information Retrieval Research. London: Butterworths,
1981; and Ruff, Imre. “Citation Analysis of a Scientific Career.” Social Studies of Science
9(Feb. 1979):81-90.
67. Ellis, P., et al. “Studies on Patent Citation Networks.” Journal of Documenfation 34(March 1978):lZ-20.
68. Small, Henry G. “Structural Dynamirs of Scientific Literature.” International
Classification 3( 1976):67-74.
. “A Co-Citation Model of a Scientific Specialty.” Social Studies of
Science 7(May 1977):189-66.
- “A &-Citation
Context Analysis and the Structure of Paradigms.”
70. ~
Journal of Documentation 36(Sept. 1980):183-96.
71. Shepherd, Robert G., and C d e . Erich. “Scientists in the Popular Press.’’New
Scientist 76(24 Nov. 1977):482-84.
72. Narin, “Evaluative Bibliometrics,” p. 334.
73. Aaronson, Steve. “The Footnotesof Science.” Mosaic G(March/April 1975):22-27;
and Wade, Nicholas. “Citation Analysis: A New Tool for Science Administrators.”
Science 188(2 May 1975):429-32.
74. Salton, Gerard. “Associative Document Retrieval Techniques Using Bibliographic Information.” Journal of the ACM lO(0ct. 1963):440-57.
75. Gray, W.A., and Harley, A.J. “Computer Assisted Indexing.” Information
Storage and Retrieval ~ ( N o v1971):167-74;
Kwok, K.1,. “The UseofTitleandCitedTitles
as Document Representation for Automatic Classification.” Information Processing and
Management 11(1975):201-06;Price, Nancy, and Schiminovich, Samuel. “A Clustering
Experiment.” Information Storage and Retrieval 4(Aug. 1968):271-80; Schiminovich,
Samuel. “Automatic Classification and Retrieval of Documents by Means of a Bibliographic Pattern Discovery Algorithm.” Information Storage and Relrieual 6(May
1971):417-35;Bichteler, Julie, and Parsons, Ronald G. “Document Retrieval by Means of
an Automatic Classification Algorithm for Citations.” Information Storageand Retrieval
10(July/Aug. 1974):267-78; Birhteler, Julie, and Eaton, Edward A. “Comparing Two
Algorithms for Document Retrieval Using Citation Links.” Journal of the ASZS 28(July
. “The Combined Use of Bibliographic Coupling and
Cocitation for Document Retrieval.” Journal of the ASIS 31( July 1980):278-82.
Citation Analysis
76. Yermish, Ira. “A Citation-Based Interactive Associative Information Retrieval
System.” Ph.D. diss., University of Pennsylvania, 1975.
77. Chapman, Janet, and Subramanyarn, K. “Cocitation Search Strategy.”
In National Online Meeting Proceedings-1981, compiled by Martha E. Williams and
Thomas H. Hogan, pp. 97-102. Medford, N.J.: Learned Information, 1981; and White,
Howard D. “Cocited Author Retrieval Online: An Experiment with the Social Indicators
Literature.” Journal of the ASIS 32(Jan. 1981):16-21.
78. Garfield, Eugene. “ISI’s On-line System Makes Searching So Easy Even a Scientist Can Do It.” Current Contents 13(26 Jan. 1981):5-8.
79. O’Connor, John. “Citing Statements: Recognition by Computer and Use to
Improve Retrieval.” Proceedings of the ASIS Annual Meeting 17(1980):177-79.
80. Cayless, C.F. “Journal Ranking and Selection,” letter to the editor.
Journal of Documentation 33(Sept 1977):243.
81. Gross, P.L.K., and Gross, E.M. “College Libraries and Chemical Education.”
Science 66(28 Oct. 1927):385-89;and Garfield, Eugene. “Citation Analysis as a Tool in
Journal Evaluation.” Science 178(3 Nov. 1972):471-79.
82. Brodman, Estelle. “Choosing Physiology Journals.” Bulletin of the Medical
Library Association 32(0ct. 1944):479-83.
83. Pritchard, Alan. “Citation Analysis vs. Use Data,” letter to the editor. Journal of
Documentation 36(Sept. 1980):268-69.
84. Singleton, Alan. “Journal Ranking and Selection: A Review in Physics.” Journal
of Documentation 32(Dw. 1976):258-89.
85. Line, Maurice B., and Sandison, Alexander. “Practical Interpretation of
Citation and Library Use Studies.” College dr Research Libraries 36(Sept. 1975):393-96.
86. Line, Maurice B. “On the Irrelevance of Citation Analyses to Practical Librarianship.” In EURIM I I , pp. 51-53; and
. “Ranked Lists Based on
Citations and Library Uses as Indicators of Journal Usage in Individual Libraries.”
Collection Management 2(Winter 1978):313-16.
87. Broadus, Robert N. “The Applications of Citation Analyses to Library Collection Building.” Advances in Librarianship 7( 1977):328.
88. Dhawan, S.M.. et al. “Selection of Scientific Journals: A Model.”
Journal of Documentation %(March 1980):24-32.
89. Kriz, Harry M. “Subscriptions vs. Books in a Constant Dollar
Budget.” College dr Research Libraries 39(March 1978):105-09.
90. See Bavelas, Janet B. “Comments on Buss’sEvaluation of Canadian Psychology
Departments.” Canadian Psychological Review 17(Oct.1976):303;and Kaplan, Abraham.
The Conduct of Inquiry: Methodology for Behavioral Science. San Francisco: Chandler,
1964, p. 28.
91. Kaplan, Norman. “The Norms of Citation Behavior: Prolegomena to the
Footnote.”American Documentation 16(July 1965):179-84.
92. Ghosh, Jata S.,and Neufeld, M. Lynne. “Uncitednessof Articles in the Journal of
the American Chemical Society.” Information Storage and Retrieval lO(Nov./Ikc.
1974):365-69; Ghosh, Jata S. “Uncitedness of Articles in Nature, A Multidisciplinary
Scientific Journal.” Information Processing and Management 1I( 1975):165-69;Garfield,
, Essays,
Eugene. “Uncitedness 111-The Importance of Not Being Cited.” In
vol. 1, pp. 413-14; and Kuch, T.D.C. “Predicting the Citedness of Sientific Papers:
Objective Correlates of Citedness in the American Journal of Physzology.”Proceedings of
the ASIS Annual Meeting 15(1978):185-87.
93. Gilbert, G. Nigel. “Referencing as Persuasion.” Social Studies of Science 7(Feb.
94. Garfield, Eugene. “Is Information Retrieval in the Arts and Humanities
Inherently Different From That in Science?” Library Quarterly 50(Jan. 1980):56.
95. Margolis, J. “Citation Indexing and Evaluation of Scientific Papers.”
Science 155(10 March 1967):1213-19.
96. Price, Derek J. de Solla. “Ethics of Scientific Publication.” Science
144(8 May 1964):655-57.
97. Kochen, Principles of Information Retrieval, p. 82.
98. Dodd, Sue A. “Bibliographic References for Numeric Social Science
Data Files: Suggested Guidelines.” journal of the ASIS SO(March 1979):77-82;and Crickman, Robin D. “The Form and Implications of Bibliographic Citations to Computer
Conference Comments.” Proceedings of the ASIS Annual Meeting 15( 1978):86-88.
99. Nelson, Theodor H. Computer Lib/Dream Machines. South Bend, Ind.:
Ted Nelson, 1980, p. DM 41.
100. Bush, Vannevar. “The Builders.” Technology Review 47(Jan. 1945):162.
Fly UP