Sign in or 
Share this
Citation Indexes, Impact Factor, and Page Rank
Citation Indexes, Impact Factor, and Page Rank are all commonly used in the sciences as ways of determining the relative importance of works of scholarship. All employ the use of citations to ascertain a particular work's importance, but in different ways. Of course, along with being helpful in the quick identification of useful articles, these practices are not without their flaws. The following is an introduction to the varying ways of determing a work's importance, as well as an explanation of why these systems have not been widely adopted in the arts and humanities.
Citation Indexes
When academics and scholars write articles and papers, they cite the resources they used in bibliographies. Citation Indexes track these citations and compile statistics that allow researchers to see which articles have cited which previous articles.
Researchers can use this information in a number of ways. Following the citations in an article or paper will lead a researcher to further resources on that topic. Likewise, a researcher can look up a particular paper in a citation index to find out authors who have cited it. Also, by using a citation index, a researcher can discover which authors and which journals are most often cited in works in the field. Thisis helpful in determining who is having the biggest effect, or impact, on that field.
For additional information on citation indexes and how to use them, the University of Texas at Austin and LibraryYale University Science Library both offer useful guides.
Two of the most notable publishers of citations indexes are Elsevier, who publish Scopus, and Thomson Scientific, who publish three indexes: Science Citation Index (SCI), Social Science Citation Index (SSCI), and Arts & Humanities Citation Index (AHCI). These indexes are available through library subscriptions. Google Scholar also provides citation count in each entry of a search result, and is freely available without a subscription.
For all of the benefit that citation indexes provide, they do not represent academic literature to its full extent. Not all journal articles are subsequently cited by other scholars. In her article, “Assessing the Value of a Journal Beyond the Impact Factor,” Anita Coleman asserts: “[T]he reality is that papers that are not cited are essentially fulfilling other roles; that is, while they may not visibly add to the communal growth of knowledge they may contribute other benefits that offset costs."
Coleman continues, "It is important to discover and measure these benefits especially in the face of large scale patterns of uncitedness. In [Library and Information Science] the rate of uncitedness is estimated at 72% . . . " Coleman recognizes that while statistics such as impact factors are one method of measuring the significance of a scholarly article or journal, other factors also contribute to the value of research publications and output.
Impact Factor
The concept of impact factor was developed by Dr. Eugene Garfield and Dr. Irving H. Sher in the early 1960s. Putting this idea into practice, Dr. Garfield went on to found the Institute for Scientific Information, now a part of Thomson Scientific. Thomson Scientific publishes these impact factors in Journal Citation Reports available through their database ISI Web of Knowledge.
A journal’s impact factor is calculated to show how frequently an article from that journal is cited in a given year. The impact factor, in the scientific and social science community, is used to evaluate a given journal’s “relative importance” in its field. The higher the impact factor, the more citations a journal has generated. Journals with many cited citations are consequently viewed as having the most prominence.
How is the impact factor calculated?
There are two factors used in this calculation, which uses data from the previous two years. First, the number of cited articles for a journal are tallied for a given year. Second, the number of articles published in this same journal are figured for that year. Data from two years is added together, then the number of cited articles are divided by the number of published articles.
A 2008 impact factor would look like this for a hypothetical journal:
These two totals are then divided (155/96) and the impact factor comes out to 1.615.
This means on average this journal over the past two years has had a published article cited just over one and a half times. Because the impact factor takes into account the total number of articles a journal publishes, as well as the number of times it is cited, smaller journals or those published less often are on a level playing field with larger journals or journals published more often. Researchers who may not be intimately familiar with a subject area can use the impact factor of a journal to make a quick determination of a particular journal’s importance to the field.
While it is important to note that impact factor does not reveal anything about the quality of the research presented in a journal, studies have been conducted that compare impact factors with perceptions of journal quality. A study published in the Journal of the Medical Library Association compared impact factors against quality rankings assigned by professionals within the field. The study designers determined that the results seemed to correlate – the same journals with significant impact factors also scored well in the more subjective survey – but recommend further research (Somnath).
In a 2005 presentation to the International Congress on Peer Review and Biomedical Publication titled “The Agony and the Ecstasy – The History and Meaning of the Journal Impact Factor,” Eugene Garfield himself addressed criticisms of impact factors while emphasizing their accuracy and significance. Responding to criticism that impact factors are based on a sampling of the literature, he reaffirms that a Journal Citation Report (including the impact factor) is based on every citation in every journal that it covers. (Garfield 5).
Another shortcoming of impact factors is the inconsistent coverage of international journals in the ISI Web of Knowledge. Researchers in Spain found that impact factors for Spanish research publications did not fully recognize all citations (Bordons).
Page Rank
Finding relevant information in the vast online universe can be somewhat difficult at times given the sheer volume of pages available. So, the creators of Google devised Page Rank, which is a system that ranks web pages using a sophisticated algorithm to help match user queries with content found on the web. It remains the secret behind Google's success as the world's foremost search engine. Google's aims are to search "more sites more quickly, delivering the most relevant results," and Page Rank is the means by which Google hopes to achieve that end. Most search engines use text matching to find results, but Page Rank also establishes a hierarchy of importance to results so users spend less time with irrelevant retrievals.
How does Page Rank decide what is relevant?
The inner workings of Page Rank get a little complicated. David Austin's article "How Google Finds Your Needle in the Web's Haystack," on the American Mathematical Society's website helps explain things in mathematical terms. According to Austin, there are several aspects to a site's Page Rank score. A website is judged for its popularity by determining how many other pages link to the page in question. The linking pages are also assessed for their importance, and if the linking pages are popular, then the importance of the page being examined will increase. Page Rank also considers the number of links on a page--if a page only has a few links on it those links are seen as more important.
The advantages of the Page Rank system are many, as Sergey Brin and Lawrence Page argue in "The Anatomy of a Large-Scale Hypertextual Web Search Engine." Brin and Page assert that Google's system is more objective than systems like impact factor. Additionally, they state that Page Rank brings up results that make more intuitive sense--if a website is well regarded and linked by other reputable sources, chances are the user will want to see that document over other untested sources. The results list from a query with Page Rank will perhaps have a higher rate of precision than other page retrieval systems.
Of course, Page Rank isn't without flaw. In "Page Rank Uncovered" by Chris Ridings and Mike Shishigin, the pair argue that one link isn't necessarily equal to a vote in a website's favor--that same link could be reciprocal deals or even spam links on a page. Perhaps the biggest weakness of Page Rank, though, is its ability to be manipulated. While manipulation of Page Rank may not be ethical, there are scores of websites available to explain how to do it. This practice is also known as Google Bombing.
Why haven't these systems -- Citation Indexes, Impact Factor and Page Rank -- been widely adopted in the arts and humanities?
One reason why citation indexes are used in the sciences and not in the arts and humanities is the comparative importance of currency of information to both fields. In the sciences, because research advances so quickly, information can quickly become out of date. Citation indexes can help a researcher identify what authors and what journals are publishing the most current and cutting-edge research, and which articles are currently the most cited and having the most impact on the field. A researcher can also determine if a paper has fallen out of date, if it has not been cited in quite some time.
This is not quite the case within the humanities. Information and research is not as likely to go out of date as quickly, if at all. A paper written in the 1950s may still be as relevant today as it was when it was first published, though this is hardly ever the case in the sciences. The impact factor is less likely to be a useful piece of information when judging published articles in the humanities field, as it is for the sciences, where currency of the information is very important.
Another reason is the use of citations. In the sciences, research is built upon previous research, and thus it is vital that the previous research be cited so others can look at these building blocks. In research in the arts and humanities, however, citation practices are not as strict. For example, a particular journal article may influence other articles, even if no actual citation is given. This kind of influence would not be measured in an impact factor or citation index.
Works Cited
Austin, David. "How Google Finds Your Needle in the Web's Haystack." American Mathematical Society. 23 April 2008. <http://www.ams.org/featurecolumn/archive/pagerank.html>.
Bordons, Maria, et al. “Advantages and Limitations in the Use of Impact Factor Measures for the Assessment of Research Performance.” Scientometrics 53.2 (2002): 195 – 206. Full text available online through Google Scholar.
Brin, Sergey and Lawrence Page. "The Anatomy of a Large-Scale Hypertextual Web Search Engine." 23 April 2008. <http://infolab.stanford.edu/~backrub/google.html>.
Coleman, Anita Sundaram. “Assessing the Value of a Journal Beyond the Impact Factor.” Journal of the American Society of Information Science and Technology 58.8 (2007): 1148-1161. Electronic preprint available from the Digital Library of Information Science and Technology.
Garfield, Eugene. "The Agony and the Ecstasy - The History and the Meaning of the Journal Impact Factor"
Presented at the International Congress on Peer Review and Biomedical Publication, Chicago, U.S.A. September 16, 2005. <http://www.garfield.library.upenn.edu/papers/jifchicago2005.pdf>.
Ridings, Chris and Mike Shishigin. "PageRank Uncovered." 23 April 2008. <http://www.voelspriet2.nl/PageRank.pdf>.
Somnath Saha, MD, MPH, et al. “Impact factor: a valid measure of journal quality?” Journal of the Medical Library Association 91.1 (2003): 42–46. Full text available online at PubMed Central.
Citation Indexes
When academics and scholars write articles and papers, they cite the resources they used in bibliographies. Citation Indexes track these citations and compile statistics that allow researchers to see which articles have cited which previous articles.
Researchers can use this information in a number of ways. Following the citations in an article or paper will lead a researcher to further resources on that topic. Likewise, a researcher can look up a particular paper in a citation index to find out authors who have cited it. Also, by using a citation index, a researcher can discover which authors and which journals are most often cited in works in the field. Thisis helpful in determining who is having the biggest effect, or impact, on that field.
For additional information on citation indexes and how to use them, the University of Texas at Austin and LibraryYale University Science Library both offer useful guides.
Two of the most notable publishers of citations indexes are Elsevier, who publish Scopus, and Thomson Scientific, who publish three indexes: Science Citation Index (SCI), Social Science Citation Index (SSCI), and Arts & Humanities Citation Index (AHCI). These indexes are available through library subscriptions. Google Scholar also provides citation count in each entry of a search result, and is freely available without a subscription.
For all of the benefit that citation indexes provide, they do not represent academic literature to its full extent. Not all journal articles are subsequently cited by other scholars. In her article, “Assessing the Value of a Journal Beyond the Impact Factor,” Anita Coleman asserts: “[T]he reality is that papers that are not cited are essentially fulfilling other roles; that is, while they may not visibly add to the communal growth of knowledge they may contribute other benefits that offset costs."
Coleman continues, "It is important to discover and measure these benefits especially in the face of large scale patterns of uncitedness. In [Library and Information Science] the rate of uncitedness is estimated at 72% . . . " Coleman recognizes that while statistics such as impact factors are one method of measuring the significance of a scholarly article or journal, other factors also contribute to the value of research publications and output.
Impact Factor
The concept of impact factor was developed by Dr. Eugene Garfield and Dr. Irving H. Sher in the early 1960s. Putting this idea into practice, Dr. Garfield went on to found the Institute for Scientific Information, now a part of Thomson Scientific. Thomson Scientific publishes these impact factors in Journal Citation Reports available through their database ISI Web of Knowledge.
A journal’s impact factor is calculated to show how frequently an article from that journal is cited in a given year. The impact factor, in the scientific and social science community, is used to evaluate a given journal’s “relative importance” in its field. The higher the impact factor, the more citations a journal has generated. Journals with many cited citations are consequently viewed as having the most prominence.
How is the impact factor calculated?
There are two factors used in this calculation, which uses data from the previous two years. First, the number of cited articles for a journal are tallied for a given year. Second, the number of articles published in this same journal are figured for that year. Data from two years is added together, then the number of cited articles are divided by the number of published articles.
A 2008 impact factor would look like this for a hypothetical journal:
| Cited Articles per Year: | Published Articles per Year: |
| 2007 = 75 | 2007 = 48 |
| 2006 = 80 | 2006 = 48 |
| Total: 155 | Total: 96 |
These two totals are then divided (155/96) and the impact factor comes out to 1.615.
This means on average this journal over the past two years has had a published article cited just over one and a half times. Because the impact factor takes into account the total number of articles a journal publishes, as well as the number of times it is cited, smaller journals or those published less often are on a level playing field with larger journals or journals published more often. Researchers who may not be intimately familiar with a subject area can use the impact factor of a journal to make a quick determination of a particular journal’s importance to the field.
While it is important to note that impact factor does not reveal anything about the quality of the research presented in a journal, studies have been conducted that compare impact factors with perceptions of journal quality. A study published in the Journal of the Medical Library Association compared impact factors against quality rankings assigned by professionals within the field. The study designers determined that the results seemed to correlate – the same journals with significant impact factors also scored well in the more subjective survey – but recommend further research (Somnath).
In a 2005 presentation to the International Congress on Peer Review and Biomedical Publication titled “The Agony and the Ecstasy – The History and Meaning of the Journal Impact Factor,” Eugene Garfield himself addressed criticisms of impact factors while emphasizing their accuracy and significance. Responding to criticism that impact factors are based on a sampling of the literature, he reaffirms that a Journal Citation Report (including the impact factor) is based on every citation in every journal that it covers. (Garfield 5).
Another shortcoming of impact factors is the inconsistent coverage of international journals in the ISI Web of Knowledge. Researchers in Spain found that impact factors for Spanish research publications did not fully recognize all citations (Bordons).
Page Rank
Finding relevant information in the vast online universe can be somewhat difficult at times given the sheer volume of pages available. So, the creators of Google devised Page Rank, which is a system that ranks web pages using a sophisticated algorithm to help match user queries with content found on the web. It remains the secret behind Google's success as the world's foremost search engine. Google's aims are to search "more sites more quickly, delivering the most relevant results," and Page Rank is the means by which Google hopes to achieve that end. Most search engines use text matching to find results, but Page Rank also establishes a hierarchy of importance to results so users spend less time with irrelevant retrievals.
How does Page Rank decide what is relevant?
The inner workings of Page Rank get a little complicated. David Austin's article "How Google Finds Your Needle in the Web's Haystack," on the American Mathematical Society's website helps explain things in mathematical terms. According to Austin, there are several aspects to a site's Page Rank score. A website is judged for its popularity by determining how many other pages link to the page in question. The linking pages are also assessed for their importance, and if the linking pages are popular, then the importance of the page being examined will increase. Page Rank also considers the number of links on a page--if a page only has a few links on it those links are seen as more important.
The advantages of the Page Rank system are many, as Sergey Brin and Lawrence Page argue in "The Anatomy of a Large-Scale Hypertextual Web Search Engine." Brin and Page assert that Google's system is more objective than systems like impact factor. Additionally, they state that Page Rank brings up results that make more intuitive sense--if a website is well regarded and linked by other reputable sources, chances are the user will want to see that document over other untested sources. The results list from a query with Page Rank will perhaps have a higher rate of precision than other page retrieval systems.
Of course, Page Rank isn't without flaw. In "Page Rank Uncovered" by Chris Ridings and Mike Shishigin, the pair argue that one link isn't necessarily equal to a vote in a website's favor--that same link could be reciprocal deals or even spam links on a page. Perhaps the biggest weakness of Page Rank, though, is its ability to be manipulated. While manipulation of Page Rank may not be ethical, there are scores of websites available to explain how to do it. This practice is also known as Google Bombing.
Why haven't these systems -- Citation Indexes, Impact Factor and Page Rank -- been widely adopted in the arts and humanities?
One reason why citation indexes are used in the sciences and not in the arts and humanities is the comparative importance of currency of information to both fields. In the sciences, because research advances so quickly, information can quickly become out of date. Citation indexes can help a researcher identify what authors and what journals are publishing the most current and cutting-edge research, and which articles are currently the most cited and having the most impact on the field. A researcher can also determine if a paper has fallen out of date, if it has not been cited in quite some time.
This is not quite the case within the humanities. Information and research is not as likely to go out of date as quickly, if at all. A paper written in the 1950s may still be as relevant today as it was when it was first published, though this is hardly ever the case in the sciences. The impact factor is less likely to be a useful piece of information when judging published articles in the humanities field, as it is for the sciences, where currency of the information is very important.
Another reason is the use of citations. In the sciences, research is built upon previous research, and thus it is vital that the previous research be cited so others can look at these building blocks. In research in the arts and humanities, however, citation practices are not as strict. For example, a particular journal article may influence other articles, even if no actual citation is given. This kind of influence would not be measured in an impact factor or citation index.
Works Cited
Austin, David. "How Google Finds Your Needle in the Web's Haystack." American Mathematical Society. 23 April 2008. <http://www.ams.org/featurecolumn/archive/pagerank.html>.
Bordons, Maria, et al. “Advantages and Limitations in the Use of Impact Factor Measures for the Assessment of Research Performance.” Scientometrics 53.2 (2002): 195 – 206. Full text available online through Google Scholar.
Brin, Sergey and Lawrence Page. "The Anatomy of a Large-Scale Hypertextual Web Search Engine." 23 April 2008. <http://infolab.stanford.edu/~backrub/google.html>.
Coleman, Anita Sundaram. “Assessing the Value of a Journal Beyond the Impact Factor.” Journal of the American Society of Information Science and Technology 58.8 (2007): 1148-1161. Electronic preprint available from the Digital Library of Information Science and Technology.
Garfield, Eugene. "The Agony and the Ecstasy - The History and the Meaning of the Journal Impact Factor"
Presented at the International Congress on Peer Review and Biomedical Publication, Chicago, U.S.A. September 16, 2005. <http://www.garfield.library.upenn.edu/papers/jifchicago2005.pdf>.
Ridings, Chris and Mike Shishigin. "PageRank Uncovered." 23 April 2008. <http://www.voelspriet2.nl/PageRank.pdf>.
Somnath Saha, MD, MPH, et al. “Impact factor: a valid measure of journal quality?” Journal of the Medical Library Association 91.1 (2003): 42–46. Full text available online at PubMed Central.
|
kkraus |
Latest page update: made by kkraus
, May 13 2008, 1:06 PM EDT
(about this update
About This Update
Moved from: Home
- kkraus
No content added or deleted. - complete history) |
|
Keyword tags:
citation indexes
impact factor
page rank
More Info: links to this page
|
There are no threads for this page.
Be the first to start a new thread.
