66:. According to the NRC, this analysis was "based on an indirect approach to determining what faculty value in a program" and was done by first asking a sample faculty group to rate a number of programs in their area, and then using a statistical analysis "to calculate how the 20 program characteristics would need to be weighted in order to reproduce most closely the sample ratings." In doing so, the rankings "attempted to understand how much importance faculty implicitly attached to various program characteristics when they rated the sample of programs." Weights were assigned to each of characteristic varied by field.
146:, one of the members of the NRC committee that produced the ranking, critiqued the final result. Cole objected to the committee's choice not to include any "measures of reputational standing or perceived quality" in the survey, which he called "the most significant misguided decision" in the recent study. Cole also critiqued the various statistical inputs and the weight assigned to each. The
121:. The Center for a Public Anthropology praised the National Research Council's 2010 rankings as "an impressive achievement" for its move away from reputational rankings and toward data-based rankings, but also noted that the lack of specific rankings reduced clarity even as it improved accuracy.
51:; rather, a statistical range was given. This was because "the committee felt strongly that assigning to each program a single number and ranking them accordingly would be misleading, since there are significant uncertainties and variability in any ranking process."
170:" in the rankings, concluding that "We caution against using the 2010 NRC data or metrics for any assessment-oriented study of research productivity." The rankings were also critiqued by sociologist Fabio Rojas.
77:-based: Faculty were "asked about the importance of 20 characteristics ... in determining the quality" of a type of program. Weights were assigned to determinant according to the results, varying by discipline.
38:
had conducted a survey, and compiled a report, on United States
Research-Doctorate Programs approximately every 10 years, although the time elapsed between each new ranking had sometimes exceeded 10 years.
93:
scores, graduate student funding, number of Ph.D.s and completion percentage, time to degree, academic plans of graduating students, student work space, student health insurance, and student activities.
154:
departments also expressed "serious concerns" about vaguely defined reporting terms leading to inconsistent data, inaccuracies in the data, and the use of bibliometrics from the
552:
47:
Data collection for the most recent report began in June 2006; it was released on
September 28, 2010. These rankings did not provide exact ranks for any university or
491:
Shortridge, Ashton; Goldsberry, Kirk; Weessies, Kathleen (2011), "Measuring
Research Data Uncertainty in the 2010 NRC Assessment of Geography Graduate Education",
187:
109:
stated that historically the NRC rankings have been the "gold standard" for academic department ratings. The rankings were also called "the gold standard" by
461:
Bernat, Andrew; Grimson, Eric (December 2011), "Doctoral program rankings for U.S. computing programs: the national research council strikes out",
89:), fraction of the faculty supported by grants and number of grants per faculty member, diversity of the faculty and students, student
297:
444:
35:
166:
A. Shortridge, K. Goldsberry, and K. Weessies found significant undercounts in the data and poor sensitivity to "
268:
406:
410:
386:
147:
429:
370:
131:
517:
327:
90:
17:
192:
210:
A Data-Based
Assessment of Research-Doctorate Programs in the United States: Frequently Asked Questions
366:"Too Big to Fail: How 'better than nothing' defined the National Research Council's graduate rankings"
344:
463:
424:
394:
118:
81:
The factors included in these computations included the number of publications per faculty member,
314:
281:
537:
294:
209:
155:
126:
222:
8:
159:
110:
63:
365:
251:
136:
122:
114:
167:
500:
472:
361:
151:
143:
532:
504:
390:
301:
238:
82:
74:
546:
477:
239:
A Data-Based
Assessment of Research-Doctorate Programs in the United States
163:
106:
86:
397:
Department of
Computer Science and Engineering, retrieved 2010-09-29.
48:
383:
425:"Computer Scientists Cry Foul Over Data Problems in NRC Rankings"
102:
The rankings have both been praised and criticized by academics.
188:"The Wait for the National Research Council Rankings Continues"
490:
295:
Ph.D. programs rank high in
National Research Council report
282:
CU awaits release of gold standard of grad school rankings
328:
Counterpoint: Doctoral-Program
Rankings—the NRC Responds
553:
University and college rankings in the United States
533:
NRC Assessment of
Research Doctoral Programs website
345:
A Critic Sees Deep
Problems in the Doctoral Rankings
204:
202:
158:despite its poor coverage of many computer science
212:" (2010). United States National Research Council.
538:1995 NRC graduate rankings (Statistical Analysis)
326:E. William Colglazier and Jeremiah P. Ostriker, "
304:" (September 28, 2010). University of California.
271:" (September 19, 2011). City College of New York.
199:
544:
460:
413:, September 28, 2010. Retrieved 2010-09-29.
284:" (September 16, 2010). Cornell University.
269:NRC Ranks CCNY PhD Program Among Best in US
445:"Dangers of Rankings with Inaccurate Data"
407:NRC Doctoral Rankings and Computer Science
476:
224:Assessment of Research Doctorate Programs
113:John M. Tarbell and in news releases by
442:
36:United States National Research Council
14:
545:
422:
384:Erroneous NRC Ranking Data for UW CSE
185:
54:Two series of rankings were offered:
520:(September 30, 2010). Orgtheory.net.
360:
317:." Center for a Public Anthropology.
85:(except in computer science and the
24:
25:
564:
526:
518:NRC Rankings: Was It a Big Fail??
423:Glenn, David (October 6, 2010),
510:
484:
454:
436:
416:
400:
377:
354:
337:
320:
411:Computing Research Association
307:
287:
274:
261:
244:
231:
215:
186:Morse, Robert (July 9, 2009),
179:
148:Computing Research Association
135:, responding to a critique by
42:
13:
1:
430:Chronicle of Higher Education
371:Chronicle of Higher Education
349:Chronicle of Higher Education
332:Chronicle of Higher Education
173:
132:Chronicle of Higher Education
129:defended the rankings in the
505:10.1080/00221341.2011.607510
193:U.S. News & World Report
97:
7:
10:
569:
443:Grimson, Eric (May 2010),
464:Communications of the ACM
227:, U.S. National Academies
83:citations per publication
395:University of Washington
347:" (September 30, 2010).
254:" (September 27, 2010).
119:University of California
478:10.1145/2043174.2043203
449:Computing Research News
330:." (October 17, 2010).
493:Journal of Geography
156:ISI Web of Knowledge
127:Jeremiah P. Ostriker
29:PhD program rankings
293:Andy Evangelista, "
111:biomedical engineer
64:regression analysis
389:2010-10-01 at the
364:(April 24, 2011),
300:2012-03-03 at the
137:Stephen M. Stigler
123:William Colglazier
115:Cornell University
362:Cole, Jonathan R.
16:(Redirected from
560:
521:
514:
508:
507:
488:
482:
481:
480:
458:
452:
451:
440:
434:
433:
420:
414:
409:, Peter Harsha,
404:
398:
381:
375:
374:
358:
352:
341:
335:
324:
318:
311:
305:
291:
285:
278:
272:
265:
259:
248:
242:
235:
229:
228:
219:
213:
206:
197:
196:
183:
152:computer science
144:Jonathan R. Cole
49:doctoral program
21:
568:
567:
563:
562:
561:
559:
558:
557:
543:
542:
529:
524:
515:
511:
489:
485:
459:
455:
441:
437:
421:
417:
405:
401:
391:Wayback Machine
382:
378:
359:
355:
342:
338:
325:
321:
312:
308:
302:Wayback Machine
292:
288:
280:Susan Kelley, "
279:
275:
266:
262:
249:
245:
236:
232:
221:
220:
216:
207:
200:
184:
180:
176:
100:
45:
30:
23:
22:
15:
12:
11:
5:
566:
556:
555:
541:
540:
535:
528:
527:External links
525:
523:
522:
509:
499:(6): 219–226,
483:
453:
435:
415:
399:
376:
353:
343:David Glenn, "
336:
319:
306:
286:
273:
260:
256:Not Even Wrong
243:
230:
214:
198:
177:
175:
172:
99:
96:
79:
78:
67:
62:were based on
44:
41:
28:
9:
6:
4:
3:
2:
565:
554:
551:
550:
548:
539:
536:
534:
531:
530:
519:
513:
506:
502:
498:
494:
487:
479:
474:
471:(12): 41–43,
470:
466:
465:
457:
450:
446:
439:
432:
431:
426:
419:
412:
408:
403:
396:
392:
388:
385:
380:
373:
372:
367:
363:
357:
350:
346:
340:
333:
329:
323:
316:
310:
303:
299:
296:
290:
283:
277:
270:
264:
257:
253:
250:Peter Woit, "
247:
240:
234:
226:
225:
218:
211:
205:
203:
195:
194:
189:
182:
178:
171:
169:
165:
161:
157:
153:
149:
145:
140:
138:
134:
133:
128:
124:
120:
116:
112:
108:
103:
95:
92:
88:
84:
76:
72:
68:
65:
61:
57:
56:
55:
52:
50:
40:
37:
32:
27:
19:
512:
496:
492:
486:
468:
462:
456:
448:
438:
428:
418:
402:
379:
369:
356:
348:
339:
331:
322:
309:
289:
276:
263:
255:
252:NRC Rankings
246:
233:
223:
217:
191:
181:
150:and various
142:Sociologist
141:
130:
104:
101:
80:
70:
59:
53:
46:
33:
31:
26:
18:NRC Rankings
164:Geographers
160:conferences
43:Methodology
174:References
107:Peter Woit
105:Physicist
87:humanities
71:S-rankings
60:R-rankings
241:" (2010).
98:Reception
547:Category
387:Archived
315:Overview
298:Archived
117:and the
75:survey
168:noise
73:were
125:and
69:The
58:The
34:The
501:doi
497:110
473:doi
91:GRE
549::
495:,
469:54
467:,
447:,
427:,
393:,
368:,
201:^
190:,
162:.
139:.
516:"
503::
475::
351:.
334:.
313:"
267:"
258:.
237:"
208:"
20:)
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.