235:
73:
175:
32:
809:– cross-platform architecture-agnostic benchmark suite based on multithreaded ray tracing performance; baselined against a VAX-11/780; and used since 1984 for evaluating relative CPU performance, compiler differences, optimization levels, coherency, architecture differences, and operating system differences.
556:
Many server architectures degrade dramatically at high (near 100%) levels of usage — "fall off a cliff" — and benchmarks should (but often do not) take that factor into account. Vendors, in particular, tend to publish server benchmarks at continuous at about 80% usage — an unrealistic situation — and
378:
Computer manufacturers are known to configure their systems to give unrealistically high performance on benchmark tests that are not replicated in real usage. For instance, during the 1980s some compilers could detect a specific mathematical operation used in a well-known floating-point benchmark and
348:
Benchmarks are designed to mimic a particular type of workload on a component or system. Synthetic benchmarks do this by specially created programs that impose the workload on the component. Application benchmarks run real-world programs on the system. While application benchmarks usually give a much
450:
tiles, and pausing the process to be able to resume without having to start over. Software can have additional features specific to its purpose, for example, disk benchmarking software may be able to optionally start measuring the disk speed within a specified range of the disk rather than the full
510:
formula. However, the costs are necessarily only partial, and vendors have been known to price specifically (and only) for the benchmark, designing a highly specific "benchmark special" configuration with an artificially low price. Even a tiny deviation from the benchmark package results in a much
490:
Qualities of service, aside from raw performance. Examples of unmeasured qualities of service include security, availability, reliability, execution integrity, serviceability, scalability (especially the ability to quickly and nondisruptively add or reallocate capacity), etc. There are often real
418:
Manufacturers commonly report only those benchmarks (or aspects of benchmarks) that show their products in the best light. They also have been known to mis-represent the significance of benchmarks, again to show their products in the best possible light. Taken together, these practices are called
410:
CPU — typically have slower clock rates than a sequential CPU with one or two execution units when built from transistors that are just as fast. Nevertheless, CPUs with many execution units often complete real-world and benchmark tasks in less time than the supposedly faster high-clock-rate CPU.
514:
Facilities burden (space, power, and cooling). When more power is used, a portable system will have a shorter battery life and require recharging more often. A server that consumes more power and/or space may not be able to fit within existing data center resource constraints, including cooling
425:
Ideally benchmarks should only substitute for real applications if the application is unavailable, or too difficult or costly to port to a specific processor or computer system. If performance is critical, the only benchmark that matters is the target environment's application suite.
571:
tends to be much more focused on the predictability of completing long-running tasks correctly before deadlines, such as end of month or end of fiscal year. Many important core business processes are batch-oriented and probably always will be, such as
476:
Vendors tend to tune their products specifically for industry-standard benchmarks. Norton SysInfo (SI) is particularly easy to tune for, since it mainly biased toward the speed of multiple operations. Use extreme caution in interpreting such
471:
Benchmarking is not easy and often involves several iterative rounds in order to arrive at predictable, useful conclusions. Interpretation of benchmarking data is also extraordinarily difficult. Here is a partial list of common challenges:
323:
advanced, it became more difficult to compare the performance of various computer systems simply by looking at their specifications. Therefore, tests were developed that allowed comparison of different architectures. For example,
414:
Given the large number of benchmarks available, a manufacturer can usually find at least one benchmark that shows its system will outperform another system; the other systems can be shown to excel with a different benchmark.
506:. Transaction Processing Performance Council Benchmark specifications partially address this concern by specifying that a price/performance metric must be reported in addition to a raw performance metric, using a simplified
336:
processors, which did not necessarily translate to more computational power; a processor with a slower clock frequency might perform as well as or even better than a processor operating at a higher frequency. See
575:
Benchmarking institutions often disregard or do not follow basic scientific method. This includes, but is not limited to: small sample size, lack of variable control, and the limited repeatability of results.
368:
of an application, it will contain the performance-sensitive aspects of that application. Running this much smaller snippet on a cycle-accurate simulator can give clues on how to improve performance.
564:
extensively for a variety of reasons, and benchmarking is still catching up to that reality where multiple applications and application tiers are concurrently running on consolidated servers.
925:– open-source cross-platform benchmarking suite for Linux, OpenSolaris, FreeBSD, OSX and Windows. It includes a number of other benchmarks included on this page to simplify execution.
535:
computing capacity. Vendors only like to report what might be narrowly required for production capacity in order to make their initial acquisition price seem as low as possible.
2157:
545:
Users can have very different perceptions of performance than benchmarks may suggest. In particular, users appreciate predictability — servers that always meet or exceed
379:
replace the operation with a faster mathematically equivalent operation. However, such a transformation was rarely useful outside the benchmark until the mid-1990s, when
480:
Some vendors have been accused of "cheating" at benchmarks — doing things that give much higher benchmark numbers, but make things worse on the actual likely workload.
567:
There are few (if any) high quality benchmarks that help measure the performance of batch computing, especially high volume concurrent batch and online computing.
1266:
538:
Benchmarks are having trouble adapting to widely distributed servers, particularly those with extra sensitivity to network topologies. The emergence of
1023:
560:
Many benchmarks focus on one application, or even one application tier, to the exclusion of other applications. Most data centers are now implementing
722:
These were the first general purpose industry standard computer benchmarks. They do not necessarily obtain high scores on modern pipelined computers.
797:– composed of a list of tests that could be mixed to create a 'load mix' that would simulate a specific computer function on any UNIX-type OS.
1972:
768:
1978:
1573:
1409:
781:
492:
349:
better measure of real-world performance on a given system, synthetic benchmarks are useful for testing individual components, like a
1322:
137:
459:, have a "quick scan" feature which measures the speed through samples of specified intervals and sizes, and allow specifying a
109:
196:
90:
45:
2337:
1930:
1488:
1469:
1234:
116:
1146: – a collection of test cases intended to show that a software program has some specified set of behaviors
1891:
907:– synthetic benchmark suite measuring performance of integer arithmetic, memory operations, and floating-point arithmetic
1296:
191:
1446:
221:
156:
123:
59:
1823:
1094:– modeling scheme used by AMD and Cyrix to reflect the relative performance usually compared to competing products.
1274:
2700:
2504:
1133:
815:– customizable, cross-platform framework to crowdsource benchmarking and optimization of user workloads (such as
812:
964:– floating-point arithmetic performance, often reported in millions of Whetstone instructions per second (MWIPS)
105:
2462:
1566:
1103:
1036:, included with Windows Vista and later releases, providing an index for consumers to rate their systems easily
1033:
94:
2352:
549:. Benchmarks tend to emphasize mean scores (IT perspective), rather than maximum worst-case response times (
515:
limitations. There are real trade-offs as most semiconductors require more power to switch faster. See also
303:
Benchmarks provide a method of comparing the performance of various subsystems across different chip/system
1848:
1787:
1064:
263:
1138:
837:– integer arithmetic performance, often reported in DMIPS (Dhrystone millions of instructions per second)
603:
Scalability: Benchmark tests should work across systems possessing a range of resources from low to high.
542:, in particular, complicates benchmarking since some workloads are "grid friendly", while others are not.
297:
51:
20:
591:
Representativeness: Benchmark performance metrics should be broadly accepted by industry and academia.
491:
trade-offs between and among these qualities of service, and all are important in business computing.
375:
to do this, although SPEC's Unix-based benchmarks were quite lengthy and thus unwieldy to use intact.
2596:
1792:
1559:
1441:. Morgan Kaufmann Series in Data Management Systems (2nd ed.). Morgan Kaufmann Publishers, Inc.
273:
is also commonly utilized for the purposes of elaborately designed benchmarking programs themselves.
1386:
2346:
2294:
2110:
2093:
1833:
1782:
1752:
1694:
867:
407:
281:
1323:"Benchmarking Contemporary Deep Learning Hardware and Frameworks: a Survey of Qualitative Metrics"
741:
used on machines with multiple cores and/or processors, or systems consisting of multiple machines
2610:
2525:
2421:
2247:
2152:
2147:
1935:
1884:
1097:
1008:
956:
952:
794:
561:
546:
507:
503:
285:
130:
83:
657:
may be used for automatic detection of computer's hardware parameters like number of registers,
2273:
2044:
308:
2137:
2059:
2039:
1941:
1712:
1373:
1028:
961:
856:
828:
710:
456:
320:
304:
1362:
395:
companies to improve not only their own benchmark scores, but real application performance.
2491:
1925:
1920:
1757:
1747:
516:
484:
460:
259:
8:
2652:
1996:
1742:
1737:
1727:
1671:
1616:
1606:
1061:- originally tested language performance, but widely used as a machine benchmark as well.
922:
550:
682:
linpack benchmark (contains basic linear algebra subroutine written in FORTRAN language)
557:
do not document what happens to the overall system when demand spikes beyond that level.
2658:
2379:
2129:
2014:
1877:
1797:
1772:
1732:
1722:
1666:
1536:
1333:
1195:
1091:
916:
885:
876:– I/O subsystem measurement and characterization tool for single and clustered systems.
637:
2317:
1807:
1633:
1601:
1596:
1528:
1484:
1465:
1442:
1435:
1230:
1214:
1187:
852:
844:
658:
532:
361:
277:
2630:
2331:
2181:
1957:
1777:
1762:
1717:
1643:
1628:
1621:
1582:
1520:
1401:
1343:
1222:
1199:
1177:
998:
974:
758:
255:
1347:
1330:
2019 IEEE First
International Conference on Cognitive Machine Intelligence (CogMI)
2568:
2431:
2397:
2236:
2142:
1802:
1128:
980:
894:
568:
276:
Benchmarking is usually associated with assessing performance characteristics of
1166:"How not to lie with statistics: the correct way to summarize benchmark results"
584:
There are seven vital characteristics for benchmarks. These key properties are:
2600:
2499:
2481:
2454:
1853:
1828:
1767:
1676:
1648:
940:
934:
910:
899:
733:
measure the throughput and response times of database management systems (DBMS)
662:
539:
342:
1508:"The Evolution of Benchmarking as a Computer Performance Evaluation Technique"
391:
technology as it related to performance. Benchmarks are now regularly used by
19:
This article is about the use of benchmarks in computing. For other uses, see
2694:
2636:
2533:
1915:
1532:
1191:
1080:
848:
816:
452:
1226:
2672:
2389:
2257:
1843:
1838:
1681:
1611:
1122:
1086:
641:
523:
499:
property tests, database scalability rules, and service level requirements.
360:, giving processor architects the ability to measure and make tradeoffs in
1399:
258:, a set of programs, or other operations, in order to assess the relative
2538:
2202:
2172:
2073:
1858:
1702:
1638:
1300:
861:
696:
take statistics of all types of operations from many application programs
531:
Vendor benchmarks tend to ignore requirements for development, test, and
447:
439:
435:
399:
1869:
1077:– A cross-platform benchmark for Windows, Linux, macOS, iOS and Android.
495:
Benchmark specifications partially address these concerns by specifying
2678:
2370:
2211:
2034:
1967:
1215:"Continuous Benchmarking: Using System Benchmarking in Build Pipelines"
1143:
1058:
1039:
1018:
985:
800:
651:
core routine consists of a relatively small and specific piece of code.
628:
443:
365:
357:
288:, but there are circumstances when the technique is also applicable to
1540:
1507:
1182:
1165:
1083:– the Intel comparative microprocessor performance, published by Intel
847:
for storage benchmarking that generates a variety of requests against
2642:
2472:
2407:
2342:
2068:
1457:
1456:
Scalzo, Bert; Kline, Kevin; Fernandez, Claudia; Burleson, Donald K.;
1074:
834:
715:
522:
In some embedded systems, where memory is a significant cost, better
487:, neglecting other important features of a computer system, such as:
350:
329:
325:
247:
234:
72:
2668:
2626:
2616:
2444:
2083:
2029:
1551:
1524:
1462:
Database
Benchmarking Practical Methods for Oracle & SQL Server
1338:
1013:
822:
633:
392:
388:
338:
293:
289:
2664:
2576:
2548:
2543:
2265:
2226:
2216:
2119:
2078:
2055:
2024:
928:
873:
840:
806:
772:
333:
943:– Telecommunication Application Transaction Processing Benchmark
2606:
2439:
2300:
2221:
2019:
2009:
1109:
1052:
1003:
993:
989:
904:
879:
776:
588:
Relevance: Benchmarks should measure relatively vital features.
328:
processors generally operated at a higher clock frequency than
1219:
2019 IEEE International
Conference on Cloud Engineering (IC2E)
1071:
programming language on various platforms. Introduced in 1983.
463:
size, meaning the number of requested bytes per read request.
2646:
2620:
2558:
2553:
2514:
2327:
2000:
1962:
1068:
919:– A set of benchmarks to measure and compare cloud offerings.
889:
763:
752:
606:
Transparency: Benchmark metrics should be easy to understand.
2158:
Princeton
Application Repository for Shared-Memory Computers
1455:
1400:
Transaction
Processing Performance Council (February 1998).
553:
perspective), or low standard deviations (user perspective).
2510:
2416:
2192:
1437:
The
Benchmark Handbook for Database and Transaction Systems
949:– An XML transaction processing benchmark for XML databases
946:
496:
403:
384:
380:
372:
371:
Prior to 2000, computer and microprocessor architects used
239:
1213:
Grambow, Martin; Lehmann, Fabian; Bermbach, David (2019).
1100:- one of the earliest microcomputer benchmarks, from 1977.
2321:
870:– designed to measure overall CPU and memory performance
434:
Features of benchmarking software may include recording/
364:
decisions. For example, if a benchmark extracts the key
937:– a simple benchmark used to test recursion performance
262:
of an object, normally by running a number of standard
1212:
759:
764:
654:
measure performance of a computer's basic components
292:. Software benchmarks are, for example, run against
1321:Dai, Wei; Berleant, Daniel (December 12–14, 2019).
1164:Fleming, Philip J.; Wallace, John J. (1986-03-01).
600:
Cost-effectiveness: Benchmark tests are economical.
97:. Unsourced material may be challenged and removed.
1506:
1434:
184:needs attention from an expert in Computer science
597:Repeatability: Benchmark results can be verified.
2692:
1479:Nambiar, Raghunath; Poess, Meikel, eds. (2009).
1332:. Los Angeles, CA, USA: IEEE. pp. 148–155.
1055:– commonly used on phones and ARM-based devices.
398:CPUs that have many execution units — such as a
1297:"Hardware Testing and Benchmarking Methodology"
968:
693:Procedure for programming synthetic benchmark:
483:Many benchmarks focus entirely on the speed of
238:A graphical demo running as a benchmark of the
1163:
594:Equity: All systems should be fairly compared.
1885:
1567:
1478:
955:(VAX unit of performance) – also called VAX
1973:Standard Performance Evaluation Corporation
1320:
769:Standard Performance Evaluation Corporation
702:write program based on the proportion above
387:architectures emphasized the importance of
60:Learn how and when to remove these messages
1979:Transaction Processing Performance Council
1892:
1878:
1574:
1560:
1505:Lewis, Byron C.; Crews, Albert E. (1985).
1504:
1410:Transaction Processing Performance Council
913:– a benchmark for realtime physics engines
782:Transaction Processing Performance Council
753:Industry standard (audited and verifiable)
579:
493:Transaction Processing Performance Council
1899:
1337:
1181:
788:
356:Benchmarks are particularly important in
222:Learn how and when to remove this message
157:Learn how and when to remove this message
819:) across hardware provided by volunteers
233:
1481:Performance Evaluation and Benchmarking
1267:"NVidia's Benchmark Tactics Reassessed"
831:– scientific HPC applications benchmark
676:normally abstracted from actual program
625:user's application software (i.e.: MIS)
2693:
1294:
1264:
636:building a large project, for example
511:higher price in real world experience.
502:In general, benchmarks do not measure
199:may be able to help recruit an expert.
2338:Data center infrastructure efficiency
1931:Data center infrastructure efficiency
1873:
1555:
1360:
803:– filesystem and hard drive benchmark
648:Component Benchmark / Microbenchmark
610:
1581:
1432:
747:
442:file, visualization such as drawing
168:
95:adding citations to reliable sources
66:
25:
2463:Physics Abstraction Layer API Bench
1112:– a virtualization benchmark suite.
685:results are represented in Mflop/s.
311:is called Continuous Benchmarking.
16:Standardized performance evaluation
13:
2248:HPC Challenge Random Memory Access
2047:(Computational Intensive Workload)
1426:
707:Types of Synthetic Benchmark are:
14:
2712:
1498:
1402:"History and Overview of the TPC"
1363:"Benchmarking network processors"
41:This article has multiple issues.
2276:(Commercial Processing Workload)
1824:Graphical user interface testing
888:– traditionally used to measure
699:get proportion of each operation
429:
173:
71:
30:
2505:Transaction Processing over XML
1134:Lossless compression benchmarks
526:can significantly reduce costs.
438:the course of performance to a
188:Outdated or deprecated sources.
82:needs additional citations for
49:or discuss these issues on the
1393:
1354:
1314:
1288:
1258:
1255:Software: HDDScan, GNOME Disks
1249:
1206:
1157:
1034:Windows System Assessment Tool
986:Underwriters Laboratories (UL)
977:: MobileMark, SYSmark, WebMark
825:– Embedded computing benchmark
679:popular kernel: Livermore loop
1:
2353:Server Efficiency Rating Tool
1348:10.1109/CogMI48466.2019.00029
1150:
466:
1849:Software reliability testing
1788:Software performance testing
1361:Ehliar, Andreas; Liu, Dake.
1065:Creative Computing Benchmark
969:Microsoft Windows benchmarks
771:(SPEC), in particular their
307:. Benchmarking as a part of
197:WikiProject Computer science
7:
2244:BSS Random Access benchmark
1139:Performance Counter Monitor
1116:
298:database management systems
186:. The specific problem is:
10:
2717:
1691:Testing types, techniques,
314:
106:"Benchmark" computing
18:
2597:Adjusted Peak Performance
2589:
2567:
2524:
2490:
2471:
2453:
2430:
2406:
2388:
2369:
2362:
2310:
2287:
2256:
2235:
2201:
2180:
2171:
2128:
2109:
2092:
2054:
1995:
1988:
1950:
1908:
1816:
1690:
1659:
1589:
1170:Communications of the ACM
1046:
485:computational performance
2295:Hierarchical INTegration
2288:Computer system (entire)
2111:Graphics processing unit
2094:Digital signal processor
1834:Orthogonal array testing
1783:Smoke testing (software)
1753:Dynamic program analysis
619:word processing software
547:service level agreements
408:reconfigurable computing
282:floating point operation
254:is the act of running a
2526:Web server benchmarking
2422:AIM Multiuser Benchmark
2153:HPC Challenge Benchmark
2148:NAS Parallel Benchmarks
1936:Giga-updates per second
1433:Gray, Jim, ed. (1993).
1227:10.1109/IC2E.2019.00039
1098:Rugg/Feldman benchmarks
1009:Superposition Benchmark
900:NAS parallel benchmarks
795:AIM Multiuser Benchmark
580:Benchmarking principles
504:Total cost of ownership
266:and trials against it.
2701:Benchmarks (computing)
2274:IBM iSeries benchmarks
2045:IBM iSeries benchmarks
1381:Cite journal requires
1295:Castor, Kevin (2006).
1125:(business perspective)
1106:– a browser speed test
882:– Filesystem benchmark
864:– an integer benchmark
789:Open source benchmarks
784:(TPC): DBMS benchmarks
353:or networking device.
309:continuous integration
243:
2492:Database transactions
2189:BreakingPoint Systems
2138:DEISA Benchmark Suite
1942:CPU power dissipation
1713:Compatibility testing
1464:. Rampant TechPress.
829:DEISA Benchmark Suite
321:computer architecture
237:
1926:Performance per watt
1921:Proprietary software
1758:Installation testing
1748:Differential testing
1545:The dates: 1962-1976
1265:Krazit, Tom (2003).
1221:. pp. 241–246.
813:Collective Knowledge
738:Parallel benchmarks
730:Database benchmarks
690:Synthetic Benchmark
622:tool software of CAD
517:performance per watt
91:improve this article
2653:Phoronix Test Suite
1997:Floating-point unit
1743:Development testing
1738:Destructive testing
1728:Conformance testing
1672:Integration testing
1617:Model-based testing
1607:Exploratory testing
923:Phoronix Test Suite
551:real-time computing
280:, for example, the
2659:Performance Rating
2380:Browser speed test
2311:Energy consumption
2130:Parallel computing
1798:Symbolic execution
1773:Regression testing
1733:Continuous testing
1723:Concurrent testing
1667:Acceptance testing
1590:The "box" approach
1092:Performance Rating
917:PerfKitBenchmarker
886:LINPACK benchmarks
673:contains key codes
611:Types of benchmark
455:reading speed and
362:microarchitectural
244:
2688:
2687:
2590:Platform specific
2585:
2584:
2318:Average CPU power
2283:
2282:
2167:
2166:
1867:
1866:
1808:Usability testing
1634:White-box testing
1602:All-pairs testing
1597:Black-box testing
1490:978-3-642-10423-7
1471:978-0-9776715-3-3
1236:978-1-7281-0218-4
1183:10.1145/5666.5673
845:Command-line tool
748:Common benchmarks
533:disaster recovery
284:performance of a
278:computer hardware
232:
231:
224:
214:
213:
167:
166:
159:
141:
64:
2708:
2631:embedded systems
2367:
2366:
2332:embedded systems
2178:
2177:
1993:
1992:
1958:BAPCo consortium
1894:
1887:
1880:
1871:
1870:
1778:Security testing
1763:Negative testing
1718:Concolic testing
1644:Mutation testing
1629:Grey-box testing
1622:Scenario testing
1583:Software testing
1576:
1569:
1562:
1553:
1552:
1544:
1510:
1494:
1475:
1452:
1440:
1420:
1419:
1417:
1416:
1397:
1391:
1390:
1384:
1379:
1377:
1369:
1367:
1358:
1352:
1351:
1341:
1327:
1318:
1312:
1311:
1309:
1308:
1299:. Archived from
1292:
1286:
1285:
1283:
1282:
1273:. Archived from
1262:
1256:
1253:
1247:
1246:
1244:
1243:
1210:
1204:
1203:
1185:
1161:
1024:Valley Benchmark
999:Heaven Benchmark
638:Chromium browser
421:bench-marketing.
256:computer program
227:
220:
209:
206:
200:
177:
176:
169:
162:
155:
151:
148:
142:
140:
99:
75:
67:
56:
34:
33:
26:
2716:
2715:
2711:
2710:
2709:
2707:
2706:
2705:
2691:
2690:
2689:
2684:
2581:
2569:X Window System
2563:
2520:
2486:
2467:
2449:
2432:Virtual machine
2426:
2402:
2398:Cycles per byte
2384:
2358:
2306:
2279:
2252:
2237:Computer memory
2231:
2197:
2163:
2143:Livermore loops
2124:
2105:
2088:
2050:
1984:
1946:
1904:
1898:
1868:
1863:
1812:
1803:Test automation
1692:
1686:
1655:
1585:
1580:
1549:
1501:
1491:
1472:
1449:
1429:
1427:Further reading
1424:
1423:
1414:
1412:
1398:
1394:
1382:
1380:
1371:
1370:
1365:
1359:
1355:
1325:
1319:
1315:
1306:
1304:
1293:
1289:
1280:
1278:
1263:
1259:
1254:
1250:
1241:
1239:
1237:
1211:
1207:
1162:
1158:
1153:
1129:Figure of merit
1119:
1067:– Compares the
1049:
981:CrystalDiskMark
971:
895:Livermore loops
857:storage devices
791:
755:
750:
613:
582:
569:Batch computing
469:
432:
317:
228:
217:
216:
215:
210:
204:
201:
195:
178:
174:
163:
152:
146:
143:
100:
98:
88:
76:
35:
31:
24:
17:
12:
11:
5:
2714:
2704:
2703:
2686:
2685:
2683:
2682:
2676:
2662:
2656:
2650:
2640:
2634:
2624:
2614:
2604:
2601:Nuclear weapon
2593:
2591:
2587:
2586:
2583:
2582:
2580:
2579:
2573:
2571:
2565:
2564:
2562:
2561:
2556:
2551:
2546:
2541:
2536:
2530:
2528:
2522:
2521:
2519:
2518:
2508:
2502:
2500:TATP Benchmark
2496:
2494:
2488:
2487:
2485:
2484:
2482:Tak (function)
2478:
2476:
2469:
2468:
2466:
2465:
2459:
2457:
2455:Physics engine
2451:
2450:
2448:
2447:
2442:
2436:
2434:
2428:
2427:
2425:
2424:
2419:
2413:
2411:
2404:
2403:
2401:
2400:
2394:
2392:
2386:
2385:
2383:
2382:
2376:
2374:
2364:
2360:
2359:
2357:
2356:
2350:
2340:
2335:
2325:
2314:
2312:
2308:
2307:
2305:
2304:
2298:
2291:
2289:
2285:
2284:
2281:
2280:
2278:
2277:
2271:
2268:
2262:
2260:
2254:
2253:
2251:
2250:
2245:
2241:
2239:
2233:
2232:
2230:
2229:
2224:
2219:
2214:
2208:
2206:
2199:
2198:
2196:
2195:
2190:
2186:
2184:
2175:
2169:
2168:
2165:
2164:
2162:
2161:
2155:
2150:
2145:
2140:
2134:
2132:
2126:
2125:
2123:
2122:
2116:
2114:
2107:
2106:
2104:
2103:
2099:
2097:
2090:
2089:
2087:
2086:
2081:
2076:
2071:
2065:
2063:
2052:
2051:
2049:
2048:
2042:
2037:
2032:
2027:
2022:
2017:
2012:
2006:
2004:
1990:
1986:
1985:
1983:
1982:
1976:
1970:
1965:
1960:
1954:
1952:
1948:
1947:
1945:
1944:
1939:
1933:
1928:
1923:
1918:
1912:
1910:
1906:
1905:
1897:
1896:
1889:
1882:
1874:
1865:
1864:
1862:
1861:
1856:
1854:Stress testing
1851:
1846:
1841:
1836:
1831:
1829:Manual testing
1826:
1820:
1818:
1814:
1813:
1811:
1810:
1805:
1800:
1795:
1793:Stress testing
1790:
1785:
1780:
1775:
1770:
1768:Random testing
1765:
1760:
1755:
1750:
1745:
1740:
1735:
1730:
1725:
1720:
1715:
1710:
1705:
1699:
1697:
1688:
1687:
1685:
1684:
1679:
1677:System testing
1674:
1669:
1663:
1661:
1660:Testing levels
1657:
1656:
1654:
1653:
1652:
1651:
1649:Static testing
1646:
1641:
1631:
1626:
1625:
1624:
1619:
1614:
1609:
1604:
1593:
1591:
1587:
1586:
1579:
1578:
1571:
1564:
1556:
1547:
1546:
1525:10.2307/249270
1500:
1499:External links
1497:
1496:
1495:
1489:
1476:
1470:
1453:
1447:
1428:
1425:
1422:
1421:
1392:
1383:|journal=
1353:
1313:
1287:
1257:
1248:
1235:
1205:
1176:(3): 218–221.
1155:
1154:
1152:
1149:
1148:
1147:
1141:
1136:
1131:
1126:
1118:
1115:
1114:
1113:
1107:
1101:
1095:
1089:
1084:
1078:
1072:
1062:
1056:
1048:
1045:
1044:
1043:
1042:(discontinued)
1037:
1031:
1026:
1021:
1016:
1011:
1006:
1001:
996:
983:
978:
970:
967:
966:
965:
959:
950:
944:
941:TATP Benchmark
938:
935:Tak (function)
932:
926:
920:
914:
908:
902:
897:
892:
883:
877:
871:
865:
859:
849:computer files
838:
832:
826:
820:
810:
804:
798:
790:
787:
786:
785:
779:
766:
761:
754:
751:
749:
746:
745:
744:
743:
742:
736:
735:
734:
728:
727:I/O benchmarks
725:
724:
723:
720:
719:
718:
713:
705:
704:
703:
700:
697:
688:
687:
686:
683:
680:
677:
674:
668:
667:
666:
663:memory latency
655:
652:
646:
645:
644:
631:
626:
623:
620:
612:
609:
608:
607:
604:
601:
598:
595:
592:
589:
581:
578:
577:
576:
573:
565:
562:virtualization
558:
554:
543:
540:grid computing
536:
529:
528:
527:
520:
512:
500:
481:
478:
468:
465:
451:disk, measure
431:
428:
343:megahertz myth
316:
313:
230:
229:
212:
211:
181:
179:
172:
165:
164:
79:
77:
70:
65:
39:
38:
36:
29:
15:
9:
6:
4:
3:
2:
2713:
2702:
2699:
2698:
2696:
2680:
2677:
2674:
2670:
2666:
2663:
2660:
2657:
2654:
2651:
2648:
2645:(Windows and
2644:
2641:
2638:
2637:iCOMP (index)
2635:
2632:
2628:
2625:
2622:
2618:
2615:
2612:
2608:
2605:
2602:
2598:
2595:
2594:
2592:
2588:
2578:
2575:
2574:
2572:
2570:
2566:
2560:
2557:
2555:
2552:
2550:
2547:
2545:
2542:
2540:
2537:
2535:
2534:Apache JMeter
2532:
2531:
2529:
2527:
2523:
2516:
2512:
2509:
2506:
2503:
2501:
2498:
2497:
2495:
2493:
2489:
2483:
2480:
2479:
2477:
2474:
2470:
2464:
2461:
2460:
2458:
2456:
2452:
2446:
2443:
2441:
2438:
2437:
2435:
2433:
2429:
2423:
2420:
2418:
2415:
2414:
2412:
2409:
2405:
2399:
2396:
2395:
2393:
2391:
2387:
2381:
2378:
2377:
2375:
2372:
2368:
2365:
2361:
2354:
2351:
2348:
2344:
2341:
2339:
2336:
2333:
2329:
2326:
2323:
2319:
2316:
2315:
2313:
2309:
2303:(CPU, memory)
2302:
2299:
2296:
2293:
2292:
2290:
2286:
2275:
2272:
2269:
2267:
2264:
2263:
2261:
2259:
2255:
2249:
2246:
2243:
2242:
2240:
2238:
2234:
2228:
2225:
2223:
2220:
2218:
2215:
2213:
2210:
2209:
2207:
2204:
2200:
2194:
2191:
2188:
2187:
2185:
2183:
2179:
2176:
2174:
2170:
2159:
2156:
2154:
2151:
2149:
2146:
2144:
2141:
2139:
2136:
2135:
2133:
2131:
2127:
2121:
2118:
2117:
2115:
2112:
2108:
2101:
2100:
2098:
2095:
2091:
2085:
2082:
2080:
2077:
2075:
2072:
2070:
2067:
2066:
2064:
2061:
2057:
2053:
2046:
2043:
2041:
2038:
2036:
2033:
2031:
2028:
2026:
2023:
2021:
2018:
2016:
2013:
2011:
2008:
2007:
2005:
2002:
1998:
1994:
1991:
1987:
1980:
1977:
1974:
1971:
1969:
1966:
1964:
1961:
1959:
1956:
1955:
1953:
1951:Organizations
1949:
1943:
1940:
1937:
1934:
1932:
1929:
1927:
1924:
1922:
1919:
1917:
1916:Free software
1914:
1913:
1911:
1907:
1903:
1895:
1890:
1888:
1883:
1881:
1876:
1875:
1872:
1860:
1857:
1855:
1852:
1850:
1847:
1845:
1842:
1840:
1837:
1835:
1832:
1830:
1827:
1825:
1822:
1821:
1819:
1815:
1809:
1806:
1804:
1801:
1799:
1796:
1794:
1791:
1789:
1786:
1784:
1781:
1779:
1776:
1774:
1771:
1769:
1766:
1764:
1761:
1759:
1756:
1754:
1751:
1749:
1746:
1744:
1741:
1739:
1736:
1734:
1731:
1729:
1726:
1724:
1721:
1719:
1716:
1714:
1711:
1709:
1706:
1704:
1701:
1700:
1698:
1696:
1689:
1683:
1680:
1678:
1675:
1673:
1670:
1668:
1665:
1664:
1662:
1658:
1650:
1647:
1645:
1642:
1640:
1637:
1636:
1635:
1632:
1630:
1627:
1623:
1620:
1618:
1615:
1613:
1610:
1608:
1605:
1603:
1600:
1599:
1598:
1595:
1594:
1592:
1588:
1584:
1577:
1572:
1570:
1565:
1563:
1558:
1557:
1554:
1550:
1542:
1538:
1534:
1530:
1526:
1522:
1518:
1514:
1513:MIS Quarterly
1509:
1503:
1502:
1492:
1486:
1482:
1477:
1473:
1467:
1463:
1459:
1454:
1450:
1448:1-55860-292-5
1444:
1439:
1438:
1431:
1430:
1411:
1407:
1403:
1396:
1388:
1375:
1364:
1357:
1349:
1345:
1340:
1335:
1331:
1324:
1317:
1303:on 2008-02-05
1302:
1298:
1291:
1277:on 2011-06-06
1276:
1272:
1268:
1261:
1252:
1238:
1232:
1228:
1224:
1220:
1216:
1209:
1201:
1197:
1193:
1189:
1184:
1179:
1175:
1171:
1167:
1160:
1156:
1145:
1142:
1140:
1137:
1135:
1132:
1130:
1127:
1124:
1121:
1120:
1111:
1108:
1105:
1102:
1099:
1096:
1093:
1090:
1088:
1085:
1082:
1079:
1076:
1073:
1070:
1066:
1063:
1060:
1057:
1054:
1051:
1050:
1041:
1038:
1035:
1032:
1030:
1027:
1025:
1022:
1020:
1017:
1015:
1012:
1010:
1007:
1005:
1002:
1000:
997:
995:
991:
987:
984:
982:
979:
976:
973:
972:
963:
960:
958:
954:
951:
948:
945:
942:
939:
936:
933:
930:
927:
924:
921:
918:
915:
912:
909:
906:
903:
901:
898:
896:
893:
891:
887:
884:
881:
878:
875:
872:
869:
866:
863:
860:
858:
854:
850:
846:
842:
839:
836:
833:
830:
827:
824:
821:
818:
817:deep learning
814:
811:
808:
805:
802:
799:
796:
793:
792:
783:
780:
778:
774:
770:
767:
765:
762:
760:
757:
756:
740:
739:
737:
732:
731:
729:
726:
721:
717:
714:
712:
709:
708:
706:
701:
698:
695:
694:
692:
691:
689:
684:
681:
678:
675:
672:
671:
669:
664:
660:
656:
653:
650:
649:
647:
643:
639:
635:
632:
630:
627:
624:
621:
618:
617:
616:Real program
615:
614:
605:
602:
599:
596:
593:
590:
587:
586:
585:
574:
570:
566:
563:
559:
555:
552:
548:
544:
541:
537:
534:
530:
525:
521:
518:
513:
509:
505:
501:
498:
494:
489:
488:
486:
482:
479:
475:
474:
473:
464:
462:
458:
454:
453:random access
449:
445:
441:
437:
430:Functionality
427:
423:
422:
416:
412:
409:
405:
401:
396:
394:
390:
386:
382:
376:
374:
369:
367:
363:
359:
354:
352:
346:
344:
340:
335:
331:
327:
322:
312:
310:
306:
305:architectures
301:
299:
295:
291:
287:
283:
279:
274:
272:
267:
265:
261:
257:
253:
249:
241:
236:
226:
223:
208:
198:
193:
189:
185:
182:This article
180:
171:
170:
161:
158:
150:
139:
136:
132:
129:
125:
122:
118:
115:
111:
108: –
107:
103:
102:Find sources:
96:
92:
86:
85:
80:This article
78:
74:
69:
68:
63:
61:
54:
53:
48:
47:
42:
37:
28:
27:
22:
2673:Motorola 68k
2390:Cryptography
2258:Input/output
1901:
1844:Soak testing
1839:Pair testing
1707:
1682:Unit testing
1612:Fuzz testing
1548:
1516:
1512:
1483:. Springer.
1480:
1461:
1436:
1413:. Retrieved
1405:
1395:
1374:cite journal
1356:
1329:
1316:
1305:. Retrieved
1301:the original
1290:
1279:. Retrieved
1275:the original
1270:
1260:
1251:
1240:. Retrieved
1218:
1208:
1173:
1169:
1159:
1123:Benchmarking
1087:Khornerstone
642:Linux kernel
583:
524:code density
470:
433:
424:
420:
417:
413:
397:
377:
370:
355:
347:
318:
302:
275:
270:
268:
251:
245:
218:
205:October 2022
202:
194:for details.
187:
183:
153:
144:
134:
127:
120:
113:
101:
89:Please help
84:verification
81:
57:
50:
44:
43:Please help
40:
2603:simulation)
2539:Curl-loader
2475:performance
2205:and storage
2203:Filesystems
2173:Peripherals
2074:Fhourstones
1900:Processing
1859:Web testing
1703:A/B testing
1639:API testing
1519:(1): 7–16.
931:– 3D render
862:Fhourstones
629:Video games
448:color-coded
444:line graphs
440:spreadsheet
400:superscalar
260:performance
2679:WorldBench
2371:JavaScript
2035:SuperPrime
1968:Futuremark
1902:benchmarks
1458:Ault, Mike
1415:2018-07-02
1339:1907.03626
1307:2008-02-24
1281:2009-08-08
1242:2023-12-03
1151:References
1144:Test suite
1059:Byte Sieve
1040:Worldbench
1019:SuperPrime
853:partitions
467:Challenges
461:data block
406:CPU, or a
366:algorithms
358:CPU design
117:newspapers
46:improve it
2681:(Windows)
2643:Novabench
2473:Recursion
2408:Multiuser
2349:software)
2343:SPECpower
2069:Dhrystone
2040:Whetstone
1989:Processor
1708:Benchmark
1533:0276-7783
1192:0001-0782
1104:SunSpider
1075:Geekbench
1029:Whetstone
962:Whetstone
835:Dhrystone
716:Dhrystone
711:Whetstone
634:Compilers
436:exporting
351:hard disk
330:Athlon XP
326:Pentium 4
294:compilers
271:benchmark
269:The term
252:benchmark
248:computing
192:talk page
147:July 2015
52:talk page
21:Benchmark
2695:Category
2669:SysSpeed
2627:Coremark
2617:BogoMips
2445:SPECvirt
2363:Software
2270:Ioblazer
2212:Bonnie++
2160:(PARSEC)
2084:CoreMark
2030:Super PI
1938:(memory)
1909:Concepts
1817:See also
1460:(2007).
1271:IDG News
1117:See also
1014:Super PI
823:Coremark
801:Bonnie++
572:billing.
477:results.
393:compiler
389:compiler
341:and the
339:BogoMips
300:(DBMS).
290:software
190:See the
2665:Sysinfo
2655:(Linux)
2639:(Intel)
2577:Xmark93
2549:OpenSTA
2544:httperf
2266:Iometer
2227:Diskspd
2217:HD Tach
2182:Network
2120:BRL-CAD
2079:SPECint
2056:Integer
2025:Prime95
2015:LINPACK
1695:tactics
1200:1047380
929:POV-Ray
874:Iometer
841:DiskSpd
807:BRL-CAD
773:SPECint
670:Kernel
457:latency
402:CPU, a
334:PowerPC
315:Purpose
131:scholar
2667:&
2607:AnTuTu
2507:(TPoX)
2440:VMmark
2410:system
2373:engine
2355:(SERT)
2301:NBench
2297:(HINT)
2222:IOzone
2020:LAPACK
2010:SPECfp
1975:(SPEC)
1541:249270
1539:
1531:
1487:
1468:
1445:
1233:
1198:
1190:
1110:VMmark
1053:AnTuTu
1047:Others
1004:PiFast
994:PCMark
990:3DMark
905:NBench
880:IOzone
777:SPECfp
665:, etc.
661:size,
242:engine
133:
126:
119:
112:
104:
2661:(AMD)
2647:macOS
2621:Linux
2559:Tsung
2554:TPC-W
2515:NoSQL
2328:EEMBC
2113:(GPU)
2096:(DSP)
2001:FLOPS
1981:(TPC)
1963:EEMBC
1537:JSTOR
1366:(PDF)
1334:arXiv
1326:(PDF)
1196:S2CID
1081:iCOMP
1069:BASIC
975:BAPCo
890:FLOPS
659:cache
264:tests
138:JSTOR
124:books
2511:YCSB
2417:SDET
2347:Java
2193:SUPS
2102:BTDi
1693:and
1529:ISSN
1485:ISBN
1466:ISBN
1443:ISBN
1387:help
1231:ISBN
1188:ISSN
957:MIPS
947:TPoX
868:HINT
775:and
497:ACID
404:VLIW
385:VLIW
383:and
381:RISC
373:SPEC
250:, a
240:OGRE
110:news
2611:ARM
2322:x86
2060:ALU
1521:doi
1406:TPC
1344:doi
1223:doi
1178:doi
953:VUP
911:PAL
855:or
640:or
508:TCO
446:or
332:or
319:As
296:or
286:CPU
246:In
93:by
2697::
1535:.
1527:.
1515:.
1511:.
1408:.
1404:.
1378::
1376:}}
1372:{{
1342:.
1328:.
1269:.
1229:.
1217:.
1194:.
1186:.
1174:29
1172:.
1168:.
992:,
988::
851:,
843:–
345:.
55:.
2675:)
2671:(
2649:)
2633:)
2629:(
2623:)
2619:(
2613:)
2609:(
2599:(
2517:)
2513:(
2345:(
2334:)
2330:(
2324:)
2320:(
2062:)
2058:(
2003:)
1999:(
1893:e
1886:t
1879:v
1575:e
1568:t
1561:v
1543:.
1523::
1517:9
1493:.
1474:.
1451:.
1418:.
1389:)
1385:(
1368:.
1350:.
1346::
1336::
1310:.
1284:.
1245:.
1225::
1202:.
1180::
519:.
225:)
219:(
207:)
203:(
160:)
154:(
149:)
145:(
135:·
128:·
121:·
114:·
87:.
62:)
58:(
23:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
