SPEC and SOAP Benchmark
SPEC develops real application benchmarks that would exercise major system components:
technologies, architectures, implementations, memory systems, I/O subsystems, OS, clock rates, bus protocols, compilers, libraries, and application software plus scalability, graphics and networking.

SPEC chooses a simple measure, elapsed time, to run benchmark. A simple speed metric and machine-independent code are keys to providing a comprehensive and fair comparison between competing machines.

SPEC measures performance by determing the times required to run a suite of applications and then comparing the time for completion of each with the time of a reference machine.

SPEC selected geometric mean-a composite metric: each benchmark carries the same weight.

SPEC's members voted not to combine the results of two fundamentally different workloads (inter and floating-point) .

CPU benchmarks suite
SPEC CPU2000 suite includes applications from the following areas:

• AI game theory
• compilers
• interpreters
• data compression
• databases
• weather prediction
• fluid dynamics
• physics
• chemistry
• image processing

Criteria SPEC considers important for the CPU benchmark suite:

• the program can be made compute bound
• the program can be made portable across different hardware architectures and operating systems
• how close the program is to the state of the art for the given field

appPlatform

Currently under development, SPECappPlatform is designed to measure the scalability and performance of enterprise platforms (such as J2EE and .NET) running web services. The benchmark will include features that are typical in customers' enterprise applications, such as transactions, persistence services, web services, and messaging.

- posted by Pop @ 9:49 AM