PnR basic - Terms and Concepts

从今天开始，正式研究PnR testing。因为暂时没有具体项目测试需要，所以，以自己的业余时间自学为主。

首先从一些基本概念和名词开始。

Concurrency - Simultaneous execution of interacting computational tasks
Throughput - Number of units of reliable work done in a given amount of time (eg KB/s or 5 Reports processed per second)
Response time - How long it takes a system to react to a given input
Latency (恢复时间) - Delay between when something is initiated to when it finishes
Availability - Percentage of time that system is operational
Reliability (可靠性) - Percentage of time, when available, in which the system delivers fault free results
Robustness (健壮性) - Horizontal Scalability - Resources are clustered across multiple distinct pieces of hardware. Usually results in higher availability and performance
Vertical Scalability - Resources are added to a single piece of hardware. Usually minimizes overhead and management costs
Linear Scalability - There is a one to one relationship (straight line) between performance and additional resources. Allows systems to grow very large without penalty
MTBF (Mean time between failures) - Expected time between any two successive failures of a system
Private Bytes (or VM Size in Windows Task Manager) - the number of bytes that the process has allocated that cannot be shared with other processes. This is a Windows only term.
Virtual Bytes (or Virtual Size on UNIX) - the amount of virtual address space (physical and page-file) allocated to the process. This is the most important metric because this determines when the process crashes AND it's always greater than or equal to Private Bytes.
Memory Usage (in Windows Task Manager) and Resident Size (on UNIX) - the amount of physical RAM actually consumed by the process memory actually located in physical RAM. This value can be useful for diagnosing low-memory performance conditions but it can actually conceal memory leaks.

Types of P&R Tests
Load - Testing to see how system reacts with large volumes of data or users
Endurance - Testing to see how system behaves over long periods of time
Stress - Testing to see how system reacts when pushed beyond normal operating conditions
Scalability - Testing to see how system performs when horizontally and vertically scaled (relationship between performance and additional resources)
Robustness - Testing to see how system reacts when given bad input