不要让自动化测试失控
2010-04-04 17:26
399 查看
I.M.Testy的一篇文章《UI Automation Out Of Control》对于不顾一切,企图自动化测试一切的人们而言应该读读:
http://blogs.msdn.com/imtesty/archive/2009/08/01/automation.aspx
作者最后归纳了几个自动化测试的要点(非技术要点):
Not all automated UI tests save time!
Tests that require constant massaging and tweaking because they constantly throw false positives take up a huge amount of a tester’s time in wasted maintenance.
考虑到维护的时间,并非所有自动化测试都能节省时间。
Sometimes a human is a more efficient oracle than a computer algorithm!
Sure, just about anything a computer does can be automated to some degree in some fashion, but there really are clearly some tests where it is more prudent and simpler to rely on a tester.
有时候,用人来判断比用计算机算法更简单、更可靠!
Don’t rely on automation to emulate your customers!
Test automation does not effectively emulate a human user. Sure, we have test methods in some of our internal automation frameworks to slow down simulated keystrokes (the actual keys are not being pressed on the keyboard), or simulate multiple or repeated clicks on a control or the mouse, and other tricks that try to emulate various user behaviors; however, test automation is generally poor at detecting behavioral issues such as usability, ease of use, or other customer value type assessments. Rely on the feedback from internal and external customers who are dog-fooding, self-hosting, and beta-testing your product (and act on it).
不要依赖自动化来模拟用户行为!尤其是在做可用性、易用性测试时。
Go under the covers! I think many testers rely too heavily on UI automation because they think it emulates user behavior (although most things such as populating a text box are simulated via Windows APIs), or perhaps because they don’t know how to dig into the product below the surface of the UI. Whatever the case, think about the specific purpose of the test. If it is easier to check a return value, or call an API to change a setting then go deep…and stop messing around on the surface. (It only complicates the test, wastes valuable machine cycles, reduces reuse across multiple versions, and often leads to long term maintenance costs. (For an example of this see my previous post.)
不要忘了我们还有单元测试!
Constantly massaging code contributes to false negatives! I have seen many cases where a tester designs a a UI automated test, and then tweaks a bit here and there to get it to run. Often times this tweaking contributes to a tests ineffectiveness in exposing problems, and may even hide other problems. Also, some tweaks are geared around synchronization issues (sync’ing the automated test with the system under test) and involve artificially slowing down the automation (usually by stopping or ‘sleeping’ the automated test process for a specific period of time). Other tweaks might hard-code parameters that then make the test fail on a different resolution or non-portable across different environments.
UI自动化测试往往倾向于不断调整脚本以适应被测试对象,这些调整往往导致问题暴露不够高效,甚至隐藏了某些问题。
STOP trying to automate every damn test!
As I stated before…just because we can automate something doesn’t mean that we should try to automate everything! We need to make rational decisions about what tests to automate, and what is the best approach to automating that test.
切忌自动化所有测试!
http://blogs.msdn.com/imtesty/archive/2009/08/01/automation.aspx
作者最后归纳了几个自动化测试的要点(非技术要点):
Not all automated UI tests save time!
Tests that require constant massaging and tweaking because they constantly throw false positives take up a huge amount of a tester’s time in wasted maintenance.
考虑到维护的时间,并非所有自动化测试都能节省时间。
Sometimes a human is a more efficient oracle than a computer algorithm!
Sure, just about anything a computer does can be automated to some degree in some fashion, but there really are clearly some tests where it is more prudent and simpler to rely on a tester.
有时候,用人来判断比用计算机算法更简单、更可靠!
Don’t rely on automation to emulate your customers!
Test automation does not effectively emulate a human user. Sure, we have test methods in some of our internal automation frameworks to slow down simulated keystrokes (the actual keys are not being pressed on the keyboard), or simulate multiple or repeated clicks on a control or the mouse, and other tricks that try to emulate various user behaviors; however, test automation is generally poor at detecting behavioral issues such as usability, ease of use, or other customer value type assessments. Rely on the feedback from internal and external customers who are dog-fooding, self-hosting, and beta-testing your product (and act on it).
不要依赖自动化来模拟用户行为!尤其是在做可用性、易用性测试时。
Go under the covers! I think many testers rely too heavily on UI automation because they think it emulates user behavior (although most things such as populating a text box are simulated via Windows APIs), or perhaps because they don’t know how to dig into the product below the surface of the UI. Whatever the case, think about the specific purpose of the test. If it is easier to check a return value, or call an API to change a setting then go deep…and stop messing around on the surface. (It only complicates the test, wastes valuable machine cycles, reduces reuse across multiple versions, and often leads to long term maintenance costs. (For an example of this see my previous post.)
不要忘了我们还有单元测试!
Constantly massaging code contributes to false negatives! I have seen many cases where a tester designs a a UI automated test, and then tweaks a bit here and there to get it to run. Often times this tweaking contributes to a tests ineffectiveness in exposing problems, and may even hide other problems. Also, some tweaks are geared around synchronization issues (sync’ing the automated test with the system under test) and involve artificially slowing down the automation (usually by stopping or ‘sleeping’ the automated test process for a specific period of time). Other tweaks might hard-code parameters that then make the test fail on a different resolution or non-portable across different environments.
UI自动化测试往往倾向于不断调整脚本以适应被测试对象,这些调整往往导致问题暴露不够高效,甚至隐藏了某些问题。
STOP trying to automate every damn test!
As I stated before…just because we can automate something doesn’t mean that we should try to automate everything! We need to make rational decisions about what tests to automate, and what is the best approach to automating that test.
切忌自动化所有测试!
相关文章推荐
- 提供 Watir 入门 PPT 下载--有兴趣了解 Web 测试自动化的朋友可以看看
- 基于Selenium2和TestNG的自动化测试
- monkeyrunner-自动化测试
- 适合自动化的测试与不适合做自动化的测试
- 软件测试:测试自动化仅仅是测试工具吗?
- 简单介绍如何使用robotium进行自动化测试
- 上海沙龙 - 接口测试自动化经验分享
- webservice接口功能自动化测试准备工作
- 关于项目自动化测试架构的改良计划 - DataProviderEngine架构
- 如何实现嵌入式软件测试的自动化
- 从自动化测试到持续部署,你需要了解这些
- 用Visual Studio做自动化测试
- Android自动化测试中AccessibilityService获取控件信息
- Gallio 自动化测试平台
- iOS_ MonkeyTalk(1)(iOS与android的自动化测试工具)
- 游戏项目中的自动化测试和持续集成
- 【自动化测试技术QTP基础系列十二】---API之Reporter对象
- 自动化测试数据驱动之xml文件读取
- 不要忽略文件测试操作符
- 闲聊软件测试自动化(2): 测试自动化的实践中我们有哪些困惑?