在一間小會議室, 二個工程師正在討論下一專案的自動化測試.
"Julian, 這次的測試自動化要怎麼做呢?" 我同事問.
"應該從前一版最花時間的測試先下手吧." 我回答.
"那應該是回歸測試(Regression Test). 這對我們是一個挑戰, 因為這次的測試工程師比上一版本時更少." 他說道.
"恩, 我們可以先將驗收測試(Acceptance Test)自動化, Acceptance test在專案中會被執行數百次." 我說道.
"自動化測試的重複使用也很重要, 我們不可能每次都砍掉重寫. 如果我是老闆, 一定不願意投資不能重複使用的自動化測試." 我馬上接著說.
"我同意, 不過這並不容易. 因為使用者介面不斷地改變, 何況我們也不清楚下一版本專案的規格, 甚至整個專案架構都會有所改變. 有太多不確定因素了." 他說道.
"你的考量很對, 但也只能先試著用對的方法去做. 一般狀況下, 越過使用者介面的自動化測試比較有機會能重複利用, 因為程式動到底層的機會比較小." 我說道.
"好, 我們應該要選擇最有價值的方法." 他同意我的說法.
你可能已經瞭解自動化測試帶來的好處, 也有決心去做好它.
但是通常會遇到的第一個問題是"要怎麼開始?"
在開始研究自動化測試的框架(framework)及可用的程式語言之前.
更重要的是瞭解測試自動化的目標.
測試自動化只是測試方法的一種, 它絕不是銀彈能拿來解決任何問題. 它有些限制: 成本昂貴及入門門檻高, 也不適合能來做某些類型的測試.
比較好的做法是, 根據之前專案的經驗, 找出最需要測試自動化的部分, 並訂定目標去解決問題.
舉例來說, 在出hotfix之前, 通常都要花費很多精力測試有沒有改出新的bug.
這個例子中, 可以將目標設定為將基本功能的測試都自動化, 這樣每次新的hotfix釋出之前, 只要跑過自動化測試就能比較安心地將hotfix給用戶.
還有一些常看到的目標: 測試平台組合太多, 需要自動化測試幫忙. 或是需要一套關鍵字測試自動化系統來節省測試工程師開發的時間. 或是高重複使用率的測試自動化.
目標決定好之後, 測試自動化的方向就不容易走偏. 可以開始尋找或是開發框架及工具, 向有同樣經驗的人諮詢, 並可以規畫整個測試自動化的策略. "目標不一定是需要達到的, 目標往往只是用來說明瞄準的方向." 李小龍說過.
第一次的測試自動化是很艱難的. 即使以失敗告終, 也不要灰心. 在過程中可以學習到很多, 而且方向對的話, 成果只會一次比一次好.
自動化測試是很值得嘗試的. 它不只能讓測試工程師多一種測試的選擇, 也能帶來一定的成就感. 當我們有更多種選擇的時候, 也代表我們能用更有效率的方法來測試. 在這個敏捷開發(Agile Development)的時代中, 測試時間永遠都是不夠, 我們應該要能選擇最適合的方法來測試軟體.
你準備好開始自動化了嗎?
--------------------------------------------------------------------------------------------------------
In a small meeting room, two engineers were discussing test automation for next project.
"Julian, what is our automation plan?" My colleague asked.
"Automate the most effort consuming work in previous project." I answered.
"I believe it is regression test effort. It is a challenge for us. In this project, there are even less test engineers than last one." He said.
"Yes, we should focus on automating acceptance test first. Acceptance test is ususally run a hundred times during development" I said.
"I suggest test automation reuse rate be considered, too, because we shouldn't redevelop test automation in every project, and then drop it. If I were boss, I don't want to invest automation which can't be reused in next version." I also said.
"I agree, but it is not easy, because user interface is always changing, and nobody knows the spec of next version now; maybe there will be code re-architecture. There are too many uncertainties." He said.
"Your concern is right. However, in general cases, automation bypassing user interface can be reused because the layer behind user interface is rarely changed, we should try to make it." I said.
"OK, we should select the most valuable way as our direction." He agreed with me.
You may understand the benefits of test automation, and have determination to adopt it.
The next problem is :"How to start it?"
Before starting to survey testing framework or scripting language for test automation.
The more important thing is to understand what is the target of automation.
Automation is one of test methodologies, however, it is not a silver bullet. Automation cost is expensive and entry point is high, and it is not suitable for some kind of testing.
Suggestion is to refer to the point of suffering in current testing, and set target to solve it.
For example, in the project of sustained product, it takes huge efforts to test side effects of hotfix, because there might be side effects of basic functions after applying hotfix.
In this case, we can set the target to automate basic function testing to reduce regression efforts for every hotfix.
There are other kinds of targets, including the target of executing tests on comprehensive testing platforms, or the target of creating key-word based test automation library for other testing engineers to make up test scripts easily, or the target of high reuse rate test automation.
Once you decide the target, it helps you find correct direction for automation.
Based on it, you can survey or develop suitable tools and testing framework, consult with experienced people, and work on automation strategy. "A goal is not always meant to be reached, it often serves simply as something to aim at." Said by Bruce Lee
The first automation is not easy. If it fails in the end, don't be dejected. There are lots of things to be learned, and it will get better and better if the target is correct.
Automation is still worthy of investment. It can complete test engineers' skill set, and it can inspire test engineers by the sense of achievement. The more skills we have, the more efficient we are. Testing time is never enough in the era of Agile development, so, we need to select the best way to test software.
Are you ready to start automation?
No comments:
Post a Comment