Key to Successful Automation II - Tool

子曰: "工欲善其事, 必先利其器"
想把測試自動化做好, 善用工具是十分重要的.
這篇文章將介紹一些值得推薦的工具, 而且好消息是 - 它們都是免費, 跨平台.

1. STAF (Software Testing Automation Framework)
http://staf.sourceforge.net/
功能非常強大的測試工具.
STAF是一隻daemon process, 提供著各式各樣的服務, 像是電腦間的溝通, 開啟process, 處理檔案系統, 資源管理, 寄電子郵件等等...
功能十分齊全, 也很適合拿來開發自己的測試架構.
某些商業測試軟體也是以STAF為基礎開發.

STAF process可跨OS溝通

網站上有著完整的使用說明, 跨平台 (Windows, Linux, Mac), 穩定度高.
提供程式介面, 可以搭配各種程式語言使用 (Java, C, C++, Python, Perl, Tcl, Rexx).
在測試項(test case)的管理上有STAX輔助, 可以用XML的格式來撰寫測試項, 當然也能與unittest framework像是JUnit搭配使用.
個人使用迄今只有遇到一個問題, 就是在Windows日文64位元平台上會crash.
在選擇開源(open source)軟體時, 最怕的就是沒有人維護, 但STAF維護的狀況相當好, 可以從SourceForge的專案狀態看到:臭蟲不斷地被修正及釋出新版.
http://sourceforge.net/project/stats/?group_id=33142&ugn=staf&type=&mode=year

由於STAF是提供服務, 所以在使用上需要寫一些程式來整合所有的服務.
以下是一段Python的sample code

from PySTAF import *
import sys
try:
# Initialize一個STAF的handle
    handle = STAFHandle("MyTest")
except STAFException, e:
    print "Error registering with STAF, RC: %d" % e.rc
    sys.exit(e.rc)
# 利用STAF的PING service測試STAF這支process是否存在
result = handle.submit("local", "ping", "ping")
if (result.rc != 0):
    print "Error submitting request, RC: %d, Result: %s" % (result.rc, result.result)
# 利用STAF的VAR service得到電腦的作業系統資訊
result = handle.submit("local", "var", "resolve {STAF/Config/OS/Name}")
if (result.rc != 0):
    print "Error submitting request, RC: %d, Result: %s" % (result.rc, result.result)
else:
    print "OS Name: %s" % result.result
# Uninitialize STAF的handle
rc = handle.unregister()
sys.exit(rc)



2. Selenium
http://seleniumhq.org/
網頁測試工具.
在開始介紹它之前, 先提供一個數據給大家參考.
從Elisabeth Hendrickson的文章"Do Testers Have to Write Code?"中.
業界徵才-測試自動化的需求中, Selenium排名高居第一, 相信大家就能瞭解這個工具有多紅.
Selenium提供了網頁測試所需的各種功能. 打開瀏覽器, 輸入文字, 點擊按鈕, 檢查文字等..
跨平台, 跨瀏覽器, 支援各種程式語言 (一覽), 提供完整的使用說明讓Selenium一炮而紅.
最貼心的是, Selenium還提供Firefox的外掛可將使用者的操作錄成程式碼, 降低了開發的門檻.

以下是官網上的Python的sample code, 結合Python的unittest.
用起來相當直覺.

from selenium import selenium
# This is the driver's import.  You'll use this class for instantiating a
# browser and making it do what you need.

import unittest, time, re
# This are the basic imports added by Selenium-IDE by default.
# You can remove the modules if they are not used in your script.

class NewTest(unittest.TestCase):
# We create our unittest test case

    def setUp(self):
        self.verificationErrors = []
        # This is an empty array where we will store any verification errors
        # we find in our tests

        self.selenium = selenium("localhost", 4444, "*firefox",
                "http://www.google.com/")
        self.selenium.start()
        # We instantiate and start the browser

    def test_new(self):
        # This is the test code.  Here you should put the actions you need
        # the browser to do during your test.

        sel = self.selenium
        # We assign the browser to the variable "sel" (just to save us from
        # typing "self.selenium" each time we want to call the browser).

        sel.open("/")
        sel.type("q", "selenium rc")
        sel.click("btnG")
        sel.wait_for_page_to_load("30000")
        self.failUnless(sel.is_text_present("Results * for selenium rc"))
        # These are the real test steps

    def tearDown(self):
        self.selenium.stop()
        # we close the browser (I'd recommend you to comment this line while
        # you are creating and debugging your tests)

        self.assertEqual([], self.verificationErrors)
        # And make the test fail if we found that any verification errors
        # were found

其它像是AutoIt(Windows使用者介面自動化), Watir(網頁測試工具, Ruby專用)也都是很常見的工具.
善用這些工具, 站在巨人的肩膀上讓我們能夠走得更遠, 更穩.

-----------------------------------------------------------------------------------
"To do a good job, one must first sharpen one's tools." Chinese philosopher Confucius said.
Good tools are prerequisite to the successful execution of test automation.
This article will discuss about some useful tools, and the good news is, they are all free and cross platform.

1. STAF (Software Testing Automation Framework)

http://staf.sourceforge.net/

STAF is a very powerful testing tool with a variety of functions.

It is a daemon process as a service provider; and it can handle communications between computers, launch process, provide file system utility, manage resource, and send email, etc.

STAF provides basic testing framework utility and can be used to develop advanced testing framework. Some commercial tools are based on STAF, too.

STAF process can communicate between different platforms

In STAF official website, a comprehensive manual can help you adopt it.
STAF is cross platform (Windows, Linux, Mac), and is very stable.
Besides, STAF provides programming interface with Java, C, C++, Python, Perl, Tcl, Rexx, etc, so you can integrate it with your familiar programming language.
About test suite management, STAX supports XML style test case description, or you can choose unittest framework like JUnit to manage test suites.
There is only one problem when I used STAF so far: It crashes on the Windows Japanese X64 platform because of some special double-byte characters.
Maintenance problem should be considered when choosing open source tool. Fortunately, from the statistics of source forge, STAF project status is excellent, bug is fixed and minor version is released frequently.
http://sourceforge.net/project/stats/?group_id=33142&ugn=staf&type=&mode=year

STAF provides services, so team 

Below is sample code in Python.

from PySTAF import *
import sys

try: 

# Initialize a STAF handle
    handle = STAFHandle("MyTest")
except STAFException, e:
    print "Error registering with STAF, RC: %d" % e.rc
    sys.exit(e.rc)

# Use STAF's PING service to test if STAF process exists

result = handle.submit("local", "ping", "ping")

if (result.rc != 0):
    print "Error submitting request, RC: %d, Result: %s" % (result.rc, result.result)

# Use STAF's VAR service to get OS information

result = handle.submit("local", "var", "resolve {STAF/Config/OS/Name}")

if (result.rc != 0):
    print "Error submitting request, RC: %d, Result: %s" % (result.rc, result.result)
else:
    print "OS Name: %s" % result.result

# Uninitialize the STAF handle

rc = handle.unregister()

sys.exit(rc)

2. Selenium

http://seleniumhq.org/

A web application testing tool.

Before introducing it, I would like to show you how "Hot" it is now.
From the article of Elisabeth Hendrickson "Do Testers Have to Write Code?", the top automaton technology required for software testers is Selenium.
Selenium provides common utilities for web testing, ex, launching browser, inputting text, clicking button, and asserting text...
It can also support cross platforms, cross browser, cross programming language, and of course, provide a comprehensive user guide.
Besides, the most amazing is that Selenium has Firefox plug-in to record user actions into test scripts, it can eliminate the barriers of adoption dramatically.


Below is the Python sample code from Selenium official website, combined with Python unittest.
The use is very intuitive.

from selenium import selenium
# This is the driver's import.  You'll use this class for instantiating a
# browser and making it do what you need.

import unittest, time, re
# This are the basic imports added by Selenium-IDE by default.
# You can remove the modules if they are not used in your script.

class NewTest(unittest.TestCase):
# We create our unittest test case

    def setUp(self):
        self.verificationErrors = []
        # This is an empty array where we will store any verification errors
        # we find in our tests

        self.selenium = selenium("localhost", 4444, "*firefox",
                "http://www.google.com/")
        self.selenium.start()
        # We instantiate and start the browser

    def test_new(self):
        # This is the test code.  Here you should put the actions you need
        # the browser to do during your test.

        sel = self.selenium
        # We assign the browser to the variable "sel" (just to save us from
        # typing "self.selenium" each time we want to call the browser).

        sel.open("/")
        sel.type("q", "selenium rc")
        sel.click("btnG")
        sel.wait_for_page_to_load("30000")
        self.failUnless(sel.is_text_present("Results * for selenium rc"))
        # These are the real test steps

    def tearDown(self):
        self.selenium.stop()
        # we close the browser (I'd recommend you to comment this line while
        # you are creating and debugging your tests)

        self.assertEqual([], self.verificationErrors)
        # And make the test fail if we found that any verification errors
        # were found

There are some excellent tools as well. For example, AutoIt is a tool for Windows GUI automation, and Watir is a web application testing tool for Ruby.
By leveraging these tools and standing on the shoulders of giants, we are able to go further.