What do you do? Checking v.s.Testing

~三年前我的一天~
10:00 AM
一如往常, 早上十點左右進辦公室,
打開電腦, 看郵件. "很好, 今天的daily build沒有build fail"
將我的虛擬機器 (Windows XP x86) 還原.
開始安裝最新的build, 等待的過程中繼續看信.
過了十分鐘左右, 安裝成功, 開始執行防火牆的基本功能測試.

11:00 AM
由於基本功能測試已經跑了幾十次, 我早已駕輕就熟.
"XP平台上的基本功能運作良好" 我心裡想
但由於產品的驅動程式在不同平台上都不同.
保險起見,
還必須測試基本功能在另外2台虛擬機器上(Windows Vista x86, Windows Vista x64).

12:00 AM
測試結束, 3個平台的基本功能皆正常.
但對於每個build的例行公事感到無奈甚至厭煩.
與同事一同去吃中餐.

~Checking v.s.Testing~
在產品開發過程中, 為了確保每天build的品質.
通常都會有所謂的acceptance test.
唯有這些基本測試項目通過後, 我們才能確定這個build是有品質的.
但由於測試項目固定.
執行數十次後, 覺得自己就像是個機器人.
不, 事實上可能比機器人還不如, 因為機器人還不會犯錯.
如此一來很可惜地浪費了人的聰明才智, 更有可能賠上的是對測試的熱情.

前陣子看到Michael Bolton的文章在探討Testing 跟Checking, 還蠻有趣的.
Checking Is Confirmation. 

"Checking是一種確定的過程"

我們已經對行為有所預期, 只是去檢查是否合乎預期.

舉個例子來說, 就像執行已經定義好的測試項: 點擊桌面的捷徑後軟體介面應該被開啟.
Testing Is Exploration and Learning.

"Testing是一種探索與學習的過程"

與軟體有所互動.

不斷地觀察, 思考, 探索, 學習, 

舉例來說, 觀察軟體在千禧年下的反應, 

從結果再去推想是否其他部分也有受到日期的影響, 

文章中有提到一個點: Checks Are Machine-Decidable.

當預期結果是可以描述的時候, 機器或是其他人已經有能力和我們作一樣的檢查.

既然如此, 

大家可以思考看看, 每天的工作中有多少是所謂的Checking?
是不是有機會自動化Checking的部分呢?
讓自己從無盡的測試中鬆一口氣, 去做更有價值的事.

----------------------------------------------------------------------------------------------------------------------------------

~One day three years ago~

10:00 AM

As usual, I arrived to office around 10 o'clock.
Powered on working machine, and checked emails. "Great, there is no failed build today"
Reverted my virtual machine (Windows XP x86) to clean snapshot.

I installed the latest build on virtual machine. During installation, I checked other emails.
After 10 minutes, installation was successful, and I began to test basic functions of firewall.

11:00 AM
Because I had executed test cases dozens of times, I was very familiar with them.
"Basic functions work well on Windows XP platform" I thought.
However, drivers of software were different in every platform.
For safety, I had to test basic functions on other 2 virtual machines (Windows Vista x86, Windows Vista x64)

12:00 AM
Tests were finished, all basic functions worked well on 3 platforms.
I was bored by these routine checks of every build.

Had launch with my colleagues.

~Checking v.s.Testing~

In software development, there is build acceptance test to make sure quality of daily build.

Only if all basic test cases pass, we can say this build is qualified.

However, these basic test cases are fixed.

After dozens of times of execution, people may feel what they act is like a robot.

No, maybe worse than a robot because people may make errors, but it doesn't.

It is a pity to waste wisdom of humans, and lose passion of testing.

I read article from Michael Bolton to discuss about Testing and Checking recently.

It is very interesting.

"Checking Is Confirmation." He said.

We already have expected result, and we just confirm if it meets our expectation.

For example, a predefined test case: Main console should be launched if user clicks shortcut.

"Testing Is Exploration and Learning." He also said.

We interact on software.

Observe, think, explore, and learn it.

For example, we observe the interaction of millennium.

Then we may be inspired and think about if other components are effect by date, too.

There are another interesting words in the article. "Checks Are Machine-Decidable."

If expected result is describable, machines or other people should already have abilities to check it.

So you can think about it, how much of your daily work is checking?

Is it possible to automate it?

Rescue yourself from endless routine checking, and do something more valuable.