Effective Software Test Automation

Rating:
Summary: A significant contribution to the software QA profession
Review: Before proceeding you need to know that this book is specific to the Microsoft .NET environment, and is not a general book on test automation. That said, this is possible one of the most important books recently published on the subject for those who develop for Microsoft technologies, and the tool it leads you through developing is worth many times the price of the book. The best way to describe this book's strengths and benefits is to align them to three different classes of reader who will benefit the most:

(1) developers or QA practitioners who need to quickly implement a testing tool that dramatically reduces the time it takes to execute test cycles. This book will service this audience as a technical manual for the tool, which can be downloaded in source and binary formats from the publisher's web site.

(2) developers who want a baseline tool that can be extended and modified to suit their specific needs, or to be integrated into (or augment) a suite of existing testing tools. The clear explanation of how this tool was designed and the code that makes up the tool will easily meet this audience segment's needs. More importantly, the book even shows how to develop assemblies with which to test the tool.

(3) developers who want to improve their own skills by examining an integrated application that has been engineered by experts. The tool upon which this book is based is one of the most elegantly designed and implemented examples of good programming practices and software engineering. Not only does it show how to harness some of the capabilities of the .NET framework and associated IDE, but also shows how to integrate into Microsoft Office applications and create a seamless enterprise application. The tool in this book is integrated into Microsoft Excel (version 2000), and the concepts, use of XML, and the way everything is tied into a coherent application that provides useful services exemplify how to develop business applications.

The tool itself is one of the most useful and clever test automation approaches I've seen in any environment. It auto-discovers what must be tested, and develops test scripts with virtually no intervention on the part of QA. It also dramatically reduces test time, and will significantly contribute to the deployment of defect-free applications, especially in a rapid development environment (such as those shops using extreme programming or agile methods). While the tool will not completely replace other testing tools, it will nicely augment them. I think the best use of this tool is in the development domain to be used for unit and integration testing. It also supports incremental regression testing, which can be effectively executed by the build manager before promoting to QA for final test and release. Of course, this tool will also fit nicely within the QA domain, especially with its ability to support both white- and black-box testing.

In combination this book and the associated tool are, in my opinion, important contributions to the QA profession, and to teams working with Microsoft technologies.

Rating:
Summary: A significant contribution to the software QA profession
Review: Before proceeding you need to know that this book is specific to the Microsoft .NET environment, and is not a general book on test automation. That said, this is possibly one of the most important books recently published on the subject for those who develop for Microsoft technologies. Moreover, the tool it leads you through developing is worth many times the price of the book. The best way to describe this book's strengths and benefits is to align them to three different classes of reader who will benefit the most:

(1) developers or QA practitioners who need to quickly implement a testing tool that dramatically reduces the time it takes to execute test cycles. This book will service this audience as a technical manual for the tool, which can be downloaded in source and binary formats from the publisher's web site.

(2) developers who want a baseline tool that can be extended and modified to suit their specific needs, or to be integrated into (or augment) a suite of existing testing tools. The clear explanation of how this tool was designed and the code that makes up the tool will easily meet this audience segment's needs. More importantly, the book even shows how to develop assemblies with which to test the tool.

(3) developers who want to improve their own skills by examining an integrated application that has been engineered by experts. The tool upon which this book is based is one of the most elegantly designed and implemented examples of good programming practices and software engineering. Not only does it show how to harness some of the capabilities of the .NET framework and associated IDE, but also shows how to integrate into Microsoft Office applications and create a seamless enterprise application. The tool in this book is integrated into Microsoft Excel (version 2000), and the concepts, use of XML, and the way everything is tied into a coherent application that provides useful services exemplify how to develop business applications.

The tool itself is one of the most useful and clever test automation approaches I've seen in any environment. It auto-discovers what must be tested, and develops test scripts with virtually no intervention on the part of QA. It also dramatically reduces test time, and will significantly contribute to the deployment of defect-free applications, especially in a rapid development environment (such as those shops using extreme programming or agile methods). While the tool will not completely replace other testing tools, it will nicely augment them. I think the best use of this tool is in the development domain to be used for unit and integration testing. It also supports incremental regression testing, which can be effectively executed by the build manager before promoting to QA for final test and release. Of course, this tool will also fit nicely within the QA domain, especially with its ability to support both white- and black-box testing.

In combination this book and the associated tool are, in my opinion, important contributions to the QA profession, and to teams working with Microsoft technologies.

Rating:
Summary: Useful, Focused and a Good .Net Hands-On Case Study
Review: Effective Software Test Automation presents a hands-on case study with code to create a test tool capable of generating test scripts, data stores and output summary. While the concepts discussed certainly apply to the development of most tools, the code provided is specific to the .Net environment using C# as the development language, and the bulk of the book is dedicated to describing how the tool is created in that environment with full code listings. I would recommend this book to testers and developers who are new to the .Net development environment or those who may have worked in it and want to start considering a tool to generate test scripts.

This book accomplishes two important goals. The first is the step by step creation of the test tool. I implemented the code to create the automated test tool to generate the data store and the test script up through chapter 7 without any problems. It worked as described, and the code, along with its description, for the remainder of the book was well presented. The second significant contribution the book makes is the tour it takes the reader through aspects of the .Net environment. It demonstrates the robust development qualities in .Net. An example of this is the hands-on discussion of the namespace system.Reflection. Further, it provides an insightful guide to referencing the MS Excel Object Library through example as the code dynamically creates a datastore in an Excel worksheet.

The authors also provide an overview on how to use the tool effectively and provide adequate notes on usage context. Towards the latter part of the book they are adamant that testers still need to analyze the requirements and understand the data used to test the application. For instance, testers still need to understand concepts such as boundary value analysis, equivalence partitions and other testing techniques which are more thoroughly addressed in other theoretical texts. It should be noted, that the tool presented by the authors does not guarantee code coverage (i.e. path analysis within a method). Readers will have a better appreciation for the tool if they study some of the classics in testing theory by Glenford Myers, Robert Poston and Paul C. Jorgensen.

The tool itself is certainly useful from the standpoint of regression testing. In other words, once an application is stable in its functionality, the test scripts generated can form a baseline of testing when changes are introduced. The authors do mention XP, but I have to comment as to the effective use of the tool in XP development. In XP, testing is based on test driven development (TDD). In TDD, the test script is written BEFORE the implementation code is ever produced. In TDD (agile development) the test harness is an up front statement of the design stating what the code should do - not what it already does because it hasn't been written yet. Their tool requires the implementation code (actually, the assembly) to be created before it creates the test script. Again, though, there are many development efforts where TDD is not used (the majority actually ) and the tool the authors present provides a good start for analysis of code completed by developers to generate an initial set of scripts when a maintenance environment receives transitioned code. Their observations of the spirit of XP are very accurate. I am only expressing a philosophical concern in the testing arena.

In essence, the tool can have benefits, but like any tool should be used in the proper context. Developers should not assume that any tool can replace human intuition and solid testing techniques.

Overall, I found the book to be well focused and clearly written. I only have two criticisms of the book itself, which is why I would rate it four out of five stars (which for me is a good rating). I think the title or cover should have noted something about .Net (e.g. "A Case Study Using .Net"), even though the authors claim the concepts are generic. The concepts are, but the implementation is not since legacy programming environments do not have support for integrating an XML framework in the code for comments. In my previous life as a UNIX and C developer and considering the cost, I would get very little benefit from the text since the "generic" part is limited after Chapter 2. Second, while the authors provided a high level discussion of the features the tool would provide in the later part of Chapter 2, it would have been helpful had there been illustrations with a brief description of how it would be used earlier. The tool creation was presented piece by piece, and while I was able to follow along with the authors, I found myself immediately skipping ahead to Chapter 7 to get an idea of what the end product would look like. After doing so, I went back to where I left off in Chapter 4 and the rest was fine. It's a small thing actually, but an early picture or two of where the code is taking the reader is helpful.

Rating:
Summary: Useful, Focused and a Good .Net Hands-On Case Study
Review: Effective Software Test Automation presents a hands-on case study with code to create a test tool capable of generating test scripts, data stores and output summary. While the concepts discussed certainly apply to the development of most tools, the code provided is specific to the .Net environment using C# as the development language, and the bulk of the book is dedicated to describing how the tool is created in that environment with full code listings. I would recommend this book to testers and developers who are new to the .Net development environment or those who may have worked in it and want to start considering a tool to generate test scripts.

This book accomplishes two important goals. The first is the step by step creation of the test tool. I implemented the code to create the automated test tool to generate the data store and the test script up through chapter 7 without any problems. It worked as described, and the code, along with its description, for the remainder of the book was well presented. The second significant contribution the book makes is the tour it takes the reader through aspects of the .Net environment. It demonstrates the robust development qualities in .Net. An example of this is the hands-on discussion of the namespace system.Reflection. Further, it provides an insightful guide to referencing the MS Excel Object Library through example as the code dynamically creates a datastore in an Excel worksheet.

The authors also provide an overview on how to use the tool effectively and provide adequate notes on usage context. Towards the latter part of the book they are adamant that testers still need to analyze the requirements and understand the data used to test the application. For instance, testers still need to understand concepts such as boundary value analysis, equivalence partitions and other testing techniques which are more thoroughly addressed in other theoretical texts. It should be noted, that the tool presented by the authors does not guarantee code coverage (i.e. path analysis within a method). Readers will have a better appreciation for the tool if they study some of the classics in testing theory by Glenford Myers, Robert Poston and Paul C. Jorgensen.

The tool itself is certainly useful from the standpoint of regression testing. In other words, once an application is stable in its functionality, the test scripts generated can form a baseline of testing when changes are introduced. The authors do mention XP, but I have to comment as to the effective use of the tool in XP development. In XP, testing is based on test driven development (TDD). In TDD, the test script is written BEFORE the implementation code is ever produced. In TDD (agile development) the test harness is an up front statement of the design stating what the code should do - not what it already does because it hasn't been written yet. Their tool requires the implementation code (actually, the assembly) to be created before it creates the test script. Again, though, there are many development efforts where TDD is not used (the majority actually ) and the tool the authors present provides a good start for analysis of code completed by developers to generate an initial set of scripts when a maintenance environment receives transitioned code. Their observations of the spirit of XP are very accurate. I am only expressing a philosophical concern in the testing arena.

In essence, the tool can have benefits, but like any tool should be used in the proper context. Developers should not assume that any tool can replace human intuition and solid testing techniques.

Overall, I found the book to be well focused and clearly written. I only have two criticisms of the book itself, which is why I would rate it four out of five stars (which for me is a good rating). I think the title or cover should have noted something about .Net (e.g. "A Case Study Using .Net"), even though the authors claim the concepts are generic. The concepts are, but the implementation is not since legacy programming environments do not have support for integrating an XML framework in the code for comments. In my previous life as a UNIX and C developer and considering the cost, I would get very little benefit from the text since the "generic" part is limited after Chapter 2. Second, while the authors provided a high level discussion of the features the tool would provide in the later part of Chapter 2, it would have been helpful had there been illustrations with a brief description of how it would be used earlier. The tool creation was presented piece by piece, and while I was able to follow along with the authors, I found myself immediately skipping ahead to Chapter 7 to get an idea of what the end product would look like. After doing so, I went back to where I left off in Chapter 4 and the rest was fine. It's a small thing actually, but an early picture or two of where the code is taking the reader is helpful.

Rating:
Summary: Somewhat limited
Review: Effective Software Test Automation walks the reader through building an automated testing tool in C#.

The book describes its target audience as intermediate to advanced programmers. This is accurate since some of the examples get quite complex, such as reflection. The book provides a quick overview of C# for programmers that know other languages. As someone who does not know C#, I found this overview sufficient for understanding the book. The book also gives step-by-step instructions on how to follow the examples using Visual Studio .NET.

The authors state that the concepts in this book can be applied to other languages, such as Java. While this is true, many of the examples are .NET specific. One of the chapters covers testing the Windows Registry. Other chapters cover how to use .NET to accomplish reflection, testing spreadsheets, etc.

The testing tool that ends up being developed is similar to a subset of commercial tools, like Insure++. The tool developed by the book looks as if it could be a good open source tool for C#. If you are a C# developer, it is worth buying the book for the source code alone. The book could be used as a user guide (or developer guide if you are extending the tool.)

I found this book to be a great walkthrough of a C# project. The authors take you through the iterations of the project as it advances. Overall, I would recommend the book for .NET developers.

Rating:
Summary: Good tool for testing in .NET
Review: Effective Software Test Automation walks the reader through building an automated testing tool in C#.

The book describes its target audience as intermediate to advanced programmers. This is accurate since some of the examples get quite complex, such as reflection. The book provides a quick overview of C# for programmers that know other languages. As someone who does not know C#, I found this overview sufficient for understanding the book. The book also gives step-by-step instructions on how to follow the examples using Visual Studio .NET.

The authors state that the concepts in this book can be applied to other languages, such as Java. While this is true, many of the examples are .NET specific. One of the chapters covers testing the Windows Registry. Other chapters cover how to use .NET to accomplish reflection, testing spreadsheets, etc.

The testing tool that ends up being developed is similar to a subset of commercial tools, like Insure++. The tool developed by the book looks as if it could be a good open source tool for C#. If you are a C# developer, it is worth buying the book for the source code alone. The book could be used as a user guide (or developer guide if you are extending the tool.)

I found this book to be a great walkthrough of a C# project. The authors take you through the iterations of the project as it advances. Overall, I would recommend the book for .NET developers.

Rating:
Summary: A great experience
Review: I just purchased and finished reading this book last week and completed the automated tool project. Now I am already using this tool for my test tasks. I want to write comments on it now in order to help others to make a decision.

This book is easy to read, has just enough examples to help me understand the contents easily.
The writer is creative, and effective. The content cannot be found from other sources.
It covers some interesting programming techniques and software test automation. Software test is really fully automated after I finished reading this book. Of course, the test procedure is very much different from the traditional methods and tools.
With regard to programming I learned from this book:
1. How to use reflection.
2. How to use late binding.
3. How to program XML and Xpath.
4. How to program Excel using win32 API.
5. How to use CodeDom to write code, which is really cool.
6. How to program to access the Windows Registry.
7. Learned some interesting math and geometry, too.

With regard to testing, what I learned from this book includes:
1. What is software test automation. The definition is unique in this book.
2. Developed a fully automated test tool. I feed the test tool with an application, receive test results and hand the bug reports to the developers to fix them.
3. Learned the way this book taught to create a complete and real object for integration testing. I also learned how to make stub and mock objects. All the methods are automated by the tool.
4. Automated Unit testing, regression testing and integration testing. After I developed the tool, I don't need to write other code to completely test a project.
5. Learned a new way to create external database for automated software test.
6. Learned how to automatically generate multiple testing cases and then generate a flawless test script. The test script generation is totally data-driven.
7. Learned how to add new testing functions to this tool. I added a function to do performance testing the first day by my own and the book did not mention.
8. It also taught me to write one test script to test an application. Test management becomes easier than ever before.

This book solves some critical software test problems. I believe the future software developments will soon adapt this book to their projects.
I recommend this book to all software programmers, software testers, managers and other interested parties.

In addition, there are some typos need to be fixed. I look forward to the 2nd edition without typos.

Rating:
Summary: We value this book useful
Review: This book fits our current and future projects and provides enough material to complete our tasks. Our group likes this book because:
1. It is easily to read. The topics are well presented and clearly defined. The examples are designed to solve problems from real world software development projects.
2. It presents software test basics in the words of programming languages.
3. The book introduces the test concepts and uses C# .NET as examples to develop a tool to do almost all the tedious jobs for software testing, which the other tools can not do, and the other books never mention.
4. It explains how and why each line of code is used to achieve the software test automation instead of specific programming technologies of a language.
5. It is not a software test dictionary (or a bible), or an inventory of available software testing tools. I don¡¯t like to read a book which degrades the works of the others.
6. It repeats little content the other books, web pages or sources cover. The testing functions are effective and different from the available testing tools as I know of (Winrunner, Rational Test)
7. It helps developers produce good code, helps testers conduct the test with ease, and helps managers setup a high code quality standard.
8. Finally, it helps us to get a really and completely automated testing tool.


Rating:
Summary: Somewhat limited
Review: This book is NOT generic to all types of platforms as it purports. In fact, if you are not programming in an Windows environment you will get very little from this book.

The test tool created is only one and claims, on the jacket cover, to be a great replacement for the major vendor tools. While I agree the major tools aren't all they claim to be, neither is this one. It's limited in usage to very specific kinds of tests and is definitely not the silver bullet the jacket would have you believe. In fact, it's a similar subset to some of the vendor tools.

There are even more areas in which the .Net platform can be used for testing. Some key areas such as debugging and tracing are missing from this text. How about the use of nUnit? What about using .Net to access databases? This is not a thorough treatment, but it's fine that it's not. It's just that it claims to be more than it is.

After all is said and done, this book is worth buying for Windows test professionals, so that's why I give it 3 stars. I just resent being told, again and again, that someone has the "answer" to software testing. Get with it folks, software testing takes discipline and rigor and there is no way to "microwave" your project through a thorough software test process. This book does provide valuable knowledge for your Windows test project and great info on .Net to ADD to your existing test expertise.