Blog article Red Badger "What is Specification by Example?" → “What are the differences between Specification by example, Behaviour-Driven development (BDD) and Acceptance Testing-Driven Development (ATDD)?” #.
“Software development collaboration tool” → “Collaboratively define Acceptance Tests” ( additionally to xUnit tests ).
“Software testing tool”.
“Wiki”.
“Web server”.
Fixture Gallery.
SourceForge "Fixture Gallery", SourceForge "Fixture Gallery" - “Simple, straight-forward examples of FIT/FitNesse fixtures for Java and .NET test environments. Documentation and examples are available in the form of a live FitNesse wiki and PDF”.
The OpenSource robotframework, Google Code "robotframework. A generic test automation framework", GitHub "robotframework" in Python - “A generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). It has easy-to-use tabular test data syntax and utilizes the keyword-driven testing approach. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new keywords from existing ones using the same syntax that is used for creating test cases”.
Manual installation: Unpack the downloads in different directories, and open a shell ( Powershell or Command.com on Windows ) in the directories, before executing the commands:
GitHub "Thucydides BDD Reporting Library" - “Thucydides is a tool that lets you use WebDriver-based unit or BDD tests to write more flexible and more reusable WebDriver-based tests, and also to generate documentation about your acceptance tests, including a narrative description of test, along with the corresponding screen shots, and also high-level summaries and aggregations of the test r… ”.
“PyHamcrest” can´t be updated by well-known automatic installation processes, on Python 2.7 even after a proper installation of the Python package “behave”. An installation attempt by “easy_install pyhamcrest” is aborted due to syntax errors in downloaded Python files and other faults . Is “PyHamcrest” just for Python 3.x or also still for Python 2.x ?!
The OpenSource Github "heynemann / pyccuracy" - “A Behaviour-Driven-Development-style tool written in Python that aims to make it easier to write automated acceptance tests. It improves the readability of those tests by using a structured natural language – and a simple mechanism to extend this language – so that both developers and customers can collaborate and understand what the tests do”.
Python Package Index "pytest-bdd" - “BDD library for the py.test runner”, “ytest-bdd implements a subset of Gherkin language for the automation of the project requirements testing and easier behavioral driven development. Unlike many other BDD tools it doesn't require a separate runner and benefits from the power and flexibility of the pytest. It allows to unify your unit and functional tests, easier continuous integration server configuration and maximal reuse of the tests setup”.
Serverspec - RSpec tests for your servers configured by Puppet, Chef or anything else - “With Serverspec, you can write RSpec tests for checking your servers are configured correctly. Serverspec tests your servers' actual state by executing command locally, via SSH, via WinRM, via Docker API and so on. So you don't need to install any agent softwares on your servers and can use any configuration management tools, Puppet, Chef, CFEngine and so on. But the true aim of Serverspec is to help refactoring infrastructure code”.
GitHub "serverspec" - “RSpec tests for your servers configured by Puppet, Chef or anything else even by hand”.
“SpecFlow aims at bridging the communication gap between domain experts and developers by binding business readable behavior specifications and examples to the underlying implementation”.
“Our mission is to provide a pragmatic and frictionless approach to Specification-By-Example for .NET projects”.
“SpecFlow also supports the concepts of Acceptance Test Driven Development (ATDD) and Behavior Driven Development (BDD), which are often used synonymously with Specification-By-Example”.
The commercial online service Relish - “Living documentation. Combine your product's documentation with cucumber tests. Publish, browse, search & share on Relishapp…”.
Opinion by Gojko Adzic: “Many people I met at conferences misunderstand and think that incrementally building up a documentation system means oging back to the Waterfall ideas of big up-front analysis. During the presentation 'How to Sell BDD to the business' in November 2009, Dan North said that BDD is effectively V-Model compressedinto two weeks” .
Suggestion by Gojko Adzic: “Instead of trying to collaborate with business users on specifying how to put things into the system, we should start with examples of output”.
I am still looking for a free PDF copy of the book. Please contact me, if you have a freely downloaded item and if you are willing to share it with me, for free.
The free PDF book with Comics Chris Matts "Agile2010 Nashville. Gaylord Opryland Resort" ( The PDF book “Real Options at Agile 2009” is not available for (free) download at this site ) → “Feature Injection and Test Driven Requirements”.
“Specifications with examples are acceptance tests”.
“Scripts are not specifications” .
“A script explains how something can be tested. It describes business functionality through lower-level interactions with the system. A script requires the reader to work back from the actions and understand what's really important. Scripts also bake the test into workflow and session constrains, which migh changein the future even when the underlaying”. business rules don´t change.
“A specification explains what the system does. It focusses on the business functionality in the most direct way possible. Specifications are shorter because because they describe buiness concepts directly. That makes them easier to read and understand than scripts”.
Suggestion by Gojko Adzic: “Avoid automating existing manual test scripts”.
Suggestion by Gojko Adzic: “Don't check business logic though the user interface.”Traditional test automation tools mostly work by manipulating user interface objects. Most automation tools for exeutable specifications can go below the user interface and talk to application programming interfaces directly ( API ) .
Suggestion by Gojko Adzic: “Automate below the skin of the application. Workflow and session rules can often be checked only against the user interface layer. But that doesn´t mean that the only option to automate those checks is to launch a browser. Instead of automating the specifications through a browser, several teams developing web applications saved a lot of time and effort going right below the skin of the application - to the HTTP layer”.
Gojko Adzic suggests: “Specify user interface functionality at a higher level of abstraction”.
Gojko Adzic discovered: “Automating user interfaces. Almost all the teams I interviewed made the same mistake early on. They specified tests intended to be automated though user interface as series of technical tips, often directly writing user interface automation commands in their specification” .
Pierre Veragen realized “User interface tests were task oriented ( click, point ) and therefore rightly coupled to the implementation of the GUI, rather thant activity oriented.... FitNesse tests wre organized according to the way UI was set up”.
“When the UI was updated, all these tests had to be updated...”.
“A small change to the GUI, adding a ribbon control, broke everything. There was no way we could update the tests”.
Lance Walton explains:
“The next stage was to understand that the workflow still had to do with the solution we designed, so actully let's forget about workflow and focus on what the user is trying to achieve”.
“So we had pages that contained the details, then we had the task level above that, and then we finally had the goal that the user is trying to achieve”.
Gojko Adzic suggests: “Avoid recorded UI tests”, “Many traditional test automation tools offer record-and-replay user interface automation”, “This is one of the areas where automation of executable specifications is quite different than traditional automatied regression testing”.
Gojko Adzic defines “Three level of user interface automation”.
Technical activity level - What are the technical steps required to exercise individual workflow steps ?
For example: Open the shop home page, log in with “test user” and “testpassword”, got to the ”/book” page, click the first image with the “book” CSS-class, wait for the page to load, click the Buy Now link, and so on.
User workflow level - How can a user exercise the functionality through the UI, on a higher activity level ?
For example: Put two books in a shoppoing cart, enter address details and verify that delivery options include free delivery.
The automation layer should handle the workflow level by combining blocks composed at the technicial activity level.
Business rule level - What is the test demonstrating or exercising ?
For example: Free delivery is offered to customers who order two or more books”. Specifications should be described at the business rule level.
internet.com "Agile Software Development eBook", sponsored by IBM - “This eBook begins with an overview of the Agile Scaling Model (ASM), a framework used to provide context to the plethora of agile methodologies available today”.
Gojko Adzic, David de Florinier "The Secret Ninja Cucumber Scrolls" - “This document is a step-by-step guide for Cucumber, a tool that is quickly becoming the weapon of choice for many agile teams when it comes to functional test automation, creating executable specifications and building a living documentation”.