Next steps in Quality Assurance

In Software Development, quality is a must. Every company, from start-ups to market leaders, seek for deliver quality. But the real question is, what quality means?

Quality? What Quality?

According ISO, quality is:

“The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs”

And it is a very comfortable definition. Only problem is that cannot be used directly to measure the quality level of a working piece of software.

So, some extra work is required. First, I will remove the “or service” part, since I would like to talk about Development.

Business, Process and Product Requirements

For each of us dealing with Software Development, it will sound familiar to say that “stated or implied needs” is an abstract way of naming the Requirements.

So, simplifying the ISO statement, we could say that quality is:

“The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs Requirements”

Now is crystal clear, isn’t it? (If your answer is ‘yes’, please stop reading this post)

Improvement areas

There are several classifications for the features and characteristic of a product, but I like the ISO 9126:1991 way:

• Functionality: A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs. (ISO 9126: 1991, 4.1)

• Reliability: A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time. (ISO 9126: 1991, 4.2)

• Usability: A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users. (ISO 9126: 1991, 4.3)

• Efficiency: A set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used, under stated conditions. (ISO 9126: 1991, 4.4)

• Maintainability: A set of attributes that bear on the effort needed to make specified modifications. (ISO 9126: 1991, 4.5)

• Portability: A set of attributes that bear on the ability of software to be transferred from one environment to another. (ISO 9126: 1991, 4.6)

What is Quality all about?

Drilling down into the definition of quality until something meaningful happened is far from easy. And the main issue about it is due to the fact that quality is a perception. One single attribute cannot define quality by itself. Deliverance of zero-bug software is a major objective but it cannot be isolated from the rest. A Java function doing nothing will accomplish the goal.

The idea then is to gather as much information from different sources, seeking for a comprehensive list of requirements to fulfill. And, after that, negotiating priorities according Company strategic goals will lead us to the next level.

In next posts we will start the process that (hopefully) will significantly improve current quality level

Automation Code released

Last months, Openbravo’s QA team developed a set of scripts using Selenium. With Selenium, we were able to automate our standard Smoke Test.

About Selenium

After surveying the cloud of automation tools, we choose Selenium for a number of reasons:

It allows to run on many operating systems. Other tools like market’s leader HP Quick Test or IBM Rational Robot work in Windows only.

It allows to use either Mozilla Firefox (versions 2 and 3) or Microsoft Internet Explorer (version 6). Other tools like Watir and Watij work with IE only (although Watir has
Firewatir, a mechanism to drive Firefox).

Scripts can be developed in Java, and using JUnit as driver, you can easly develop a framework using Eclipse Classic IDE.

Last but not least, it’s open source software.

Do you like automation? Try it!

Now the whole community can access the automation code branches for 2.3x and 2.40 stable branches. A tag for testing 2.40 community edition is also available, and the current development Main is available as well. For more information you can visit the Project Page at our Forge.
If you want to know more about the process, you may check Automation main page in our wiki. There are also a lot of useful pages grouped in an Automation Category.

If you are interested in running the scripts, you may check this wiki page. We encourage you to use automation to check stability of any Openbravo ERP version you have.

About Automated Software Testing

A key process for any QA process is reliable automation, so virtually a continuous quality assurance cycle is inserted to find any stability issue on early stages. In Openbravo’s QA team, this is performed with a combination of Java, Selenium and Ant. Java and Selenium are used for user interface web-based functional testing. Other Java processes execute queries to the database to ensure correct non UI observable results. And by using Ant, all this processes are linked among them as well as added to daily build tasks.

The most important aspect of UI based automation is the high volatility of the resultant scripts. Simple changes that real users may not notice, like changes in button HTML identifiers, can broke an automated test. Also, functional changes are part of the normal development, like a new requirement of adding a new mandatory field on a form.

A failed script execution on a daily build, fires a maintenance tasks for automated scripts. Since new functionalities are not included on automation, only changes to current functionality or unexpected behavior can affect Smoke Test. In the latter case, a bug should be fed. If the change is because a planned modification, both online documentation and automated scripts are updated and the Smoke Test is run again.

Next Steps

This version of the scripts are somewhat basic regarding dynamic execution. That means that test cases must be executed from first to last, since previous generated data is a precondition. For example, you may note that create a Sales Order requires to have a specific Customer, as well as a specific Product. Dynamic scripts are under development right now, which will allow to decouple modules to fit any given data.
We are currently enlarging the scope of automation for trunk version. Contributors are welcomed. If you have some knowledge on QA processes, automation and programming skills, contact us at automation _at_ openbravo _dot_ com
Additionally, you may want to develop your own scripts to verify custom code. We will gladly help you if you contact us at our Automation forum at the Forge.

Openbravo ERP Password Remember

If you are using Openbravo ERP since 2.35 or before, you may miss autocompletion because of this 2.40 defect. For testing purposes, entering several usernames and passwords is painful without this feature, so there is a Firefox tweak  that help us.

As you may know, autocompletion is an additional property that can be set on and off for each field on any given html page. So, we need a fix for Firefox in order to forget that restriction.

Just in case, there is also a more elegant hack that will work in other applications (not Openbravo)