What is black box testing?

Black box vs. white box testing

Black box and white box can be viewed as testing methods that are opposites. In black box testing, the idea is to test an application's functionality without knowledge of its internal design; in white box testing, the person testing knows the internal workings of the application and that's what's being analyzed.

In black box testing, the primary concern is to determine if the application is functioning as intended, from the user's point of view; in white box testing, the primary concern is how the application is performing, to ensure functionality and efficiency.

In black box testing, tests are designed based on outcome classes -- similarities in application performance based on test inputs; in white box testing, tests are designed around validation of application logic.

In black box testing, the goal is for application performance to be as flawless as possible from the user's perspective; in white box testing, the goal is to achieve the highest possible quality and integrity of the application's code.

Grey box testing

Grey box testing sits somewhere between black box and white box testing. While white box testing assumes full knowledge of an application's inner workings, grey box works with partial knowledge, enough to facilitate integration testing, end-to-end system tests and penetration testing.

A grey box tester might not know the full intricacies of an application's code but would be aware of the data structures and data flow in use to set up appropriate test cases. This allows for a greater pool of potential testers.

Types of black box testing

The following three main types of test cases are used in black box testing:

• Functional testing.
• Nonfunctional testing.
• Regression testing.

Functional testing

Functional testing includes the validation of an application's functionality: Is the application doing what it's supposed to do, working as intended? Put another way, does the application being tested match its specifications and requirements?

Nonfunctional testing

Nonfunctional testing isn't concerned with an application's features or user-readiness, but instead is focused on its more technical aspects -- performance, usability and scalability.

Regression testing

Another approach to black box testing is to plan defined test cases that target a specific feature or provide regression testing. For example, in his book Black-Box Testing: Techniques for Functional Testing of Software and Systems, Boris Beizer, software engineer and author, creates examples where the entire test plan derives from the application's specification, without looking at how the code operates.

Predefined cases reduce test result variation, which leads to the minefield problem, also known as reduced application test coverage over time. Preplanned tests also prohibit a tester from improvising according to the results of a given test, commonly referred to as exploratory testing.

Black box testing techniques

The multitudinous combinations of possible inputs, states and time sequences make 100% complete testing of software programs impossible. The software program variations make it a challenge to choose the most powerful black box tests. So, testers must draw conclusions from that limited subset of tests.

The following black box testing techniques reduce the input space into groups so testers can take a subset of each group to examine:

• Equivalence class partitioning divides software data into equal partitions to test each section.
• Boundary value analysis tests for errors within input values that range from one end of a boundary to the other.
• All-pairs testing uses any combination in pairs of data inputs to check parameters for bugs.

Benefits of black box testing

Black box testing provides the following advantages:

• It separates the tester from the code creator.
• The software testing technique forces the team to see it from an outsider's view.
• A black box test assesses the software from the user's perspective. This creates both social distance and critical distance between software development and testing, which makes it more likely that the tester will manipulate the application, referred to as the box, in a manner its creator hadn't considered.

Clear box testing requires setup and instrumentation, or at least poring over code, while most black box techniques can begin immediately; the operator simply tries to use the software. A system could behave correctly as a black box, but still contain defects in the code itself.

Challenges of black box testing

Black box testing isn't without its challenges. These should be considered before deciding whether to add it to the organization's testing methods. These include the following:

• It's difficult to automate black box testing, making it labor-intensive and more time-consuming than other common testing methods.
• Root cause analysis can require intervention from more technical testers when a failure occurs.
• It can be challenging to create a test program covering all eventualities, as the testing is based on potential user behavior and subjective user experience.

Black boxes outside of software

Black boxes exist beyond the world of software applications. The general idea -- you can't see what's in there -- can apply in all sorts of industries, including the following:

• Telecommunications. A black box is a resistor that connects to a phone line that enables the telephone company's equipment to detect when a call has been answered.
• Data mining. A black box is an algorithm or a technology that doesn't provide an explanation of how it works.
• Film making. A black box is a dedicated hardware device, or equipment that's specifically used for a particular function.
• Theater and television. A black box is an unfurnished studio.
• Finance. A black box is a computerized trading system that doesn't make its rules easily available.

Redundancy in software testing can risk creating inefficiencies in development. Learn how testing teams can make smart decisions during the testing process