The journey of a testbench developer

In this post, I’m talking to you, testbench developer (some might say engineers, but I’m in the no-OIQ, no title category - thanks Quebec).


“I’d rather start programming in 15 minutes in a familiar environment, using a language I know and without having to ask for a budget” - All developers, since the beginning of time

Why is it that a majority of companies do not use a test solution such as TestStand to execute their test sequences? I have a theory.


As a developer of production tests, it is often our duty to find the best way to verify systems on the production line. Afterall, the requirement is simple: “All units delivered to our clients must be fully working”. Alright, but:

  • Who defines what is a fully working unit?

  • What happens if a defective unit, or a 99%-tested unit, slips through?

  • Will R&D cooperate to simplify our work and create a testable system ?

  • Will the tests be executed in-house or by a third-party (Contract Manufacturer) ?

  • How do we support the production line if its located at many hours of distance of our offices ?

Our work often pushes us to strike for the best: 100% test coverage, results exported to cloud, on-demand adjustment and validation of criteria values, custom-built test rig, etc. Obviously, we tell ourselves that surely, someone wanted to do that before us.


And so, as any good programmer, we look for available test solutions. Why reinvent the wheel?


And so it’s here, 75% of the time, that we - developers - decide not to use a commercial solution and to build a custom testbench.

Contender #1: TestStand. Looks OK from the first look. The GUI is straight out of 2003, but I can deal with that. Let’s see how much it costs… Ouch. 6665$ for a development license.


What exactly does a Starting from 6665$ price tag implies ? Most will say “You save 20k$ of development”. That’s true. But what we’re thinking of as a developer is:

  • The week of boss-convincing we’ll have to go through to have him accept to put forward that cash.

  • The additional week that will be required for the transaction to go through and to finally have the “license” on our PC.

  • The month - and often more - of learning that will be required for us to be comfortable with NI’s proprietary development environment.

But why go through all that! All we want is to do our job: verify systems!


“I’d rather start programming in 15 minutes in a familiar environment, using a language I know and without having to ask for a budget” - All developers, since the beginning of time


And so it’s here, 75% of the time, that we decide not to use a commercial solution and to build a custom testbench. I’ve done it. I know plenty who’ve done it. Most companies do it that way. And it’s fine! … at the start.

  • Week 2 - Working tests. “Wow. Developing a testbench is quick!”

  • Week 4 - A GUI. “Easy.”

  • Week 6 - Standardized test outputs, user inputs. “Development is slower than I thought”

  • (...)

And as time goes by, we realize that, finally, there are a lot of features in a test platform! Problems arise:

  • How to deploy multiple copies of the testbench on different production PCs?

  • How to update the testbench without significantly interrupting production?

  • How to debug the testbench under pressure when deliveries depend on it?

  • How to create and maintain complete documentation that evolves with the testbench itself, in order to train technicians and testers?

  • How to guide technicians in debugging failing units?

  • How to efficiently unify and analyse production data of multiple testbenches?

The difference in development time between answering 60% and 90% of these needs is HUGE.


Why is it that all of these features are developed independently by all of these companies?

Moreover and unfortunately, testbenches are often developed with a limited care for code quality: technical debt rapidly accumulates and changes are more and more difficult and risky.


The result is a working testbench. It is transferred to production successfully and we, as a developper, find a new job and challenge and so it is that, 2 years later, the testbench source code becomes a mystical and dangerous creature that should not be touched.


It’s bullshit. Why is it that a majority of companies with which I have discussed follow that route? Why is it that all of these features are developed independently by all of these companies?


And what if I told you there’s an open-source framework that already has 80% of those features?


A framework that allows us, developers, to jump right in test code without a single dollar of budget. That allows us to use Python and our favorite IDE without any proprietary environment.


Well, Google created it and we’ve made it better. It’s called Spintop-OpenHTF. Check it out: https://www.tackv.ca/testbench-dev.


It’s a central part of our vision here at TackV: to give electronics manufacturers the power to truly understand and optimize their production line. Learn more on our homepage: https://www.tackv.ca/.


TackV enables intelligence for electronics-manufacturing companies.

Our aim is to facilitate access and understanding of the production line information by all stakeholders, leading to increased velocity and reduced costs linked to quality control and operations optimization.

From Quebec City, Canada.

© 2019-2020 Tack Verification Inc - All rights reserved.
Privacy Policy