What is a test bed?

The formal definition from wikipedia: "A platform for experimentation of large development projects."



There are many ways in which testbeds are used, specifically the one I work in. There are 3 ways in which we use it:
1. The development and deployment of new ideas or products in real-time outside of operations,
2. Testing the usefulness or goodness of pre-existing ideas or products, and
3. Setting up an Education/Discussion environment

With new ideas or products, come new challenges. Much like we did last year with 1 hour total severe weather probabilities, the goals are to generate guidance and make forecasts. The discussion of this process could/should generate insight about what is needed, what connects and what doesn't, and what our limitations are.

For existing ideas you iterate or make improvements following the above. Whether that is changing the guidance to meet your needs, simplifying the forecast process, or just really putting it through the grinder to find out what need or want it does or does not meet, and qualitatively or quantitatively assessing its value and usefulness. Again we use 3) to discuss the ways in which it works, the ways in which it doesn't, and if there is opportunity to further refine or improve it.

But we are lost in the weeds in that assessment. Which is fine when you are starting out and trying to make progress on a topic you know little about. Its more of a learning environment which transforms into a testbed. And we are at that transformation point.

If you ask me, our testbed is now developing expert decision systems, albeit without the system. We are thinking up new techniques, doing data mining, creating knowledge, constructing context, and assembling information. We are also trying to communicate said data, knowledge, context, and information. In some cases it is through visual displays and some ways it is through our discussions.

This is very much like the diffusion of innovations. A very interesting treatise on the subject is by Everett Rogers, a book I am currently reading.

We are trying to elicit good ideas, make an attempt at innovating and in the process attract early adopters to test our inventions. And we struggle with the same things that all innovators struggle with:
1. What is a good idea?
2. What makes an innovation?
3. Who will like and adopt an innovation?
4. How will an innovation be utilized or will it be reinvented?
5. Should we adopt an innovation? what are the consequences or at least ramifications?

Part of the struggle is being able to critically evaluate what is needed, what is wanted, what are we capable of doing? On the other end of the spectrum, what is the value of an innovation, what are the metrics of success, and more fundamentally what are the stages of innovation (the framework) for knowing how they are progressing? Without a framework, there is little to help to answer these questions objectively.






This is of huge importance in the coming years, due to the availability of grant funding sources through NWS for research to operations or through OAR for the HWT or HMT.  At some level, other people will use funding instrument criteria to determine what is innovative and what isnt or we will be beholden to someone elses' metrics of success.