Building a Proper LabVIEW State Machine Design Pattern – Pt 1

Posted on 2015-02-232015-03-14 by mlportersr

The other day I was looking at the statistics for this site and I noticed that one of the most popular post with readers was the one I wrote on the producer/consumer design pattern. Given that level of interest, it seemed appropriate to write a bit about another very common and very popular design pattern: the state machine. There’s a good reason for the state-machine’s popularity. It is an excellent, often ideal, way to deal with logic that is repetitive, or branches through many different paths. Although, it certainly isn’t the right design pattern for every application, it is a really good technique for translating a stateful process into LabVIEW code.

However, some of the functionality that state machines offer also means they can present development challenges. For example, they are far more demanding in terms of the design process, and consequently far less forgiving of errors in that process. As we have seen before with other topics, even the most basic discussion of how to properly design and use state machines is too big for a single post. Therefore, I will present one post to discuss the concepts and principles involved, and in a second post present a typical implementation.

State Machine Worst Practices

For some reason it seems like there has been a lot of discussions lately about state machine “best practices”. Unfortunately, some of the recommendations are simply not sound from the engineering standpoint. Meanwhile others fail to take advantage of all that LabVIEW offers because they attempt to mimic the implementation of state machines in primitive (i.e. text-based) languages. Therefore, rather than spinning out yet another “best practices” article, I think it might interesting to spend a bit of time discussing things to never do.

In describing bad habits to avoid, I think it’s often a good idea to start at the most general level and work our way down to the details. So let’s start with the most important mistake you can make.

1. Use the state machine as the underlying structure of your entire application

State machines are best suited for situations where you need to create a complex, cohesive, and tightly-coupled process. While an application might have processes within it that need to be tightly-coupled, you want the application as a whole to exhibit very low levels of coupling. In fact, much of the newest computer science research deprecates the usage of state machines by asserting that they are inherently brittle and non-maintainable.

While I won’t go that far, I do recognize that state machines are typically best suited for lower-level processes that rarely, if ever, appear to the user. For example, communications protocols are often described in terms of state machines and are excellent places to apply them. One big exception to this “no user-interface” rule is the “wizard” dialog box. Wizards will often be built around state machines precisely because they have complex interface functionality requirements.

2. Don’t start with a State Diagram

Ok, so you have a process that you feel will benefit from a state machine implementation. You can’t just jump in and start slinging code right and left. Before you start developing a state machine you need to create a State Diagram (also sometimes called a State Transition Diagram), to serve as a road-map of sorts for you during the development process. If you don’t take the time for this vital step, you are pretty much in the position of a builder that starts work on a large building with no blueprint. To be fair, design patterns exist that are less dependent upon having a completed, through design. However, those patterns tend to be very linear in structure, and so are easy to visualize in good dataflow code. By contrast, state machines are very non-linear in their structure so can be very difficult to develop and maintain. To keep straight what you are trying to accomplish, state machines need to be laid out carefully and very clearly. The unfortunate truth, however, is that state machines are often used for the exact opposite reason. There is a common myth that state machines require a minimum of design because if you get into trouble, you can always just, “add another state”. In fact, I believe that much of the bad advice you will get on state machines finds its basis in this myth.

But even if we buy the idea that state machines require a more through design, why insisted on State Diagrams? One of the things that design patterns do is foster their own particular way of visualizing solutions to programming problems. For example, I have been very candid about how a producer/consumer design pattern lends itself to thinking about applications as a collection of interacting processes. In the same way, state machines foster a viewpoint where the process being developed is seen as a virtual machine constructed from discrete states and the transitions between those states. Given that approach to problem solving, the state diagram is an ideal design tool because it allows you to visually represent the structure that the states and transitions create.

So what does it take to do a good state-machine design? First you need to understand the process — a huge topic on its own. There are many good books available on the topic, as well as several dedicated web sites. Second, having a suitable drawing program to create State Diagrams can be helpful, and one that I have used for some time is a free program called yEd. However fancy graphics aren’t absolutely necessary. You can create perfectly acceptable State Diagrams with nothing more than a paper, a pencil and a reasonably functional brain. I have even drawn them on a white board during a meeting with a client and saved them by taking a picture of them with my cell phone.

Moreover, drawing programs aren’t much help if you don’t know what to draw. The most important knowledge you can have is a firm understanding of what a state machine is. This is how Wikipedia defines a state machine:

A finite-state machine (FSM) or finite-state automaton (plural: automata), or simply a state machine, is a mathematical model of computation used to design both computer programs and sequential logic circuits. It is conceived as an abstract machine that can be in one of a finite number of states. The machine is in only one state at a time; the state it is in at any given time is called the current state. It can change from one state to another when initiated by a triggering event or condition; this is called a transition.

An important point to highlight in this description is that a state machine is at its core is a mathematical model — which itself implies a certain level of needed rigor in their design and implementation. The other point is that it is a model that consists of a “finite number of steps” that the machine moves between on the basis of well-defined events or conditions.

3. Ignore what a “state” is

Other common problems can arise when there is confusion over what constitutes a state. So let’s go back to Wikipedia for one more definition:

A state is a description of the status of a system that is waiting to execute a transition.

A state is, in short, a place where the code does something and then pauses while it waits for something else to happen. Now this “something” can be anything from the expiration of a timer to a response from a piece of equipment that a command had been completed successfully (or not). Too often people get this definition confused with that for a subVI. In fact one very common error is for a developer to treat states as though they were subroutines that they can simply “call” as needed.

4. Use strings for the state variable

The basic structure behind any state machine is that you have a loop that executes one state each time it iterates. To define execution order there is a State Variable that identifies the next state the machine will execute in response to an event or condition change. Although I once saw an interesting object-oriented implementation that used class objects to both identify the next state (via dynamic dispatch) and pass the state machine operational data (in the class data), in most cases there is a much simpler choice of datatype for the State Variable: string or enumeration.

Supposedly there is an ongoing discussion over which of these two datatypes make better state variables. The truth is that while I have heard many reasons for using strings as state variables, I have yet to hear a good one. I see two huge problems with string state variables. First, in the name of “flexibility” they foster the no-design ethic we discussed earlier. Think about it this way, if you know so little about the process you are implementing that you can’t even list the states involved, what in the world are you doing starting development? The second problem with state strings is that using them requires the developer to remember the names of all the states, and how to spell them, and how to capitalize them — or in a code maintenance situation, remember how somebody else spelled and capitalized them. Besides trying to remember that the guy two cubicles down can never seem to remember that “flexible” is spelled with an “i” and not an “a”, don’t forget that there is a large chunk of the planet that thinks words like “behavior” has a “u” in them…

By the way, not only should the state variable be an enumeration, it should be a typedef enumeration.

5. Turn checking the UI for events into a state

In the beginning, there were no events in LabVIEW and so state machines had to be built using what was available — a while loop, a shift register to carry the state variable, and a case structure to implement the various states. When events made their debut in Version 6 of LabVIEW, people began to consider how to integrate the two disparate approaches. Unfortunately, the idea that came to the front was to create a new state (typically called something like, Check UI) that would hold the event structure that handles all the events.

The correct approach is to basically turn that approach inside out and build the state machine inside the event structure — inside the timeout event to be precise. This technique as a number of advantages. To begin with, it allows the state machine to leverage the event structure as a mechanism for controlling the state machine. Secondly, it provides a very efficient mechanism for building state machines that require user interaction to operate.

Say you have a state machine that is basically a wizard that assists the user in setting up some part of your application. To create this interactivity, states in the timeout event would put a prompt on the front panel and sets the timeout for the next iteration to -1. When the user makes the required selection or enters the needed data, they click a “Next” button. The value change event handler for the button knows what state the state machine was last in, and so can send the state machine on to its next state by setting the timeout back to 0. Very neat and, thanks to the event-driven programming, very efficient.

On the other hand, if you are looking for a way to allow your program to lock-up and irritate your users, putting an event structure inside a state is a dandy technique. The problem is that all you need to stop your application in its tracks is one series of state transitions where the “Check UI” state doesn’t get called often enough, or at all. If someone clicks a button or changes something on the UI while those states are executing, LabVIEW will dutifully lock the front panel until the associated event is serviced — which of course can’t happen because the code can’t get to the event structure that would service it. Depending on how bad the overall code design is and the exact circumstances that caused the problem, this sort of lock-up can last several seconds, or be permanent requiring a restart.

6. Allow default state transitions

A default state transition is when State A always immediately transitions to State B. This sort of design practice is the logical equivalent of a sequence structure, and suffers from all the same problems. If you have two or more “states” with default transitions between them, you in reality have a single state that has been arbitrarily split into multiple pieces — pieces that hide the true structure of what the code is doing, complicates code maintenance and increases the potential for error. Plus, what happens if an error occurs, there’s a shutdown request, or anything else to which the code needs to respond? As with an actual sequence structure, you’re stuck going through the entire process.

7. Use a queue to communicate state transitions

Question: If default transitions are bad, why would anyone want to queue up several of them in a row?
Answer: They are too lazy to figure out exactly what they want to do so they create a bunch of pieces that they can assemble at runtime — and then call this kind of mess, “flexibility”. And even if you do come up with some sort of edge case where you might want to enqueue states, there are better ways of accomplishing it than having a queue drive the entire state machine.

Implementation Preview

So this is about all we have room for in this post. Next Monday I’ll demonstrate what I have been writing about by replacing the random number acquisition process in our testbed application with an updated bit of LabVIEW memorabilia. Many years ago the very first LabVIEW demo I saw was a simple “process control” demo. Basically it had a chart with a line on it that trended upwards until it reached a limit. At that point, an onscreen (black and white!) LED would come on indicating a virtual fan had been turned on and the line would start trending back down. When it hit a lower limit, the LED and the virtual fan would go off and the line would start trending back up again. With that early demonstration in mind, I came up with this State Diagram:

When we next get together, we’ll look at how I turn this diagram into a state-machine version of the original demo — but with color LEDs!

Until next time…

Mike…

Managing Shared Resources in a Subpanel-Based Interface

Posted on 2015-02-162015-03-14 by mlportersr

We have seen that building a system out of autonomous interacting processes can be a powerful architectural concept. However, there can be opportunities to simplify the operation of the system as a whole (and make it more robust) by designing one of those interacting processes to create common shareable resources that can be reused throughout system.

This time out we will consider a couple simple ways to implement this functional reusability without creating a brittle application.

Creating Reusable Resources

So the first thing we need to do is figure out exactly what we mean by “Reusable Resources”. In a sense we are creating a reusable resource every time we build a reusable library of drivers or functions, but that sort of reuse is not what I am talking about here. The distinction is that things like libraries are for really tools for the developer and users seldom even realize they exist. What I am talking about in this post are aspects of the program that are user facing.

For example, say you have an application that has 4 or 5 screens and they all need a couple buttons and a menu or two, but the buttons and menus on each screen do different things. One solution would be to simply duplicate the button and menu logic on each screen. but there are limitations to what you can do in this way. To begin with, forget the idea of having menus. Windows in subpanels technically aren’t open, so they don’t have menu bars and can’t have menus.

A far more flexible solution is to let the main GUI carry these reusable, or shared resources and let the subpanel VIs interact with them via references. Right now, that approach will be the focus of our attention.

Basic Concepts

One of the beautiful aspects of VI Server is that it allows you to do nearly anything with references that you can do directly. Therefore, our basic approach will be to pass references to the shared resources to the subpanel VI when it becomes active (i.e. visible in the subpanel) so it can register for, and respond to, the events associated with those resources. As a demonstration of how this works, we’ll add a button and a very simple menu example. The menu example will be simple because I am planning a complete discussion of menus for later time.

A New Requirement

The application to this point has provided a means for looking at data, or at least simulated data. It does not, however, have the ability to store any data. What we want to add is the ability to press a button, or make a menu selection, and have the data source we are looking at save the current chart contents to a text file. In addition, to raise the bar a bit higher, let’s also include in this requirement that the button and menu have to reconfigure themselves based on which data source is selected. In other words, if we are looking at the sine data source, the button and menu should say something like, “Save Sine Data”, or if viewing the ramp data, “Save Ramp Data”.

So what do we need to do to implement this functionality? To answer that question, let’s consider what we know. We know that a big reason for using subpanels in the first place is to eliminate data transfers. Therefore the data needs to be saved from the acquisition process itself — what would be the point of moving it somewhere else first?

Moreover, if each process is going to be handling its own data, doesn’t it make sense to let it handle the button clicks and menu selections too? The alternative is for the GUI to detect the button click or the menu selection and, in response, generate another event of some kind to let the acquisition processes know what happened. But there is no benefit to having the GUI acting as a middle man, mediating the communications. Better to go with the simple way.

And this is how you do it:

Dynamically Managing Events

If the easy way to handle this requirement is to let the acquisition processes handle the buttons and menus, we are going to need a way to turn events of all kinds on and off — not just timeouts. After all, there are three acquisition processes and we only want one of them at a time managing the shared resources.

To see how that works, let’s revisit for a moment how you register dynamic events. Here is a typical snippet of code (in fact, a subVI named (re)create reuse events.vi) that is registering a couple events. The basic idea is pretty simply. You wire a reference of some kind to the node and LabVIEW will show you the specific events that you can associate with that reference. In this case, we are creating a value change event from a Boolean control reference, and a menu selection event derived from a VI reference.

By the way, if you plan on building this subVI from scratch, be aware that this situation is one of the few places where LabVIEW cares about the order in which you wire things. To duplicate my results, first create and name the input controls, then wire them to the Register for Events node, finally create the output event refnum indicator, and then duplicate it to create the corresponding input. Once you have the subVI (or at least me implementation of it), this is what it looks like installed into one of the acquisition processes. It’s the one with the red banner across the top.

If you are observant you might be wondering how this is going to work because neither of the reference inputs that the subVI uses to create the registration, are connected to anything — which means that the references they are passing to the registration node aren’t valid. The thing is, when LabVIEW creates an event registration it doesn’t check for the references to be valid. Used in this way, the references only have two functions: Define the type of reference associated with the event, and name the event.

This fact is important because it is central to what we are trying to accomplish. As long as the references associated with the registration are invalid, the events are in essence turned off because you can’t fire an invalid event. To turn them on, all you have to do is modify the registration to use valid references. This snippet shows how I have modified the Change Source event and its event handler.

First, I moved the My Name control outside the event loop because I an going to be needing it several places. Second, I modified the event data to pass three references in addition to the name of the source being selected. Two of those references relate to the values we used to initialize the registration. The third reference is a menu reference that we will discuss in a bit. Third, I modified the event handler such that if My Name matches the selector string from the event data, the registration is recreated using the two real references, thus allowing this VI to receive those two events. If My Name does not match the selector string from the event data it executes an instance of the registration VIs that has no references wired to its inputs. Fourth, what in the world is that single frame sequence structure with the wires passing through it for?

In the finished code you will see a subVI sitting there, but I wanted to pass on a tip. Many times when working I will realize that I will need a subVI that I haven’t created yet. To allow me to continue work, I will drop down a single frame sequence structure and wire up the data inputs and outputs that I know it will need. Once that it done, I turn it into a subVI by selecting it and then choosing the Create SubVI option from the window’s Edit menu. As a reminder that I have to go back and finish it, I will leave the new subVI with the default icon.

We’ll get to what this mystery VI does in a bit. But it should be obvious that these changes are useless if there are no error handlers, so I added them in too. Here is what the handler for the button press looks like.

We first call our little library function to pop the button back out (since we obviously can’t have the terminal for the button here) and then calls a new subVI to save the data to a text file. This is what the data saving VI looks like on the inside:

Note the comment I put in the code. I will be wanting to come back and reimplement this data saving better so I put a bookmark in the code to remind me.

We are also going to need an event handler for the menu event as well, but before we can build that code, you need to know a bit more about how menus work in LabVIEW.

The 411 on Menus

As I stated before, I am going to devote an entire post to menus in the near future, so the following discusses a very basic implementation.

The first thing you need to know about menus is that to create and manipulate runtime menus you need a menu reference — no surprise there. Unfortunately, LabVIEW doesn’t treat menu references like it does references to other parts of a VI. For example, if you need a reference to VI’s front panel, you can get it from a reference to the VI. Not so with menu references. The only way to get a menu reference is from a special node that has to reside on the block diagram of the VI that will be showing the menu. This “quirk” is why we need to pass two different references to the subpanel VI that both deal with menus: the VI reference allows us to register an event for the menu, but we need the menu reference in order to make changes to the menu contents.

Next, you need to know that every menu item has a Name and a Tag. In many ways these properties are analogous to a control’s Caption and Label. Like a control’s label, a menu item’s tag cannot be changed dynamically because it is the way that LabVIEW uniquely identifies that particular object. However, the menu item name is the visible string you see when you operate the menu and (like a control’s caption) can and often does change during the execution of an application.

Here is the code in the GUI that will define our very basic menu and its three items. Note that I have packaged this logic in a subVI so when I upgrade the menu generation, it will be easier to do. Remember that a good principle for when to make something a subVI is to ask yourself is this functionality (or the implementation of the functionality) likely to change?

Breaking down the logic, the first node deletes all the existing menus. The next node creates a new top level File menu. Finally, the third node is in a loop and it creates a selection to save the data (tagged: Save Data, a dividing line and a Quit selection to stop the application. As I stated before, the tag for each item must be unique. To make meeting this requirement easy, I create hierarchical tags that are a colon delimited listing of the item’s parent tags. For example, the tag for the Quit selection on the File is File:Quit. This structure also makes the tag easy to parse in event handlers for the menu.

The Menu Event Handlers

Given the menu is itself a shared resource, the response to the menu events is also shared. Here is the menu event handler that is in the GUI. It handles the event that related to the application as a whole — Quit. As you would expect, all it does is run the VI that verifies that the user wants to quit before firing the standard Stop Application event.

Likewise, there is the menu handler that resides in the acquisition processes. It simply calls the same data saving VI we used in the button press event handler.

In both handlers, menu events for which they are not responsible, are handled by an (empty) default case.

Changing Resource Appearances

Remember the dummy VI we created earlier? Its job is to implement the requirement that the markings on the button and menu change to reflect the process that is managing them. Here is what it looks like:

The My Name value is used to generate the new marking which is then applied to the button’s Boolean Text property and a node that uses it to set the menu item’s name. The node uses the item tag File:Save Data to identify the specific item it is changing. If you check out this node using the online help you’ll notice that it also let’s you enable and disable items, and add or remove check marks in front of items.

Code Testing

After making the required modifications to the other two data sources, we are ready to test the code. As you go from one data source to the next you’ll notice that the menu selection and button text change to reflect the name of the source that is being displayed. While you’re at it, test the data saving and see that the application remembers the last place that you saved a file.

This is all on subpanels for now. I have gotten some really good comments from several of you and will rolling some of those questions into future posts. In addition, when you download the code for a post, be sure to go through all of it. There are a lot of things going on in the code that I comment on in the VIs, but don’t have space for in the post.

Testbed Application — Release 10

Toolbox — Release 5

Oh yes, a preview… In the next post I will start looking at another popular design pattern and what a proper LabVIEW implementation of it looks like.

Until next time…

Mike…

Building a Subpanel-Based User Interface

Posted on 2015-02-092015-03-14 by mlportersr

Having discussed some of the main issues related to incorporating modularity into your application’s GUI, we are now ready to look at how to tweak what we have already built into an application that incorporates those ideas. From the user’s standpoint, the program will operate very much as it did before, but with no data transfer hassles: there’s not any data transfers!

Let’s See Them Work!

To see how subpanels work we will modify our existing testbed application. However, these modifications will only require minor changes to the application. Remember, it is not an accident that this sort of modification can be accomplished without a major rework of the code. This sort of adaptability results from a conscious decision to use techniques and methods that are inherently adaptable. Always be thinking ahead…

And if you do think about what we are doing here you realize the required modifications will be minor in scope because there really isn’t very much in the code that fundamentally cares how the interface is structured. In fact, there are only two places that will need to be changed. The GUI needs to know what to do when a different screen is selected — this logic needs to change. Likewise, the acquisition processes (which previously ran unseen in the background) now need to be prettier, but on the whole they will be doing less than they did before.

Making Background Task Presentable

So let’s start by turning the background tasks into something we would want the world to see on our application’s main interface. It only takes a glance to notice that the existing graphs are too small, so the first step in the transformation is to resize the graphs on our three data sources and make them all the same size as the existing graph in the GUI. In addition, because all three processes will be going into the same subpanel, we also need to make sure the front panels of the three acquisition processes are all the same size: slightly larger than the graph. While we are doing cosmetic things, we also want to make sure that the front panels are the same color as the main GUI.

The functional modification needed to make these VIs live happily in a subpanel is to go into their VI properties and in the Window Appearance section, turn off both scrollbars.

We now turn to the code changes. In order to put a VI into a subpanel you write a reference to the VI to a subpanel control method called Insert VI. The thing is, retaining references to background processes that you launch can get tricky. This point is especially true when you need to launch multiple instances of a reentrant VI. Although the testbed isn’t (currently) using reentrant processes, the software does support it so we need to think about how to get references to the processes that will be going into the subpanel. One simple way to accomplish this goal is to have the processes “publish” a reference to themselves that can be used elsewhere in the code. To implement this approach, I created a buffer that has two public interface VIs. One appends a reference to the buffer contents and one reads the buffer contents. Note that we won’t take the time to look at the code for these functions here because we have examined this structure before.

To use these VIs, we put an instance of the insert VI at the very beginning of the call chain in each of the acquisition processes. Like so:

The only other changes we need to make to the acquisition logic both relate to the timeout value for the loop. We no longer need the acquisition to wait before starting so a single keystroke will change the “-1” timeout in the initialization logic back to “1”.

Likewise, we don’t need the Change Source event right now, but we may need it again (soon) so we’ll leave the event handler in place, but modify the code so it doesn’t do anything.

…and that is all we have to do to the acquisition processes.

Adding the Subpanel to the GUI

With the acquisition processes modified, all we have to left to do is change the front panel control on the GUI, and alter what happens when you make a Data Source selection.

Changing the front panel control consists of simply removing the graph and replacing it with a subpanel. One thing you need to think about is the size that you are going to make the subpanel. I have found through experience that you get a better looking result if you make the subpanel a skosh bigger than the front panel of the VIs that you are putting in it. For the purposes of this discussion a “skosh” is an empirically determined constant equal to 4 pixels, so make the subpanel 4 pixels taller and wider than the acquisition process front panels.

You’ll note that when you place a subpanel on a front panel, LabVIEW also installs an invoke node for it on the block diagram, we’ll use that in a little bit. The only other thing you need to do that is related to the graph is remove the VIs implementing the update UDE, and delete the associated event handler.

Note: Normally I would put a screenshot here illustrating the change, but in this case I can’t think of a good way of showing something which is no longer present, and isn’t being replaced by something…

Concerning the Data Source value change event, this is where we get to use the invoke node that LabVIEW created when you instantiated the subpanel. Remove the VI for firing the UDE, and wire in the invoke node in its place. As you can see the node has one input that is a reference to the VI that is to appear in the subpanel. Now, our technique for getting that reference is remarkably similar to what we did to get the data value for the UDE. The difference is that instead of selecting one element from an array of strings, the same array index node is now selecting one element from an array of VI references — references that come from the read VI for the reference buffer we discussed earlier.

Running the Result

If you run the application that we have created, you will notice that (with the exception of a couple points) the operation is very similar to the way it worked in the last release. As you make selections, you can observe that the data displayed still changes as it did before. But if you watch carefully, you will notice that the plugins continue to acquire data even when you aren’t watching them. Moreover, if you go back to a screen after having not looked at it for few seconds, you will notice that it will initially show the previous data and them update all the “missing” datapoints at once. This might seem strange, but it is the side effect of a very good thing that LabVIEW does for you.

Knowing that updating screens that aren’t currently visible can use a lot of computer resources unnecessarily (especially if they have charts on them), LabVIEW keeps track of front panels that are visible and only takes the time to update the data displays if the screen is visible. The delayed update that you see happening all at once is the result of LabVIEW realizing that the front panel is now visible and sending all the updates to the chart at once. On simple controls and indicators, these updates typically happen so fast that you have to really be looking for them, but I wanted you to see the effect.

Note that in the previous paragraph I did not refer to front panels that are “open”. The thing is, not all screens that are open are visible. For example, VI’s running in the background (like the data acquisition processes used to be) are open but Hidden, i.e. not visible. Conversely, the front panels of VIs in subpanels are visible, but they aren’t open. In any case, here’s the updated code.

Testbed application — Release 9

Project Toolbox — Release 4 (No Change)

Before we move on to the next thing I want to cover, I’m going to hang-out with subpanels for one more post. Specifically, designing VIs in subpanels to be completely autonomous processes is a very powerful technique, but sometimes a good way to foster reusability is to have the main GUI provide some standardized, reusable resources that the various subpanel plugins can use. To cover that situation, next time I’ll look at ways to manage shared resources like buttons and menus from subpanels.

Until next time…

Mike…

Modularization – It’s not just for Block Diagrams (Front Panels can play too!)

Posted on 2015-02-022015-05-10 by mlportersr

At the very end of my last post I suggested that perhaps the best way to display user data on your application’s GUI is to avoid sending data to the GUI in the first place. Of course the big question is how the GUI can display data that you don’t send to it?

The programmatic slight of hand that explains this paradox is a feature of LabVIEW called a subpanel. To understand how subpanels fit into the overall LabVIEW “ecosystem”, let’s review for a moment. As anyone who has used LabVIEW for more than 5 minutes will recall, every VI you create has 3 main parts:

The Front Panel — This part provides the VI with a user interface for interacting with the routine.
The Block Diagram — This window is where you put the graphical source code that describes what you are trying to do.
The Connector Pane/Icon — This part allows you to call your VI from the block diagram of another VI.

Now all programming environments have places to enter source code and methods for calling or reusing other code, but the Front Panel is really unique to LabVIEW. With most languages, you have to write code to create the GUI, but with LabVIEW, the GUI is part of the code. Just as calling one VI from the block diagram of another VI is a way to modularize your program’s logic, so subpanels provide a way to modularize your program’s user interface by incorporating the front panel of one VI into the front panel of another VI.

But Why Subpanels?

To be honest, there are other techniques for “modularizing” user interfaces within LabVIEW, and we’ll take a quick look at a couple — though to be honest we should call the result of one technique “pseudo modularity” because while the interface looks modular, it’s really not.

First we want to consider XControls. A couple years ago at NIWeek, I presented a session on how to use XControls, so it’s a topic with which I am familiar. XControls approach the question of GUI modularization by providing a mechanism for creating custom front panels controls that implement custom functionality not found in standard controls. Although they are very powerful, they have some issues:

Problems with XControls

Complexity: XControls are difficult to create and require both greater skill on the part of the developer and advanced knowledge of how LabVIEW works internally. As you can imagine, an XControl of even modest complexity can take a long time to develop. The result is that the functionality they encapsulate must be sufficiently reusable to justify the effort required to create the XControl.
Can Complicate Debugging: One of the “interesting” aspects of using an XControl is that they are almost always running. Of course, when you think about it, this makes sense. A control obviously has to exhibit some useful behavior at run time. However, during development they must also be able respond to changes that the developer is making. For an XControl this requirement means that, on some level, it is running as soon as you drop it down on a front panel. While this fact obviously complicates the process of debugging the XControl itself, it can also adversely effect the debugging of the program you are trying to create — especially if the XControl uses some of the same libraries that you use in creating your program. You can see limited access to subVIs because they are being called in the XControl and so are already running, or are at least reserved for execution.
Support is Limited: Actually, “stinks on ice” is probably closer to the truth. Documentation is poor, examples are practically nonexistent, and because NI considers it an advanced topic, tech support is often no real help.

Tab Controls — Just Say “No”

This technique is the one I described as “pseudo modularity”. As a control metaphor, the tab is familiar to anyone who has been around computers for a while. However, operating system designers have been moving away from tab interfaces for a variety of reasons. To be fair, I would like to start this section with some of the positive things about tab controls, but to tell the truth I can’t think of any…

Problems with Tab Controls

Complexity: While a tab control might help visually organize the 30 or 40 controls and indicators on your front panel, the block diagram can rapidly become a mess as you try to deal with the functionality associated that many terminals. Moreover, because the tabs hide complexity, they increase the likely that your front panel will have 30 to 40 controls and indicators.
No Reusability: Let’s say you create really nice a user interface on one tab of your application. Now let’s further postulate that the interface is so good that another internal customer wants you to build them an application that uses the same interface. How do you do it? Well, with tabs, your option is to cut and paste the controls and then rebuild the logic on the block diagram.

Of course that’s just the beginning of the challenges. You still have to verify that the code works the same in the new context, and you have to hope that the new program’s timing doesn’t effect how the interface works.
Compatibility Problems: It is unfortunately not uncommon to see posts on the forum where someone is wanting to know why common controls and indicators don’t work as expected when they are on tab controls. Such questions are not uncommon because, to be perfectly frank, LabVIEW has a long history of tab controls causing a variety of screen update and performance problems.
Confused VI Server Interface: Normally when working with controls and indicators the terminals of which are on your block diagram, finding there references programmatically is easy. With a reference to the VI’s front panel you can get an array of the controls on the front panel, and the reference to the control will be in that array.

But a tab control complicates the whole process. First you get a reference to the tab control, from that you get an array of references to the pages in the tab control, and once you have found the right page you can get the array of control references containing the control reference you want.

A Good Solution for GUI Modularity

But there’s another way to answer the question: “But Why Subpanels?”: the advantages that subpanels offer. But please note that you can only expect to realize these advantages if you use subpanels properly — which is to say, VIs appearing in the subpanels are autonomous processes that encapsulate the logic for a single operation and, when necessary, display the results of those operation on their own front panels.

Subpanel Advantages

Subpanels Simplify Code, Reduce Memory Footprint and Improve Efficiency: When using subpanels you are able to realize these benefits because the minimal code required implement them is almost always smaller than the code you take out that implemented the redundant communication and display formatting operations. In addition, remember that because you are reducing the data manipulation the code is performing, you are also reducing the memory (and CPU) needed to manage it.
Subpanels Make Code Robust: When designing a process VI that will run in a subpanel, it is easier to create highly cohesive code with very low coupling between it and the remainder of the application.
Subpanels Generalize GUI Structure: Working with subpanels inherently puts you into a mode where you start thinking about the GUI at a higher level. One side-effect of thinking in this way is that so much application-specific logic is moved into subpanels that, in the end, the GUI itself doesn’t know or care what application it is running.

This point is particularly interesting because the interface could get so generic that it could be divided up into separate dedicated areas like this:

Normally the subpanel frames would not be visible, and there would be no annotations in each section, but then the front panel would be basically blank, and very uninteresting to look at. To get an idea of what could go in each section, consider these descriptions. Remember, not all of these sections would necessarily be on every screen, and their sizes can be adjusted to fit requirements.

Header: The intent of the header subpanel is to show VIs that present information that is unlikely to change as a function of the screen being shown in the body. This area could include things like the current time, the operator who is logged in, total operating hours or the name of the display.

Footer: This subpanel extends across the entire bottom of the screen. It could hold VIs that provide things like a status display, or a scrolling alarm status list. Like the header, this section ideally wouldn’t change during the course of program execution.

Body: The application’s main display VIs would go in this subpanel.

Sidebar: The VIs providing this section’s contents of could include things like navigation buttons, or supplemental information that is related to the main display.

If you have ever done any web development, this basic layout might look a little familiar, which is appropriate since I “appropriated” the idea. One of the key concepts of the web is to separate content from where and how the content is displayed to the user. This approach in LabVIEW is a (small) step in that direction with the subpanels providing the display structure.

So combine these advantages these with a host of others, and you can begin to get a feel for why I consider subpanels to be among the top 5 features added to LabVIEW — ever.

Let’s See Them Work!

To get our first taste of subpanels in action, we won’t go to the extent of creating a fully abstracted interface like we just saw. Rather, I will show how to manage one subpanel — and you can expand it from there, however this post is already getting long, so we’ll have to wait until next time to look at that code.

Until next time…

Mike…

Turning a VI On and Off in Software

Posted on 2015-01-262015-06-15 by mlportersr

Something you hear about a lot on the LabVIEW users forum are people who want to stop a VI and then restart it. They will often say things like,

“I have a VI that acquires some data and displays it to the user just the way I want. Now I want to run it several times, but change the test setup between runs – and my customer wants it all done automatically. I have code that can do the test setup, but now I need to run my acquisition and display VI and I can’t figure out how to programmatically click the ‘Abort’ and ‘Run’ buttons.”

Clearly, they can’t actually click the button, but if you consider the situation for a moment you realize that they don’t actually need to start and stop any VIs to obtain the desired behavior. All you really need is a way to put VIs into a mode where they don’t do anything until you are ready for them to continue. It’s, likewise, not unusual to hear from people who are trying to sort out how to add interactivity to a state-machine the normally runs without human intervention. Of course this is why the whole question is interesting: We all run into situations where we want the code to stop and wait until something happens.

Obviously, if you put the problem that way its easy to see that we have just described an event-driven application – which we have just spent several weeks building.

A Little Review

However, to fully realize our goal of implementing “stop-start” behavior, we need to give further consideration to the timeout events we have been using. If a VI is truly going to be dormant when we “turn it off” and then go back to normal when “turned on” we need to first have the code that defines the VI’s “normal” operation (whether it be sequential logic, a state-machine, or something in between) located in the timeout case. Then second, we need to be able to disable and then re-enable the timeout event.

Already in the testbed code we have seen that you can use a negative timeout to turn off the timeout event – as we do in some of the logic for the error checking initialization code. The thing is, this technique has a lot more to offer than as simply a way to trap startup errors. As an example of what else it can do, we are going to expand our testbed application such that instead of simply reading data from one process, it will be capable of drawing data from three different processes. One will return random numbers, one a sine wave and one a ramp signal. To select the input to read, we’ll have a pop-up menu on the display process’s front panel. Although this would seem to be a pretty significant change, you’ll see that it’s really not.

Modifying The DAQ Process

The first thing to note when considering the changes we will need to make, is that the basic infrastructure for this new capability already exists. We already have a structure where we are dynamically changing the timeout event’s timeout.

When it the acquisition process starts, the timeout is set to zero.
If initialization is successful, the timeout is set to 1 so acquisition is started.
Each time a new datapoint is acquired and sent to the GUI, the timeout is set to the value read from a FGV.

All we have to do is expand on this existing functionality, and the first step in that process is to change the behavior in Item 2 above. Currently, the code wants to start acquiring data as soon as it can, so when the error check finishes, the timeout is set to 1. We want to change that behavior because we don’t want the process to start acquiring data until it’s turned on. To implement that change requires a single key-stroke to change the “1” to a “-1”.

Now that the code is set to wait until it is enabled, we need a way to turn it on – and by extension turn off all the processes that are not being enabled. Hopefully, you are thinking ahead and already see where I am going with this: another UDE. Let’s call it something logical like Change Source, and make the data it carries a string named Source Name. Normally, you might think that this would be a place to use something like an enumeration, and normally you would be right. However, there are exceptions to all rules, and as you will see this is an interesting exception.

With the new UDE created, here is its event handler…

…and as you can see it couldn’t be simpler. We compare the Source Name value to the contents of a string control called My Name and if they match, the timeout is set to 1, otherwise it’s set to -1. Because all three data sources will incorporate this same logic, the effect is that only one of the three – the one whose name matches the Source Name value – will be enabled.

But where does My Name come from? I added this control to the front panel and connected it to a terminal on the connector pane. We will need to modify the startup code to populate this just before running the process VI – and that’s our next stop.

Oh, one more thing: Now that we have the acquisition process modifications done,(yes, that is all there is to it…) I made two copies of it changed the data generation code in the copies to produce a ramp and a sine wave, and added the new acquisition VIs to the project.

Passing a Startup Value

To this point, the process VIs haven’t needed any information when they were launched, but now the acquisition processes at least need to know in essence “who they are”. Here is the modified testbed.vi code:

The VI reference to the Open VI Reference node now shows the added string input terminal, which also appears on the Start Asynchronous Call node. To that new input, I have wired the Label value from the launch process data. Remember that, it will become important in just a moment.

The only other modification required is to add a VI, after the launch loop, that broadcasts a signal to the system telling it that the launch process is finished. Let’s take a quick look at why that signalling is needed and how it works.

Signalling Completion

During startup it can be critical for things to happen in the right order. Most of the time you can ensure proper sequencing by simply launching processes in the right order. Sometimes, however, conflicting requirements can leave you with no “right order”. As you might imagine, there are many possible scenarios requiring this sort of synchronization, and just as many potential solutions from which to choose. Moreover, as with Boolean logic, the “correct” solution will sometimes depend on how you are thinking about the problem.

In our testbed, we see a common situation: I like to have the display process launch very early in the startup process. In fact, the only thing that will typically launch before the GUI is the exception handler – which always comes first because the error handling has to be in place before you do anything. The GUI comes next because it is often the logical place to initialize common system resources that the acquisition processes will need when they start. The conflict lies in the need for the GUI to initialize its own front panel. Now that there are three acquisition processes running, the GUI had to select which one to display first. However, before one can be selected, the process VI has to be running and it won’t launch until after the GUI.

The solution to this conundrum is another type of “stop-start” situation. We need to let the GUI initialize everything except its front panel and then make it pause and wait until it receives a signal telling it that the launcher is done. Once that signal is received, the GUI can finish its initialization and commence normal operation.

To implement this functionality in a generic, reusable way, I have created two simple VIs that encapsulate a named LabVIEW notifier. Here is the routine that waits for the notification:

The VI is contained in a library that creates a name space for the logic. The VI first acquires a reference to a notification, the name of which is derived from the name of the library containing it &ndash: in this case Launch Completed. It then waits for that notification to occur, but with a timeout specified. To make this code easier to use, I modified the API slightly to allow the developer to define the timeout in units of time. Finally, if the wait ends in a timeout, I generate an error to that effect.

The VI for sending the notification is equally simple:

Located in the same library as the wait VI, the sender also derives the notification name from the library name and then sends the notification. An important point to remember is that notifiers are often likened to queues because on the surface they seem very similar. However, in addition to the fact that a new notification will overwrite an old one, there is another big difference. If you have multiple listeners monitoring the same queue, LabVIEW channels the enqueued data to the various VIs in round-robin fashion. For example, say you have three VIs all pulling data from the same queue, any one VI will only see every third item enqueued. By contrast, a notifier works more like an event in that if you have three VIs waiting for notifications the same notifier, they will all see every notification.

Adding Control to the GUI

So to recap, we have modified our existing acquisition process, created two new ones using it as a template, and modified the startup logic to pass a new piece of data that the acquisition processes need. The only thing left now is to update the GUI.

Start on the front panel (always a good place to start), add a test ring control and label it Data Source. But a text ring in the current state isn’t very useful because it doesn’t contain anything. To fix that, we will (using a property node) insert into the control an array of strings that we want to appear as selections. And where do we get the array of strings? Here’s the code:

Using data from the same VI that the launcher used to obtain a list of startup processes, we generate an array of all the process labels, delete the first two elements (as discussed earlier, the exception handler and display processes) and write the remainder of the array to the ring control’s Strings[] property. The result is a popup menu containing the names of the acquisition data sources.

Next, we add an event case for a value change event on the ring control.

The ring value is a numeric, so the event handler uses it to index an element from the array of process labels, and uses the resulting string to fire the Change Source event. This is why we can use a string as the event data: Both the string we are using the fire the event, and the strings we are using to tell the processes who they are, derive from the same source: the process labels. Consequently, it is impossible for them to not match.

Finally, we need to look at the GUI initialization logic.

The first subVI called is the wait routine discussed above. When that notification is received, all the processes are loaded so the GUI can finish initializing itself by firing two value change events: one to set the sample period and a second to select the initial data source.

Summing Up

As usual, the completed code is below and when you run it you will now see that you can select between the three data sources we discussed. However, there is another lesson to be learned. I started this discussion saying that one might feel justified in saying that it would be a major undertaking to make such fundamental changes in the way an application operates. But we have just seen that it really is not. I can honestly say that describing what I did took much longer than making the modifications – and the reason is simple. I didn’t have to write very much code. Instead I was able to leverage features that were already built into the code, and reuse ideas that had already served me well in the past.

The sort of maintainability that allows our code to be nimble and not brittle should be a goal of all that you do, but it takes practice. Practice, and an understanding that learning never stops because there is always more to learn about the craft that we practice.

http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 8
http://svn.notatamelion.com/blogProject/toolbox/Tags/Release 4

Next time, we are going to take this line of work a step further and consider a more complex case. Sometimes you don’t want the data sources to stop acquiring data when they are “inactive”. Rather you want them to continue monitoring their respective inputs even when they are running in the background. But, how do you then manage the data transfer to the GUI? Well, perhaps the best solution for effectively displaying process data on the GUI is to never pass process data to the GUI. Sound confusing?

Until Next Time…

Mike…

One quick “PS”. Now that I am getting a good number of these posts done, I have been thinking I would like to have them available in languages other than English. I’m not exactly sure how it would work yet, but if you would like to help with the effort, please contact me.

Talking to a Database from LabVIEW

Posted on 2015-01-192015-06-08 by mlportersr

Ok, this is the third part of our discussion of how to effectively utilize databases in a LabVIEW-based application. To recap, In Part 1 we have covered the basic justifications for utilizing databases, and checked-out some drivers that implement basic database connectivity. Part 2 expanded on that information by going into the (very basic) basics of how to design and build a database. I also provided some links for further reading, and a LabVIEW tool for executing an SQL script to turn it into an actual database. What you are looking at now is Part 3 and it will present code that implements the LabVIEW side of the data management capability we discussed earlier. If you haven’t read the other two sections, I would recommend that you do so before continuing — don’t worry, we’ll wait for you.

Those of you who did read the previous portions, feel free to talk among yourselves until the others get back.

Everybody back? Good! Let’s continue.

Example Code

The database we have implemented so far covers three basic areas in the Testbed application. Something that I didn’t mention before is that these areas were not picked at random, or arbitrarily. Rather, if you look at them you see that each one presents an example of one kind of database operation while presenting useful concepts that you will want to know about in the future.

Processes to Launch: This section demonstrates data that has an inherent structure as embodied in its one foreign key relationship.
Event Recording: Here we considered a table to which applications will eventually write. It also shows a little more structure in that it relates to two different header tables: one that identifies the application generating the event and one that identifies the type of event that is being recorded.
Default Sample Period: Although much of the data in a system will be structured, there is still a place for simple setting such as you might store in an INI file. This last example showed such a situation.

Carrying forward with this idea of demonstrating a variety of concepts, as we go through the code that implements the LabVIEW side of the connection, I will point out a few different techniques that you will find useful. The thing to remember is that there are engineering decisions that you have to make and no one technique or approach will serve in every possible situation.

Reading Processes to Launch

The new VI that performs this operation is still named Configuration Management.lvlib:Get Processes to Launch.vi and has the same basic structure as the INI file version (which I renamed), but the configuration file IO logic is replaced with the database IO drivers I presented earlier.

Although the structure is basically the same as before, there are a few changes due to the improved structure that the database provides. First, there is a new input (Launch Condition) that is tied internally to a database search term. Second, the output data structure is modified to utilize the enumeration for the launch mode, replacing the boolean value used before.

In terms of the query code itself, the large SQL statement in the string constant is for the most part pretty standard code. The statement specifies what columns we want (label, item_path, launch_mode), the table they are in (launch_item) and the WHERE clause provides the search terms that define the output dataset we want. Likewise, note that although I never read the launch_order value, I use it to sort the order of the results. If you have data that needs to be in a specific order this is an important point. Unless you explicitly tell the DBMS how to order the results, the sequence of records is totally undefined. The only real complication is contained in the WHERE clause.

You will recall from our discussion of normalization that the two primary search terms we will be using are stored as ID numbers that reference a pair of header tables. These header tables contain the human-readable labels that we want to use for our searches. This code demonstrates one approach to resolving the ID references through the use of subqueries. A subquery is (as its name suggests) a small query that occurs inside the main query. They are typically employed to lookup data that the calling application doesn’t directly know. In this example, we know that the application name and launch condition, but we don’t know what ID numbers are associated with those inputs. The subqueries look up those values for us so the main query can use them to get the data we want.

The advantage of subqueries is that they allow you to specify what you want in the terms that are meaningful to the calling application. The disadvantage is that they can complicate SQL code and the resulting code can be more DBMS-specific. In addition, with some DBMS (like for example, Jet) there can be a significant performance hit involved in subqueries. You’ll note that the example code mitigates this potential issue by buffering the search results, thus ensuring that the “hit” will only occur once.

Saving Errors to the Database

This operation is performed by a new version of Startup Processes.lvlib:Store Error.vi. As with the original version, the input error cluster provides the information for the error record, and the output error cluster is the result of the record insert.

This code shows an alternative to using subqueries to obtain the ID references. The two subVIs look-up the required IDs in the database the first time they are called and buffers the results for later reuse. The two routines are very similar, so here is what the application ID VI looks like:

The advantage of this approach is that the required SQL is much simpler and very standard. The only real “disadvantage” is that you have to create these buffers — which really isn’t very much of a disadvantage. I really like this technique for situations where there are common IDs that are used over and over again. Unfortunately, this sort of modularization isn’t always possible, in most real-world applications you will need to know both techniques.

Read Default Sample Period

You will recall that this is new functionality so there is no “old code” to compare it to. Here is one option for what Configuration Management.lvlib:Get Default Sample Period.vi could look like.

The code here is very similar to that for reading the processes to launch, the main difference being that two of the search terms are in essence hardcoded. Current, there is only one parameter being stored in this table, but that situation could change at any time. While replicating this VI for each additional parameter would work, the result could be a lot of duplicated code. Consequently, I typically prefer to start with a lower-level VI that is something like this…

…and then make the VIs like Configuration Management.lvlib:Get Default Sample Period.vi simple wrappers for the buffer.

Hopefully by now you are beginning to see a bit of a pattern here. Every type of access doesn’t require a unique technique. In fact, there are really very few unique use cases — and frankly we just covered most of them. The real key is to get comfortable with these basic methods so you can customize them to address your specific requirements.

Moving On…

So we now have our database, and the VIs for accessing it. How do we roll this into out application? One way is to simply replace the old INI file versions of the VIs with the new database-oriented ones — which is what I have done for now. I say. “…for now…” because there is a better solution, but that will have to wait for another day. Right now, the solution we have is serviceable so I want to get back to a couple more important topics. Here’s the updated application:

http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 7

Until next time…

Mike…

Building a Testbed Database

Posted on 2015-01-122015-05-10 by mlportersr

As we continue our exploration of using databases to store data in your applications, the next thing we need to do is consider what the basic structure off a database should look like and how to construct a simple one that would be suitable for our Testbed application. Considering what we have assembled so far, we can immediately see two existing areas where a database would be useful:

Lookup for processes to launch: Currently this information is being stored in the INI file.
Recording errors that occur: Currently errors are being logged to a test file that is stored in the same directory as the application.

In both cases, moving this data into a database will offer a number of benefits in terms of configurability and reliability. But in addition to these two existing opportunities, there is a third value to implement that is new.

The default sample period: In the existing code, this value is defined by the default value of a control on the front panel of the Display Data process. This technique works, and in many case is a perfectly fine solution, but there are also cases where your customers will want to be able to change a value without you needing to modify the code. So we’ll make that change too.

As we go through the following discussion, an important point to remember is that project development is — in addition to everything else — a process of ongoing refinement. As you work through a project you are always going to be learning more about what it is that you are trying to accomplish and you need to be willing to incorporate those “lessons-learned” in your code. Of course this point assumes that your code is modularized such that this sort of “mid-course correction” can be implemented without doing violence to what you already have in place.

In order to break this work up into bite-sized pieces, this post will address the structure of the database itself. The VIs needed to implement this configurability will be addressed in the next installment.

Why be Normal(ized)?

According to Wikipedia, normalization (in the database sense) is “…the process of organizing the fields and tables of a relational database to minimize redundancy.” Basically, this means that a given piece of information, like a person’s name, should only be stored in one place in the database. Normalization is important because it is one of the major ways that databases ensure consistency in the data they contain. For example, if an operator’s name is only stored in one place, you don’t have to worry after the fact if “Robert Porter” and “Robert Portor” are really the same person.

Beyond this basic explanation, there is a lot of detail that we don’t need to discuss right now. In fact, there are whole books written on the topic of normalization. For your further reading, I have included links to two very good ones are at the end on the post. Our goal right now is to simply cover the basics so as you can understand what a real database developer puts together for you. Moreover, if the need should ever arise for you to create a database for yourself, this information certainly won’t make you an expert, but it will help you avoid a few major pitfalls.

The first step in creating a database is called Data Modelling. Data modelling is a process of looking at your data in order to understand its inherent structure. Ideally, data modelling should be something that you do early in your design process, regardless of whether you are going to be using a database. If you think about it, this point is particularly true when doing LabVIEW development. After all, LabVIEW’s core paradigm is one of dataflow, so how can you expect to properly define an application’s dataflows if you don’t understand the data’s inherent structure? Naturally therefore, when starting work on this testbed, I did some basic data modelling. Let’s go over what I discovered.

Supporting Many Applications

When studying the data that an application needs to handle, a good place to start is with a list of the data items that the code will need to complete the intended tasks. Some of this information will be common to all the tasks the code will be supporting, and some will be unique. At first glance, it might not seem like there is any information that the various functions have in common, but if we think ahead a bit, we see that there is one. To this point, we have been talking about the program that we are creating, as if it is the only one we are ever going to build and install on a user’s computer. But is this really a reasonable assumption?

In a way, this is sort of like talking about a computer that will only have one program installed on it. In addition, we have talked before about the advantages of seeing an “application” as the aggregation of the independent behaviors of many different processes that operate more or less independent of one another. It seems to me unlikely that the assumption we have been making will continue to hold up in the future.

In the world governed by INI files, this reality is addressed by simply having a separate INI file for each program. Databases accomplish the same task by adding a field that identifies which application is associated with (or in database-speak, related to) each data record. But how should these relationships be recorded in the database? Well, we could just include a field in each table called something like application_name, but this solution has two problems. First, remembering what we said about normalization, this approach would produce a lot of redundant data and a lot of opportunities for things to be misspelled, or mislabeled. Second, since we would be using that value to qualify the queries we perform in order to extract just the data related to one particular application, those string fields will end up being search terms and indexing or searching strings is inherently very inefficient.

Both problems are solved by creating a separate table that uniquely identifies each application. Primarily, it would hold the application name, but could contain other information such as version number, who wrote it, and when it was modified. In this table, each application will have one record and each record has a unique number associated with it called the Primary Key. Any other table that needs to relate to its data to a specific application can simply include a field that holds this number — and numbers are very fast to search and index. Here is the SQL code needed to create this type of table, sometimes called a header table:

CREATE TABLE appl (
    id       AUTOINCREMENT PRIMARY KEY,
    label    TEXT(40) WITH COMPRESSION,
    ver_nbr  TEXT(10) WITH COMPRESSION,
    dev_name TEXT(50) WITH COMPRESSION
  )
;

The first line contains the command (CREATE TABLE) and specifies the new table’s name (appl). Everything between the parentheses is a comma-delimited list of the columns that the table will contain. Each column definition consists of a column name and a definition of the column’s contents. Note that column names can contain spaces, but it complicates the syntax, so I don’t use them.

The first column is named id and the column definition presents the Jet DBMS syntax for identifying a column that you want to be an automatically generated integer primary key. Other DBMS also have mechanisms for accomplishing the same thing, but the syntax varies. The next three columns are defined as variable-length text fields of differing sizes. Note that defined in this way, the indicated count specifies the maximum number of characters that the field can contain, but the DBMS only stores the actual number of characters you put in it. Finally, the WITH COMPRESSION keywords are needed because a few versions ago, Jet was converted to storing UNICODE characters, which are 2-bytes long. The effect of this change is that ASCII strings started taking up twice as much memory as was needed. The WITH COMPRESSION keywords tell Jet to store the strings as 1-byte values.

One more point. The last field (dev_name) is intended to contain the name of the developer that last modified the application. But, wouldn’t what we said earlier about normalization apply to this information as well? Yes it would. To be formally correct, there should be another table of developers to which this table should relate. However, I put this in here to highlight the important point that in database design — as with LabVIEW programming — there trade-offs and so we need to ask ourselves whether the added normalization provides sufficient benefit to justify the added complication of another table. In this case, the answer I came up with was, “No”.

Processes to Launch

When we consider the information needed to launch the startup processes with the desired user feedback, we find that there are three things we need — plus of course a reference to the application identified in the table we just created. This is what we have.

appl_id: This is the numeric reference to the table identifying the applications.
label: This is the string that will be displayed to the user while the process is being launched. Before it was derived from the name of the VI. This approach is more flexible.
item_path: As when this data was stored in the INI file, this string is a relative path to the VI to be launched.
item_mode: This string indicates the mode used to launch the process VI. It is associated with an enumeration in LabVIEW that (for now at least) simply indicates whether the VI is reentrant or non-reentrant.
launch_order: This is an integer column we will use to sort the query results so the process VIs are launched in the right order.

So with this information in hand we are ready to define our next table, right? Well, not quite. We need to consider another assumption that we have made. To this point, the code we have created only uses VIs that are dynamically loaded in one place: at program startup. Will this assumption always be true? Simple logic would say that dynamic loading is a very powerful technique, so we will very likely be wanting to use it in the future. So let’s create another header table that will store a string indicating the condition where the VI will be launched. Right now the table will only have a single value in it, “Startup”.

Turning all this discussion into SQL commands to create the needed tables, this code creates the new header table:

CREATE TABLE launch_cond (
    id                AUTOINCREMENT PRIMARY KEY,
    launch_condition  TEXT(128) WITH COMPRESSION
  )
;

…then this code inserts the one record we need now…

INSERT INTO launch_cond (launch_condition) VALUES ('Startup');

…and finally we can create the table for holding the launch items:

CREATE TABLE launch_item (
    id              AUTOINCREMENT PRIMARY KEY,
    appl_id         INTEGER NOT NULL,
    launch_cond_id  INTEGER NOT NULL,
    label           TEXT(40) WITH COMPRESSION,
    item_path       TEXT(128) WITH COMPRESSION,
    launch_mode     TEXT(40) WITH COMPRESSION,
    launch_order    INTEGER,
    CONSTRAINT launch_cond_FK FOREIGN KEY (launch_cond_id) REFERENCES launch_cond(id),
    CONSTRAINT launch_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id)
  )
;

This table contains all the same stuff we saw in the other table creation commands, but with a couple additions. The last two items in the column list aren’t columns. Rather, they create the relational links between this table and the two header tables by defining a pair of foreign key constraints. The name of the constraints are important because they will be used in error messages associated with the constraint. The constraint definitions themselves are straight-forward, specifying a column that references a specified column in a specified table.

Storing Errors

The foregoing case demonstrates how to deal with tables that store configuration data. Now we turn our attention to a table that the application writes to while running. Specifically, we will look at the table for storing errors that occur during program execution. Here are the pieces of information we know we will need going in:

appl_id: This is the numeric reference to the table identifying the applications. In this case it is the application that generated the associated event.
evt_dttm: This field holds timestamp values indicating when the events occurred.
evt_type_id: In addition to knowing when an event occurred, you also need to capture what type of event it was. Starting off we can imagine three types of events (errors, warnings and notes) but there could be more, so we’ll store the type in another header table and reference it here.
evt_code: This integer column hold the error code from the LabVIEW error cluster.
evt_source: Another text column, this field holds the Source string from the LabVIEW error cluster.

First comes the new header table and its three records. Note that these event types correlate to the three values of a LabVIEW enumeration (Event Types.ctl).

CREATE TABLE event_type (
    id     AUTOINCREMENT PRIMARY KEY,
    label  TEXT(40) WITH COMPRESSION
  )
;

INSERT INTO event_type (label) VALUES ('Error');
INSERT INTO event_type (label) VALUES ('Warning');
INSERT INTO event_type (label) VALUES ('Note');

And then the table for storing the events…

CREATE TABLE event (
    id             AUTOINCREMENT PRIMARY KEY,
    appl_id        INTEGER NOT NULL,
    event_type_id  INTEGER NOT NULL,
    evt_dttm       DATETIME DEFAULT now() NOT NULL,
    evt_code       INTEGER,
    evt_source     MEMO WITH COMPRESSION,
    CONSTRAINT event_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id),
    CONSTRAINT event_event_type_FK FOREIGN KEY (event_type_id) REFERENCES event_type(id)
  )
;

The first new idea on display here is in the definition of the evt_dttm field. In addition to stating the data type (DATETIME), it also specifies that the field cannot be empty (NOT NULL) and tells Jet what value to use if an insert statement does not contain a time value: now(). This built-in Jet constant returns the current date and time. You also see that I introduced a new datatype MEMO to hold the textual description of the error. Given that it uses the same WITH COMPRESSION keywords as we use with the TEXT, you might assume that it is another way of storing textual data — and you’d be right. The difference between the two is that while both are variable length fields, a TEXT field can only hold a maximum of 255 characters. By contrast a MEMO field can (in theory at least) hold strings as long as 2.14-Gbytes.

The Default Sample Period

Finally there are always simple, unstructured setup values, like the default sample period, so let’s set up a table for them too. Note that I picked column names that reflect the simple organization of an INI file with sections, keys and values.

CREATE TABLE misc_setting (
    id         AUTOINCREMENT PRIMARY KEY,
    appl_id    INTEGER NOT NULL,
    p_section  TEXT(50),
    p_key      TEXT(50) WITH COMPRESSION,
    p_value    TEXT(255) WITH COMPRESSION,
    CONSTRAINT miscsettings_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id)
  )
;

INSERT INTO misc_setting (appl_id, p_section, p_key, p_value)
   SELECT id, 'Data Acquisition','Sample Period','1000'
     FROM appl
    WHERE label = 'Testbed'
;

The syntax used for inserting the record we need is a bit different because it has to look up the value that will go into the appl_id field using something called a subquery. The syntax for the Jet implementation of this type of operation is rather obscure and is in no way compliant with the SQL standard, but this is what it looks like. Unfortunately, it is not the only place where Jet runs counter to the standard. For example, getting the event table to automatically insert the event timestamp value using by using a default value of now() sidesteps the problem of formatting time values, which is also very non-standard.

A Few More Things

So there we have it: the structure for our simple database — or as much of it as we need right now. But you may be wondering what you are supposed to do with all that SQL code? I’m glad you asked. I have created a simple utility that executes the code to create the database, and you can find it (and the complete SQL source code file) here:

http://svn.notatamelion.com/blogProject/local database builder/Tags/Release 1

When you run the utility, it builds the new database file in the directory where the source code file is located. While I’m linking to things, it can be helpful sometimes to simply open a database file and look at its contents. If you have Access installed on your computer you can look at the resulting database file immediately, otherwise I can recommend a small, lightweight utility called Database .Net. It doesn’t require an installer, supports a bunch of different DBMS, and does not impact a computer’s registry so can run from anywhere. I like to keep it on a USB thumb-drive in case I need to take a quick look at the contents of a database.

Finally, here are a couple more links for books that I recommend for learning more about the issues we rushed through in this post. I really like The Practical SQL Handbook because it covers all the important points in a readable, entertaining way. A dry college text, this is not. Designing Quality Databases With IDEF1X Information Models, on the other hand, is a college text that goes into a tremendous amount of detail on the topic of data modelling and a common technique for documenting what you find — which of course makes it very through. This book is not a quick read, but it is worth the effort.

All we have left to cover now is the LabVIEW side of the job, which we’ll get into with the next post.

Until next time …

Mike…

Managing Data — the Easy Way

Posted on 2015-01-052015-05-18 by mlportersr

As I work on this blog there are times when I have to put certain topics on hold because the infrastructure doesn’t yet exist to allow me to effectively cover the topic. Well, this is one of those times. What I want to do is start getting into some topics like advanced user interfaces. However, these capabilities presume the existence of a data management capability that our testbed does not yet possess. To begin filling in that blank, the next few posts are going to start looking at techniques for utilizing databases as a data storage mechanism for LabVIEW-based applications.

But why databases? Wouldn’t it be easier to store all our configuration or test data in text files? When considering the storage of any sort of data the first thought for many developers is often to just, “…throw it in a text file…”. Therefore, this question needs to be the first thing that we discuss.

The Ubiquitous INI File

The INI file, as a method for storing configuration data, is a standard that has been around for a long time. Its main advantage is that it is a simple human-readable format that anyone with a text editor can manipulate. However, INI files also have a significant downside, and its number one problem is security. Simply put, INI files are inherently insecure because they use a simple human-readable format that anyone with a text editor can manipulate. Anybody who knows how to use Notepad can get in and play around with your program’s configuration, so unless you are very careful in terms of data validation, this openness can become an open door to program instability and failure.

A second issue is that while INI files can be subdivided into sections, and each section can contain key name and value pairs; beyond that simplistic formatting they are largely unstructured. This limitation shouldn’t be a surprise, after all the format was developed at a time when programs were very simple and had equally simple configuration requirements. However, with many modern applications one of the big challenges that you have to face when using INI files is that it is difficult, if not impossible, to store configuration data that exhibits internal relationships.

For example, say you have 2 or 3 basic types of widgets that you’re testing and each type of widget comes in 2 or 3 variants or models. In this scenario, some of the test parameters will be common for all widgets of a specific type while others are specific to a particular model. If you want to capture this data in a text file, you have two basic options. First you can put everything in one big table — and so run the risk of errors resulting from redundant data. Second, you can store it in a more normalized form and try to maintain the relationships manually — which is a pain in the neck, and just as error prone.

Text Data Files?

OK, so text files aren’t the best for storing configuration data. What about test data? Unfortunately, you have all the same problems — in spades — plus a few new ones: For example, consider Application Dependency.

Application dependency means that the data format or structure is specific to the program that created the file. Basically, the limitation is that outside programs have to know the structure of your file ahead of time or it won’t be able to effectively read your data. Depending upon the complexity of your application, the information required to read the data may expose more about the inner workings of your code than you really feel comfortable publishing.

Another problem is numeric precision. You can’t save numbers in a text file, just string representations of the numbers. Consequently, numeric precision is limited to whatever the program used when the file was created. So if you think that all you need when saving the data is 3 decimal places, and then find out later that you really need 4, you’re pretty much hosed since there is no way to recreate the precision that was thrown way when the data was saved.

Finally, data in a text file usually has no, or at least inadequate, context. Everybody worries about the accuracy of their test results, but context is just as important. Context is the information that tells you how to interpret the data you have before you. Context includes things like who ran a test, when it was run, and how the test was configured. Context also tells you things like what unit of measure to use in reading numeric data, or when the instruments were calibrated.

The Case for a Database

Due to all the foregoing issues, my clear preference is to use databases to both manage configuration data and store test results. However some LabVIEW developers refuse to consider databases in a misguided effort to avoid complication. Out of a dearth of real information they raise objections unencumbered by facts.

My position is that when you take into consideration the total problem of managing data, databases are actually the easiest solution. Of course that doesn’t mean that there won’t be things for you to learn. The truth is that there will be things to learn regardless of the way you approach data management. My point is that with databases there is less for you to learn due to the outside resources that you can leverage along the way. For one simple (but huge) example, consider that you could figure out on your own the correct ways to archive and backup all your test data — or you could put the data into a database and let the corporate IT folks, who do this sort of thing for a living, handle your data as well.

So let’s get started with a few basic concepts.

What is a DBMS?

One common point of confusion is the term DBMS, which stands for DataBase Management System. A DBMS is software that provides a standardized interface for creating, maintaining and using databases. In a sense, the relationship between a DBMS and a database is exactly the same as the relationship between a word-processor, like Microsoft Word, and a letter home to your Mom. Just as a word-processor is a program for creating textual documents, so a DBMS can be seen as a program for creating databases. One big difference though, is that while word-processors are programs that you have to explicitly start before you can use them, a DBMS will typically run in the background as a Windows service. For example, if you are reading this post from a Windows-based computer, there is at least one DBMS (called Jet) running on your computer right now.

What is a database?

A database is a structured collection of data. In that definition, one word is particularly important: “structured”. One of the central insights that lead to the development of databases was that data has structure. Early on, people recognized that some pieces of information are logically related to other pieces and that these relationships are just as important as the data itself. The relationships in the data are how we store the data context we discussed earlier.

By the way, as an aside, people who talk about an “Access Database” are wrong on two counts since Access is neither a database or a DBMS. Rather is it an application development environment for creating applications that access databases. By default, Access utilizes the Jet DBMS that is built into Widows, but it can access most others as well.

How do you communicate with a database?

It wasn’t that long ago that communicating with a database from LabVIEW was painful. The first product I ever sold was an add-on for the early Windows version of LabVIEW that was built around a code interface node that I had to write using the Watcom C compiler. Actually, looking back on it, “painful” is an understatement…

In any case, things are now dramatically different. On the database side the creation of standards such as ODBC and later ADO (also called OLE DB) provided standardized cross-vendor interfaces. For their part, National Instruments began providing ways of accessing those interfaces from within LabVIEW (starting with Version 5). Today accessing databases through ActiveX or .net interfaces is a breeze. To demonstrate this point, I’ll present a package of drivers that I have developed and posted on the user forum several years ago.

Getting Connected…

The VIs we will now discuss are the core routines that you will need to interact with any DBMS that supports an ADO interface — which is basically all of them. The only common DBMS that doesn’t support ADO is SQLite. Instead, it has its own DLL that you have to access directly. Still, if you want a very lightweight database engine that will run on nearly anything (including some real-time hosts) it is a good choice and there are driver packages available through the forum.

Getting back to our standard interface, the following six routines provide all the support most applications will ever need. One thing to notice is that with the exception of one subVI that requires added code to work around a bug in certain version of SQL Server, most of the code is very simple, which is as it should be.

To start that exploration, we’ll look at the routine that is the logical starting place for any database interaction — and incidentally is the only routine that cares what database you are using.

ADO Database Drivers.lvlib:Acquire Connection.vi

The start of any interaction with a database, naturally involves establishing a connection to the DBMS that is managing it, and then identifying which specific database you want to use. This VI calls the ADO method that performs that operation.

You’ll notice that the ActiveX method only has one required input: the Connection String. This string is a semicolon-delimited list of input parameters. Although the exact parameters that are required depends on the DBMS you are accessing, there is one required parameter, Provider. It tells the ADO functionality what driver to use to access the DBMS. This is what a typical connection string would look like for connecting to a so-called Access database.

Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\mydatabase.mdb;User Id=;Password=;

Note that in this case the Data Source consists of a path to a specific file. For another DBMS this same parameter might point to an IP address, a logical name the driver defines or the name of a Windows service. But what if you don’t know what connection string to use? Well you can ask the DBMS vendor — or you can check: www.ConnectionStrings.com. Yes, there is an entire website dedicated to listing connection strings, and it covers nearly every ADO-compatible DBMS on the planet.

ADO Database Drivers.lvlib:Execute SQL Command.vi

OK, you are connected to your DBMS and you have a reference to your database. The next question is what do you want to do now? One answer to that question would be to send a command to the DBMS telling it to do something like create a new table in which you can store data, or add a new record to an existing table. This VI meets that need by sending to the DBMS command strings consisting of statements written in a language called SQL.

The core of this routine is the connection Execute method. Because this method could be used to execute a command that returns data, the 0x80 input optimizes operation by telling the method to not expect or handle any returned data. The method, instead, returns a count of the number of rows that the last command impacted.

ADO Database Drivers.lvlib:Create and Read Recordset.vi

While sending commands is important, the most common thing to do with a database connection is to use it to read data from the database. To optimize operation when you have multiple users concurrently accessing the same database, ADO creates something called a recordset. A recordset is a memory-resident copy of the requested data that you can access as needed without requiring further interaction with the database itself.

The three subVIs shown create a recordset using a source string consisting of an SQL query, reads the recordset’s contents, and then closes the recordset to free its memory. For details of how these subVIs work, checkout the comments on their block diagrams.

ADO Database Drivers.lvlib:Release Connection.vi

Once you are finished using a connection, you need to tell ADO that you are done with it.

Note that although the method called is named Close, it doesn’t really close anything. Thanks to built-in functionality called connection pooling the connection isn’t really closed, rather Window just marks the connection as not being used and puts it into a pool of available connections. Consequently, the next time there is a request for a connection to the same database, Windows doesn’t have to go to the trouble of opening a new connection, it can just pull from the pool a reference to a connection that isn’t currently in use.

ADO Database Drivers.lvlib:Start Transaction.vi

A topic that database folks justifiably spend a lot of time talking about is data integrity. One way that a DBMS supports data integrity is by ensuring that all operations are “atomic”, i.e. indivisible. In simple terms, this means that for a given operation either all the changes to the database are successful, or none of them are. This constraint is easy to enforce when inserting or modifying a single record, but what about the (common) case where a single logical update entails adding or modifying several individual records?

To handle this situation, DBMS allows you to define a transaction that turns several database operations into a single atomic operation. This VI marks the start of a transaction.

To mark the other end of the transaction, you have to perform either a Commit (make all the changes permanent) or a Rollback (undo all the changes since the transaction’s start).

ADO Database Drivers.lvlib:Rollback Transaction on Error.vi

This VI combines the Commit and Rollback operations into a single routine. But how does it know which to do? Simple, it looks at the error cluster. If there is no error it commits the transaction…

…but if there is an error it will rollback the transaction.

What’s Next

Clearly the foregoing six VIs are not a complete implementation of all that is useful in the ADO interface — for instance, they totally ignore “BLOBs”. However I have found that they address 95% of what I do on a daily basis, and here is a link to the new version of the toolbox that includes the drivers.

Toolbox Release 3

But let’s be honest, having an appropriate set of drivers is in some ways the easy part of the job. You still need a database to access and you need to know at least a bit about the SQL language. This is where what I said earlier I said about leveraging the time and talents of other people come into play. Your corporate or institutional IT will often assist you in setting up a database — many times, in fact, they will insist on doing it for you. Likewise, with an interface that is based on command strings written in SQL, you will find a wealth of talent willing to lend a hand, perhaps in-house but certainly online. Still, having said all that, it’s good for you to understand at least a bit about how these other aspects operate. Therefore, next time we’ll create a small local database (using Jet) and start implementing the data management for our testbed application.

Until next time…

Mike…

Advantages of Using Alternative Logic Symbols

Posted on 2014-12-292014-12-19 by mlportersr

There is a feature hidden on the LabVIEW function palettes that implements a very useful bit of functionality, but unfortunately is not being used nearly enough. I am speaking of the Compound Arithmetic node which can be found on either the Numeric or Boolean palette. Despite its name, the node does far more than simple “Arithmetic”. In this post we’ll consider how it can and should be used to help us implement logic operations.

Most folks use it when they need to save space while implementing a logic gate with more than 2 inputs, but there is a lot more functionality hidden inside it. For example, have you ever gotten tied up in the logic of what you feel should be a pretty simple operation, but end up spend a lot of time figuring it out? Well this little node can help you define logic using a simple, straightforward process.

Ever look at a logical operation in a piece of inherited code (or worse code you wrote yourself six months ago) and sat scratching your head trying to figure out what the original developer was trying to do? Well this node can help you learn not only what the developer wanted, but what they were thinking.

Augustus De Morgan: The Other Big Name in Binary Logic

I first learned about binary and logic gates while going through electronics school in the United States Air Force. However, when I got out and found my first civilian job, I saw a gate on a schematic that looked like something like this:

I of course knew what an AND gate was, but what was the deal with all the extra bubbles? When I asked an engineer I worked with about the symbol, he explained that it’s just another way of drawing an OR gate. When I asked why the designer didn’t just draw an OR gate, he explained (somewhat irritated) that the designer couldn’t because an OR gate means something different. So I asked one more question, “Well, if it’s the same as an OR gate how can it mean something different?”. The engineer responded (this time, decidedly irritated), “Because one is an OR gate and one is an AND gate!” At this point he stomped off praying under his breath for some unnamed deity to give him strength when dealing with, “stupid technicians”.

OK, so this mystery gate with all the extra bubbles is just another way of drawing an OR gate, but somehow it also managed to mean something different from an OR gate. In other words, it is simultaneously the same-as and different-from an OR gate. At this point, my head began to hurt as I wondered how functions that allowed this sort of confusion could be called “logic”.

Somewhat later I discovered Augustus De Morgan, and specifically De Morgan’s Law which stated (in the linguistic gobbledygook that mathematicians seem to love):

The negation of a conjunction is the disjunction of the negations.
The negation of a disjunction is the conjunction of the negations.

Uh, right… or in normal in English: If you draw the truth-tables for an AND gate with all its inputs and the output inverted and an OR gate you will see that they are in fact identical. Likewise, the opposite is also true. The truth table for an OR gate with all its inputs and output inverted is the same as that for an AND gate. Well, that explained the bit about the modified gate symbol being, “…another way of drawing an OR gate.”

But what about the part where they mean different things? Again De Morgan provided the answer, not in the form of another pithily worded law, but in the basic meaning of the word “meaning”. Coming from the background I did, the output of a logic gate was either high or low, on or off, +4.5V or 0V. It never occurred to me that these signals could have a broader meaning — like a low on this input means that something important has happened, or even basic concepts like true and false. This realization made the engineer’s last statement make sense. An OR gate and an AND gate express fundamentally different logical ideas, regardless of how their inputs and output might be inverted.

At its most basic, an OR gate says, I want a given output when any input is in the indicated state. By contrast, the message an AND gate delivers is that I want a given output when all inputs are in the indicated states. So they really do mean different things. Still while this discussion may have been interesting, and perhaps even intellectually stimulating, if there is no practical applications of the knowledge, we have wasted our time. However the preceding discussions has several very practical implications. To begin with, it points the way to the possibility of creating code that not only defines a given functionality, but can actually provide insight into what you were thinking as you were designing and implementing the code. Moreover, due to the way LabVIEW implements the Compound Arithmetic node, you can easily build-up even complex logic operations through a simple 3-step process. To see how this works, let’s walk through a simple example that shows all the steps.

Setting-up the Example

The example we’ll consider is developing the logic that either allows a while loop to keep running or causes it to stop — depending upon your point of view. We know that, by default, passing a Boolean true to the conditional node causes the loop to stop at the end of the current iteration. Now we can change that behavior by right-clicking on the conditional node — but why bother? Here is skeleton of the loop that we’ll be controlling.

The goal will be to control the operation of the loop where we have two values upon which to make our control decisions: a scaled random number that is measured against a range of 0 to 975, and the number of loop iterations which tested to determine whether or not it is equal to 5o.

Defining the Output

Our first challenge is to decide what we are wanting to do: Let the loop run until something is true, or let the loop run as long as something is not true? Either one would work equally well, but given the specifics of the code you’re working on, one may represent how you are thinking about the task at hand better than the other. You want to use the one that better fits your thinking.

To implement this first decision, place a Compound Arithmetic node on the diagram and if you are thinking that the loop should run until something happens leave the output as is, otherwise right click on the node’s output terminal and select Invert from the pop-up menu. I’ll invert the output.

Defining the Core Logic

To this point, we have been talking in generalities about “something” happening or not happening. Now we need to drill down a bit and start defining the “something”. Specifically, do we want the event to be flagged when all the inputs are in particular states or when any of the inputs are in particular states? The answer to this question tells us whether we fundamentally want an AND operation or an OR operation. Looking at the logic, I see the two parameters and I want the loop to continue if they are both valid — an AND operation. Now if you get your Compound Arithmetic node from the Boolean palette, the default operation is OR. On the other hand, if you get the node from the Numeric palette you’ll note that the default operation is ADD. In either case you will need to right-click on the node and select AND from the Change Mode… submenu. While we’re at it, let’s wire up our two conditions.

Note that although the run arrow is no longer broken, we aren’t done yet. We still have one more step.

Defining the Inputs

I said earlier that I wanted the gate to test whether all the inputs are valid. So now we need to look at each of the two inputs and identify what the valid state for each. The top terminal is wired to the In Range? output from the In Range and Coerce node. Because this output is true when the input number is in range (and therefore, valid) we will leave its input as is. However the bottom input is coming from logic that is comparing the loop counter to a limit (50). Since that value is a limit, a valid input is one that has not yet reached the limit. Consequently, in this case, a valid number is indicated by the output of the comparison being false. To indicate that fact, right-click on the lower input terminal and select Invert from the popup menu. Here is the finished code.

If you run it you’ll notice that the loop counter indicator will show a variety of values, depending on which of the two conditions stopped the loop, but the count will never be greater than 50.

Admittedly, at first glance the logic gate looks a little strange, but if you deconstruct it by stepping through the foregoing process backwards you can easily what it’s doing. More importantly, though, the structure gives you an insight into what I was thinking when I created the logic. Also note that depending on how you think about what it is that you are doing, there are several other ways the gate could be defined that would give the same functionality.

Until next time…

Mike…

When are UDEs not the Right Answer?

Posted on 2014-12-222014-12-19 by mlportersr

As we have seen, UDEs can be a powerful way of passing data between processes, but they are no Silver Bullet, no Panacea, no Balm of Gilead, no … well, you get the point. Given that we aren’t going to do something silly like only use one method for passing all our data, we need to ask an important question: When should you consider the other techniques?

Knowing What is Important

The reason people choose one thing over another (or at least why people should choose one thing over another) is that they are looking for better performance — however you might define that word. But before we can say one technique is better than another, we need to understand the key features of each technique. With that knowledge in hand we can then apply those techniques in ways where the features work to our advantage, but we avoid problematic side-effects. So, since our present concern is data passing, let’s consider the issue I call, immediacy.

There are times when something needs to happen as quickly as possible after an input value changes. In the hardware world this sort of condition is called an edge trigger and as a system architect working with them you are often primarily interested in when the change occurred. Of course an event can carry data, but the key distinguishing factor for this sort of communications is timing. These are the sorts of signals for which UDEs are an excellent choice because UDEs are very good at telling remote processes that something has changed.

On the other hand there are signals where functionally it doesn’t matter really when they changed. All the code really cares about is what their value is the next time they are used. These are the signals that you could pass with a UDE, but to do so would be wasteful of the receiving process’ time. Remember that when an event fires any process registered to receive that event has to stop what was doing to handle the event and then return to what it was doing before it was interrupted — even though it fundamentally doesn’t care when the value changes. Although these diversions might be small, they do take some time and so can effect loop timing. Moreover, if you have something running in a Timeout event a UDE (or any other type of event) can drastically effect the timing of when the next timeout occurs. Remember that a timeout, of say 1000 msec, does not mean that the code in it will execute every 1000 msec. Rather, the event triggers when the time since the last occurrence of any event exceeds 1000 msec.

In fact, in this situation, if you have a UDE that fires every 500 msec, the timeout will never occur. To summarize this point, a good way to think about this is that an event actually carries two pieces of information: data and timing. What we need sometimes, though, is just the data.

Just the Data

The good news is that LabVIEW offers a variety of options for passing just the data. The oldest (and still very useful) way is with a Functional Global Variable (FGV), also sometimes called a LabVIEW 2 style global. A FGV uses an uninitialized shift register to hold data in memory. More recent additions to our arsenal include feedback nodes, data value references (DVRs) and shared variables.

Your choice between these options should be driven by memory and performance concerns. A FGV is easy to create and very flexible, but if the data being buffered is large, they can suffer performance issues due to copying of memory buffers. A DVR on the other hand, is a little trickier to use because there is a reference you have to pass around, but is very efficient. With large datasets i will often split the difference and combine the two: I will use the DVR to store the data, but use a FGV to buffer and store the DVR reference. Although it is sort of overkill for the data that I will transferring, I’ll show you the technique in the following example.

What We’ll Build

Right now in the testbed application the delay between sampled is hardcoded to 1000 msec. So to demonstrate the passing of non-time-critical data, let’s modify our code to allow the UI process to vary the delay between data samples.

The first thing we want to do is create the buffer we will use to transfer the data. So in the project directory create a subdirectory named _buffers and inside it create a subdirectory named Sample Rate. Finally inside that subdirectory create a LabVIEW library named Sample Rate.lvlib and two more subdirectories called _subVIs and _typedefs. If you have been following this blog from the beginning, this arrangement should look familiar as it is very similar to what we did for organizing UDEs — and we are repeating it here for all the same reasons (unique name space, access control for subVIs, etc.).

The Buffer

The first real code will be the buffer and this is what it looks like:

As promised, it has the basic structure of a FGV, but the global data is the DVR reference. Note also that the DVR’s datatype is a cluster (typedef’d of course) that contains a single integer. The typedef is saved in the typedef directory with the name Data.ctl and the buffer is saved in the subVI directory with the name Buffer.vi. The two subdirectories have been added to the library and the access scope for the subVI directory is set to Private.

Reading and Writing the Buffer

With the buffer itself created we need to add to the library a pair of publicly accessible VIs for accessing it — and here they are. First the read:

…and now the write:

Note that if anything about the data or how it is stored would ever need to change in the future, this one library would be the only place in the code that would be impacted.

Modifying the Testbed

All we have to do now is modify the testbed to add in the logic that we just created, but thanks to the infrastructure that we have created, that will be an easy task. In fact it will take exactly three changes — total modification time, less than 5 minutes.

Add an I32 control to the front panel of the display process and create a value change event for it. In the event handler for the value change, wire the NewVal event data node to the input of the buffer write VI.
Setup the initialization by adding a property node to fire the value change event using the default control value.
Add the buffer read to the acquisition VI as shown.

That’s all there is to it. Another good point for this approach is that because the DVR’s datatype is defined as a typedef, you can make changes to the data in the buffer without recreating everything. In addition, because the data is a cluster, the existing code won’t break if you add another value to the cluster. Go ahead and give it a try. Open the typedef and add another control to the cluster. After you have saved the change note that the the existing code did not break.

Check here for the updated project:
http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 6

Oh yes, one more thing. Remember how we created a zip archive holding the basic template files for a UDE? It might be a good idea to do the same thing for this structure. Until next time…

Mike…

State Machine Worst Practices

1. Use the state machine as the underlying structure of your entire application

2. Don’t start with a State Diagram

3. Ignore what a “state” is

4. Use strings for the state variable

5. Turn checking the UI for events into a state

6. Allow default state transitions

7. Use a queue to communicate state transitions

Implementation Preview

Share this:

Creating Reusable Resources

Basic Concepts

A New Requirement

Dynamically Managing Events

The 411 on Menus

The Menu Event Handlers

Changing Resource Appearances

Code Testing

Share this:

Let’s See Them Work!

Making Background Task Presentable

Adding the Subpanel to the GUI

Running the Result

Share this:

But Why Subpanels?

Problems with XControls

Tab Controls — Just Say “No”

Problems with Tab Controls

A Good Solution for GUI Modularity

Subpanel Advantages

Let’s See Them Work!

Share this:

A Little Review

Modifying The DAQ Process

Passing a Startup Value

Signalling Completion

Adding Control to the GUI

Summing Up

Share this:

Example Code

Reading Processes to Launch

Saving Errors to the Database

Read Default Sample Period

Moving On…

Share this:

Why be Normal(ized)?

Supporting Many Applications

Processes to Launch

Storing Errors

The Default Sample Period

A Few More Things

Share this:

The Ubiquitous INI File

Text Data Files?

The Case for a Database

What is a DBMS?

What is a database?

How do you communicate with a database?

Getting Connected…

ADO Database Drivers.lvlib:Acquire Connection.vi

ADO Database Drivers.lvlib:Execute SQL Command.vi

ADO Database Drivers.lvlib:Create and Read Recordset.vi

ADO Database Drivers.lvlib:Release Connection.vi

ADO Database Drivers.lvlib:Start Transaction.vi

ADO Database Drivers.lvlib:Rollback Transaction on Error.vi

What’s Next

Share this:

Augustus De Morgan: The Other Big Name in Binary Logic

Setting-up the Example

Defining the Output

Defining the Core Logic

Defining the Inputs

Share this:

Knowing What is Important

Just the Data

What We’ll Build

The Buffer

Reading and Writing the Buffer

Modifying the Testbed

Share this: