Implementing Dynamic User-Defined Events

Posted on 2016-04-122016-04-12 by mlportersr

In the previous post, we started looking at how to deal with the situation where you have reentrant VIs that need to have UDEs that are specific to particular instances of the VI. That post covers a lot of the basic theory and design issues, so if you haven’t read that post, please take a few moments to check it out as much of the following won’t make sense without that background.

Filling in the Blanks

The basic approach we took in designing our example consisted of first defining the program’s overall intended operation. With that theory sorted-out, we then began creating the program’s high-level structure, being sure to leave blanks or prototypes as placeholders for details that we hadn’t yet defined. At one time this sort of approach would have been considered heresy. The feeling was you hade to have everything worked out in detail before you wrote even one line of code or created even a single VI.

Clearly there are a lot of practical problems with this approach, but for one of the biggest (in terms of long-term impact on the project) is that it leads to a break-down in the walls that information hiding is trying so hard to erect. Think about it for a second, if an integral part of designing a process launcher includes figuring out the internal operation of the processes it is going to launch, it’s going to be very difficult to keep that knowledge from contaminating your launcher design. Or to put it another way, there will be a strong tendency to create code that depends on the process VIs behaving in a particular way.

You can see a better approach in the way we designed the launcher for our reentrant VI. Last time we defined a VI that (beyond the mechanics of how to launch a reentrant VI) only concerned itself with the data that the process VIs need to do their work. Based on a black-box description of what the routines were supposed to do, we able to create the logic for providing that data without worrying about what the routines were going to do with it. This is a Very Good Approach.

Getting Registered

One of the “blanks” that we left to be filled until now was a VI called DAQ Operations.lvlib:Interval Registry.vi. When we first considered this VI’s external persona we noted that, when passed an enumerated value Check and an interval, it would return a boolean flag indicating whether or not that interval already existed. However, we made no assumptions about how it might go about completing that task. Having taken, so to speak, a step inside the veil we can look at how it works.

Interval Registry - Check

This is the logic that completes the Check function, and as it turns out, there isn’t really very much to it. In fact, we see logic that implements a simple FGV. All the function in question really does is search an array of intervals to see if the one we asked about exists. Given this level of simplicity, a logical question would be, “Why bother with the subVI? The process manager has a loop, why not just use it to maintain the list of active intervals?”

This approach would handle half of the problem quit well – the half dealing with whether or not a particular interval was launched. But if you think about it, that isn’t what we really need to know. Operationally, we don’t care if a particular interval was ever launched. We want to know whether the interval is still running, and the interval’s current state is something of which the manager has no knowledge. Remember that when we defined the rules governing the high-level behavior of the acquisition subsystem, we said that when an acquisition process sees that it has no addresses left to poll it will shut itself down. How is the process manager supposed to know that happened?

Beyond this practical consideration, there is also a conceptual problem associated with storing the interval list in the process manager. You might not realize it, but all data has an associated “scope” that defines the area, or context, which owns the data. If we store the interval list in the process manager we are, conceptually, giving ownership of a piece of information that should belong to the subsystem as a whole, to one specific VI. Moreover, that one specific VI would then be required to manage all accesses to that data. Now while that functionality could certainly be implemented, it would be at the cost of additional complications in the form of additional signalling, and event handlers to respond to those messages.

By contrast, moving the array into a FGV that any VI in the subsystem can access, makes the data immediately available to the subsystem as a whole: No event handlers, no added signalling. As we get into the VI that performs the (simulated) Modbus IO Collect Data.vi, we see how the FGV’s remaining two functions work.

Keeping Time

Next, let’s look at the reentant VI (Metronome.vi) that is responsible for triggering acquisitions at a fixed interval. You will recall that the VI is passed three parameters when it is launched: The desired interval between acquisitions, the event to fire when the interval times out, and the event that will fire when the interval is shutting down. Here is the logic for initializing the VI.

Metronome - Initialization

We see that the interval value drives the Timeout node of an event loop, the shutdown event is registered, and the event that this VI will fire, is simply passed into the event loop. As you might expect, the event loop itself only has two events. Here is the Timeout event:

Metronome - Timeout

No surprises here: all the event needs to do is fire the acquisition event – which this code does with alacrity. The shutdown event (which handles both types of shutdown) is likewise uncomplicated:

Metronome - Stop Interval

It destroys the two dynamic UDEs and stops the event loop. The acquisition event is destroyed because once a shutdown is initiated, it is no longer needed. The interval stop event is destroyed because it is only fired in the acquisition VI, so by the time we get to this point in the code it has already been fired by the only other VI that needs it. The small delay between the two operations is to ensure that the acquisition VI has time to start shutting itself down.

Getting into the Publishing Business

The last code we need to look at is the reentrant VI that reads the simulated data and publishes it for use (Collect Data.vi). Again starting with the initialization logic, we see something very similar to the metronome VI:

Collect Data - Initialization

The main difference is that this VI needs to register to receive both dynamic UDEs. Although the VI will be generating the Delete Interval event it also needs to be able to respond to it because that context is the best place put the logic for deinitializing any acquisition logic that might be used. Another conceptual difference from the metronome is that this VI relates to the interval input in a way that is fundamentally different. For the metronome the interval is a number that has a specific meaning: it is the number of milliseconds between triggers. For this data collection routine however, it is simply a number that provides it way for it to identify itself. The broader point is that you need to remember to manage the expectations when different processes look at the same value and see different things.

Because this VI is going to be acquiring data, and performing other operations, its event loop is also more complex. Let’s look at the 5 events it will handle:

Timeout – This application is only generating simulated data, but in most other situations, there will need to be some sort of initialization performed, like opening DAQ references or establishing connections to one or more Modbus devices, and this event is a good place to handle such initialization. The way this logic is written, it is easy to make the initialization run just once, but still have it readily available if it ever needs to be run again.

Collect Data - Timeout

However, there is other initialization that needs to be performed even for simulated data. Specifically, if the initialization logic completes with no errors, we need to tell the rest of the application that this process is open and ready for business. To record that state change, we use our “repository” VI again, but this time running the Insert logic…

Interval Registry - Insert

…which is the very soul of simplicity.

Start Addresses – This event’s purpose is to maintain the process’ internal list of addresses in response to the user starting additional addresses. In completing this work, there are two cases that it will need to address.

Collect Data - Start Addresses - Adding

First there is the case where the interval number matches the value that the process is using to identify itself. In that situation the code needs to add the new addresses to the existing list, or more correctly, it needs to add addresses to the array that don’t already exist in the array. This logic protects the logic from what would be a very common operator mistake: adding addresses tht already exist. The second situation is one where the interval number does not match the value that the process is using to identify itself.

Collect Data - Start Addresses - Delete Case

In this case, we need to enforce the rule that only one process can be polling a given address. Hence, the logic needs to remove from its array any addresses any addresses contained in the new event. Of course this operation raises the spectre of the entire polling array being emptied out, with the contingent requirement that the interval shut itself down. The logic handles that scenario with a two-step process. First it tells the rest of the application that its stopping by calling the registry VI using the delete logic.

Interval Registry - Delete

After removing itself from the registry, the VI fires the Stop Interval UDE to close both itself and its associated metronome process.

Stop Addresses – This event has logic that is similar to the previous event’s delete logic, but is simpler because the only thing that matters is whether the indicated address is in the process’ address list.

Collect Data - Stop Addresses

Get Data – This event generates an array of simulated data and passes it to the static Publish Data UDE, along with an array of addresses associated with the data values. In addition for troubleshooting purposes, the code also writes the data to a front panel table.

Collect Data - Get Data

Stop Interval; Stop Application – Finally, this event stops this VI if either the interval or the application as a whole is shutting down.

Collect Data - Stop Interval

As the comment says, if the acquisition logic needs to be deinitialized, this in the place to put that logic. But why is the code interested in the front panel’s state? Although this VI usually runs in the background unseen, there are times that you want to be able to view its front panel so you can verify its operation. In this example, I implemented that functionality using the VI properties to force the front panel open when it starts running. This logic checks to see if the front panel is open and, if so, closes it.

Testing the Code

So with all the code implemented, here are the Subversion links to the application an the toolbox of reusable code:

Dynamic Registration – Release 1
Toolbox – Release 18

The first thing I would recommend after downloading the code, it to go through it while re-reading both this post and the previous one. Often times it is easier to understand things when looking at the code, that are otherwise a bit obscure when all you have are pictures of the code.

Next run the top-level VI (Dynamic Registration.vi). When the front panel opens, click on the Add Addresses button to define some addresses. For the purpose of this example, I created a simple dialog box that lets you specify a starting address, the number of consecutive addresses to collect, and the sample interval. The starting address needs to be between 40000 and 49999, the number of consecutive address must be less than 1000 and the sample interval needs to be between 300 and 5000 (milliseconds). These parameter limits are set in the dialog box code – feel free to change them as you desire. Likewise, the output of this dialog box is a list of addresses, so you can also change the selection interface if you so desire.

To get started, define 5 addresses starting at 40000 with a sample interval of 1000-msec. You should see the front panel of acquisition VI pop open showing the 5 addresses being updated once per second. In addition, the main GUI should show data from the same 5 addresses.

Click the Add Addresses button again, but this time define 5 addresses starting at 40008, and updating every 2000-msec. A second acquisition VI windows will open showing the 5 new addresses, and the main GUI will show a total of 10 addresses with the results changing at different rates.

Let’s next see what happens if you specify addresses that are already being polled. Click the Add Addresses button one more time and define 5 addresses starting at 40004, and updating every 3000-msec. This action defines a range where 40004 is already being polled once a second and 40008 is being polled every 2 seconds. In response you will see a third acquisition window open, but you will observe that one address is removed from each of the two existing polling lists, and that the main GUI shows a total of 13 addresses being polled.

Finally, to test the auto-shutdown operation click the Delete Addresses button and in the resulting dialog box, tell the system to delete 5 addresses starting at 40008. Because the 2000-msec interval is emptied out, the acquisition VI window associated with that interval will close. Finally, the polling list for the 3000-msec interval will be reduced by one.

The Big Tease

So that’s about all for now on this topic, but what’s in store for next time? One of the things that I like to talk about in this venue are things that can cause unexpected complications, so next time I’m going to discuss what happens when a DLL misbehaves as you are trying to close it.

Until Next Time…
Mike…

Dynamic UDEs: the Power for Reentrant Processes

Posted on 2016-03-222016-03-23 by mlportersr

If you have an application that you want to construct from multiple parallel processes, a key requirement is signalling – telling the various parts of the application that something important has happened, or is happening, in one of the other processes. When it comes to fulfilling that requirement, a valuable tool to have at hand is the User Defined Event, or UDE. In fact, over the past year this blog has considered a variety of ways to use this tool. However, all these implementations have one thing in common: They all use static UDEs. In other words, the event that will be used to signal a particular occurrence is decided when the code is created and it never changes.

But what if you don’t know until runtime what event you want to use? For example, it is common with reentrant code that you won’t have a fixed set of UDEs because there isn’t a fixed set of VIs that are running. Sometimes you need to be able to send a message to one particular clone. In such a situation, you aren’t just sending general signals, but signals that are unique to a particular instance of a VI. The solution is to use UDEs that are dynamically generated in the same way that the reentrant VIs are.

To demonstrate this technique, the next couple posts will highlight this use case, starting this time with a discussion of some of the design considerations. Our into to this exploration is an application that I was once assigned to maintain.

The Problem Defined

The job asked me to expand an existing application that had been developed by a large LabVIEW consulting firms located here in the US. The problem is that the software wasn’t designed for expandibility. Specifically, a key part of the program was a subsystem that polled a user-defined list of Modbus registers at a rate that was also user-defined. Because the user-defined inputs could change at any time, the decision was made to make the acquisition loops event-driven, and create a separate “metronome” process that would fire an acquisition event at the user defined rate. So far, so good. The real issue is with the implementation of this concept.

Apparently, there was originally only going to be one timed interval but, as you might expect, a requirement was later added to create a second one. To meet this scope change, with as little effort as possible, the decision was made to simply duplicate all the VIs used for the first process while appending “2” to the end of the names – an expedient that is unfortunately common in code developed by “large LabVIEW consulting firms”. To make matters worse, the modularization was poor so the program was basically built around a huge cluster containing literally dozens of references for UDEs, notifiers and queues that ran through nearly every VI in the application.

In the end, the only way to implement the required functionality was to completely redesign and reimplement that portion of the code. The really ironic part is that it took me less time to implement the functionality correctly, than it did to do it poorly the first time. Using this description as a jumping-off point, I obviously won’t be discussing the solution that I implemented for the original application. What we will do is use it as “inspiration” for examining techniques that could cover a wide range of similar requirements.

Getting Moving in the Right Direction

First, we should recognize that while our earlier conversation incorporates a pretty good description of what the code basically needs to do, we do need to flesh it out a bit: At run time, we need to be able to create multiple independently-timed data reads with varying intervals between reads. In addition, the results from these timed acquisitions need to be “published” somehow so they can be used by the rest of the program.

With this broader functional definition in place, we can begin to consider the appropriate API for accessing that functionality. As I have said many times before, one of the corner stones of a good API is the concept of information hiding – the process of deciding what information to expose to the calling code, and what information to keep private. So like a politician running for reelection, our next job is to decide what to hide and where to hide it.

The basic principle in play is to hide any information that the calling code either doesn’t need to know, or which would be counter-productive for it to know. If we think about it, there are exactly three pieces of information that the calling VI actually needs to know in order to define and use an acquisition task:

The Modbus address to read
The interval between reads
How to receive the published data

On the other hand however, there is a (much) large list of things that the calling code doesn’t need to know, among which are things like:

How the Modbus is read
How the timers work
The signalling that the timers use
How many acquisition processes there are
How the timing is implemented
Internal data structures
etc…

Now that we have a handle on what we want to expose – and just as importantly, what we do not want seen – we can start designing the outward interface.

The Outside View

The obvious place to start is with the VI that will configure or setup the polling. Given that we have already decided that we only want to expose two parameters (addresses to read, and the read interval) we can go ahead and create a placeholder VI that provides the appropriate IO, but which is for now empty.

Start Addresses

Note that with the exception of the error cluster, this VI has no other outputs. Remember that all this VI is doing is identifying a group of addresses that some hidden “something” is going to read at the specified interval. Consequently, the only response that this VI can give is whether or not the specification process was successful. The assumption is that other parts of the software will independently report errors that occur during the data reads. In the same way, we are also going to need a VI to tell the “something” that is doing the reading that we are no longer wishing to poll particular addresses.

Stop Addresses

The interface to this VI is even more basic than the one for starting the polling of addresses, and the reason is simple. At this point we don’t care what rate at which a given address is being polled, we just want it to stop. You could argue that knowing the polling interval would make it easier for the code to find the addresses to delete, while that is true, it would also mean that the calling code would have to keep track of the addresses that it is monitoring – which could quickly become an awful lot of redundant information for the caller to remember and manage.

Last but not least, the third interface VI that we need to specify is the data publication mechanism. To keep things simple, we should use a technique that is easy to implement. So I am picking the logical equivalent of the callback techniques apparent in other languages: a User Define Event. For this application, the event will return an array of address/value pairs that the calling application can use as it desires. Note that in this implementation, all the acquisition processes will be sending their data to the same place, so this can be a static UDE.

The Test Application

Finally, while we’re talking about interfaces, let’s also look at the calling application. Because all the “heavy lifting” will be encapsulated in subVIs, the calling code can be very simple. It has buttons for identifying addresses that we wish to poll, addresses we want to stop polling, and stopping the application. To display the results, the application’s front panel incorporates a table that shows the addresses in ascending order, and the last data value acquired for that address.

Main GUI

The block diagram is, likewise, pretty plain. It is event-driven with one event for each of the three buttons on the front panel, plus one to handle the UDE that publishes the data. You can check out its code in the source later.

Crawling Under the Covers

With the front end interfaces thus defined, we now can start thinking about code that will make the interfaces do something useful. The simple part is the UDE for publishing the results because it uses the same technique that we have used many times over the past year. To summarize the implementation, a library named for the event (Publish Data.lvlib) has four subVIs: One (structured as a FGV) for creating/buffering a reference to the event, and one each for registering to receive the event, generating the event, and destroying the event. In addition, it incorporates a typedef that defines the event data.

Publish Data Event Library

The process for managing the addresses to read requires a bit more thought. To begin with, we know that there are only two address management operations: adding and removing addresses. However, we also know that if we are to conform to our API, we need to hide those explicit operations from the calling application. This situation is one of those development scenarios where the words that we use to talk about what we are doing can help or hinder our understanding of what we are trying to accomplish. To see what I mean, let’s consider a similar case that is part of LabVIEW itself: the logic for handling queues.

You may have noticed that with the built-in API you don’t “create” or “destroy” queues. Rather, you “acquire” and “release” references to the queue. While you may wonder what difference this wording makes, we need to remind ourselves that it isn’t simply a matter of an API developer running amuck with a thesaurus. It actually describes a very real and very important distinction. Instead of creating a queue, you are simply telling LabVIEW that you want to acquire a reference to a queue. Now, when you make this request, there are two possible situations:

The queue doesn’t currently exist and LabVIEW needs to create one.
The queue already exists so all you need is to get a reference to the one that is there.

Likewise, you don’t destroy a queue when you are done using it, you simply tell LabVIEW that you have no further need for it by releasing the reference you previously acquired. Because LabVIEW keeps track of how many open references are associated with each queue, LabVIEW can tell when the queue is no longer in use and destroy it automatically. Now consider for a moment the degree to which this hidden functionality simplifies your code. You no longer need to worry about what or who is using the queue, and when it is safe to destroy it. All that potentially complex logic is hidden in the way that the functionality is encapsulated, and the difference in terminology highlights that difference.

As we are designing our API, we need to adhere to the same idea. So instead of “adding” and “removing” address, let’s think about this problem in terms of “starting” and “stopping” acquisition from lists of addresses. To grasp the benefits that we can garner from this change in language, lets consider what actually needs to happen behind the scenes for each of these operations. Just to be clear, this complexity has nothing to do with how we choose to implement the functionality, it is inherent in what we are trying to do. This logic will have to be created regardless of how we structure the code.

Starting Acquisition: This might seem to be pretty easy, but what if users start acquiring data from an address at one rate, but then later changes their mind (or makes a mistake) and starts the same address at a different rate. There is no point to have the same address being read at two different rates. Likewise, it is not clear that this action should be considered an error. Therefore, to do what the user is requesting you first have to remove that address from the process that is currently polling it, and then reassign it to a different (perhaps new, perhaps preexisting) acquisition process. Taking the point further, what if the address you remove from a process is the only address that it is currently polling. Removing that address would leave that acquisition loop with nothing to do, and so we need some way to stop it.

Stopping Acquisition: For its part, stopping the acquisition of addresses can hide some complexity of its own. For example, say the user identifies a list of 4 addresses to be stopped. There is no guarantee that all the addresses are being polled by the same acquisition process – and even if they are all together, we don’t know which process is reading them. This fact implies a need to be able to search all the acquisition processes for a particular address. Plus, as before if we remove all the addresses associated with a particular interval we need to stop that acquisition task.

Remember, this functionality will always be needed, it is simply more robust (and therefore smarter) to hide it from the calling application by encapsulating it in our API.

Getting Down With the Acquisition

To this point we have described the acquisition processes as having two loops: One performs the acquisition and one is a “metronome” function that periodically fires an event that causes the acquisition loop to acquire one scan of all the channels contained in its current configuration. In addition, both of these processes need to be reentrant so multiple copies of each can be launched as needed. Now we need to refine that basic description be specifying the rules that will govern their operation.

First, let’s state that each process is self maintaining both in terms of its own operation and its data. What that requirement means is that each instance of the acquisition process will maintain for itself a list of the addresses that it is polling. Consequently, when a process receives a system message (via UDE) to stop polling on one or more addresses, it will examine its own list of addresses and remove any that are in the “stop polling” list.

Second, we will state that there will only one acquisition process running for each acquisition interval. Hence, if a process receives a system message specifying its own polling interval, it will add those addresses to the list of addresses it is already polling. For example, say there is a process running that is acquiring data from 4 addresses every 1000 msec and it receives a system message that the user wants to start an additional 5 addresses at that same sample interval. The code will add those 5 new addresses to the 4 that it is already polling.

Third, if an acquisition process receives a system start message that does not specify its polling interval, it will prevent duplicate polling by automatically search its configuration for the addresses in the message and delete any that it finds.

Fourth, if after stopping one or more addresses a process finds that its polling list is empty, it will shut itself down by firing an event that is unique to that particular instance.

Fifth, in the event that the user starts addresses for an interval that is not currently running, the logic incorporates a manager function that will start-up a new acquisition process to handle that interval.

Let’s Build Some Code

Finally we are ready to start writing some code to materialize what has to this point been mostly words. A good place to start this work is with the manager VI that will launch new acquisition processes for us. I like this approach because it will give us the opportunity look at how the pieces fit together.

How the Pieces Fit Together

The operational rules we listed earlier provide a number of clues as to where we are going next. To begin with, we talked a lot about messages. This information by itself is enough to let us design and implement the inner workings of the two interface routines we prototyped earlier. They are simply the event generation VIs of two more static UDEs (Start Addresses and Stop Addresses) – and we already know how to build those.

Next, because we have defined what the manager basically needs to do we can create an event-driven shell for it that can respond to two events: Stop Application and Start Addresses. Again, if you have been reading this blog for a while, the Stop Application event is an “old friend” so we will concentrate on the manager’s response to the Start Addresses event. Since this response will vary depending upon whether or not the specified interval already exists, we need to design a way for the code to make that determination, while continuing to bear in mind the principles of information hiding, to wit, the manager doesn’t need to know how the determination was made, just what the result was.

Start Addresses Event Handler

To support this functionality, we will create a VI (called Interval Registry.vi) that the acquisition processes will maintain as they start and stop. This function will support three operations: Check, Insert and Delete. For the Check operation we see here, the routine checks to see if there is already a process running at the indicated sample interval. If there isn’t, the VI returns a false Boolean value that causes the manager to call a launcher VI (Process Launcher.vi) that kicks off all the VIs needed to service the interval. If, however, the interval is already running there is nothing more for the manager to do, so the true case (not shown) does nothing.

Turning our attention now to the VI that launches the new process VIs, we have built this sort of launcher several times before, so we already know its basic structure. What we need clarity on is the details of the data that needs to be passed to the two VIs.

Starting with the simpler of the two (which we will call Metronome.vi) it’s only purpose is to fire an acquisition event at some predefined interval. However, if it is going to stop when it has nothing more to do, it also needs to know what event will tell it to stop. Note that both of these events are specific to each interval that is created. Consequently, they both need to be created on the fly when the acquisition process is launched with their references passed into the new clone as a parameters.

In the same way, looking at the acquisition VI (we’ll call it Modbus Reader.vi) we see that it is going to be receiving the acquisition event that the metronome fires and is also going to need to shut down like the metronome, so it will need to get references to the same to events. The only other messages it will need to receive are the static ones that we have discussed earlier, but because they are static we don’t have to be concerned with them here.

So we add in the event definition logic, and this is what our finished launcher VI looks like:

Process Launcher

The Big Tease

The next thing we need to look into is the VIs that are being launched and the logic that resides inside Interval Registry.vi, but this post is getting long so that will have to wait until next time.

Until Next Time…
Mike…

If the socket fits, wear it…

Posted on 2015-12-072015-12-08 by mlportersr

One of this blog’s recurring themes is the importance of modularity as an expression of the age-old tactic of “divide and conquer”. What is perhaps new (or at least daunting) to some readers is the idea of spreading tasks across not just separate processes on the same computer, but across multiple networked computers. Of course if this strategy is to be successful, the key is communications and to that end we have been examining ways of incorporating remote access capabilities into out testbed application.

Last time out, we implemented the first interface for remote applications to monitor and control our application. That interface took the form of a custom TCP protocol that used packets of JSON data to carry messages over a vanilla TCP connection. I started there because it provides a simplified mechanism for exploring some of the issues concerning basic code structure. Although this interface worked well, and in fact would prove adequate for a wide variety of applications, it did exhibit one big issue. To wit, clients had to be written in a specific way in order to use it. This fact is a problem for many applications because users are growing increasingly reticent about installing special software. They want to know why they need to load special code to do a job? The way they see it, their PCs (and cell phones for that matter) come with a bunch of networking software preloaded on them – and they have a valid point! Why should they have to install something new?

A complete answer to that question is far beyond the scope of this post, but we can spend a few useful moments considering one small niche of the overall problem, and a standardized solution to that problem. Specifically, how can we leverage some of those networking tools (read: browsers) to support remote access to our testbed application? As we have discussed before, the web environment provides ample tools for creating some really nice interfaces. The real sticking point is how that “really nice” interface can communicate with the testbed application. You may recall that a while back we considered one technique that I characterized as a “drop box” solution. The idea was to take advantage of the database underlying a web application by using it to mediate the communications. In other words, the LabVIEW application writes new data to the database and the web application reads and displays the data from the database – hence the “drop box” appellation.

While we might be able to force-fit this approach into providing a control capability, it would impose a couple big problems: First, it would mean that the local application would have to be constantly polling a remote database to see if there have been any changes. Second, it would be really, Really, REALLY slow. We need something faster. We need something more interactive. We need WebSockets.

What are WebSockets?

Simply put, the name WebSockets refers to a message-based protocol that was standardized in 2011 as RFC 6455. The protocol that the standard defines is low-overhead, full-duplex and content agnostic, meaning that it can carry data of any type – even JSON-encoded text data (hint, hint).

An interesting aspect of this protocol is that its default port for establishing a connection is port 80 – the same as the default port for HTTP. While this built-in conflict might be confusing, it actually makes sense. You see when a client initiates an HTTP connection, the first thing it does is pass to the server a number of headers that provide information on the requested connection. One of those headers allows the client to request an Upgrade connection. The original purpose of this header was to allow the client to request an upgraded connection with, for example, enhanced security. However, in recent years it has become a mechanism to allow multiple protocols to listen to the same port.

The way the process works is simple: The client initiates a normal HTTP connection to the server but sets the request headers to indicate that it is requesting a specific non-HTTP protocol. In the case of a request for the WebSockets protocol, the upgrade value is websocket. The server responds to this request with a return code of 101 (Switching Protocols). From that point on, all further communications are made using the WebSockets protocol. It is important to note that while this initial handshake leads some to assume that WebSockets in some ways dependent upon, or rides on top of the HTTP protocol, such is not the case. Aside from the initial connection handshake, the WebSockets protocol is a distinct process that shares nothing with HTTP. Consequently, while the most common application of the technique might be web-based client-server operation, the WebSockets protocol is equally well-suited for peer-to-peer messaging. The only limitation is that one of the two peers needs to be able to respond correctly to the initial handshake.

It is also worth understanding why the basic idea of using Port 80 for the initial connection is significant. A conversation on Stackoverflow gives a pretty good explanation of several issues, but for me the major advantage of using port 80 is that it avoids IT-induced complications. Many corporate IT departments will lock down ports that they don’t recognize. While there are some that try to lock down port 80, it is much less common. Before continuing on, if you’re interested, you also can find the details of the initial handshake here.

The LabVIEW Connection

Ok, so it sounds like WebSockets could definitely have a place in our communications toolbox, but how are we going to take advantage of it from LabVIEW? The answer to that question lies in the work of LabVIEW CLA Sam Sharp. He has developed a set of “pure G” VIs that allows you to implement either side of the connection. Because these are written in nothing but G, there are no DLLs involved so they can run equally well on any supported LabVIEW platform. Making the deal even sweeter, he has documented his code, created a tutorial on them, released his VIs for anyone to use, and all the compensation he requests is “…it would be great if you credit me…”. So, Sam, may you have a million click-throughs.

The following discussion is written assuming Sam’s VIs which I have converted to LabVIEW 2015. One quick note, if you don’t or can’t use the VIPM, you can still use the *.vip file, all you have to do is change the “v” to a “z” and you are good to go. As a first taste of how these VIs work, let’s look (like we did with the TCP example last time) at an over-simplified example to get a sense of the overall logical flow.

For our purposes here, the testbed application will be the “server” so our code starts by listening for a connection attempt on the default Port 80. When it receives a connection, a reference to that connection is passed to a VI (DoHandshake.vi) that implements the initial handshake to activate the WebSockets protocol. Note that a key part of this process is the passing of a couple of “magic strings” between the client and server to validate the connection and protocol selection.

With the handshake completed and both ends of the connection satisfied that the WebSockets protocol is going to be used, the following subVI (Read.vi) reads a data packet from the client that, in our application, represents a data or control request. Next comes the subVI (Write.vi) that writes a response back to the client. Finally the code calls a subVI (Close.vi) that sends a WebSockets command to close the connection, and then closes the TCP connection reference that LabVIEW uses.

Building the Interface

To build this bare logic into something usable, the structure of the server task is essentially identical to that of the TCP process we built last time. In fact, the only difference between the two is ports to which they are listening, and the specific reentrant handlers that they launch in response to a TCP connection. So let’s concentrate on that alternate process. During initialization, the handler calls the subVI that implements the initial handshake.

In addition to the connection reference, this routine also outputs a string that is the URI that was used to establish the connection. Although we don’t need it for our application, it could be used to pass additional information to the server. Once initialization is complete the main event loop starts, but unlike the TCP handler we wrote earlier, it is not based around a state-machine structure.

While we could have broken up the process into separate states, the fact that Sam has provided excellent subVIs implementing the read and write functionality makes such a structure feel a bit contrived – or at least to me it does. When the timeout event fires, the code waits for 500 msecs for the first user data coming from the connection. If the read times-out, the loop waits for another 500 msec and then tries again. This polling technique is important because it allows other things (like the system shutdown event) to interrupt the process. Likewise, because we are waiting for a response that is, at least potentially, coming from a remote computer the polling allows us to wait as long as necessary for the response.

When the request data does arrive, the JSON data string is processed by a pair of subVIs that we originally created for the TCP protocol handler. They create the appropriate Remote Access Commands object and pass it on to the dynamic dispatch VI (Process Command.vi) that executes the command and returns the response. The response data is next flattened to a JSON string and written to the connection. Because the current implementation assumes a single request/response cycle per connection, the code closes the WebSockets connection and the TCP connection reference. However, it would be easy to visualize a structure that would not close the connection, but rather repeat one of the data read commands at a timed interval to create a remote “live” interface.

In terms of the errors that can occur during this process, the code has to correctly respond to two specific error codes. First is error code 56, a built-in LabVIEW error that flags a network operation timeout. Because this is the error that is generated if server hasn’t yet received the client’s request, the code basically ignores it. Second is error code 6066, which is a WebSockets-specific error defined in RFC 6455 to flag the situation where the remote client closes a WebSockets connection. Our code responds by closing the TCP connection reference and stopping the loop.

Testing our Work

Now that we have our new server up and running we need to be able to test its operation. However, rather than creating another LabVIEW application to act as the test platform, I built it into a web application. The interface consists of a main screen that provides a pop-up menu for selecting what you want to do and 5 other screens, each of which focus on a specific control action. As these things go, I guess it’s not a great web application, but it is serviceable enough for our purposes. If you need a great application, talk to Sam Sharp – that’s what his company does.

The HTML and CSS

As I have preached many times before, one of the things that makes web development powerful is the strict “division of labor” between its various components: the HTML defines the content, the CSS specifies how the content should look, JavaScript implements client-side interactivity and a variety of languages (including JavaScript!) providing server-side programmability. So lets start with a quick look at the HTML that defines my web interface, and CSS that makes it look good in spite of me… In order to provide some context for the following discussion, here is what the main screen looks like:

It has a title, a header and a pop-up menu from which you can select what you want to do. As a demonstration of the effect that CSS can have, here’s the part of the HTML that creates the pop-up menu.

<button class="btn btn-default dropdown-toggle" type="button" data-toggle="dropdown">Available Actions<span class="caret"></span></button>
<ul class="dropdown-menu">
  <li><a href="ReadGraphData.html">Read Graph Data</a></li>
  <li><a href="ReadGraphImage.html">Read Graph Image</a></li>
  <li class="divider"></li>
  <li><a href="SetAcquisitionRate.html">Set Acquisition Rate</a></li>
  <li><a href="SetDataBufferDepth.html">Set Data Buffer Depth</a></li>
  <li><a href="SetTCParameters.html">Set TC Parameters</a></li>
</ul>

You’ll notice that pop-up menu is constructed from two separate elements: A button and an unordered list – normally a set of bullet points – where each item in the list is defined as an anchor with a link to one of the other pages. However, as the picture shows, when this code runs we don’t see a button and a set of bullet points, we see one pop-up menu. How can this be? The magic lies in CSS that dramatically changes the appearance of these elements to give them the appearance of a menu. Likewise, some custom JavaScript makes the visually manipulate elements work like a menu. What is very cool, however, is that the resources making this transformation possible are part of a standard package, called Twitter Bootstrap, that is free for anyone to use. In a similar vein, let’s look at the page that displays a plot of data acquired from the testbed application:

At the top of the screen there’s a small form where the user enters information defining the task to be performed, and a button to initiate the operation that the user is requesting. Below that form, is a blank area where the software will draw the graph of the acquired data. Let’s look at two specific bits of HTML, first the code that builds the data entry form…

<form>
  <fieldset class="input-box">
    <legend>View Graph Data</legend>
    <input type="text" class="str-input" id="ipAddr" value="localhost">  Host</input><br>
    <input type="number" class="num-input" id="portNum" value="80">  Port Number</input><br>
    <select id = "targetPlugin">
      <option value = "Sine Source">Sine Source</option>
      <option value = "Ramp Source">Ramp Source</option>
      <option value = "Hen House TC">Hen House TC</option>
      <option value = "Dog House TC">Dog House TC</option>
      <option value = "Out House TC">Out House TC</option>
    </select><label>  Select Target for Action</label><br>
    <input type="button" id="just-submit-button" value="Send Command">
  </fieldset>
</form>

…and now the code that defines the graph:

<div id="container" style="min-width: 310px; height: 400px; margin: 0 auto"></div>

But, something seems to be missing. The first snippet will create data-entry fields and a button, but what happens when the button is clicked? Apparently, nothing. Likewise, the consider the graphing element. We can see how large the area is to be, but where is the data coming from? And where are the graphing operations? To answer those questions, we need to look elsewhere.

The JavaScript

The power behind much of the web in general – and our application in particular – is the interpreted language JavaScript. In addition to being able to access all resources on your computer, JavaScript can interact directly with web pages and their underlying structures. For folks that like to split hairs, JavaScript is “object-based” because it does support the concept of object, but it is not “object-oriented” because it doesn’t explicitly support classes.

More important for what we are going to be doing is that it supports the concept of “callbacks” (read: User Defined Events). In other words, you can tell JavaScript to automatically performs functions when certain events occur. For example, our JavaScript code is going to be interacting with the web page that loaded it, we need to be sure that the page is fully loaded before that program starts. In order to accomplish that goal, the JavaScript file associated with the page includes this structure:

$(window).load(function() {
	...  // a lot of stuff goes here
});

This code creates a callback for the .load() event. The parameter passed to the .load() event is a reference to the function that JavaScript will run when the event fires. As is common in JavaScript, the code declares the function in line so everything between the opening and closing curly brackets will be executed when the event fires. So after declaring a few variables the code includes this:

$("#just-submit-button").click(function(){
  //The code here retrieves all of the input data and formats the request.
  target = $("#targetPlugin").val();
  remAddr = $("#ipAddr").val();
  remPort = $("#portNum").val();
  jsonData = '\"Read Graph Data\":' + JSON.stringify({"Target":target}); 

  // the websocket logic
  wc_connect(remAddr, remPort, parseData);
  wc_send(jsonData);
});

So the first thing the code does when the page finishes loading is register another callback, but this one defines what JavaScript will do when the user clicks the button in the form. The first three lines read the values of the form data entry fields, and the fourth assembles that data into the JSON string that will be sent to the server. The last two lines are the interface to the WebSockets logic. The first of these lines establishes the connection to the server, while the other one sends the command. But what about the response? Shouldn’t there be a line with a command like wc_receive? You really should be expecting this by now: Inside the wc_connect command the code registers another callback to handle the response.

The event (called onmessage) that is tied to this callback fires when a message is received from the server. The code implementing the callback resides in the file websockets.js (in case you’re curious) and its job is to read the JSON response data packet, check for errors, parse the data and generate the output – the graph. The only question now is, “How does it know how to parse the data and generate the graph?” And the answer is (all together now): “There’s another callback!” See the third parameter of wc_connect, the one named parseData? That value is actually a reference to a function contained in the JavaScript code for this particular page, and is an example of how JavaScript implements a “plugin architecture”. So here is how the data parser for this page starts…

var parseData = function(rawData){
  var plotData = JSON.parse(rawData);
  // trim decimal places
  plotData.forEach(function(element, index, array){
    plotData[index] =  Number(element.toFixed(3));
  });

At this point in the process, the data portion of the response is still a string, so to make processing the data easier, we first parse it to convert it into a JSON object. In the case of this particular response, the resulting object is the array of numbers expressed as strings. Really long strings. You see when LabVIEW encodes a number as a JSON string it includes far more digits of precision than are really needed, so forEach element in the array, I convert the value to a number with 3 decimal places. Here’s the rest of the code:

  // logic for drawing the graph
  $('#container').highcharts({
    title: { text: 'Recent Data', align: 'center' },
    subtitle: { text: 'System: '+remAddr+':'+remPort, align: 'center' },
    xAxis: { title: { text: 'Samples' }, tickInterval: 1 },
    yAxis: { title: { text: 'Amplitude' }, gridLineColor: "#D8D8D8" },
    tooltip: { headerFormat: '<small>Sample: {point.key}</small><br>' },
    series: [{ turboThreshold: 0, name: target, data: plotData, lineWidth: 1, marker:{enabled: false}, color: '#000000' }]
  });
}

This is the code that does the plotting, and as we shall see in a moment, this small amount of code produces a beautiful and highly functional chart that displays the values of individual points in a tooltip when you hover over them with the mouse and even provides a pop-up menu that allows you to save the plot image in a variety of image formats. This functionality is possible thanks to a plotting library called Highcharts that uses the structure defined in the HTML as a placeholder for what is going to draw. I have used this library before in demonstrations because in my experience it is stable, easy to use, and very well-documented. I also like the fact that regardless of what kind of plot I am trying to create they have a demo online that gets me about 95% of the way to my goal. Please note that this library is a commercial product, but they make it available for free for “non-Commercial” applications – however even for commercial usage, the one-time license fee is really pretty reasonable. Finally, even though it doesn’t appear that they actively police their licensing with things like crippled versions or the like, if you are using this on a professional project, pay the people. They have certainly earned their bread.

Testing the Pages

So at last we have our server in place and some test web pages (and supporting code) created. We need to consider how to run the web client. Here you have three options: First, you could just double-click the top-level file in Windows Explorer and Windows will dutifully open the file in your browser and everything will work as it should. Second, if you have access to an existing web server you can copy the dozen or so files to it and test it from there. Third, you could create a small temporary server strictly for testing. If you choose that path, a good option is a server called Express.js. As it name implies, it is written in JavaScript, which means it runs under the Node.JS execution engine. You can set one up sufficient to test our current code in about 10 minutes – including the time required to download the code.

The overall test process is similar to what we did to test the custom TCP server last time. The only significant change is the interface. First, test things that should work to make sure they do. Second, test the things that shouldn’t work and make sure they don’t. Here are examples of what you can expect to see on the graphing and image-fetch screens:

Testbed App – Release 20
Toolbox – Release 17
WebSockets Client – Release 1

Big Tease

So what’s next? We have looked at access via a custom TCP interface and the standard WebSockets interface. How about next time, we look at how to do embed this connectivity in a C++ program using a DLL?

Until Next Time…
Mike…

It’s a big interconnected world – and LabVIEW apps can play too

Posted on 2015-11-032015-11-02 by mlportersr

If I were asked to identify one characteristic that set modern test systems apart from their predecessors, my answer would be clear: distributed processing. In the past, test applications were often monolithic programs – but how things have changed! In the past we have talked several times about how the concept of distributed processing has radically altered our approach to system architecture. In fact, the internal design of our Testbed Application is a very concrete expression of that architectural shift. However, the move away from monolithic programs has had a profound impact in another area as well.

In the days of yore, code you wrote rarely had to interact with outside software. The basic rule was that if you wanted your program to do something you had to implement the functionality yourself. You had to assume this burden because there was no alternative. There were no reusable libraries of software components that you could leverage, and almost no way to pass data from one program to another. About all you could hope for were some OS functions that would allow you to do things like read and write disk files, or draw on the screen. In this blog we have looked at ways that our LabVIEW applications can use outside resources through standardized interfaces like .NET or ActiveX. But what if someone else wants to write a program that uses our LabVIEW code as a drop in component? How does that work?

As it turns out, the developers saw this possibility coming and have provides mechanisms that allow LabVIEW application to provide the same sort of standardized interface that we see other applications present. Over the next few posts, we are going to look at how to use incorporate basic remote interfaces ranging from traditional TCP/IP-based networking to building modern .NET assemblies.

What Can We Access?

However, before we can dig into all of that, we need to think about what these interfaces are going to access. In our case, because we have an existing application that we will be retrofitting to incorporate this functionality, we will be looking at the testbed to identify some basic “touchpoints” that a remote application might want to access. By contrast, if you are creating a new application, the process of identifying and defining these touchpoints should be an integral part of you design methodology from day one.

The following sections present the areas where we will be implementing remote access in our testbed application. Each section will describe the remote interface we desire to add and discuss some of the possible implementations for the interface’s “server” side.

Export Data from Plugins

The obvious place to start is by looking at ways of exporting the data. This is, after all, why most remote applications are going to want to access our application: They want the data that we are gathering. So the first thing to consider is, where does the data reside right now? If you go back and look at the original code, you will see that, in all cases, the primary data display for a plugin is a chart that is plotting one new point at a time. Here is what the logic looked like in the Acquire Sine Data.vi plugin.

As you can see, the only place the simulated data goes after it is “acquired” is the chart. Likewise, if you looked at the code for saving the data, you would see that it was getting the data by reading the chart’s History property.

Now, we could expand on that technique to implement the new export functionality, but there is one big consequence to that decision. Approaching the problem in this way would have the side-effect of tying together the number of data points that are saved to the chart’s configuration. Hence, because the amount of data that a chart buffers can’t be changed at runtime, you would have to modify the LabVIEW code to change the amount of data that is returned to the calling application.

A better solution is to leverage what we have learned recently about DVRs and in-place structures to create a storage location the size of which we can control without modifying the application code. A side-effect of this approach is that we will be able to leverage it to improve the efficiency of the local storage of plugin data – yes, sometimes side-effects are good.

To implement this logic we will need three storage buffers: One for each of the two “acquisition” plugins and one for the reentrant “temperature controller” plugin. The interface to each storage buffer will consist of three VIs, the first one of which is responsible for initializing the buffer:

This screenshot represents the version of the initialization routine that serves the Ramp Signal acquisition process. The basic structure of this code is to create a circular buffer that will save the last N samples – where “N” is a reconfigurable number stored in the database. To support this functionality, the DVR incorporates two values: The array of datapoints and a counter that operates as a pointer to track where the current insertion point is in the buffer. These VIs will be installed in the initialization state of the associated plugin screen’s state machine. With the buffer initialized, we next need to be able to insert data. This is typical code for performing that operation:

Because the DVR data array is initialized with the proper number of elements at startup, all this logic has to do is replace an existing value in the array with a newly acquired datapoint value, using the counter of course to tell it which element to replace. Although we have a value in the DVR called Counter we can’t use it without a little tweaking. Specifically, the DVR’s counter value increments without limit each time a value is inserted, however, there is only a finite number of elements in the data array. What we need for our circular buffer is a counter that starts at 0, counts to N-1 and then returns to 0 and starts over. The code in the image shows the easiest way to generate this counter. Simply take the limitless count and modulo divide it by the number of points in the buffer. The output of the modulo division operation is a quotient and a remainder. The remainder is the counter we need.

Modulo division is also important to the routine for reading the data from the buffer, but in this case we need both values. The quotient output is used to identify when the buffer is still in the process of being filled with the initial N datapoints:

During this initial period, when the quotient is 0, the code uses the remainder to trim off the portion of the buffer contents that are yet to be filled with live data. However, once the buffer is filled, the counter ceases being the a marker identifying the end of the data, and it becomes a demarcation point between the new data and the old data. Therefore, once the quotient increments past 0, a little different processing is required.

Once the circular buffer is full, the element that the remainder is pointing at is the oldest data in the array (chronologically speaking), while the datapoint one element up from it is newest. Hence, while the remainder is still used to split the data array, the point now is to swap the two subarrays to put the data in correct chronological order.

Retrieve Graph Images from Plugins

The next opportunity for remote access is to fetch not the data itself, but a graph of the data as it is shown on the application’s GUI. This functionality could form the basic for a remote user interface, or perhaps as an input to a minimalistic web presentation. Simplifying this operation is a control method that allows you to generate an image of the graph and the various objects surrounding it like the plot legend or cursor display. Consequently, the VI that will be servicing the remote connections only needs to be able to access the chart control reference for each plugin. To make those references available, the code now incorporates a buffer that is structurally very similar to the one that we use to store the VI references that allow the GUI to insert the plugins into its subpanel. Due to its similarity to existing code, I won’t cover it in detail, but here are a few key points:

Encapsulated in a library to establish a namespace and provided access control
The FGV that actually stores the references is scoped as Private
Access to the functionality is mediated though publicly-scoped VIs

This FGV is the only new code we will need to add to the existing code.

Adding Remote Control

One thing that both of the remote hooks we just discussed have in common is that they are both pretty passive – or to make this point another way, they both are monitoring what the application is doing without changing what it is doing. Now we want to look at some remote hooks that will allow remote applications control the application’s operation, at least in a limited way.

Since the way the application works is largely dependent upon the contents of the database, it should surprise no one that these control functions will need to provide provisions for the remote application to alter the database contents in a safe and controlled way.

Some Things to Consider

The really important words in that last sentence are “safe” and “controlled”. You see, the thing is that as long as you are simply letting people read the data you are generating, your potential risk is often limited to the value of the data that you are exposing. However, when you give remote users or other applications the ability to control your system, the potential exists that you could lose everything. Please understand that I take no joy in this conversation – I still remember working professionally on a computer that didn’t even have a password. However, in a world where “cyber-crime”, “cyber-terrorism” and “cyber-warfare” have become household terms, this conversation is unavoidable.

To begin with, as a disclaimer you should note that I will not be presenting anything close to a complete security solution, just the part of it that involves test applications directly. The advice I will be providing assumes that you, or someone within your organization, has already done the basic work of securing your network and the computers on that network.

So when it comes to securing applications that you write, the basic principle in play here is that you never give direct access to anything. You always qualify, error-check and validate all inputs coming from remote users or applications. True, you should be doing this input validation anyway, but the fact of the matter is that most developers don’t put a lot of time into validating inputs coming from local users. So here are a couple of recommendations:

Parametrize by Selecting Values – This idea is an expansion on a basic concept I use when creating any sort of interface. I have often said that anything you can do to take the keyboard out of your users’ hands is a good thing. By replacing data that has to be typed with data menus from which they can select you make software more robust and reduce errors. When working with remote interfaces, you do have to support typed strings because unless the remote application was written in LabVIEW, typing is the only option. But what you can do is limit the inputs to a list of specific values. On the LabVIEW-side the code can convert those string values into either a valid enumeration, or a predefined error that cancels the operation and leaves your system unaltered. When dealing with numbers, be sure to validate them also by applying common-sense limits to the inputs.

Create Well-Defined APIs – You want to define a set of interfaces that specify exactly what each operation does, and with as few side-effects as possible. In fancy computer-science terms, this means that operations should be atomic functions that either succeed or fail as a unit. No half-way states allowed! Needless to say, a huge part of being “well-defined” is that the APIs are well-documented. When someone calls a remote function, they should know exactly what is expected of them and exactly what they will get in response.

Keep it Simple – Let’s be honest, the “Swiss Army Knife” approach to interface design can be enticing. You only have to design one interface where everything is parametrized and you’re done, or at least you seem to be for a while. The problem is that as requirements change and expand you have to be constantly adding to that one routine and sooner or later (typically sooner) you will run into something that doesn’t quite fit well into the structure that you created. When that happens, people often try to take the “easy” path and modify their one interface to allow it to handle this new special case – after all, “…it’s just one special case…”. However once you start down that road, special cases tend to multiply like rabbits and the next thing you know, your interface is a complicated, insecure mess. The approach that is truly simple is to create an interface that implements separate calls or functions for each logical piece of information.

With those guidelines in mind, let’s look at the three parameters that we are going to be allowing remote users or applications to access. I picked these parameters because each shows a slightly different use case.

Set the Acquisition Sample Interval

One of the basic ways that you can store a set of parameters is using a DVR, and I demonstrated this technique by using it to store the sample rates that the “acquisition” loops use to pace their operation. In the original code, the parameter was already designed to be changed during operation. You will recall that the basic idea for the parameter’s storage was that of a drop box. It wasn’t important that the logic using the data know exactly when the parameter was changed, as long as it got the correct value the next time it tried to use the data. Consequently, we already have a VI that writes to the DVR (called Sample Rate.lvlib:Write.vi) and, as it turns out, it is all we will need moving forward.

Set Number of Samples to Save

This parameter is interesting because it’s a parameter that didn’t even exist until we started adding the logic for exporting the plugin data. This fact makes it a good illustration of the principle that one change can easily lead to requirements that spawn yet other changes. In this case, creating resizable data buffers leads to the need to be able change the size of those buffers.

To this point, the libraries that we have defined to encapsulate these buffers each incorporate three VIs: one to initialize the buffer, one to insert a new datapoint into it, and one to read all the data stored in the buffer. A logical extension of this pattern would be the addition of a fourth VI, this time one to resize the existing buffer. Called Reset Buffer Size.vi these routines are responsible for both resizing the buffer, and correctly positioning the existing data in the new buffer space. So the first thing the code does is borrow the logic from the buffer reading code to put the dataset in the proper order with the oldest samples at the top and the newest samples at the bottom.

Next the code compares the new and old buffer sizes in order to determine whether the buffer is growing larger, shrinking smaller or staying the same size. Note that the mechanism for performing this “comparison” is to subtract the two value. While a math function might seem to be a curious comparison operator, this technique makes it easy to identify the three conditions that we need to detect. For example, if the values are the same the difference will be 0, and the code can use that value to bypass further operations. Likewise, if the two numbers are not equal, the sign of the result will indicate which input is larger, and the absolute magnitude of the result tells us how much difference there is between the two.

This is the code that is selected when the result of the subtraction is a positive number representing the number of element that are to be added to the buffer.

The code uses the difference value to create an array of appropriate size and then appends it to the bottom of the existing array. In addition, the logic has to set the Counter value point to the first element of the newly appended values so the next insert will go in the correct place. By contrast, if the buffer is shrinking in size, we need to operate on the top of the array.

Because the buffer is getting smaller, the difference is a negative number representing the number of elements to be removed from the buffer data. Hence, the first thing we need to do is extract the number’s absolute value and use it to split the array, effectively removing the elements above the split point. As before, we also need to set the Counter value, but the logic is a little more involved.

You will remember that the most recent data is on the bottom of the array, so where does the next data point need to go? That’s right, the buffer has to wrap around and insert the next datapoint at element 0 again, but here is where the extra complexity comes in. If we simply set Counter to 0 the data insert logic won’t work correctly. Reviewing the insert logic you will notice that the first pass through the buffer (modulo quotient = 0) is handled differently. What we need is to reinitialize Counter with a number that when subjected to the modulo division will result in a remainder of 0, and a quotient that is not 0. An easily derived value that meets that criteria is the size of the array itself.

Finally we have to decide what to do when the buffer size isn’t changing, and here is that code. Based on our discussions just now, you should be able to understand it.

Set Temperature Controller Operating Limits

Finally, there are two reasons I wanted to look at this operation: First, it is an example of where you can have several related parameters that logically form a single value. In this case, we have 5 separate numbers that, together, define the operation of one of the “temperature controller” processes. You need to be on the look-out for this sort of situation because, while treating this information as 5 distinct value would not be essentially wrong, that treatment would result in you needing to generate a lot of redundant code.

However, this parameter illustrates a larger issue, namely that changes in requirements can make design decisions you made earlier – let’s say – problematic. As it was originally designed, the temperature controller values were loaded when the plugins initialized, and they were never intended to be changed during while the plugin was running. However, our new requirement to provide remote control functionality means that this parameter now needs to be dynamic. When confronted with such a situation, you need to look for a solution that will require the least rework of existing code and the fewest side-effects. So you could:

Redesign the plugin so it can restart itself: This might sound inviting at first because the reloading of the operating limits would occur “automatically” when the plugin restarted. Unfortunately, it also means that you would have to add a whole new piece of functionality: the ability for the application to stop and then restart a plugin. Moreover, you would be creating a situation where, from the standpoint of a local operator, some part of the system would be restarting itself at odd intervals for no apparent reason. Not a good idea.
Redesign the plugin to update the limits on the fly: This idea is a bit better, but because the limits are currently being carried through the state machine in a cluster that resides in a shift-register, to implement this idea we will need to interrupt the state machine to make the change. Imposing such an interruption risks disrupting the state machine’s timing.

The best solution (as in all such cases) is to address the fundamental cause: the setups only load when the plugin starts and so are carried in the typedef cluster. The first step is to remove the 5 numbers associated with the temperature controller operating parameters from the cluster. Because the cluster is a typedef, this change conveniently doesn’t modify the plugin itself, though it does alter a couple of subVIs – which even more conveniently show up as being broken. All that is needed to repairs these VIs is to go through them one by one and modify the code to read the now-missing cluster data values with the corresponding values that the buffered configuration VI reads from the database. Said configuration VI (Load Machine Configuration.vi) also requires one very small tweak:

Previously, the only time logic would force a reload of the data was when the VI had not been called before. This modification adds an input to allow the calling code to request a reload by setting the new Reload? input to true. To prevent this change from impacting the places where the VI is already being called, the default value for this input is false, the input is tied to a here-to-fore unused terminal on the connector pane, and the terminal is marked as an Optional input.

Building Out the Infrastructure

At this point in the process, all the modifications that need to be done to the plugins themselves have been accomplished, so now we need is a place for the external interface functionality itself to live. One of the basic concepts of good software design is to look at functionality from the standpoint of what you don’t know or what is likely to change in the future, and then put those things into abstracted modules by themselves. In this way, you can isolate the application as a whole, and thus protect it from changes that occur in the modularized functionality.

The way this concepts applies to the current question should be obvious: There is no way that we can in the here and now develop a compete list of the remote access functionality that users will require in the future. The whole question is at its essence, open-ended. Regardless if how much time you spend studying the issue, users have an inherently different view of your application than you do and so they will come up with needs that you can’t imagine. Hence, while today we might be able to shoe-horn the various data access and control functions into different places in the current structure, to do so would be to start down a dead-end road because it is unlikely that those modifications would meet the requirements of tomorrow. What we need here is a separate process that will allow us to expand or alter the suite of data access and control functionality we will offer.

Introducing the Remote Access Engine

The name of our new process is Remote Access.vi and (like most of the code in the project) it is designed utilizing an event-drive structure that ensures it is quiescent unless it is being actively accessed. The process’ basic theory of operation is that when one of its events fire, it performs the requested operation and then sends a reply in the form of a notification. The name of the notification used for the reply is sent as part of the event data. This basic process is not very different from the concept of “callbacks” used in languages such as JavaScript.

Although this process is primarily intended to run unseen in the background, I have added three indicators to its front panel as aides in troubleshooting. These indicators show the name of the last event that it received, the name of the plugin that the event was targeting, and the name of the response notifier.

The Read Graph Data Event

The description of this event handler will be longer than the others because it pretty much sets the pattern that we will see repeated for each of the other events. It starts by calling a subVI (Validate Plugin Name.vi) that tests to see if the Graph Name coming from the event data is a valid plugin name, and if so, returns the appropriate enumeration.

The heart of this routine is the built-in Scan from String function. However, due to the way the scan operation operates, there are edge conditions where it might not always perform as expected when used by itself. Let’s say I have a typedef enumeration named Things I Spend Too Much Time Obsessing Over.ctl with the values My House, My Car, My Cell Phone, and My House Boat, in that order. Now as I attempt to scan these values from strings, I notice a couple of “issues”. First there is the problems of false positives. As you would expect, it correctly converts the string “My House Boat” into the enumerated value My House Boat. However, it would also convert the string “My House Boat on the Grand Canal” to the same enumeration and pass the last part of the string (” on the Grand Canal”) out its remaining string output. Please note that this behavior is not a bug. In fact, in many situations it can be a very useful behavior – it’s just not the behavior that we want right now because we are only interested in exact matches. To trap this situation, the code marks the name as invalid if the remaining string output is not empty.

The other issue you can have is what I call the default output problem. The scan function is designed such that if the input string is not scanned successfully, it outputs the value present at the default value input. Again, this operation can be a good thing, but it is not the behavior that we want. To deal with this difference, the code tests the error cluster output (which generates and error code 85 for a failed scan) and marks the name as invalid if there is an error.

When Validate Plugin Name.vi finishes executing, we have a converted plugin name and a flag that tells us whether or not we can trust it. So the first thing we do is test that flag to see whether to continue processing the event or return an error message to the caller. Here is the code that executes when the result of the name validation process is Name Not Valid.

If the Response Notifier value from the event data is not null, the code uses it to send the error message, “Update Failed”. Note that this same message is sent whenever any problem arises in the remote interface. While this convention certainly results in a non-specific error message, it also ensures that the error message doesn’t give away any hints to “bad guys” trying to break in. If the Response Notifier value is null (as it will be for local requests) the code does nothing – remember we are also wanting to leverage this logic locally.

If the result of the name validation process is Name Valid, the code now considers the Plugin Name enumeration and predicates its further actions based on what it finds there. This example for Sine Source shows the basic logic.

The code reads the data buffer associated with the signal and passes the result into a case structure that does one of two different things depending on whether the event was fired locally, or resulted from a remote request. For a remote request (Response Notifier is not null), the code turns the data into a variant and uses it as the data for the response notifier. However, if the request is local…

…it sends the same data to the VI that saves a local copy of the data.

The Read Graph Image Event

As I promised above, this event shares much of the basic structure as the one we just considered. In fact, the processing for a Name Not Valid validation result is identical. The Name Valid case, however, is a bit simpler:

The reason for this simplification is that regardless of the plugin being accessed, the datatypes involved in the operation are always the same. The code always starts with a graph control reference (which I get from the lookup buffer) and always generates an Image Data cluster. If the event was fired locally, the image data is passed to a VI (Write PNG File.vi) that prompts the user for a file name and then saves it locally. However, if instead of saving a file, you are wanting to pass the image in a form that is usable in a non-LabVIEW environment, a bit more work is required. To encapsulate that added logic, I created the subVI Send Image Data.vi.

The idea is to convert the proprietary image data coming from the invoke node into a generic form by rendering it as a standard format image. Once in that form, it is a simple matter to send it as a binary stream. To implement this approach, the code first saves the image to a temporary png file. It then reads back the binary contents of the file and uses it as the data for the response notifier. Finally, it deletes the (now redundant) temporary file.

The Set Acquisition Rate Event

This event is the first one to control the operation of the application. It also has no need to be leveraged locally, so no dual operation depending on the contents of the Response Notifier value.

Moreover, because the event action is a command and not a request, the response can only have one of two values: “Update Failed” or “Update Good”. The success message is only be sent if the plugin name is either Sine Source or Ramp Source, and no errors occurs during the update. While on the topic of errors, there are two operations that need to be performed for a complete update: the code must modify both the database and the buffer holding the live copy of the setting that the rest of the application uses. In setting the order of these two operations, I considered which of the two is most likely to generate an error and put it first. When you consider that most of the places storing live data are incapable of generating an error, the database update clearly should go first.

So after verifying the plugin name, the subVI responsible for updating the database (Set Default Sample Period.vi) looks to see if the value is changing. If the “new” value and the “old” value are equal, the code returns a Boolean false to its Changed? output and sets the Result output to Update Good. It might seem strange to return a value to the remote application that the update succeeded when there was no update performed, but think about it from the standpoint of the remote user. If I want a sample period of 1000ms, an output of Update Good tells me I got what I wanted – I don’t care that it didn’t have to change to get there. If the value is changing…

…the code validates the input by sending it to a subVI that compares it to some set limits (500 < period < 2500). Right now these limits are hardcoded, and in some cases that might be perfectly fine. You will encounter many situations where the limits are fixed by the physics of a process or a required input to some piece of equipment. Still, you might want these limits to be programmable too, but I’ll leave that modification as, “…as exercise for the reader.” In any case, if the data is valid, the code uses it to update the database and sets the subVI’s two outputs to reflect whether the database update generated an error. If the data is not valid, it returns the standard error message stating so.

The Set Data Buffer Depth Event

The basic dataflow for this event is very much like the previous one.

The primary logical difference between the two is that all plugins support this parameter. The logic simply has to select the correct buffer to resize.

The Set TC Parameters Event

With our third and (for now at least) final control event, we return to one that is only valid for some of the plugins – this time the temperature controllers.

The interesting part of this event processing is that, because its data was not originally intended to be reloaded at runtime, the live copy of the data is read and buffered in the object-oriented configuration VIs.

Consequently, the routine to update the database (Save Machine Configuration.vi) first creates a Config Data object and then use that object to read the current configuration data. If the data has changed, and is valid, the same object is passed on to the method that writes the data to the database. Note also, that the validation criteria is more complex.

In addition to simple limits on the sample interval, the Error High Level cannot exceed 100, the Error Low Level cannot go below 30, and all the levels have to be correct relative to each other.

Testing

With the last of the basic interface code written and in place, we need to look at how to test it. To aide in that effort, I created five test VIs – one for each event. The point of these VIs is to simply exercise the associated event so we can observe and validate the code’s response. For instance, here’s the one for reading the graph data:

It incorporates two separate execution paths because it has two things that it has to be doing in parallel: Sending the event (the top path) and waiting for a response (the bottom path). Driving both paths, is the output from a support VI from the notifier library (not_Generic Named Notifier.lvlib:Generate Notifier Name.vi). It’s job is to generate a unique notifier name based on the current time and a 4-digit random number. Once the upper path has fired the event, it’s work is done. The bottom path displays the raw notifier response and graphs of the data that is transferred. Next, the test VI for reading the graph image sports similar logic, but the processing of the response data is more complex.

Here, the response notifier name is also used to form the name for a temporary png file that the code uses to store the image data from the event response. As soon as the file is written, the code reads it back in as a png image and passes it to a subVI that writes it to a 2D picture control on the VI’s front panel. Finally, the three test VIs for the control operations are so similar, I’ll only show one of them.

This exemplar is for resizing the data buffers. The only response processing is that it displays the raw variant response data.

To use these VIs, launch the testbed application and run these test VIs manually one at a time. For the VIs that set operating parameters, observe that entering invalid data generates the standard error message. Likewise, when you enter a valid setting look for the correct response in both the program’s behavior and the data stored in the database. For the VI’s testing the read functions, run them and observe that the data they display matches what the selected plugin shows on the application’s GUI.

Testbed Application – Release 18
Toolbox – Release 15

The Big Tease

In this post, we have successfully implemented a remote access/control capability. However, we don’t as of yet have any way of accessing that capability from outside LabVIEW. Next time, we start correcting that matter by creating a TCP/IP interface into the logic we just created. After that introduction, there will be posts covering .NET, ActiveX and maybe even WebSockets – we’ll see how it goes.

Until Next Time…
Mike…

Tree-Control Menus: A Case Study in Data Management

Posted on 2015-09-282015-09-28 by mlportersr

The last time we were together, we discussed the first of two common use cases for tree controls: displaying tabular data. This time out, we are going to look at the other major use case: using tree controls as a sort of menu system to control an application’s operation – or at least its GUI.

The Problem We’re Solving

If you look at the testbed application that we have been working on for almost a year, it’s pretty clear that much of the work has been going on “behind the scenes” and not in the GUI. Oh it is nicely modularized thanks to a structure built around a subpanel interface, but the actual controls are really pretty bare bones. A good example of the utilitarian, but unsophisticated structure is the usage of a simple pop-up menu to select the screen to view. Right now it works pretty well because there is only a handful of plugin screens from which we can choose. However, it doesn’t take much imagination to visualize the mess that would result if there were a dozen, or even hundreds of screens available. We need better organization.

Fixing the Data

The biggest conceptual difference between our current goal, and the one we worked on last time is that in our earlier discussion we were displaying data that already existed outside the application. In other words, my disk has directories that contain files and other directories whether or not I chose to create a program that can read and display the directory’s contents. By contrast, the data we are going to be displaying now only exists within the context of our program, or perhaps within the context of our test environment as a whole. One big consequence of this fact is that we a lot more freedom to define the data’s structure and presentation.

For instance, when our testbed runs right now, there are two “acquisition” processes and three “temperature controllers”. Let’s say, for the sake of argument, that the controller functions are dispersed geographically, and what we see on the local interface is status data from three remote processes. In such a situation, we can observe that there is no “correct” way of viewing that overall structure. Depending upon who the user is and what they need to do there are (at least) two ways that these systems could be organized.

One user, might want to see a top-level breakdown that groups systems based on the function they perform. With this approach to organization, you would have sections for “Data Sources” and “Temperature Controllers”. The individual screen would then be grouped under one or the other of those headings:

Alternatively, a different user might want to see the network resources grouped primarily by each system’s geographical location, with the functions for each site then grouped together like so:

However, as I said before, neither view is any more “correct” than the other. Therefore. we need to be able to support either one – and any other structure that our customers request, as well. Although this level of flexibility might seem to be a tall order, the truth of the matter is that the tree control’s basic operation is very simple, so all we are really talking about is a matter of data management. Moreover, we already have in our hands the tools we need to accomplish the job. I am talking, of course about our database.

Creating the Data Management Structures

So in defining our data structures, we can start with what we already know: The user needs to be able to select basic menu structures by changing a single value. From this requirement it’s obvious that we’re going to need a table to identify the menu’s basic context. We will then use the values stored in that table to qualify the menu item groupings. Here is the definition for this table, and the three records we are going to insert into it:

CREATE TABLE menu_context (
  id     AUTOINCREMENT PRIMARY KEY,
  label  TEXT(100),
  CONSTRAINT contextlabel_uc UNIQUE(label)
  )
;

INSERT INTO menu_context (id, label) VALUES (0,'NULL');
INSERT INTO menu_context (label) VALUES ('By Function');
INSERT INTO menu_context (label) VALUES ('By Location');

The second table we need to define, will hold the records that describe the actual menu entries. Each record defines one line of the tree control’s contents.

CREATE TABLE menu_group (
  id           AUTOINCREMENT PRIMARY KEY,
  context_id   INTEGER NOT NULL,
  item_name    TEXT(100),
  parent_id    INTEGER NOT NULL,
  sort_order   INTEGER,
  CONSTRAINT context_group_fk FOREIGN KEY (context_id) REFERENCES menu_context(id),
  CONSTRAINT self_ref_fk FOREIGN KEY (parent_id) REFERENCES menu_group(id)
  )
;

As is typical, the data for each record incorporates a primary key that uniquely identifies it. Next, comes a foreign key value that relates each record to one of the menu context values defined in the menu_context table. The last three fields store the data that controls the entry’s appearance in the tree control. The item_name field contains the text that will appear for the item’s entry in the tree control. The parent_id is the ID key for the item’s parent. A key value of 0 indicates a top-level item. Note that this values relates to the id field value in the same table. This sort of self-referential relationship is common when creating tables that are, in essence, linked lists. Finally, the sort_order field defines the order in which the menu entries will be added to the tree control. This last field is necessary because we are storing the configuration data in a database – and as you will recall DBMS make no promises about the order of data in queries unless you explicitly include an ORDER BY clause in the query.

Now that we have a table defining the overall tree control menu structure, we need to be able to insert into that structure the entries that will represent the plugin screens. In order to accomplish that task we need a table that relates data we already have in the database (the contents of the launch_item table) to the specific tree control entries that will be their parents in the tree control. The following table fulfills that task:

CREATE TABLE subpanel_group_xref (
  id               AUTOINCREMENT PRIMARY KEY,
  launch_item_id   INTEGER NOT NULL,
  menu_group_id    INTEGER NOT NULL,
  menu_context_id  INTEGER NOT NULL,
  CONSTRAINT launchid_subpanel_FK FOREIGN KEY (launch_item_id) REFERENCES launch_item(id),
  CONSTRAINT groupid_subpanel_FK FOREIGN KEY (menu_group_id) REFERENCES menu_group(id),
  CONSTRAINT contextid_subpanel_FK FOREIGN KEY (menu_context_id) REFERENCES menu_context(id)
  )
;

This table might seem a strange candidate for implementing this crucial bit of functionality because it doesn’t appear to actually store any data. The table only has 4 fields and they are all seem to be holding integers. The distinction here is that while most of the tables we have considered serve to store data, this table stores relationships – specifically the 3-way relationship that defines where each plugin will appear in the menu for each menu context. To see how these bits fit together we need to start considering the LabVIEW code that will read these structures and build the tree control based menus.

Creating the LabVIEW

The basic approach that we will take in creating the entries for the tree control is going to incorporate two distinct phases.

Draw the menu structure
Fill in the entries associated with the plugins

Reading the Data

The VI that is responsible for reading the menu data from the database (Config Data_DB_ADO:Read Tree Menu Structure.vi) has an enumerated input that selects the menu context the code will display. This value drives a subVI (Get Menu Context ID.vi) that reads and buffers the id value associated with the desired menu context.

In one sense, this subVI really isn’t necessary because you could theoretically perform this look-up operation using a so-called “subquery”, but this approach is far less efficient because it forces the DBMS to repeat the look-up with each query. In addition, these values are not going to change, so better to let LabVIEW remember them. To my way of thinking, however, the biggest issue with this approach is that it complicates the query itself. Given that this is the logic that maintainers (who may not be knowledgeable in SQL) are going to see, it’s a good idea to keep the SQL logic as simple as possible. The other thing to notice about the query is that it puts the entries in the correct order for display by incorporating the clause ORDER BY parent_id, sort_order ASC. Finally, you can see that I built this logic inside the generic ADO database subclass of the existing Config Data object structure.

For reading the tree entries associated with the plugins we use this VI, which is similar to the one for reading the main menu structure, but with some important differences.

The first obvious thing is that the query is much more complex because the primary table being queried is a cross-reference table. Consequently, we have to de-reference the id numbers to derive the data we need to build the menu entries. In learning how this de-referencing works, it’s important to remember that SQL is a language created by a mathematician – specifically a mathematician who specialized in a branch of mathematics called “Set Theory”. His (incredibly optimistic) idea was that if he could create a language based on mathematic principles, he would be able to prove, in the mathematical sense, that the program was correct (read: bug free).

While his grand hope evaporated in the face of the harsh reality that most programming has surprising little to do with mathematics (i.e. computing an answer), the set orientation of SQL has survived. For example, when you perform a query, what you are really doing is SELECTing a data subset FROM a larger set of data – which is typically a table. However, sometimes you need to gather data from a still larger set of data that is spread across multiple tables. To do that, you need to temporarily JOIN those tables together into one large virtual dataset based ON some criteria, like matching id numbers. Get the idea?

A not-so-obvious thing about this LabVIEW code is where it is located in the Config Data object structure. Unlike the routine for reading the basic menu structure, this VI is not located in the generic ADO database subclass. Instead it can be found in the JET database subclass, and the reason for this placement lies in the query. Unlike the other query operation which was implemented in generic SQL, there are aspects of this query that utilize JET-specific syntax (specifically, all the parentheses).

Generating the Menu Tags

With the data in hand that defines the tree-control menus, we now need to turn that data into menu entries. The first step in that transformation is to process the raw data we have acquired from the database to generate the tags that are needed to properly organize the tree entries. I won’t take up the room to show the code for this VI (Parse Tree Management Data.vi) because it’s easy to explain what it does – but feel free to check it out in the code. The VI’s primary program structure is a while loop that iterates through the raw tree-definition data generating the tags and formatting the data to generate the tree items. The loop has on it two shift registers: one holds an array of ID numbers that the loop has already processed, the other holds an array of clusters. Each element contains the four items that we will need to define a menu item (Parent Tag, Child Name, Child Tag and Child Only?).

With each iteration, the loop extracts the top element from the array and tests the Parent ID. If it is zero, the item is a top-level entry so the code builds its entry and continues with the next iteration. If Parent ID is anything other than 0, it searches the array of processed IDs to see if its parent has already been processed. When the comparison finds the new entry’s Parent ID it uses its tag value to synthesize the new entry’s child tag. When the new entry’s Parent ID is not found, the code adds the entry’s element back onto the bottom of the array of entries to be processed so it can be retried later. Normally, this search should never fail because the ordering in the queries should put the elements in the correct order, but this is just in case. This operation continues until there are no more entries left to process.

Finally, in terms of tree control infrastructure, the only thing we have left is to actually insert the entries that we have defined into the tree control. By the time we get to this point in the code we have gotten the definitions in the correct order so all we have left to do is disable front panel updates, clear the tree control, add the new tree entries and re-enable front panel updates. Again, this code is very simple so to save space I will refer you to the last post (https://www.notatamelion.com/2015/09/14/a-tree-grows-in-brooklyn-labview/) for details on the call and how it works.

Integration with the Testbed

To integrate this code with the existing testbed application requires very little work. First off on the front panel, we remove the existing ring control that we were using to select screens and add the tree control (I’m using he one from the System-themed palette), and a System-themed enumeration that will allow the operator to switch between the two menu context values. Note that this control could also be defined as a ring with the String[] control populated at run time to show the available options. This implementation would be useful if you want to provide the ability in your program to either allows the users to dynamically configure the basic structure of the tree control menus, or provide different options depending on who is using the system.

On the block diagram, the front panel changes impact the program logic in two places. First, we need to create a new Value Changeevent to handle the Menu Context control. This event (which is also fired when the GUI initializes itself) is responsible for rebuilding the tree-control menu display.

The event handler starts by calling a subVI (Get Tree Menu Data.vi) that accepts as an input a Menu Context value and internally calls the two database query VIs we discussed above. After concatenating the arrays that it gets from the two routines, it passes the raw data to the VI (Parse Tree Management Data.vi) I described that converts the raw data into tree control entries. Finally it returns the array of tree control entries to the event handler, which passes it, and two references to the subVI (Draw Menu.vi) which does exactly what it name says. The first of the references is, obviously, a control reference to the tree control. The other is a VI reference to the GUI itself so the subVI can defer and then re-enable front panel updates.

The other block diagram change is to purpose an existing value change event. The event n question used to handle the ring control that changed screens and while it will still be a value change event, it will be a value change event on the aptly labeled tree control, Tree.

The original logic that occupied this space took the string value of the selected ring item and used it to look up the name of the associated screen in an array of strings. The string array consisted of screen labels that were generated when the GUI loaded the subpanel VIs into memory and started them running. The resulting index was then used to index the screen’s VI reference from an array of plugin screen VI references. This VI reference would, in turn, drive the subpanel’s Insert VI method to make that screen visible in the subpanel.

The modified form works basically the same, but with a couple minor differences. Although the tree control’s value is a string, the string is the tag associated with the entry. Since the part we need to perform our search is the last item in the colon-delimited list, the first thing we need to do is strip off everything up to, and including, the lastcolon in the string. Moreover because we want this operation to be efficient as possible – so no looping. A very efficient solution is to use the built-in Match Pattern function with the rather curious-looking pattern shown. To see how it works, consider that a dot (“.”) is a special character matches any character. Next, the asterisk (“*”) is a special character that matches the longest sequence of the token that came just before it. Hence, it will match the longest sequence of any characters. Finally, the colon is not a special character so it will match just a colon. The end result is that the complete pattern will match the longest sequence of characters that are followed by a colon, and it works the same whether there is one colon in the string or a dozen. The string I want will be what is left after the match.

The other change that was needed for the tree control, is the case structure, which is there to work around a bug in the way LabVIEW handles value change events with tree controls. I configured the tree control such that only entries that are marked as Child Only are selectable. The bug is that when you click on one of the parent items, LabVIEW still fires the value change event even though the value of the tree control isn’t changing. To work around this issue, the event handler bypasses all further event processing of the “selected” item when it isn’t found in the list of screens.

Testing the Interface

As always, the “Proof of the pudding is in the eating” so let’s try running the application with its new GUI feature. One minor difference in behavior is that the subpanel now remains empty at startup until the user makes a selection. However, from the time the application starts, the user has visible a complete list of all the display screens that are available. In addition, the tree will automatically reconfigure itself when a different context is selected.

Testbed Application – Release 17
Toolbox – Release 14

The Big Tease

At one time when I started doing this work, test systems were surprisingly homogeneous. While it was true that instruments came from many different vendors, the software environment was pretty monolithic. Today, however, things have really changed. Every day it is becoming more common and accepted to have multiple applications running in parallel that were developed using a variety of development tools ranging from C++ to Java to C# and F.

In the past we have talked about creating a LabVIEW-based backend application with the main GUI built using standard web tools such as HTML, CSS and JavaScript (https://www.notatamelion.com/2015/06/08/building-a-web-backend-in-labview/). Over the next few posts I want to consider some of the other ways that your LabVIEW application can work with external applications.

Until Next Time…
Mike…

A Brief Introduction to .NET in LabVIEW

Posted on 2015-07-022015-10-06 by mlportersr

From the earliest days of LabVIEW, National Instrument has recognized that it needed the ability to incorporate code that was developed in other programming environments. Originally this capability was realized through specialized functions called Code Interface nodes, or CINs. However as the underlying operating systems continued to develop, LabVIEW acquired the ability to leverage such things as DLLs, ActiveX controls and .NET assemblies. Unfortunately, while .NET solves many of the problems that earlier efforts to standardize sharable code exhibited, far too many LabVIEW developers feel intimidated by what they see as unmanageable complexity. The truth, however, is that there are many well-written .NET assemblies that are no more difficult to use than VI Server.

As an example of how to use .NET, we’ll look at an assembly that comes with all current versions of Windows. Called NotifyIcon, it is the mechanism that Windows itself uses to give you access to programs through the part of the taskbar called the System Tray. However, beyond that usage, it is also an interesting example of how to utilize .NET to implement an innovative interface for background tasks.

The Basic Points

Given that the whole point of this lesson is to learn about creating a System Tray interface for your application, a good place to start the discussion is with a basic understanding of how the bits will fit together. To begin with, it is not uncommon, though technically untrue, to hear someone say that their program was, “…running in the system tray…”. Actually, your program will continue to run in the same execution space, with or without this modification. All this .NET assembly does is provide a different way for your users to interact with the program.

But that explanation raises another question: If the .NET code allows me to create the same sort of menu-driven interface that I see other applications using, how do the users’ selections get communicated back to the application that is associated with the menu?

The answer to that question is another reason I wanted to discuss this technique. As we have talked about before, as soon as you have more than one process running, you encounter the need to communicate between process – often to tell another process that something just happened. In the LabVIEW world we often do this sort of signalling using UDEs. In the broader Windows environment, there is a similar technique that is used in much the same way. This technique is termed a callback and can seem a bit mysterious at first, so we’ll dig into it, as well.

Creating the Constructor

In the introduction to this post, I likened .NET to VI Server. My point was that while they are in many ways very different, the programming interface for each is exactly the same. You have a reference, and associated with that reference you have properties that describe the referenced object, and methods that tell the object to do something.

To get started, go to the .NET menu under the Connectivity function menu, and select Constructor Node. When you put the resulting node on a block diagram, a second dialog box will open that allows you to browse to the constructor that you want to create. The pop-up at the top of the dialog box has one entry for each .NET assembly installed on your computer – and there will be a bunch. You locate constructors in this list by name, and the name of the constructor we are interested in is System.Windows.Forms. On your computer there may be more than one assembly with this basic name installed. Pick the one with the highest version (the number in parentheses after the name).

In the Objects portion of the dialog you will now see a list of the objects contained in the assembly. Double click on the plus sign next to System.Windows.Forms and scroll down the list until you find the bullet item NotifyIcon, and select it. In the Constructors section of the dialog you will now see a list of constructors that are available for the selected object. In this case, the default selection (NotifyIcon()) is the one we want so just click the OK button. The resulting constructor node will look like this:

But you may be wondering how you are supposed to know what to select. That is actually pretty easy. You see, Microsoft offers an abundance of example code showing how to use the assemblies, and while they don’t show examples in LabVIEW, they do offer examples in 2 or 3 other languages and – this is the important point – the object, property and method names are the same regardless of language so it’s a simple matter to look at the example code and, even without knowing the language, figure out what needs to be called, and in what order. Moreover, LabVIEW property and invoke nodes will list all the properties and methods associated with each type of object. As an example of the properties associated with the NotifyIcon object, here is a standard LabVIEW property node showing four properties that we will need to set for even a minimal instance of this interface. I will explain the first three, hopefully you should be able to figure out what the fourth one does on your own.

Starting at the top is the Text property. It’s function is to provide the tray icon with a label that will appear like a tip-strip when the user’s mouse overs over the icon. To this we can simply wire a string. You’ll understand the meaning of the label in a moment.

Giving the Interface an Icon

Now that we have created our NotifyIcon interface object and given it a label, we need to give it an icon that it can display in the system tray. In our previous screenshot, we see that the NotifyIcon object also has a property called Icon. This property allows you to assign an icon to the interface we are creating. However, if you look at the node’s context help you see that its datatype is not a path name or even a name, but rather an object reference.

But don’t despair, we just created one object and we can create another. Drop down another empty .NET constructor but this time go looking for System.Drawing.Icon and once you find the listing of possible constructors, pick the one named Icon(String fileName). Here is the node we get…

…complete with a terminal to which I have wired a path that I have converted to a string. In case you missed what we just did, consider that one of the major failings of older techniques such as making direct function calls to DLLs was how to handle complex datatypes. The old way of handling it was through the use of a C or C++ struct, but to make this method work you ended up needing to know way too much about how the function worked internally. In addition, for the LabVIEW developer, it was difficult to impossible to build these structures in LabVIEW. By contrast, the .NET methodology utilizes object-oriented techniques to encapsulate complex datatypes into simple-to-manipulate objects that accept standard data inputs and hide all the messy details.

Creating a Context Menu

With a label that will provide the users a reminder of what the interface is for, and an icon to visually identify the interface, we now turn to the real heart of the interface: the menu itself. As with the icon, assigning a menu structure consists of writing a reference to a property that describes the object to be associated with that property. In this case, however, the name of the property is ContextMenu, and the object for which we need to create a constructor is System.Windows.Forms.ContextMenu and the name of the constructor is ContextMenu(MenuItem[] menuItems).

From this syntax we see that in order to initialize our new constructor we will need to create an array of menuItems. You got to admit, this makes sense: our interface needs a menu, and the menu is constructed from an array of menu items. So now we look at how to create the individual menu items that we want on the menu. Here is a complete diagram of the menu I am creating – clearly inspired by a youth spent watching way too many old movies (nyuk, nyuk, nyuk).

Sorry for the small image, but if you click on the image, you can zoom in on it. As you examine this diagram notice that while there is a single type of menuItem object, there are two different constructors used. The most common one has a single Text initialization value. The NotifyIcon uses that value as the string that will be displayed in the menu. This constructor is used to initialize menu items that do not have any children, or submenus. The other menuItem constructor is used to create a menu item that has other items under it. Consequently in addition to a Text initialization value, it also has an input that is – wait for it – an array of other menu items. I don’t know if there is a limit to how deeply a menu can be nested, but if that is a concern you need to be rethinking your interface.

In addition to the initialization values that are defined when the item is created, a menuItem object has a number of other properties that you can set as needed. For instance, they can be enabled and disabled, checked, highlighted and split into multiple columns (to name but a few). A property that I apply, but the utility which might not be readily apparent, is Name. Because it doesn’t appear anywhere in the interface, programmers are pretty much free to use is as they see fit, so I going to use it as the label to identify each selection programmatically. Which, by the way, is the next thing we need to look at.

Closing the Event Loop

If we stopped with the code at this point, we would have an interface with a perfectly functional menu system, but which would serve absolutely no useful purpose. To correct that situation we have to “close the loop” by providing a way for the LabVIEW-based code to react in a useful way to the selections that the user makes via the .NET assembly. The first part of that work we have already completed by establishing a naming convention for the menu items. This convention largely guarantees menu items will have a unique name by defining each menu item name as a colon-delimited list of the menu item names in the menu structure above it. For example, “Larry” and “Moe” are top-level menu items so their names are the same as their text values. “Shep” however is in a submenu to the menu item “The Other Stooge” so its name is “The Other Stooge:Shep”.

The other thing we need in order to handle menu selections is to define the callback operations. To simplify this part of the process, I like to create a single callback process that services all the menu selections by converting them into a single LabVIEW event that I can handle as part of the VI’s normal processing. Here is the code that creates the callback for our test application:

The way a callback works is that the callback node incorporates three terminals. The top terminal accepts an object reference. After you wire it up, the terminal changes into a pop-up menu listing all the callback events that the attached item supports. The one we are interested in is the Click event. The second terminal is a reference for the VI that LabVIEW will have executed when the event you selected is fired. However, you can’t wire just any VI reference here. For it to be callable from within the .NET environment it has to have a particular set of inputs and a particular connector pane. To help you create a VI with the proper connections, you can right-click on the terminal and select Create Callback VI from the menu. The third terminal on the callback registration node is labelled User Parameters and it provides the way to pass static application-specific data into the callback event.

There are two important points here: First, as I stated before, the User Parameters data is static. This means that whatever value is passed to the terminal when the callback is registered is from then on essentially treated as a constant. Second, whatever you wire to this terminal modifies the data inputs to the callback VI so if you are going to use this terminal to pass in data, you need to wire it up before you create the callback VI.

In terms of our specific example, I have an array of the menu items that the main VI will need to handle so I auto-index through this array creating a callback event for each one. In all cases, though, the User Parameter input is populated with a reference to a UDE that I created, so the callbacks can all use the same callback VI. This is what the callback VI looks like on the inside:

The Control Ref input (like User Parameter) is a static input so it contains the reference to the menu item that was passed to the registration node when the callback was created. This reference allows me to read the Name property of the menu item that triggered the callback, and then use that value to fire the SysTray Callback UDE. It’s important to remember when creating a callback VI to not include too much functionality. If fact, this is about as much code as I would ever put in one. The problem is that this code is nearly impossible to debug because it does not actually execute in the LabVIEW environment. The best solution is to get the selection into the LabVIEW environment as quickly as possible and deal with any complexity there. Finally, here is how I handle the UDE in the main VI:

Here you can see another reason why I created the menu item names as I did. Separating the different levels in the menu structure by colons allows to code to easily parse the selection, and simultaneously organizes the logic.

Future Enhancements

With the explanations done, we can now try running the VI – which disappears as soon as you start it. However, if you look in the system tray, you’ll see its icon. As you make selections from its menu you will see factoids appear about the various Stooges. But this program is just the barest of implementations and there is still a lot you can do. For example, you can open a notification balloon to notify the user of something important, or manipulate the menu properties to show checkmarks on selected items or disable selections to which you want block access.

The most important changes you should make, however, are architectural. For demonstration purposes the implementation I have presented here is rather bare-bones. While the resulting code is good at helping you visualize the relationships between the various objects, it’s not the kind of code you would want to ship to a customer. Rather, you want code that simplifies operation, improves reusability and promotes maintainability.

Stooge Identifier — Release 1

The Big Tease

So you have the basics of a neat interface, and a basic technique for exploring .NET functionality in general. But what is in store for next time? Well I’m not going to leave you hanging. Specifically, we are going to take a hard look at menu building to see how to best modularize that functionality. Although this might seem a simple task, it’s not as straight-forward as it first seems. As with many things in life, there are solutions that sound good – and there are those that are good.

Until Next Time…

Mike…

Building a Proper LabVIEW State Machine Design Pattern – Pt 2

Posted on 2015-03-022015-03-14 by mlportersr

Last week’s post was rather long because (as is often the case in this work) there was a lot we had to go over before we could start writing code. That article ended with the presentation of a state diagram describing a state machine for a very simple temperature controller. This week we have on our plate the task of turning that state diagram into LabVIEW code that implements the state machine functionality.

The first step in that process is to consider in more detail the temperature control process that the state diagram I presented last week described in broad terms. By the way, as we go through this discussion please remember that the point of this exercise is to learn about state machines — not temperature control techniques. So unless your application is to manage the internal temperature of a hen-house, dog-house or out-house; don’t use this temperature control algorithm anywhere.

How the demo works

The demonstration is simulating temperature control for an exothermic process — which is to say, a process that tends to warm or release heat into the environment over time. To control the temperature, the system has two resources at its disposal: an exhaust fans and a cooler. Because the cooler is actively putting cool air into the area, it has a very dramatic effect on temperature. The fan, on the other hand, has a much smaller effect because it is just reduces the heat through increased ventilation.

When the system starts, the state machine is simply monitoring the area temperature and as long as the temp is below a defined “High Warning Level” it does nothing. When that level is exceeded, the system turns on the fan, as shown by the fan light coming on. In this state, the temperature will continue to rise, but at a slower rate.

Eventually the temperature will, exceed the “High Error Level” and when it does, the system will turn on the cooler (it has a light too). The cooler will cause the temperature to start dropping. When the temperature drops below the “Low Warning Level” the fan will turn off. This action will reduce the cooling rate, but not stop it completely. When the temperature reaches the “Low Error Level”, the cooler will turn off and the temperature will start rising again.

So let’s look at how our state machine will implement that functionality.

“State”-ly Descriptions

As I stated last week, the basic structure is an event structure with most of the state machine functionality built into the timeout case. To get us started with a little perspective, this is what the structure as a whole looks like.

Obviously, from the earlier description, this state machine will need some internal data in order to perform its work. Some of the data (the 4 limit values and the sample interval) is stored in the database, while others are generated dynamically as the state machine executes. Regardless of its source, all of this data is stored in a cluster, and you can see two of the values that it contains being unbundled for use. Timeout is the timeout value for the event structure and is initialized to zero. The Mode value is an enumeration with two values. In the Startup case is the logic that implements startup error checking and reads the initial setup values from the database. When it finishes, it sets Mode to its other value: Run. This is where you will find the bulk of the state machine logic. Note that while I don’t implement it here, this logic could be expanded to provide the ability to do such things as pause the state machine.

The following sections describe the function of each state and shows the code that implements it.

Initialization

This state is responsible for getting everything initialized and ready to run.

In a real system, the logic represented here by a single hardware initialization VI would be responsible for initializing the data acquisition, verifying that the system is capable of communicating with the fan and cooler, and reading their operational states. Consequently, this logic might be 1 subVI or there might be 2 or 3 subVIs. The important point is to not show too much detail in various states. Use subVIs. Likewise, do not try to expand on the structure by adding multiple initialization states.

Finally, note that while the selection logic for the next state may appear to be a default transition, it isn’t. The little subVI outside the case structure actually creates a two-way branch in the logic. If the incoming error cluster is good, the incoming state transitions (in this case, Read Input) is passed through unmodified. However, if an error is present, the state machine will instead branch to the Error state where the problem can be addressed as needed.

Read Input

This state is responsible for reading the current temperature (referred to as the Process Variable) and updating the state machine data cluster.

In addition to updating the last value, there are a couple other values that it sets. Both of these values relate to how the acquisition delay is implemented in the state machine. The first of these is the Timeout value, and since we want no delay between states, we set this to zero. The other value is the Last Sample Time. It is a timestamp indicating when the reading was made. You’ll see in a bit how these values are used.

This state also updates two front panel indicators, the graph and a troubleshooting value.

Test Fan Limits

This state incorporates a subVI that analyzes the data contained in the state machine data to determine whether or not the fan needs to change state.

The selector in this state results in what is the potential for a 3-way branch. If a threshold has been crossed, the next state will be Set Fan, if it has not, the next state will be Test Cooler Limits, and if an error has occurred, the next state will be Error.

Set Fan

Since the fan can only be on or off, all this state needs to do is reverse its current operating condition.

In addition to the subVI toggling the fan on or off, the resulting Fan State is unbundled from the state machine data and the value is used to control the fan LED.

Test Cooler Limits

This state determines whether the cooler needs to be changed. It can also only be on or off.

The logic here is very similar to that used to test the fan limits.

Set Cooler

Again, not unlike the corresponding fan control state, this state changes the cooler state and sets the cooler LED as needed.

Acquisition TO Wait

This state handles the part of the state machine that is in often the Achilles Heel of an implementation. How do we delay starting the control sequence again without incurring the inefficiency of polling?

The answer is to take advantage of the timeout that event structures already have. The heart of that capability is the timeout calculation VI, shown here:

Using inputs from the state machine data, the VI adds the Sample Interval to the Last Sample Time. The result is the time when the next sample will need to be taken. From that value, the code subtracts the current time and converts the difference into milliseconds. This value is stored back into the state machine data as the new timeout. This technique is very efficient because it requires no polling.

But wait, you say. This won’t work if some other event happens before the timeout expires! True. But that is very easy to handle. As an example, here is the modified event handler for the save data button.

It looks just as it did before, but with one tiny addition. After reading, formatting and saving the data, the event handler calls the timeout calculation VI again to calculate a new timeout to the intended next sample time.

Error

This state handles errors that occur in the state machine. That being the case, it is the new home for our error VI.

Deinitialize

Finally, this state provides the way to stop the state machine and the VI running it. To reach this state, the event handler for the UDE that shuts down the application, will branch to this state. Because it is the last thing to execute before the VI terminates, you need to be sure that it includes everything you need to bring the system to a safe condition.

With those descriptions done, let’s look at how the code works.

The Code Running

When you look at the new Temperature Controller screen you’ll notice that in addition to the graph and the indicators showing the states of the fan and cooler, there are a couple of numbers below the LEDs. The top one is the amount of elapsed between the last two samples, the bottom one is the delay calculated for the acquisition timeout.

If you watch the program carefully as its running, you’ll notice something a bit odd. The elapsed time indicator shows a constant 10 seconds between updates (plus or minus a couple of milliseconds — which is about all you can hope for on a PC). However, the indicator showing the actual delay being applied is never anywhere near 10,000 milliseconds. Moreover, if you switch to one of the other screens and the back, the indicated delay can be considerably less than 10,000 milliseconds, but the elapsed time never budges from 10 seconds. So what gives?

What you are seeing in action is the delay recalculation, we talked about earlier. In order to better simulate a real-world system, I put a delay in the read function that pauses between 200- and 250-msec. Consequently when execution reaches the timeout calculation, we are already about 1/4 of a second into the 10 second delay. The calculation, however, automatically compensates for this delay because the timeout is always referenced to the time of the last measurement. The same thing happens if another event comes between successive data acquisitions.

On Deck

As always, if you have any questions, concerns, gripes or even (gasp!) complements, post ’em. If not feel free to use any of this logic as you see fit — and above all, play with the code see how you might modify it to do similar sorts of things.

Stay tuned. Next week we will take a deeper look at something we have used before, but not really discussed in detail: Dynamically Calling of VIs. I know there are people out there wondering about this, so it should be fun.

Until next time…

Mike…

Building a Proper LabVIEW State Machine Design Pattern – Pt 1

Posted on 2015-02-232015-03-14 by mlportersr

The other day I was looking at the statistics for this site and I noticed that one of the most popular post with readers was the one I wrote on the producer/consumer design pattern. Given that level of interest, it seemed appropriate to write a bit about another very common and very popular design pattern: the state machine. There’s a good reason for the state-machine’s popularity. It is an excellent, often ideal, way to deal with logic that is repetitive, or branches through many different paths. Although, it certainly isn’t the right design pattern for every application, it is a really good technique for translating a stateful process into LabVIEW code.

However, some of the functionality that state machines offer also means they can present development challenges. For example, they are far more demanding in terms of the design process, and consequently far less forgiving of errors in that process. As we have seen before with other topics, even the most basic discussion of how to properly design and use state machines is too big for a single post. Therefore, I will present one post to discuss the concepts and principles involved, and in a second post present a typical implementation.

State Machine Worst Practices

For some reason it seems like there has been a lot of discussions lately about state machine “best practices”. Unfortunately, some of the recommendations are simply not sound from the engineering standpoint. Meanwhile others fail to take advantage of all that LabVIEW offers because they attempt to mimic the implementation of state machines in primitive (i.e. text-based) languages. Therefore, rather than spinning out yet another “best practices” article, I think it might interesting to spend a bit of time discussing things to never do.

In describing bad habits to avoid, I think it’s often a good idea to start at the most general level and work our way down to the details. So let’s start with the most important mistake you can make.

1. Use the state machine as the underlying structure of your entire application

State machines are best suited for situations where you need to create a complex, cohesive, and tightly-coupled process. While an application might have processes within it that need to be tightly-coupled, you want the application as a whole to exhibit very low levels of coupling. In fact, much of the newest computer science research deprecates the usage of state machines by asserting that they are inherently brittle and non-maintainable.

While I won’t go that far, I do recognize that state machines are typically best suited for lower-level processes that rarely, if ever, appear to the user. For example, communications protocols are often described in terms of state machines and are excellent places to apply them. One big exception to this “no user-interface” rule is the “wizard” dialog box. Wizards will often be built around state machines precisely because they have complex interface functionality requirements.

2. Don’t start with a State Diagram

Ok, so you have a process that you feel will benefit from a state machine implementation. You can’t just jump in and start slinging code right and left. Before you start developing a state machine you need to create a State Diagram (also sometimes called a State Transition Diagram), to serve as a road-map of sorts for you during the development process. If you don’t take the time for this vital step, you are pretty much in the position of a builder that starts work on a large building with no blueprint. To be fair, design patterns exist that are less dependent upon having a completed, through design. However, those patterns tend to be very linear in structure, and so are easy to visualize in good dataflow code. By contrast, state machines are very non-linear in their structure so can be very difficult to develop and maintain. To keep straight what you are trying to accomplish, state machines need to be laid out carefully and very clearly. The unfortunate truth, however, is that state machines are often used for the exact opposite reason. There is a common myth that state machines require a minimum of design because if you get into trouble, you can always just, “add another state”. In fact, I believe that much of the bad advice you will get on state machines finds its basis in this myth.

But even if we buy the idea that state machines require a more through design, why insisted on State Diagrams? One of the things that design patterns do is foster their own particular way of visualizing solutions to programming problems. For example, I have been very candid about how a producer/consumer design pattern lends itself to thinking about applications as a collection of interacting processes. In the same way, state machines foster a viewpoint where the process being developed is seen as a virtual machine constructed from discrete states and the transitions between those states. Given that approach to problem solving, the state diagram is an ideal design tool because it allows you to visually represent the structure that the states and transitions create.

So what does it take to do a good state-machine design? First you need to understand the process — a huge topic on its own. There are many good books available on the topic, as well as several dedicated web sites. Second, having a suitable drawing program to create State Diagrams can be helpful, and one that I have used for some time is a free program called yEd. However fancy graphics aren’t absolutely necessary. You can create perfectly acceptable State Diagrams with nothing more than a paper, a pencil and a reasonably functional brain. I have even drawn them on a white board during a meeting with a client and saved them by taking a picture of them with my cell phone.

Moreover, drawing programs aren’t much help if you don’t know what to draw. The most important knowledge you can have is a firm understanding of what a state machine is. This is how Wikipedia defines a state machine:

A finite-state machine (FSM) or finite-state automaton (plural: automata), or simply a state machine, is a mathematical model of computation used to design both computer programs and sequential logic circuits. It is conceived as an abstract machine that can be in one of a finite number of states. The machine is in only one state at a time; the state it is in at any given time is called the current state. It can change from one state to another when initiated by a triggering event or condition; this is called a transition.

An important point to highlight in this description is that a state machine is at its core is a mathematical model — which itself implies a certain level of needed rigor in their design and implementation. The other point is that it is a model that consists of a “finite number of steps” that the machine moves between on the basis of well-defined events or conditions.

3. Ignore what a “state” is

Other common problems can arise when there is confusion over what constitutes a state. So let’s go back to Wikipedia for one more definition:

A state is a description of the status of a system that is waiting to execute a transition.

A state is, in short, a place where the code does something and then pauses while it waits for something else to happen. Now this “something” can be anything from the expiration of a timer to a response from a piece of equipment that a command had been completed successfully (or not). Too often people get this definition confused with that for a subVI. In fact one very common error is for a developer to treat states as though they were subroutines that they can simply “call” as needed.

4. Use strings for the state variable

The basic structure behind any state machine is that you have a loop that executes one state each time it iterates. To define execution order there is a State Variable that identifies the next state the machine will execute in response to an event or condition change. Although I once saw an interesting object-oriented implementation that used class objects to both identify the next state (via dynamic dispatch) and pass the state machine operational data (in the class data), in most cases there is a much simpler choice of datatype for the State Variable: string or enumeration.

Supposedly there is an ongoing discussion over which of these two datatypes make better state variables. The truth is that while I have heard many reasons for using strings as state variables, I have yet to hear a good one. I see two huge problems with string state variables. First, in the name of “flexibility” they foster the no-design ethic we discussed earlier. Think about it this way, if you know so little about the process you are implementing that you can’t even list the states involved, what in the world are you doing starting development? The second problem with state strings is that using them requires the developer to remember the names of all the states, and how to spell them, and how to capitalize them — or in a code maintenance situation, remember how somebody else spelled and capitalized them. Besides trying to remember that the guy two cubicles down can never seem to remember that “flexible” is spelled with an “i” and not an “a”, don’t forget that there is a large chunk of the planet that thinks words like “behavior” has a “u” in them…

By the way, not only should the state variable be an enumeration, it should be a typedef enumeration.

5. Turn checking the UI for events into a state

In the beginning, there were no events in LabVIEW and so state machines had to be built using what was available — a while loop, a shift register to carry the state variable, and a case structure to implement the various states. When events made their debut in Version 6 of LabVIEW, people began to consider how to integrate the two disparate approaches. Unfortunately, the idea that came to the front was to create a new state (typically called something like, Check UI) that would hold the event structure that handles all the events.

The correct approach is to basically turn that approach inside out and build the state machine inside the event structure — inside the timeout event to be precise. This technique as a number of advantages. To begin with, it allows the state machine to leverage the event structure as a mechanism for controlling the state machine. Secondly, it provides a very efficient mechanism for building state machines that require user interaction to operate.

Say you have a state machine that is basically a wizard that assists the user in setting up some part of your application. To create this interactivity, states in the timeout event would put a prompt on the front panel and sets the timeout for the next iteration to -1. When the user makes the required selection or enters the needed data, they click a “Next” button. The value change event handler for the button knows what state the state machine was last in, and so can send the state machine on to its next state by setting the timeout back to 0. Very neat and, thanks to the event-driven programming, very efficient.

On the other hand, if you are looking for a way to allow your program to lock-up and irritate your users, putting an event structure inside a state is a dandy technique. The problem is that all you need to stop your application in its tracks is one series of state transitions where the “Check UI” state doesn’t get called often enough, or at all. If someone clicks a button or changes something on the UI while those states are executing, LabVIEW will dutifully lock the front panel until the associated event is serviced — which of course can’t happen because the code can’t get to the event structure that would service it. Depending on how bad the overall code design is and the exact circumstances that caused the problem, this sort of lock-up can last several seconds, or be permanent requiring a restart.

6. Allow default state transitions

A default state transition is when State A always immediately transitions to State B. This sort of design practice is the logical equivalent of a sequence structure, and suffers from all the same problems. If you have two or more “states” with default transitions between them, you in reality have a single state that has been arbitrarily split into multiple pieces — pieces that hide the true structure of what the code is doing, complicates code maintenance and increases the potential for error. Plus, what happens if an error occurs, there’s a shutdown request, or anything else to which the code needs to respond? As with an actual sequence structure, you’re stuck going through the entire process.

7. Use a queue to communicate state transitions

Question: If default transitions are bad, why would anyone want to queue up several of them in a row?
Answer: They are too lazy to figure out exactly what they want to do so they create a bunch of pieces that they can assemble at runtime — and then call this kind of mess, “flexibility”. And even if you do come up with some sort of edge case where you might want to enqueue states, there are better ways of accomplishing it than having a queue drive the entire state machine.

Implementation Preview

So this is about all we have room for in this post. Next Monday I’ll demonstrate what I have been writing about by replacing the random number acquisition process in our testbed application with an updated bit of LabVIEW memorabilia. Many years ago the very first LabVIEW demo I saw was a simple “process control” demo. Basically it had a chart with a line on it that trended upwards until it reached a limit. At that point, an onscreen (black and white!) LED would come on indicating a virtual fan had been turned on and the line would start trending back down. When it hit a lower limit, the LED and the virtual fan would go off and the line would start trending back up again. With that early demonstration in mind, I came up with this State Diagram:

When we next get together, we’ll look at how I turn this diagram into a state-machine version of the original demo — but with color LEDs!

Until next time…

Mike…

A “Finished” Testbed Application

Posted on 2014-10-302015-08-28 by mlportersr

As I’ve worked on the past several posts, I had a very specific milestone in mind. You see, I am wanting to use this LabVIEW blog to demonstrate and explain a lot of things that I have learned over the years, but the proper presentation of those topics typically requires a certain amount of pre-existing infrastructure.

That is what I have been doing through the preceding posts — building the conceptual basis for a testbed application that embodies what we have discussed so far. Having this infrastructure is critical for thorough, disciplined learning because without it you are left with example code that may demonstrate one point well, but violates a dozen other principles that are standard “best practices”.

Having said that, however, remember the quotation marks around the word Finished in the title. The marks are there to highlight the fact that, as with all LabVIEW applications, this testbed will never really, and finally finished. In fact, it is planned that it will get better over time, and support more functionality. The best we can say is that the testbed is done for now.

Come next week though, who knows? There’s still a lot to do.

The Testbed Project

Like the UDE template, the current version of the testbed and all its associated VIs is available through the website’s subversion repository at:

http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 1

You can use the web client as you did before, but considering the number of files, you will find it to be less time-consuming (and error prone) to use a dedicated client application like TortoiseSVN on Windows. Regardless of how you get the testbed project onto your computer, the first thing that you see with any application is the project file that is its home. When you open testbed.lvproj note that it embodies two of the conventions that I use on a regular basis.

There are virtual folders for two types of items — both of which are contained in LabVIEW libraries:
The _ude folder contains the libraries associated with the two UDEs that are currently being used. The _libraries folders contains the two libraries embodying the main functional areas that the application currently uses. The Configuration Management library houses the files that the application uses to fetch configuration information (right now it only needs to read the launch processes). The Startup Processes library contains the VIs that will be launchable when the application starts, and the process-specific VIs that they use. Finally, the Path Utilities library contains the VIs that the code uses to locate the internal project home directory in an executable.
The launcher VI is directly in My Computer, along with its INI file, testbed.ini.
As I have discussed elsewhere, this naming convention simplifies a variety of tasks such as locating the application INI file.

Looking at the VIs

The remainder of this post will look at the VIs used in creating the testbed application. Right now the discussion will be at a rather high-level as we will be digging into them on a more detailed basis in the future. With each VI, I also list a few of the future topics that I will use that VI to demonstrate. If you would like to see others, let me know in the comments.

Testbed.vi

Here we see pretty much what you would expect. As discussed in the previous post, the code starts by calling a subVI that stores the launcher’s path to a FGV that will make it available to the rest of the application. It then calls the subVI that reads the INI file to get the list of processes to launch.

The array of process descriptions thus retrieved is passed to the main loop that processes each element individually. The LabVIEW documentation does a good job of describing asynchronous call-and-forget so I won’t repeat it here. Instead I will point out the 1 second delay that follows the dynamic call logic. Often times, a new process starting up has a lot it needs to do. This delay gives the new process a 1 second window where the launcher is not doing any potentially time-consuming tasks — like loading into memory the next process it will launch. The delay may not be technically necessary, but I have found that it can sometimes make things run a bit smoother.

Finally, you may be wondering, why there is logic to explicitly close the launcher window? Isn’t there a VI property that tells LabVIEW to automatically close the window if it was originally closed? Well, yes there is, and you would think that it would handle this situation. In fact, it does — but only in the development environment. When you build the application into an executable, the auto-close doesn’t work for the top-level VI window, or at least doesn’t with LabVIEW 2014.

Future Enhancements

Showing the “busy” cursor
Enhancing user feedback
How to launch other standalone executables
How to launch VIs that need input parameters

Startup Processes.lvlib:Acquire Data.vi

This is the VI that the original producer loop turned into. The most noticeable change has been the addition of an event structure that both allows the process to respond to a application stop event, and ( via the timeout event) provides the acquisition timing.

Future Enhancements

Handling errors that occur during initialization
Turning acquisition on and off
Changing acquisition parameters
Correcting for timing “slips” when using the timeout event

Startup Processes.lvlib:Display Data.vi

This VI is the new consumer loop. This VI retains the event loop it had before, but adds an event for stopping the application.

Future Enhancements

Using subpanels to generalize your interface code — and avoid tab controls
Implementing dialog boxes that don’t stop everything while they are open.
The advantages of the “System”-themed controls

In addition to these VIs, there’s no rule that says there can’t be more — like say for handling errors. You may also need additional processes for handling tasks that have differing timing requirements, or communicate through different interfaces. We’ll look at those too.

But for now, feel free to poke around through the code, and if you have any questions ask in the comments. Also try building the application and perhaps even the installer.

Until next time…

Mike…

Making UDEs Easy

Posted on 2014-10-232015-12-17 by mlportersr

For the very first post on this blog, I presented a modest proposal for an alternative implementation of the LabVIEW version of the producer/consumer design pattern. I also said that we would be back to talk about it a bit more — and here we are!

The point of the original post was to present the modified design pattern in a form similar to that used for the version of the pattern that ships with LabVIEW. The problem is that while it demonstrates the interaction between the two loops, the structure cannot be expanded very far before you start running into obstacles. For example, it’s not uncommon to have the producer and consumer loops in separate VIs. Likewise, as with a person I recently dealt with on the forums, you might want to have more than one producer loop passing data to the same consumer. In either case, explicitly passing around a reference complicates matters because you have to come up with ways for all the separate processes to get the references they need.

The way around this conundrum lies in the concept of information hiding and the related process of encapsulation.

Moving On

The idea behind information hiding is that you want to hide from a function any information that it doesn’t need to do its job. Hiding information in this sense makes code more robust because what a routine doesn’t know about it can’t break. Encapsulation is an excellent way if implementing information hiding.

In the case of our design pattern, the information that is complicating things is the detailed logic of how the user event is implemented, and the resulting event reference. What we need is a way to hide the event logic, while retaining the functionality. We can accomplish this goal by encapsulating the data passing logic in a set of VIs that hide the messy details about how they do their job.

Standardizing User-Defined Events

The remainder of this post will present a technique that I have used for several years to implement UDEs. The code is not particularly difficult to build, but if you are a registered subscriber the code can be downloaded from the site’s Subversion SCC server.

The first thing we need to do is think a bit and come up with a list of things that a calling program would legitimately need to do with an event — and it’s actually a pretty short list.

Register to Receive the Event
Generate the Event
Destroy the Event When the Process Using it Stops

This list tells us what VIs the calling VI will need. However, there are a couple more objects that those VIs will be needed internally. One is a VI that will generate and store the event reference, the other is a type definition defining the event data.

Finally, if we are going to be creating 4 VIs and a typedef for each event in a project, we are going to need some way of keeping things organized. So let’s define a few conventions for ourselves.

Convention Number 1

To make it easy to identify what event VI performs a given function, let’s standardize the names. Thus, any VI that creates an event registration will be called Register for Event.vi. The other two event interface VI will, likewise, have simple, descriptive names: Generate Event.vi and Destroy Event.vi. Finally, the VI that gets the event reference for the interface VIs, shall be called Get Event Reference.vi and the typedef that defines the event data will be Event Data.ctl.

But doesn’t LabVIEW require unique VI names? Yes, you are quite right. LabVIEW does indeed require unique VI names. So you can’t have a dozen VIs all named Generate Event.vi. Thus we define:

Convention Number 2

All 5 files associated with an event shall be associated with a LabVIEW library that is named the same as the event. This action solves the VI Name problem because LabVIEW creates a fully-qualified VI name by concatenating the library name and the VI file name. For example, the name of the VI that generates the Pass Data event would have the name:
Pass Data.lvlib:Generate Event.vi
While the VI Name of the VI that generates the Stop Application event would be:
Stop Application.lvlib:Generate Event.vi

The result also reads pretty nice. Though, it still doesn’t help the OS which will not allow two files with the same name to coexist in the same directory. So we need:

Convention Number 3

The event library file, as well as the 5 files associated with the event, will reside in a directory with the same name as the event — but without the lvlib file extension. Hence Pass Data.lvlib, and the 5 files associated with it would reside in the Pass Data directory, while Stop Application.lvlib and its 5 files would be found in the directory Stop Application.

So do you have to follow these conventions? No of course not, but as conventions go, they make a lot of sense logically. So why not just use them and save your creative energies for other things…

The UDE Files

So now that we have places for our event VIs to be saved, and we don’t have to worry about what to name them, what do the VIs themselves look like? As I mentioned before, you can grab a working copy from our Subversion SCC server. The repository resides at:

http://svn.NotaTameLion.com/blogProject/ude_templates

To get started, you can simply click on the link and the Subversion web client will let you grab copies of the necessary files. You’ll notice that when you get to the SCC directory, it only has two files in it: UDE.zip and readme.pdf. The reason for the zip file is that I am going to be using the files inside the archive as templates and don’t want to get them accidentally linked to a project. The readme file explains how to use the templates, and you should go through that material on your own. What I want to cover here is how the templates work.

Get Event Reference.vi

This VI’s purpose is to create, and store for reuse, a reference to the event we are creating. Given that description, you shouldn’t be too surprised to see that it is built around the structure of a functional global variable, or FGV. However, instead of using an input from the caller to determine whether it needs to create a reference, it tests the reference in the shift-register and if it is invalid, creates a reference. If the reference is valid, it passes out the one already in the shift-register.

If you consider the constant that is defining the event datatype, you observe two things. First, you’ll see that it is a scalar variant. For events that essentially operate like triggers and so don’t have any real data to pass, this configuration works fine. Second, there is a little black rectangle in the corner of the constant indicating that it is a typedef (Event Data.ctl). This designation is important because it significantly simplifies code modification.

If the constant were not a typedef, the datatype of the event reference would be a scalar variant and any change to it would mean that the output indicator would have to be recreated. However, with the constant as a typedef, the datatype of the event is the type definition. Consequently you can modify the typedef any way you want and every place the event reference is used will automatically track the change.

Register for Event.vi

This VI is used wherever a VI needs to be able to respond to the associated event. Due to the way events operate, multiple VIs can, and often will, register to receive the same event. As you look at the template block diagram, however, you’ll something is missing: the registration output. The reason for this omission lies in how LabVIEW names events.

When LabVIEW creates an event reference it naturally needs to generate a name for the event. This name is used in event structures to identify the specific particular event handler that will be responding to an event. To obtain the name that it will use, LabVIEW looks for a label associated with the event reference wire. In this case, the event reference is coming from a subVI, so LabVIEW uses the label of the subVI indicator as the event name. Unfortunately, if the name of this indicator changes after the registration reference indictor is created, the name change does not get propagated. Consequently, this indicator can only be created after you have renamed the output of the Get Event Reference.vi subVI to the name that you wish the event to have.

The event naming process doesn’t stop with the event reference name. The label given to the registration reference can also become important. If you bundle multiple references together before connecting them to an event structure’s input dynamic event terminal, the registration indicator is added to the front of the event name. This fact has two implications:

You should keep the labels short

You can use these labels to further identify the event

You could, for example, identify events that are just used as triggers with the label trig. Alternately, you could use this prefix to identify the subsystem that is generating the event like daq or gui.

Generate Event.vi

The logic of this template is pretty straight-forward. The only noteworthy thing about it is that the event data (right now a simple variant) is a control on the front panel. I coded it this way to save a couple steps if I need to modify it for an event that is passing data. Changing the typedef will modify the front panel control, so all I have to do is attach it to a terminal on the connector pane.

Destroy Event

Again, this is very simple code. Note that this VI only destroys the event reference. If it has been registered somewhere, that registration will need to be destroyed separately.

Putting it all Together

So how would all this fit into our design pattern? The instructions in the readme file give the step-by-step procedure, but here is the result.

As explained in the instructions, I intend to use this example as a testbed of sorts to demonstrate some things, so I also modified the event data to be a numeric, and changed the display into a chart. You’ll also notice that the wire carrying the event reference is no longer needed. With the two loops thus disconnected from each other logically, it would be child’s play to restructure this logic to have multiple producer loops, or to have the producer loop(s) and the consumer loop(s) in separate VIs.

By the way, there’s no reason you can’t have multiple consumer loops too. You might find a situation where, for example, the data is acquired once a second, but the consumer loops takes a second and a half to do the necessary processing. The solution? Multiple consumer loops.

However, there is still one teeny-weensy problem. If you now have an application that consists of several parallel processes running in memory, how do you get them all launched in the proper order?

Until next time…

Mike…

Filling in the Blanks

Getting Registered

Keeping Time

Getting into the Publishing Business

Testing the Code

The Big Tease

Share this:

The Problem Defined

Getting Moving in the Right Direction

The Outside View

The Test Application

Crawling Under the Covers

Getting Down With the Acquisition

Let’s Build Some Code

How the Pieces Fit Together

The Big Tease

Share this:

What are WebSockets?

The LabVIEW Connection

Building the Interface

Testing our Work

The HTML and CSS

The JavaScript

Testing the Pages

Big Tease

Share this:

What Can We Access?

Export Data from Plugins

Retrieve Graph Images from Plugins

Adding Remote Control

Some Things to Consider

Set the Acquisition Sample Interval

Set Number of Samples to Save

Set Temperature Controller Operating Limits

Building Out the Infrastructure

Introducing the Remote Access Engine

The Read Graph Data Event

The Read Graph Image Event

The Set Acquisition Rate Event

The Set Data Buffer Depth Event

The Set TC Parameters Event

Testing

The Big Tease

Share this:

The Problem We’re Solving

Fixing the Data

Creating the Data Management Structures

Creating the LabVIEW

Reading the Data

Generating the Menu Tags

Integration with the Testbed

Testing the Interface

The Big Tease

Share this:

The Basic Points

Creating the Constructor

Giving the Interface an Icon

Creating a Context Menu

Closing the Event Loop

Future Enhancements

The Big Tease

Share this:

How the demo works

“State”-ly Descriptions

Initialization

Read Input

Test Fan Limits

Set Fan

Test Cooler Limits

Set Cooler

Acquisition TO Wait

Error

Deinitialize

The Code Running

On Deck

Share this:

State Machine Worst Practices

1. Use the state machine as the underlying structure of your entire application

2. Don’t start with a State Diagram

3. Ignore what a “state” is