Month: May 2015

Using VI Server to Interact with Executables

Posted on by mlportersr

We all want to be able to reuse code and a good way of accomplishing that goal is by repurposing executables that you wrote for other projects. The problem is how you control them. Last week we started addressing this challenge by looking at some of the general tools that are at our disposal for manipulating executables — regardless of where you got them. This time out we will complete the discussion by looking at some of the things you can do that are specific to LabVIEW-created executables.

First, we need an executable

As the title says, if we are going to talk about making VI Server calls to an executable, the first thing we need is an executable — and an executable, we have. Although the functionality it implements is, to be honest, rather sparse, it is sufficient to demonstrate what we need. Here’s what its front panel looks like:

A Small Test Executable - FP

Starting in the top left, it sports an indicator where you can see the command line that was used to launch it. Immediately below that string is an indicator showing the current time, a button for stopping the program manually, and a pair of LEDs that indicate when two different events are triggered: Application Instance Close? and Panel Close?.

To the right is a path indicator that displays one of two different paths depending on the state of the checkbox that is next to it. Below the path indicator are two numerics. One is the PID of the instance that is running. The other is the TCP/IP port number that was assigned to the executable when it was launched. Note that if you don’t provide a port number in the command line parameters, the executable will terminate almost immediately — though you may see it briefly appear in the Windows Task Manager.

Handling the front panel

Most of the code that makes this interface work is pretty straight forward, so I won’t take the time to describe it. The one exception is this bit in the initialization logic:

Intialization Logic

The reason that I’m pulling it aside for special attention is that it illustrates (at least part of) the solution for a problem that you will encounter the first time you create a VI that is designed to run entirely in the background. The heart of this problem is mismatched expectation: When LabVIEW runs an executable it expects to open the window associates with the top-level VI. You, on the other hand, wanting the executable is run unseen in the background, expect the window to stay closed. Consequently, what happens is that LabVIEW opens the window and starts the VI running in that order and your code immediately hides the front panel. What the user sees is a windows that open and then immediately closes without explanation, and if there is one things that worries users more than things happening too slowly, its things happening too fast — like windows flashing open and then closing.

The solution lies in a VI property called Transparency. The setting to control it can be found in the custom appearance dialog.

Transparency Dialog

When the box is checked and the percentage is set to 100, the window will be open but totally transparent. Hence, when the runtime engine launches the application, the window will still open but it will be invisible. A moment later, the code above will hide the window and set the transparency to 0 so that when we do decide to open it, we will be able to see it.

VI Server Operations

Last time, I presented this block of settings from the application’s INI file. Before we continue, we need to take a moment all consider what these settings mean — at least to the extent that anyone knows what they mean…

server.tcp.enabled=True
; server.tcp.port=3363
; server.tcp.serviceName=""
server.tcp.acl="290000000A000000010000001D00000003000000010000002A10000000030000000000010000000000"
server.vi.access="+*"
server.vi.callsEnabled=True
server.app.propertiesEnabled=True
server.vi.propertiesEnabled=True
server.control.propertiesEnabled=True

The first four parameters in this list control overall access to the application via TCP/IP. Consequently, they are the four that you are most likely need to muck with:

  • server.tcp.enabled=True: This setting enables the TCP/IP interface that VI Server uses. If this setting is False, nothing is happening.
  • ; server.tcp.port=3363: This setting specifies the port that the associated TCP/IP listener will be monitoring for a connection. Note that I have this line commented out because we will be assigning this value via command line parameter.
  • ; server.tcp.serviceName="": Also commented out, this optional parameter allows you to define a name that you can then use to reference the application, instead of a Port number.
  • server.tcp.acl=???: This setting defines the TCP/IP access control list (ACL) — or who is allowed to connect to the application. Already I can hear you wondering, what is the deal with that long string? Well, if you ever find out be sure and let me know. The bottom line is that the original interface included an ACL that simply listed the IP addresses that were and were not allowed to access the application. For reasons unknown, NI decided to change this common sense approach to something more enigmatic. So how do you generate this string? Glad you asked. According to LabVIEW’s documentation, you need to set up your development environment to have the same access list as you want your application to have, and then copy and paste the resulting string from LabVIEW’s INI file to your application’s INI file, seriously…

The remaining parameters specify in one way or another the specific resources that the remote program can access in the target application.

  • server.vi.access="+*": This parameter contains a list of VIs that are accessible through the VI Server interface. The default value shown allows access to all VIs.
  • server.vi.callsEnabled=True: This parameter specifies whether the remote program is allowed to run VIs contained in the executable. We leave this True so I can demonstrate that ability.
  • server.app.propertiesEnabled=True: This parameter gives the remote program access to the executable’s application-class properties — like the names of all the VIs currently in memory.
  • server.vi.propertiesEnabled=True: This parameter gives the remote program access to the VI-class properties of individual VIs. This category include things like a reference to the VI’s front panel.
  • server.control.propertiesEnabled=True: This parameter gives the remote program access to the properties associated with individual controls on VI front panels. For example, you need to have this parameter enabled to do things like programatically set the value of a control. This value is True as I will be demonstrating this ability as well.

Finally, I want to state one thing that I hope is obvious. All these “security” settings are contained in a plain text file that can be edited by anyone who knows how to use a simple text editor. The point here is that while recent versions of Windows are making it harder and harder to modify files in the “Programs” directories, it is not by any stretch of the imagination bullet-proof. Hence, if there are truly sensitive things that need restricted access, don’t depend on these settings.

What we can do with these controls

So let’s put some of what we have been learning about into action. If you download the code from the SVN repository you will find, in addition to the source code, a compiled executable. For the following tests, you can either run the executable I have included, or compile it on your own — it’s up to you. You will also want to be sure to update your copy of the toolbox as I have added a couple useful VIs. One of the executable management VIs is where we will start:

Launching the Executable

Start by opening small test executable.lvproj and then open the routine Launch 3x.vi. It’s job is to launch three copies of the test application (small test executable.exe) so on the front panel click the path browser button next to the path control and navigate to and then select the test application. Now, run the VI.

When it finishes, launch the Windows Task Manager. Nothing new under Apps, or Programs (depending on your version of Windows), so look in the Background Processes. Ah, there’s the executable, but why is it listed here? And why is there only one instance? The VI clearly looped 3 times, and there were no errors. Go to the directory where the test application is located and open its INI file. There are your answers. There is only one instance running because the INI file has multiple instances turned off, and the one executable that did launch shows up as a background process because the INI file also says to hide the root window. Leave the root window setting the way it is, but change the AllowMutipleInstances to True, and save and close the file.

Now back in the Task Manager, abort the one instance of the test application that is running now, and rerun Launch 3x.vi. You should see when it finishes that there are now 3 instances of the test application running. The instances were given sequential TCP/IP port numbers from 3365 to 3367.

Firing Remote Events

The next thing I want to do is open the front panels of the 3 instances so we can observe their operation. Now if you look at the test application’s source code, there is a UDE that will make the front panel visible, so all I need to do is fire that event. But wait, those are three instances of a compiled executable — you can’t fire events in other executables! Well actually, you can. To see how, open the test VI, Open the Executable’s Window.vi.

Open Executable Front Panel

No magic here. All the code is doing is dynamically running a VI. But check out the function before Open VI Reference, it’s called Open Application Reference. Its job is to open a reference to a copy of LabVIEW or the LabVIEW runtime engine that is running somewhere else. That “somewhere else” is defined in terms of a machine name and a port number or service name. The machine name can be a DNS name, an IP address or (as in our case here) localhost to point to the local computer.

By the way, if you think it sounds like I just said that you could make this same code access an executable residing on a remote computer by simply changing localhost to an IP address, your right. I did just say that.

But as cool as that feature might be, how does it allow me to fire an event in a compiled executable? Look at the name of the file being run: Open Window.lvlib:Generate Event.vi. It’s the VI that fires the event, and since VIs called in this way actually execute in the remote LabVIEW environment, the event gets fired in the targeted executable.

To see this code in action, run it three times with the port number 3365, 3366 and 3367. Three windows will open.

Setting Control Values

Another way of interacting with an executable is to directly manipulate controls on its front panel. However, if the target VI is event-driven like our test application, we need to remember that there is a difference between setting a value and firing any value change events associated with that control. If all you need to do is set a value, there is a VI method called Control Value:Set that will do the job nicely. However, if you want to fire the value change event you have to set the control’s Value(Signaling) property — which frankly is a bit more work.

set the selector control value with signalling

This picture is the block diagram of the test VI Toggle the selector checkbox.vi, but the good news is that for this little bit of extra effort, you can set (or read) any control property that can be changed while a VI is running.

Shutting Down the Executable

Finally, we need to be able to stop an application that is running. But the problem here is figuring out how to test it such that we can see that it really did what it was supposed to do. The solution is to turn to the trace technique we discussed a while back when we were learning about command line arguments. I have written the code such that if the executable is run with the argument “d1” in the command line, the code will write a line to the trace file saying how the instance was stopped. And to help demonstrate how this works, I have created a test VI (Stop the Executable.vi) that can execute some of these termination paths.

To start off, leave both controls in their default state, and run the VI. This example stops the targeted executable by clicking the Stop button on its front panel. The instance with the port number 3365 will immediately close.

Now increment the Port Number to 3366 and set the Method control to Windows shutdown - Forced. This example stops the targeted executable by telling to Windows to abort it. The instance with the port number 3366 will immediately close.

Finally, we want to test the remaining instance’s response to the Application Instance Close event. To do that, restart your computer now. (That’s right, restart your computer. Don’t stop anything, don’t shut anything down — just restart.) When your computer is restarted and you are logged back in, go to the directory where the test application is installed and open the trace file. You will see two lines that look something like this:

07:29:39 05/24/2015 -- Shutdown 3365 -- Just Stop
07:42:52 05/24/2015 -- Shutdown 3367 -- Appl Inst Close

The second line shows that when you restarted your computer Windows did in fact generate the Application Instance Close? event and the application caught the event. You’ll note that there is no entry for the instance with the port number 3366. Remember, we stopped it by forcing an abort and a Windows abort is very much like clicking the red abort button when LabVIEW is running a VI: It just stops. No orderly shutdown. No deinitialization.

A Small Test Executable — Release 1
Toolbox — Release 9

The Big Tease

So that was, I hope, interesting. Starting next time I’m going to start delving into how to use LabVIEW code as the data collection and control backend for an application that has as its only customer-facing interface a web site. While there are many companies offering options that claim to be developer-friendly I have found that many of the marketing claims are largely based on FUD (Fear, Uncertainty and Doubt). Simply put, they build up a “strong man” of supposed complexity and complication, and then tell you that the best (and perhaps, only) way to get past this obstacle is to buy their product. The truth, however, is that their “strong man” is really made of straw, and if you understand how it all fits together, doing it yourself isn’t really very hard.

Until Next Time…

Mike…

Managing Standalone Executables

Posted on by mlportersr

A while back when we were discussing opportunities for code modularization, I made the comment that it is even possible to incorporate processes into a new application that have already been compiled into standalone executables. At the time we didn’t have the time to go into detail on the point, but now we will take the time to consider the various tasks required to manage an existing standalone executable that you wish to leverage for a new application.

So What’s to Manage?

When I talk about managing an executable, I’m including all those tasks needed to control the executable from an operating system level. Therefore, to complete these tasks successfully we need a certain basic understanding of Windows and how it interacts with the programs that it is running. As you might imagine, most of these tasks will involve simply starting and stopping programs, but there are nuances that are important too. For example, when I double-click on a program the first time, Windows will launch it, but what if I double-click on it again? In this situation Windows has two possible options: launch another instance of the program or activate (bring to the front) the instance that is already running. One of the things that we will look at is how to specify that action for the executables we create,

Likewise, sometimes it isn’t enough to simply launch a program, sometimes we need to take high level look at its operation to determine such things as: How many instances of it are currently running or how much memory it is using. This information can help us decide how to best husband the computer resources that our application as a whole uses. Consequently, we will spend a little time considering the alternatives.

Finally, there can even be subtleties when it comes to stopping an executable. Can we simply signal the executable that it needs to stop, or do we need to force it to stop Now.

Getting Things Started

The first thing we want to cover is how to programmatically launch standalone applications. So to state the obvious, this topic is first and foremost about how to tell the operating system to launch the application for us. Luckily, this task is easily handled by the built-in System Exec.vi. This function allows you to do from within LabVIEW anything you would do from a command prompt — though at times the operations aren’t done in the exact the same way. In fact, for some older versions of Windows the differences can be significant. But even in Windows 8 there can be small inconsistencies. For example, say you have a program called Small Test Executable.exe that can accept a couple of command-line arguments like p3365 fpd2. You could launch this program from the command line like so:

C:\"Program Files\Test Executable\Small Test Executable.exe" -- p3365 fpd2

Unfortunately, this doesn’t work with System Exec.vi, which simply returns an (undocumented) error. To get the command to work you have to either write it to a temporary batch file and then execute the batch file, or modify the command like this:

"C:\Program Files\Test Executable\Small Test Executable.exe" -- p3365 fpd2

Don’t see a difference? Check out the location of the first double quote… To simplify dealing with these variations, I have created a subVI to encapsulate the functionality called Launch Executable w-Command Line Parameters.vi and included it in the new release of the toolbox.

Launch EXE with Parameters

The other thing about getting an executable running is that there are a few important launch parameters that we can only set in the application INI file, so we’ll deal with those next.

A Windows Convention

One of the conventions that Windows imposes is that the INI for an executable will have the name as the executable (but with the file extension ini) and the standard configuration parameters will reside is a section with the same name as the file (but minus the file extension). It is in this section that the runtime engine expects to find the parameters that it needs to successfully launch the executable.

The first such parameter we need to discuss controls the behavior when the executable is called more than once. By default, when you create an executable with LabVIEW only one copy of it will launch. If you call it again, Windows will simple bring to the front the one instance of the program that is already running. However, you can change this behavior by including this line in the INI file:

AllowMultipleInstances = TRUE

With this parameter set as shown, Windows will launch another instance of the program each time it is called. An important point to remember, however, is that no matter how many instances you launch, they all share the same INI file. Consequently, if you want to pass unique parameters to each instance, you will have to do so through the command line when the instances are launched.

The second INI file parameter I want to mention controls the program’s visibility to the computer user. Regardless of whether its GUI is visible, when a program normally launches it has a tab that appears on the taskbar. But what if you are creating a program that is intended to run unseen in the background? This setting handles that case.

HideRootWindow = True

When this setting is true, the program can still have a user interface but there is no tab associated with its window on the taskbar. As a side effect, the program also doesn’t appear in the task manager as an “App”, but rather lists it as a “Background Process”. Please note that this terminology is what Windows 8.x uses, older versions make the same distinction but use the word “Programs” and “Processes” to describe it.

The third INI file setting to discuss, is really not so much of a setting, as it is a family of settings that work together to control the program’s ability to respond to external connections via TCP/IP and VI Server.

server.tcp.enabled=True
server.tcp.port=3363
server.tcp.serviceName=""
server.tcp.acl="290000000A000000010000001D00000003000000010000002A10000000030000000000010000000000"
server.vi.access="+*"
server.vi.callsEnabled=True
server.app.propertiesEnabled=True
server.vi.propertiesEnabled=True
server.control.propertiesEnabled=True

Some of these settings are easy to understand, while others are rather obtuse. For right now, don’t worry about what these settings all mean. Next week we will.look at these parameters in detail.

Passing Parameters

To reiterate a point I made earlier, creating a standalone executable that will support multiple instances pretty much necessitates the use of command line arguments. In some ways, this situation is analogous to the situation concerning reentrant clones. The instance needs to be able to identify itself in the midst of a cloud of identical instances.

Given that point, one of the things you should consider when creating such an application is how the program should respond if the required command line parameters are missing. The answer to that question depends, to a large extent, on the nature of the parameters. If the input is essentially a customization for which there is a valid default value, it might be acceptable to simply accept that default and go on. However, in some situations there is no default value possible, so you should consider shutting down the program if the command line parameter is invalid or missing.

Getting Status

After we have gotten the instance (or instances) of a program up and running, one of the things that we might want to do is check on how, and perhaps what, it is doing. For now we will consider this matter in terms of the performance aspects that are visible to Windows. It is certainly possible to access the executable programmatically to obtain internal status information, and even control its operation, but that discussion will have to wait until next week.

General Information

One of the most basic indicators of the health of a program is its memory usage. You should expect the memory that a program has allocated will rise and fall over time. It should not treat trend steadily upwards without regard for what the program is actually doing. If a program’s memory consumption does consistently increase over time, that condition is referred to as a “memory leak” and it is a condition that needs to be addressed. Typically this remediation takes the form of figuring out why the program wants ever-increasing amounts of memory (like programming errors resulting in file or I/O being left open) and fixing the problem. Occasionally however, you will find that the leak is occurring in a place where you have no visibility — like inside LabVIEW itself. In those situations all you can do is try forcing Windows to deallocate the memory or change the way you are doing something so the problem doesn’t arise.

A simple way to read memory usage is to call the simple command line function tasklist and parse the result. Here is the code I wrote to do the job. Its name is Read Task List.vi.

Read Task List

If you leave the Image Name control empty, the call returns information on all the tasks currently running. However, if you populate that control with the name of a program, it returns a list of all the programs with that name. For example, on my computer right now LabVIEW.exe returns 1 item, while chrome.exe returns 13. (Why does Chrome need 13 instances of itself running in the background?)

The two most valuable pieces of information that you can get from this command is the application’s memory usage (expressed in kilobytes) and its PID. The PID is a numeric tag that uniquely identifies each program that is currently running. It is important because other commands or system calls will often require the PID of the program that you are wanting to check.

Digging for the Details

Beyond this basic information there is a variety of other details that you can find, For example, here you can find a community-developed VI that makes direct calls to a .NET assembly in order to return a plethora of information about your computer in general, and the programs that are running in particular. From the standpoint of accessing this information programmatically, this VI has two big problems:

  1. The code is at least 7 years old: This is a problem because there are places where the same operations could be done much more efficiently using techniques that LabVIEW has introduced in the intervening time.
  2. It’s essentially undocumented: To figure out how to really use it you will have to spend time researching the calls and the specific parameters — which can vary from one version of Windows to the next.
  3. It was designed for manual operation: In other words, it was designed to be used interactively, and not programmatically.

Because we need something that is usable from within a program, I created two subVIs that essentially repackage the functionality in a more program-friendly form. The first subVI is called Get Available Instances.vi. To see why this VI is needed, think back to the output of the tasklist function. In that output multiple instances of the same executable (like the 13 copies of chrome) all had the same name. They were distinguished from one another by a numeric id number called a PID. The designers of the .NET interface must have thought this approach confusing because they took a different approach that did away with the PID in favor of a name that was modified to make it unique — as in chrome through chrome#12. Likewise, any file extension was also dropped from the name.

Get Available Instances

The other subVI (Get Task Performance.vi) accepts as inputs an instance name and an enumeration that lists the available performance parameters that the VI can fetch under Windows 8.

Get Task Performance

But what is the deal with the loop and all the “extra” logic? This loop is needed because, while some of the parameters are instantaneous values that can be read at any time, others are not. As a case in point, consider the processor usage measurements. Everyone knows (or at least everyone in this business should know) that a CPU can really only do one thing at a time. In fact, the illusion that is central to much of modern technology is the programmatic slight of hand that makes it look like computers are doing multiple things at once.

Now since a CPU can only do one thing at a time, logically there are only two possible immediate answers to the question of how much processor are you’re using. Either you are not executing (in which case you have 0%) or you are executing (resulting in 100% usage). In short, for this type of measurement to have any sort of meaning it has to be approached as a statistic, not an absolute measurement. In the VI, this task is implemented by making two calls to the same measurement counter. The first call — which always returns a 0 — starts the measurement process, with the second and subsequent reads returning the statistical results for the previous measurement period.

Oh, and the logic for stopping the loop after just one iteration? That’s to optimize operation for parameters that are absolute.

Stopping: Fast and Half-Fast

It has been said that, “All good things must come to an end” and this is true for programs as well. However Windows actually provides two different types of shutdown events. The first causes Windows to simply abort the targeted program. This approach, while very fast, is logically equivalent to clicking the abort button on a LabVIEW program — and carries with it all the same dangers. The other type of shutdown event takes longer, but is much safer. It basically sends a message to the targeted program that requests it to stop. However, to reiterate something we learned before, for this messaging to work properly the executable must be written such that the Application Instance Close event will always initiate an orderly shut down.

In addition to using these messages for its own purposes, Windows also provides a command line function that allows users and individual programs to generate these messages as well. I implement both types of shut down requests in the subVI Task Killer.vi.

As a polymorphic VI, the routine incorporates instances that can shut down processes by name and by PID. You simply pick the version that gives you the level of control that you need. For instance, if I wanted to shut down all 13 instances of chrome, using the name version of the code would get the job done all at once. However, if I wanted to stop just a few select instances, I would use the version that uses the PID input. Here is the name version of the code.

task killer

But why start and stop at all?

Before ending this time, there’s one more idea that we should consider. To this point we have been thinking about the management of executables in a rather conventional sort of way: Start something just before we use it and close it when we are done. But might not there be situations where the “conventional” doesn’t make sense? I would assert that there are more than you might at first imagine.

The whole point of interacting with a process that is deployed as a separate standalone executable is that it obviously implements some bit of functionality that more than one application would need. Given that fact, starting and stopping it — especially stopping it — can get complicated. Just because one application is stopping and no longer needs it, that doesn’t mean there might not still be another application that does. One obvious solution to this problem is to simply let Windows handle it all. Install the background task such that it starts when either the computer boots, or the user logs in; and then stops when Windows shuts down. This solution might seem wasteful, but what is the real impact of such a solution? Oh sure, the program will be in memory, but if it isn’t being used it will probably be shuffled off to the page file pretty quickly. Likewise, if the executable is written correctly, it will be event driven which means that unless it is accessed by a program that is using it, the process won’t be burning up any CPU cycles either — so what is there to lose?

Just something to be thinking about.

Toolbox Release 8

The Big Tease

Ok, this is all for now. Next time I’ll introduce a little executable that will let us practice some of the theory we have examined today — and expand the conversation to include how to use VI Server to interact with executables created in LabVIEW.

Until Next Time…

Mike…

Expanding Data Processing Bandwidth — Automatically

Posted on by mlportersr

Well-written software can typically deal with any performance requirement pretty easily as long as the requirement is constant. It’s when requirements change over time that things can get dicey. For example, if your test system generates a new data packet to process every second and it consistently takes 5 seconds for the data to be processed, a little simple math will tell you how much processing bandwidth you need to create to keep up with the flow of data. But how are you to properly size things when variability is inserted into the process? What if the time between data packets can vary between 100 msec and several minutes? Or what if the data processing time can change dramatically due to things like network traffic?

These are the kind of situations where the processing needs to be more than simply “flexible”, it has to be able to automatically maintain its own operation and reconfigure itself on the fly. To demonstrate one possible implementation of this “advanced” technique, we will build on the simple pieces that we have learned in the past. In some ways, good software design techniques are like Lego blocks. Each one by itself is not very impressive, but when you stick them together, magic happens. But before we can stick anything together, we need to understand…

…what we’re going to do.

You’ll notice that any of the bandwidth management challenges that I mentioned earlier can be addressed by either adding more data processing clones, or removing existing ones that are being underutilized. Consequently, the question of how to implement this self-maintenance functionality really gets down to a matter of how to dynamically manage the number of data processing clones that are currently available — which in turn boils down to answering two very simple questions:

1. How do we know we need more?

Given that the whole point of the exercise is to manage a queue, the current state of that queue will give us all the information that we need to answer this question. Specifically, we can know when more processing bandwidth is needed by monitoring how many items are currently in the queue waiting processing. When the code starts to see the depth going steadily up, it can launch additional processes to handle the data backlog. Of course, this functionality assumes that there is a process that is constantly monitoring the queue and managing that aspect of its operation — which we actually have already in the test code from last week (Data Processing Queue Handler.vi). All we have to do is repurpose this VI to be a permanent part of the final application.

2. How do we know a clone is no longer needed?

The one part of the system that knows whether a clone is being under-utilized is, in fact, the clone itself. As a part of its normal operation, it knows and can keep track of how often is it being used. Having said that, there are (at least) a couple of ways to quantify how much a clone is being utilized. We could, for instance, consider how much time the clone is spending processing data versus how much time it spends waiting to receive data to process. If the utilization percentage drops below a given limit, the clone could then shut itself down. However, for this demonstration, I’m going to use a much simpler criteria that, quite frankly, works pretty well. The code will simply keep track of how many times in a row it goes to the queue and doesn’t find any data.

Code Modifications

Before I start describing the changes that will we will need to make in order to fashion this new ability, I want to consider for a moment the thing that won’t have to change: Queue Test.vi. You might be tempted to say, “Well big deal. All it does is stuff some data into the queue every so often. Who cares if it doesn’t have to change? It’s not even deliverable code”

While that is undoubtedly true, the fact of the matter is that this test routine is important, but not because of what it is or what it does. Rather we care about Queue Test.vi because in our little test environment, it represents the rest of our application — or at least that part of it that is generating data. Consequently, the fact that it doesn’t need modification means that your main application, likewise, won’t need modification if you decide to upgrade from a data processing environment that uses a fixed number of data processors to one that dynamically manages itself.

Data Processing Queue Handler.vi

First, note that previously this routine’s primary job was to simply report how deep the data queue was — a bit of functionality that would likely have not been needed in a real application. Now however, this routine is going to be taking an active part in the process, so I started the modifications by adding the error reporting VI that will transfer errors it generates to the exception handler.

New Queue Handler Timeout Case

In addition, because the software will initially only start a single data processing clone, I also modified the timeout event handler that performs the VI’s initialization, by removing the loop around the clone launching subVI.

The remainder of the modifications to this routine occurs in the event handler for the Check Queue Size UDE. Previously this event only reported how deep the queue was getting. While it still performs that function, that queue depth information in addition now drives the logic that determines whether we have enough processing bandwidth online.

New Queue Handler Queue Size Check Case

Note that the queue depth is compared to a new configuration value called Max Queue Size that defines how large the queue can grow before a new data processing clone is launched. Regardless of whether it launches a new clone, event handler calls another new subVI that returns the number clones that are currently running. As you will see in a moment, one of the modifications to the data processing VI is the addition of logic that keeps track of the names of the clones that are running. The subVI that we are calling here returns a count of the number of names that have been recorded so far.

Data Processor.vi

Turning now the data processing code itself, the first stop is in the state-machine’s Initialize state. Here we have all the same logic that existed before, but with a couple minor additions

New Data Processor Initialize

First, there is a new subVI that registers a clone is starting up. This subVI writes the clone’s name to the FGV that is maintaining the clone count. Second, there is also a new shift register carrying a cluster of internal data that clone will need to do its work. All that is needed during initialization is to set a timestamp value. The Check for Data state is next and it has likewise seen some tweaks — the most significant of which is moving the logic for responding to the dequeue operation into a subVI.

New Data Processor Check for Data

The justification for this move lies in the fact that this logic is now also responsible for determining whether or not the clone is being adequately utilized. As I stated before, each time the clone goes to the queue and comes up empty, the logic will increment a counter that is being carried in the new shift register’s data. If this count exceeds a new configuration value Clone Idle Count, the code will branch to a new state that will shutdown the clone. Likewise, anytime the clone does get data to process, it will reset the count to 0. The changes to the Process Data state, which comes next, are pretty trivial.

New Data Processor Process Data

All that happens here is that the timestamp extracted from the data to be “analyzed” updates the new cluster data — as well as the indicator on the front panel. Finally, there is the new state: Self Shutdown…

New Data Processor Self Shutdown

…which simply calls a subVI that removes the clone’s name from FGV maintaining a list of all running clones, and stops the event loop.

Let’s talk about “Race Conditions”

All we have left to do now is test this work and see the differences that it makes, but before we can do that, we need to have a short conversation about race conditions. Very often developers and instructors (myself included) will talk about the necessity of avoiding race conditions. The dirty little secret is that as long as you have multiple things happening in parallel, race conditions will always be present. The real point that these admonitions attempt to make is that you should avoid the race conditions that are unrecognized and potentially problematic.

I bring this point up because as you do the following testing, you may get the chance to see this concept in action. The way it will appear is that the system will launch a new clone immediately after one kills itself off for being underutilized. The reason for this apparent logical lapse is that a race condition exists between the part of the code that is checking to see if another clone is needed and the several places where the clones are deciding whether or not they are being used. There are two causes for this race condition, one we can ameliorate a bit and one over which we have no possible control.

Starting with the cause we can’t control, a simple immutable law of nature is that no matter how sophisticated our logic or algorithms might be, they can not see so much as a nanosecond into the future. Consequently, the first source of a race condition is that when the queue checking logic sees that there are three elements enqueued, it has no way of knowing that a currently active clone will be available in a few milliseconds. While it is true that under certain circumstances it might be possible to provide this logic with a bit of “foresight”, there is no generalized solution to this aspect of the problem. Consequently, this is an issue that we may just have to live with.

The news, however, is better for the second cause. Here the problem is that with all the clones having the same timeout between data checks, it is probable that sooner or later one of the clones is going to become “synchronized” with the others such that it is always checking the queue just after it was emptied by one of the others. However the solution to this problem lies in its very definition. The cure is to see to it that the clones do not have constant timeouts from one check to the next. To implement this concept in our test code I modified the routine that returns the delay to add a small random difference that changes each time it’s called.

The bottom line is that while completely removing all race conditions is not possible, they can be managed such that their impacts are minimized.

New Tests for New Code

The testing of the modified code starts the same as it did before: open and launch Data Processing Queue Handler.vi and Queue Test.vi. The first difference that you will notice is that only 1 clone is initially launched, but at the default data rate, 1 clone is more than adequate.

Now decrease the delay between data packets to 2 seconds. Here the queue depth will bounce around a bit but the clone count will stabilize at 3 or 4.

Finally, take the delay all the way down to 1 second. Initially the clone count may shoot up to 8 or 9, but on my system the clone count eventually settled down to 6 or 7.

At this point, you can begin increasing the delay again and the slowly the clones will start dropping out from disuse. Before you shutdown the test, however, you might want to set the delay back to 2 seconds and leave the code running while you go about whatever else you have to do today. It could be instructive to notice how other things you are doing on the same computer effect the queue operation. You might also want to rerun the test but start Queue Test.vi first and let it run for a minute or so before you startData Processing Queue Handler.vi — just to see what happens.

Further Enhancements?

So we have our basic scalable system completed, but are there things we could still do to improve its operation? Of course. For example, we know that due to timing issues which we can only partially control, the number of clones that is running at one time can vary a bit, even if the data rate is constant. One thing that could be done to improve efficiency would be to change the way clones are handled. For example, right now a data processor is either in memory and running or it is closed. One thing you could do is create a new state that a clone could be in — like loaded into memory, but inactive. You could implement this zombie state by setting the timeout to 0, thus effectively turning off the state machine.

It might also be helpful to change to queue depth limit at which a new clone is created by making it softer. Instead of launching a new clone anytime the queue depth exceeds 3, it might be useful in some situations to maintain a running average and only create a new clone if the average queue depth over the last N checks is greater than 3.

Who knows? Some of you might think of still other modifications and enhancements. The point is to experiment and see what works best for your specific application.

Parallel Data Processing — Release 2

The Big Tease

So what is in store for next time? Well in the past we have discussed how to dynamically launch and use VIs that run as separate processes. But what if the code you want to access dynamically like this happens to exist in a process that you have already compiled into a standalone application? If this application is working you don’t want to risk breaking something by modifying it. As it turns out there are ways to manage and reuse that code as well, even if it was created in an older version of LabVIEW. Next time we’ll start exploring how to do it.

Until Next Time…

Mike…

Expandable Data Processing

Posted on by mlportersr

When creating an application a common approach to managing application computer bandwidth is to structurally isolate the portions of the application that are doing the data processing from those implementing the data acquisition. The goal of this segmentation is to prevent the operation of the one from impacting the speed of the other. But if we think about what we learned from the posts preceding this one, we can see that this use case is just a specialized case of a general principle. Namely, that allowing processes to run in parallel reduces the likelihood that their execution will impede the operation of their peers by improving the overall utilization of the computer’s resources. Consequently, it behooves us to consider how we can allow data processing to run in parallel with the rest of the application’s code — and so garner a few of the many advantages that this approach offers.

Parallel Data Processing

It first should be noted that we already have most of the tools and concepts that we will need to build this parallel data processing capability. In fact, one possible implementation is not so different from our general approach to building state machines. So with that idea in mind, let’s consider our perspective data processor’s high-level functional requirements.

Just the One?

First, since the idea is to improve efficiency by allowing more things to happen in parallel, we need to ask ourselves the obvious question, “Why stop at just one data processor?” This question is particularly urgent if you already know that your data processing takes longer than the time required to acquire another dataset. It is an appealing idea to have 2 or more data processors that can share the processing load. However, if you are going to be running multiple instances simultaneously, the process has to be reentrant. But that’s not a problem, we’ve done reentrant processes before and know how to make that work.

State-ly Expandability

Second, we need to consider the data processor’s internal structure. I have given this point away already by talking about state-machines, so let’s consider why this choice is a good one. At first glance it might seem like using a state-machine might be overkill for this application, after all you might figure that there is at most 2 states: Wait for Data and Process Data. This line of reasoning is valid, but it assumes that the data processing is a one-step process, so are there circumstances where this assumption isn’t valid?

If your processing is going to involve the complex analysis of potentially large datasets, you might not want to spend a lot of time processing flawed or invalid data. Hence before jumping right into the full-blown analysis it would be reasonable to have a state that does a quick sanity check on the data — and then another to handle the error that results if the data is bad. Next, consider what you are doing with the data when you are done with the analysis. There could be added limit checking of the results, and perhaps transfer of the results to a database or automated report generator — all tasks that could (and probably should) be expressed as separate states. Looking back now, by my count we are now up to 6 states — and we are still assuming that the analysis process itself is a single atomic operation that you will not want or need to breakup.

My point is that if you start with the basic outline of a state-machine and end up never having more than the two states, there’s no harm done because that basic structure is pretty lean. However if, as your project progresses, you discover that more states are needed, having that infrastructure already be in place can save a lot of time.

By the way, as a quick aside, I have often had the experience of having a customer request a significant change and then have them be amazed in how little time it takes to implement the modification. I have even had customers comment on how “lucky” I was that the code was structured in such a way that the change was even possible. Well let me tell you in the strongest terms possible that if luck was involved, it was luck of my own making. The better you design your code up front, the more “lucky” you will be when the time comes to make modifications.

The Right Kind of Communications

Third, we have to look at how we are going to communicate with these cloned state-machines. In past work on our testbed application, we have used UDEs and FGVs for communications between processes, and they might work here as well, but there would be problems. If you have one UDE or FGV feeding data to all the cloned data processors, you would have the problem of deciding which clone handles the each new update. While you don’t want to miss any updates, you certainly don’t want to process the same dataset twice either. The whole process would be fraught with opportunities for errors and race conditions. Of course you could solve that problem by creating a separate UDE or FGV for each clone, but then scalability goes down the tubes as the process(es) generating the data must keep track of how many clones there are any which one will get the next dataset.

Our way out of this conundrum is to properly apply a structure that people so often misuse: The Queue. This is the use case for which queues were born. The key feature that makes them so good in this sort of situation, is that if you have multiple receivers waiting to dequeue data from the same queue, LabVIEW guarantees that each new item inserted will only be seen by one of the receivers, and that enqueued data will be distributed to all the receivers on a first come, first served, basis. Moreover, because queues can be named, different processes running in the same instance of LabVIEW don’t need the queue reference to be sent to them. All they need to know is the name of the queue and they can get their own reference.

Yes, queues do still require polling, but at least there’s only one part of your code involved, and not the whole application. Moreover, there are two other mitigating factors present. First, in this sort of application the poll rate can often be much slower, on the order of once every few seconds. Second, because the state machine resides inside an event structure, it would take but a moment to implement logic that would allow the polling to be turned off altogether.

Let’s Look at the Test Code

As we start considering the code to implement this functionality, let’s first look at the test routines that we will use to evaluate the queue functionality. The VI Data Processing Queue Handler.vi has the responsibility of starting everything off by initializing the data processing queue and launching a predetermined number of data processing clones. This logic takes place in the VI’s Timeout event handler.

Queue Handler Timeout Case

You will notice that to simplify the process of obtaining a queue reference, I have encapsulated that logic in a set of subVIs for interacting with the queue. The other event of significance is the handler for the Check Queue Size UDE. While a test is running, we want to be able to monitor the number of items in the queue, but rather than simply polling the queue status, I created a UDE that flags the handler every time a new item is enqueued.

Queue Handler Queue Size Check Case

When the event fires, the handler calls the built-in queue function that returns, in addition to some other stuff that we don’t need, the number of items that are currently in the queue. Next, to feed data to the queue, I created a second test VI called Queue Test.vi. It’s whole job is to wait a delay period specified on the front panel, and then enqueue an item into the data processing queue.

Queue Test

You can see that at this point, the only value in the queue data is a timestamp, but the data is defined as a typedef. Remember! Any time you are creating a datatype that will be accessed by reference, whether it be a UDE, a queue, a notifier or an event, always make the data structure a typedef.

After the data value is enqueued, the code fires the event that tells the handler to check the queue size.

Introducing the Data Processor

Turning finally to the reentrant data processing state-machine (Data Processor.vi) we see that in addition to an event for stopping, the VI’s Timeout event handler includes the logic for three states, the first of which is Initialize.

Data Processor Initialize

This state’s job is to get the clone ready to start processing data. Consequently, it initializes the shift register holding the queue reference, and a second shift-register carrying a boolean value that we will discuss in a moment. The state also sets the next state to be executed to Check for Data, and retains a timeout value of 0 ms so, assuming that there are no errors during initialization, the state machine will immediately start waiting for data to process. Note that the Initialize state also opens the VI’s front panel. You probably would not want this feature in deliverable code except as, perhaps, a debugging option. I have included it here to make it easier for you to see the code at work.

Data Processor Check for Data

The Check for Data state starts by calling the queue subVI that is responsible for dequeuing an item. Inside this subVI, the dequeuing function is given a timeout of 0 ms so if there is not any data immediately available, the call will terminate with the timeout flag asserted. This Boolean value is inverted and passed out of the subVI to indicate to the calling code whether there is any data that needs processing. If the Check for Data state logic finds this bit set, the code sets the next state for execution to Process Data and sets the timeout value of 0 ms. If the bit indicating that data is available is not set, the code retains Check for Data as the next state to execute but sets the timeout to a longer value (5 sec) read from the application’s INI file.

Before we go on to talk about the Process Data state, we need to have a quick conversation about the boolean shift register. Normally, when the standard Stop Application event fires, a VI wants to immediately stop what it’s doing and abort. However in one significant way, this is not a “normal” VI. In order to protect the data that has been acquired, this process should only stop if the queue driving it is empty. To create that functionality, the VI incorporates deferred shutdown logic in the form of this shift-register. Because the value is initialized to false the loop will, during normal operation, continue regardless of whether data is available or not. However, when a shutdown is requested, the event handler does not immediately stop the loop, but instead sets the shift-register value to true and branches to the Check for Data state with a 0 ms timeout. If the queue is empty, the process will end at that time. However, if the queue is not empty, the VI will continue toggling between the Check for Data and Process Data states until the queue contents are exhausted.

Data Processor Process Data

As you would expect, the Process Data state basically consists of processing the last data dequeued and branching back to the Check for Data state to look for more. However, given that the only data in our test queue is a timestamp, you have probably guessed that the actual data processing to be done isn’t very expansive — and you’re right. In fact, the “data processing” consists largely of a wait, the duration of which varies at random between 4 and 6 seconds.

Putting it to the Test

To test this code, open and run Data Processing Queue Handler.vi. You will immediately see the front panels of three data processing clones open. Move them so they aren’t overlapping each other or anything else.

Now open and run Queue Test.vi. After a few seconds it will enqueue an item and then enqueue a new one every 6 seconds. Note that as each clone handles an item it will display that item’s timestamp on its front panel. Note also that the indicated queue depth never exceeds 1.

Now change the delay on Queue Test.vi from 6 sec to 2 sec. You will notice that the queue depth chart is now updating faster. Likewise, the queue depth will begin to show momentary increases to a depth of two, but the chart will always drop back to 1. In other words, there might be slight delays now and again, but for the most part three clones can keep up with the flow of data.

Finally, drop the delay to 1.5 seconds. With data coming at this rate, the queue depth will continue to go up and down, but now it is always going up more than it is going down. This overall upward trend shows us that we have reached the point where three clones are getting overwhelmed by amount of data that is being enqueued.

Queue Overrun

Now if you increase the delay back to 2 seconds, the queue depth will gradually begin to decrease as the slower flow of new data allows the clones to begin catching up on the backlog. Alternatively, if you just click the stop button, the two test VIs will stop and close immediately, but the clones will continue running until the queue empties out.

Parallel Data Processing — Release 1

The Big Tease

So we have learned the basics behind creating an environment for an application that supports an expandable data processing capability. For many applications this simple structure will be more than adequate, but (as we have seen) if the data starts coming too fast the queue can grow without limit. Of course this isn’t necessarily a problem if the periods of high data generation are interspersed with periods of comparative idleness. However this sort of variability can be a bit of a two-edged sword. The periods of low data throughput can give the system time to recover from a large backlog of data, but this variability can also make it difficult to estimate how many copies of the data processing process will be needed. Pick a number that is too high, and you’re wasting computer resources. Pick a number that is too low and you could still end up with a situation where too much data is being queued — perhaps to the point of running out of memory.

Well, the next time we get together we’ll look at how to modify the basic structure we have created thus far to add the ability for the software to decide on the fly when more clones are needed, and when to kill off existing ones that aren’t being used.

Until Next Time…

Mike…