It’s a big interconnected world – and LabVIEW apps can play too

If I were asked to identify one characteristic that set modern test systems apart from their predecessors, my answer would be clear: distributed processing. In the past, test applications were often monolithic programs – but how things have changed! In the past we have talked several times about how the concept of distributed processing has radically altered our approach to system architecture. In fact, the internal design of our Testbed Application is a very concrete expression of that architectural shift. However, the move away from monolithic programs has had a profound impact in another area as well.

In the days of yore, code you wrote rarely had to interact with outside software. The basic rule was that if you wanted your program to do something you had to implement the functionality yourself. You had to assume this burden because there was no alternative. There were no reusable libraries of software components that you could leverage, and almost no way to pass data from one program to another. About all you could hope for were some OS functions that would allow you to do things like read and write disk files, or draw on the screen. In this blog we have looked at ways that our LabVIEW applications can use outside resources through standardized interfaces like .NET or ActiveX. But what if someone else wants to write a program that uses our LabVIEW code as a drop in component? How does that work?

As it turns out, the developers saw this possibility coming and have provides mechanisms that allow LabVIEW application to provide the same sort of standardized interface that we see other applications present. Over the next few posts, we are going to look at how to use incorporate basic remote interfaces ranging from traditional TCP/IP-based networking to building modern .NET assemblies.

What Can We Access?

However, before we can dig into all of that, we need to think about what these interfaces are going to access. In our case, because we have an existing application that we will be retrofitting to incorporate this functionality, we will be looking at the testbed to identify some basic “touchpoints” that a remote application might want to access. By contrast, if you are creating a new application, the process of identifying and defining these touchpoints should be an integral part of you design methodology from day one.

The following sections present the areas where we will be implementing remote access in our testbed application. Each section will describe the remote interface we desire to add and discuss some of the possible implementations for the interface’s “server” side.

Export Data from Plugins

The obvious place to start is by looking at ways of exporting the data. This is, after all, why most remote applications are going to want to access our application: They want the data that we are gathering. So the first thing to consider is, where does the data reside right now? If you go back and look at the original code, you will see that, in all cases, the primary data display for a plugin is a chart that is plotting one new point at a time. Here is what the logic looked like in the Acquire Sine plugin.

Simple Acquisition and Charting

As you can see, the only place the simulated data goes after it is “acquired” is the chart. Likewise, if you looked at the code for saving the data, you would see that it was getting the data by reading the chart’s History property.

Save Chart Data to File

Now, we could expand on that technique to implement the new export functionality, but there is one big consequence to that decision. Approaching the problem in this way would have the side-effect of tying together the number of data points that are saved to the chart’s configuration. Hence, because the amount of data that a chart buffers can’t be changed at runtime, you would have to modify the LabVIEW code to change the amount of data that is returned to the calling application.

A better solution is to leverage what we have learned recently about DVRs and in-place structures to create a storage location the size of which we can control without modifying the application code. A side-effect of this approach is that we will be able to leverage it to improve the efficiency of the local storage of plugin data – yes, sometimes side-effects are good.

To implement this logic we will need three storage buffers: One for each of the two “acquisition” plugins and one for the reentrant “temperature controller” plugin. The interface to each storage buffer will consist of three VIs, the first one of which is responsible for initializing the buffer:

Initialize Buffer

This screenshot represents the version of the initialization routine that serves the Ramp Signal acquisition process. The basic structure of this code is to create a circular buffer that will save the last N samples – where “N” is a reconfigurable number stored in the database. To support this functionality, the DVR incorporates two values: The array of datapoints and a counter that operates as a pointer to track where the current insertion point is in the buffer. These VIs will be installed in the initialization state of the associated plugin screen’s state machine. With the buffer initialized, we next need to be able to insert data. This is typical code for performing that operation:

Insert Data Point

Because the DVR data array is initialized with the proper number of elements at startup, all this logic has to do is replace an existing value in the array with a newly acquired datapoint value, using the counter of course to tell it which element to replace. Although we have a value in the DVR called Counter we can’t use it without a little tweaking. Specifically, the DVR’s counter value increments without limit each time a value is inserted, however, there is only a finite number of elements in the data array. What we need for our circular buffer is a counter that starts at 0, counts to N-1 and then returns to 0 and starts over. The code in the image shows the easiest way to generate this counter. Simply take the limitless count and modulo divide it by the number of points in the buffer. The output of the modulo division operation is a quotient and a remainder. The remainder is the counter we need.

Modulo division is also important to the routine for reading the data from the buffer, but in this case we need both values. The quotient output is used to identify when the buffer is still in the process of being filled with the initial N datapoints:

Read All Data.1

During this initial period, when the quotient is 0, the code uses the remainder to trim off the portion of the buffer contents that are yet to be filled with live data. However, once the buffer is filled, the counter ceases being the a marker identifying the end of the data, and it becomes a demarcation point between the new data and the old data. Therefore, once the quotient increments past 0, a little different processing is required.

Read All Data.2

Once the circular buffer is full, the element that the remainder is pointing at is the oldest data in the array (chronologically speaking), while the datapoint one element up from it is newest. Hence, while the remainder is still used to split the data array, the point now is to swap the two subarrays to put the data in correct chronological order.

Retrieve Graph Images from Plugins

The next opportunity for remote access is to fetch not the data itself, but a graph of the data as it is shown on the application’s GUI. This functionality could form the basic for a remote user interface, or perhaps as an input to a minimalistic web presentation. Simplifying this operation is a control method that allows you to generate an image of the graph and the various objects surrounding it like the plot legend or cursor display. Consequently, the VI that will be servicing the remote connections only needs to be able to access the chart control reference for each plugin. To make those references available, the code now incorporates a buffer that is structurally very similar to the one that we use to store the VI references that allow the GUI to insert the plugins into its subpanel. Due to its similarity to existing code, I won’t cover it in detail, but here are a few key points:

  • Encapsulated in a library to establish a namespace and provided access control
  • The FGV that actually stores the references is scoped as Private
  • Access to the functionality is mediated though publicly-scoped VIs

This FGV is the only new code we will need to add to the existing code.

Adding Remote Control

One thing that both of the remote hooks we just discussed have in common is that they are both pretty passive – or to make this point another way, they both are monitoring what the application is doing without changing what it is doing. Now we want to look at some remote hooks that will allow remote applications control the application’s operation, at least in a limited way.

Since the way the application works is largely dependent upon the contents of the database, it should surprise no one that these control functions will need to provide provisions for the remote application to alter the database contents in a safe and controlled way.

Some Things to Consider

The really important words in that last sentence are “safe” and “controlled”. You see, the thing is that as long as you are simply letting people read the data you are generating, your potential risk is often limited to the value of the data that you are exposing. However, when you give remote users or other applications the ability to control your system, the potential exists that you could lose everything. Please understand that I take no joy in this conversation – I still remember working professionally on a computer that didn’t even have a password. However, in a world where “cyber-crime”, “cyber-terrorism” and “cyber-warfare” have become household terms, this conversation is unavoidable.

To begin with, as a disclaimer you should note that I will not be presenting anything close to a complete security solution, just the part of it that involves test applications directly. The advice I will be providing assumes that you, or someone within your organization, has already done the basic work of securing your network and the computers on that network.

So when it comes to securing applications that you write, the basic principle in play here is that you never give direct access to anything. You always qualify, error-check and validate all inputs coming from remote users or applications. True, you should be doing this input validation anyway, but the fact of the matter is that most developers don’t put a lot of time into validating inputs coming from local users. So here are a couple of recommendations:

Parametrize by Selecting Values – This idea is an expansion on a basic concept I use when creating any sort of interface. I have often said that anything you can do to take the keyboard out of your users’ hands is a good thing. By replacing data that has to be typed with data menus from which they can select you make software more robust and reduce errors. When working with remote interfaces, you do have to support typed strings because unless the remote application was written in LabVIEW, typing is the only option. But what you can do is limit the inputs to a list of specific values. On the LabVIEW-side the code can convert those string values into either a valid enumeration, or a predefined error that cancels the operation and leaves your system unaltered. When dealing with numbers, be sure to validate them also by applying common-sense limits to the inputs.

Create Well-Defined APIs – You want to define a set of interfaces that specify exactly what each operation does, and with as few side-effects as possible. In fancy computer-science terms, this means that operations should be atomic functions that either succeed or fail as a unit. No half-way states allowed! Needless to say, a huge part of being “well-defined” is that the APIs are well-documented. When someone calls a remote function, they should know exactly what is expected of them and exactly what they will get in response.

Keep it Simple – Let’s be honest, the “Swiss Army Knife” approach to interface design can be enticing. You only have to design one interface where everything is parametrized and you’re done, or at least you seem to be for a while. The problem is that as requirements change and expand you have to be constantly adding to that one routine and sooner or later (typically sooner) you will run into something that doesn’t quite fit well into the structure that you created. When that happens, people often try to take the “easy” path and modify their one interface to allow it to handle this new special case – after all, “…it’s just one special case…”. However once you start down that road, special cases tend to multiply like rabbits and the next thing you know, your interface is a complicated, insecure mess. The approach that is truly simple is to create an interface that implements separate calls or functions for each logical piece of information.

With those guidelines in mind, let’s look at the three parameters that we are going to be allowing remote users or applications to access. I picked these parameters because each shows a slightly different use case.

Set the Acquisition Sample Interval

One of the basic ways that you can store a set of parameters is using a DVR, and I demonstrated this technique by using it to store the sample rates that the “acquisition” loops use to pace their operation. In the original code, the parameter was already designed to be changed during operation. You will recall that the basic idea for the parameter’s storage was that of a drop box. It wasn’t important that the logic using the data know exactly when the parameter was changed, as long as it got the correct value the next time it tried to use the data. Consequently, we already have a VI that writes to the DVR (called Sample and, as it turns out, it is all we will need moving forward.

Set Number of Samples to Save

This parameter is interesting because it’s a parameter that didn’t even exist until we started adding the logic for exporting the plugin data. This fact makes it a good illustration of the principle that one change can easily lead to requirements that spawn yet other changes. In this case, creating resizable data buffers leads to the need to be able change the size of those buffers.

To this point, the libraries that we have defined to encapsulate these buffers each incorporate three VIs: one to initialize the buffer, one to insert a new datapoint into it, and one to read all the data stored in the buffer. A logical extension of this pattern would be the addition of a fourth VI, this time one to resize the existing buffer. Called Reset Buffer these routines are responsible for both resizing the buffer, and correctly positioning the existing data in the new buffer space. So the first thing the code does is borrow the logic from the buffer reading code to put the dataset in the proper order with the oldest samples at the top and the newest samples at the bottom.

Put the Data in Order

Next the code compares the new and old buffer sizes in order to determine whether the buffer is growing larger, shrinking smaller or staying the same size. Note that the mechanism for performing this “comparison” is to subtract the two value. While a math function might seem to be a curious comparison operator, this technique makes it easy to identify the three conditions that we need to detect. For example, if the values are the same the difference will be 0, and the code can use that value to bypass further operations. Likewise, if the two numbers are not equal, the sign of the result will indicate which input is larger, and the absolute magnitude of the result tells us how much difference there is between the two.

This is the code that is selected when the result of the subtraction is a positive number representing the number of element that are to be added to the buffer.

Add points to Buffer

The code uses the difference value to create an array of appropriate size and then appends it to the bottom of the existing array. In addition, the logic has to set the Counter value point to the first element of the newly appended values so the next insert will go in the correct place. By contrast, if the buffer is shrinking in size, we need to operate on the top of the array.

Remove points from buffer

Because the buffer is getting smaller, the difference is a negative number representing the number of elements to be removed from the buffer data. Hence, the first thing we need to do is extract the number’s absolute value and use it to split the array, effectively removing the elements above the split point. As before, we also need to set the Counter value, but the logic is a little more involved.

You will remember that the most recent data is on the bottom of the array, so where does the next data point need to go? That’s right, the buffer has to wrap around and insert the next datapoint at element 0 again, but here is where the extra complexity comes in. If we simply set Counter to 0 the data insert logic won’t work correctly. Reviewing the insert logic you will notice that the first pass through the buffer (modulo quotient = 0) is handled differently. What we need is to reinitialize Counter with a number that when subjected to the modulo division will result in a remainder of 0, and a quotient that is not 0. An easily derived value that meets that criteria is the size of the array itself.

Finally we have to decide what to do when the buffer size isn’t changing, and here is that code. Based on our discussions just now, you should be able to understand it.

buffer size not changing

Set Temperature Controller Operating Limits

Finally, there are two reasons I wanted to look at this operation: First, it is an example of where you can have several related parameters that logically form a single value. In this case, we have 5 separate numbers that, together, define the operation of one of the “temperature controller” processes. You need to be on the look-out for this sort of situation because, while treating this information as 5 distinct value would not be essentially wrong, that treatment would result in you needing to generate a lot of redundant code.

However, this parameter illustrates a larger issue, namely that changes in requirements can make design decisions you made earlier – let’s say – problematic. As it was originally designed, the temperature controller values were loaded when the plugins initialized, and they were never intended to be changed during while the plugin was running. However, our new requirement to provide remote control functionality means that this parameter now needs to be dynamic. When confronted with such a situation, you need to look for a solution that will require the least rework of existing code and the fewest side-effects. So you could:

  1. Redesign the plugin so it can restart itself: This might sound inviting at first because the reloading of the operating limits would occur “automatically” when the plugin restarted. Unfortunately, it also means that you would have to add a whole new piece of functionality: the ability for the application to stop and then restart a plugin. Moreover, you would be creating a situation where, from the standpoint of a local operator, some part of the system would be restarting itself at odd intervals for no apparent reason. Not a good idea.
  2. Redesign the plugin to update the limits on the fly: This idea is a bit better, but because the limits are currently being carried through the state machine in a cluster that resides in a shift-register, to implement this idea we will need to interrupt the state machine to make the change. Imposing such an interruption risks disrupting the state machine’s timing.

The best solution (as in all such cases) is to address the fundamental cause: the setups only load when the plugin starts and so are carried in the typedef cluster. The first step is to remove the 5 numbers associated with the temperature controller operating parameters from the cluster. Because the cluster is a typedef, this change conveniently doesn’t modify the plugin itself, though it does alter a couple of subVIs – which even more conveniently show up as being broken. All that is needed to repairs these VIs is to go through them one by one and modify the code to read the now-missing cluster data values with the corresponding values that the buffered configuration VI reads from the database. Said configuration VI (Load Machine also requires one very small tweak:

Reload Enable

Previously, the only time logic would force a reload of the data was when the VI had not been called before. This modification adds an input to allow the calling code to request a reload by setting the new Reload? input to true. To prevent this change from impacting the places where the VI is already being called, the default value for this input is false, the input is tied to a here-to-fore unused terminal on the connector pane, and the terminal is marked as an Optional input.

Building Out the Infrastructure

At this point in the process, all the modifications that need to be done to the plugins themselves have been accomplished, so now we need is a place for the external interface functionality itself to live. One of the basic concepts of good software design is to look at functionality from the standpoint of what you don’t know or what is likely to change in the future, and then put those things into abstracted modules by themselves. In this way, you can isolate the application as a whole, and thus protect it from changes that occur in the modularized functionality.

The way this concepts applies to the current question should be obvious: There is no way that we can in the here and now develop a compete list of the remote access functionality that users will require in the future. The whole question is at its essence, open-ended. Regardless if how much time you spend studying the issue, users have an inherently different view of your application than you do and so they will come up with needs that you can’t imagine. Hence, while today we might be able to shoe-horn the various data access and control functions into different places in the current structure, to do so would be to start down a dead-end road because it is unlikely that those modifications would meet the requirements of tomorrow. What we need here is a separate process that will allow us to expand or alter the suite of data access and control functionality we will offer.

Introducing the Remote Access Engine

The name of our new process is Remote and (like most of the code in the project) it is designed utilizing an event-drive structure that ensures it is quiescent unless it is being actively accessed. The process’ basic theory of operation is that when one of its events fire, it performs the requested operation and then sends a reply in the form of a notification. The name of the notification used for the reply is sent as part of the event data. This basic process is not very different from the concept of “callbacks” used in languages such as JavaScript.

Although this process is primarily intended to run unseen in the background, I have added three indicators to its front panel as aides in troubleshooting. These indicators show the name of the last event that it received, the name of the plugin that the event was targeting, and the name of the response notifier.

The Read Graph Data Event

The description of this event handler will be longer than the others because it pretty much sets the pattern that we will see repeated for each of the other events. It starts by calling a subVI (Validate Plugin that tests to see if the Graph Name coming from the event data is a valid plugin name, and if so, returns the appropriate enumeration.

Validate plugin name

The heart of this routine is the built-in Scan from String function. However, due to the way the scan operation operates, there are edge conditions where it might not always perform as expected when used by itself. Let’s say I have a typedef enumeration named Things I Spend Too Much Time Obsessing Over.ctl with the values My House, My Car, My Cell Phone, and My House Boat, in that order. Now as I attempt to scan these values from strings, I notice a couple of “issues”. First there is the problems of false positives. As you would expect, it correctly converts the string “My House Boat” into the enumerated value My House Boat. However, it would also convert the string “My House Boat on the Grand Canal” to the same enumeration and pass the last part of the string (” on the Grand Canal”) out its remaining string output. Please note that this behavior is not a bug. In fact, in many situations it can be a very useful behavior – it’s just not the behavior that we want right now because we are only interested in exact matches. To trap this situation, the code marks the name as invalid if the remaining string output is not empty.

The other issue you can have is what I call the default output problem. The scan function is designed such that if the input string is not scanned successfully, it outputs the value present at the default value input. Again, this operation can be a good thing, but it is not the behavior that we want. To deal with this difference, the code tests the error cluster output (which generates and error code 85 for a failed scan) and marks the name as invalid if there is an error.

When Validate Plugin finishes executing, we have a converted plugin name and a flag that tells us whether or not we can trust it. So the first thing we do is test that flag to see whether to continue processing the event or return an error message to the caller. Here is the code that executes when the result of the name validation process is Name Not Valid.

Name Not Valid

If the Response Notifier value from the event data is not null, the code uses it to send the error message, “Update Failed”. Note that this same message is sent whenever any problem arises in the remote interface. While this convention certainly results in a non-specific error message, it also ensures that the error message doesn’t give away any hints to “bad guys” trying to break in. If the Response Notifier value is null (as it will be for local requests) the code does nothing – remember we are also wanting to leverage this logic locally.

If the result of the name validation process is Name Valid, the code now considers the Plugin Name enumeration and predicates its further actions based on what it finds there. This example for Sine Source shows the basic logic.

Name Valid - Remote

The code reads the data buffer associated with the signal and passes the result into a case structure that does one of two different things depending on whether the event was fired locally, or resulted from a remote request. For a remote request (Response Notifier is not null), the code turns the data into a variant and uses it as the data for the response notifier. However, if the request is local…

Name Valid - Local

…it sends the same data to the VI that saves a local copy of the data.

The Read Graph Image Event

As I promised above, this event shares much of the basic structure as the one we just considered. In fact, the processing for a Name Not Valid validation result is identical. The Name Valid case, however, is a bit simpler:

Read Graph Image

The reason for this simplification is that regardless of the plugin being accessed, the datatypes involved in the operation are always the same. The code always starts with a graph control reference (which I get from the lookup buffer) and always generates an Image Data cluster. If the event was fired locally, the image data is passed to a VI (Write PNG that prompts the user for a file name and then saves it locally. However, if instead of saving a file, you are wanting to pass the image in a form that is usable in a non-LabVIEW environment, a bit more work is required. To encapsulate that added logic, I created the subVI Send Image

Send Image Data

The idea is to convert the proprietary image data coming from the invoke node into a generic form by rendering it as a standard format image. Once in that form, it is a simple matter to send it as a binary stream. To implement this approach, the code first saves the image to a temporary png file. It then reads back the binary contents of the file and uses it as the data for the response notifier. Finally, it deletes the (now redundant) temporary file.

The Set Acquisition Rate Event

This event is the first one to control the operation of the application. It also has no need to be leveraged locally, so no dual operation depending on the contents of the Response Notifier value.

Set Acquisition Rate

Moreover, because the event action is a command and not a request, the response can only have one of two values: “Update Failed” or “Update Good”. The success message is only be sent if the plugin name is either Sine Source or Ramp Source, and no errors occurs during the update. While on the topic of errors, there are two operations that need to be performed for a complete update: the code must modify both the database and the buffer holding the live copy of the setting that the rest of the application uses. In setting the order of these two operations, I considered which of the two is most likely to generate an error and put it first. When you consider that most of the places storing live data are incapable of generating an error, the database update clearly should go first.

So after verifying the plugin name, the subVI responsible for updating the database (Set Default Sample looks to see if the value is changing. If the “new” value and the “old” value are equal, the code returns a Boolean false to its Changed? output and sets the Result output to Update Good. It might seem strange to return a value to the remote application that the update succeeded when there was no update performed, but think about it from the standpoint of the remote user. If I want a sample period of 1000ms, an output of Update Good tells me I got what I wanted – I don’t care that it didn’t have to change to get there. If the value is changing…

Set Default Sample Period

…the code validates the input by sending it to a subVI that compares it to some set limits (500 < period < 2500). Right now these limits are hardcoded, and in some cases that might be perfectly fine. You will encounter many situations where the limits are fixed by the physics of a process or a required input to some piece of equipment. Still, you might want these limits to be programmable too, but I’ll leave that modification as, “…as exercise for the reader.” In any case, if the data is valid, the code uses it to update the database and sets the subVI’s two outputs to reflect whether the database update generated an error. If the data is not valid, it returns the standard error message stating so.

The Set Data Buffer Depth Event

The basic dataflow for this event is very much like the previous one.

Set Data Buffer Depth

The primary logical difference between the two is that all plugins support this parameter. The logic simply has to select the correct buffer to resize.

The Set TC Parameters Event

With our third and (for now at least) final control event, we return to one that is only valid for some of the plugins – this time the temperature controllers.

Set TC Parameters

The interesting part of this event processing is that, because its data was not originally intended to be reloaded at runtime, the live copy of the data is read and buffered in the object-oriented configuration VIs.

Save Machine Configuration

Consequently, the routine to update the database (Save Machine first creates a Config Data object and then use that object to read the current configuration data. If the data has changed, and is valid, the same object is passed on to the method that writes the data to the database. Note also, that the validation criteria is more complex.

Validate TC Parameters

In addition to simple limits on the sample interval, the Error High Level cannot exceed 100, the Error Low Level cannot go below 30, and all the levels have to be correct relative to each other.


With the last of the basic interface code written and in place, we need to look at how to test it. To aide in that effort, I created five test VIs – one for each event. The point of these VIs is to simply exercise the associated event so we can observe and validate the code’s response. For instance, here’s the one for reading the graph data:

Test Read Graph Data

It incorporates two separate execution paths because it has two things that it has to be doing in parallel: Sending the event (the top path) and waiting for a response (the bottom path). Driving both paths, is the output from a support VI from the notifier library (not_Generic Named Notifier.lvlib:Generate Notifier It’s job is to generate a unique notifier name based on the current time and a 4-digit random number. Once the upper path has fired the event, it’s work is done. The bottom path displays the raw notifier response and graphs of the data that is transferred. Next, the test VI for reading the graph image sports similar logic, but the processing of the response data is more complex.

Test Read Graph Image

Here, the response notifier name is also used to form the name for a temporary png file that the code uses to store the image data from the event response. As soon as the file is written, the code reads it back in as a png image and passes it to a subVI that writes it to a 2D picture control on the VI’s front panel. Finally, the three test VIs for the control operations are so similar, I’ll only show one of them.

Test Resizing Data Buffers

This exemplar is for resizing the data buffers. The only response processing is that it displays the raw variant response data.

To use these VIs, launch the testbed application and run these test VIs manually one at a time. For the VIs that set operating parameters, observe that entering invalid data generates the standard error message. Likewise, when you enter a valid setting look for the correct response in both the program’s behavior and the data stored in the database. For the VI’s testing the read functions, run them and observe that the data they display matches what the selected plugin shows on the application’s GUI.

Testbed Application – Release 18
Toolbox – Release 15

The Big Tease

In this post, we have successfully implemented a remote access/control capability. However, we don’t as of yet have any way of accessing that capability from outside LabVIEW. Next time, we start correcting that matter by creating a TCP/IP interface into the logic we just created. After that introduction, there will be posts covering .NET, ActiveX and maybe even WebSockets – we’ll see how it goes.

Until Next Time…

Building a Web Backend in LabVIEW

When building a web-based application, one of the central goals is often to maximize return on investment (ROI) by creating an interface that is truly cross-platform. One of the consequences of this approach is that if you are using a LabVIEW-based program to do so-called “backend” tasks like data acquisition and hardware control, the interface to that program will likely be hidden from the user. You see the thing is, people like consistent user interfaces and while LabVIEW front panels can do a pretty good job as a conventional application, put that same front panel in a web page and it sticks out like the proverbial sore thumb.

Plus simply embedding a LabVIEW front panel in a web pages breaks one of the basic tenets of good web design by blurring the distinction between content and appearance. Moreover, it makes you — the LabVIEW developer — at least partially responsible for the appearance of a website. Trust me, that is a situation nobody likes. You are angered by what you see as a seemingly endless stream of requests for what to you are “pointless” interface tweaks, “So big deal! The control is 3 pixels too far to the left, what difference does that make?” And for their part, the full-time web developers are rankled by what they experience as a lack of control over an application for which they are ultimately responsible.

If you think about it, though, this situation is just another example of why basic computer science concepts like “low coupling” are, well, basic computer science concepts. Distinguishing between data collection and data presentation is just another form of modularity, and we all agree that modularity is A Very Good Thing, right?

It’s All About the (Data)Base

So if the concept that is really in play here is modularity, then our immediate concern needs to be the structure of the interface between that part of the system which is our backend responsibility, and that which is the “web guy’s” customer facing web-based GUI.

Recalling what we have learned about the overall structure of a web application, we know that the basic way most websites work is that PHP (or some other server-side programming tool) dynamically builds the web pages that people see based on data that they extract from a database. Consequently, as LabVIEW developer trying to make data available to the web, the basic question then is really, “How do I get my data into their database?”

The first option is to approach communications with the website’s database as we would with any other database. Although we have already talked at length about how to access databases in general, there is a problem. While most website admins are wonderful people who will share with you many, many things — direct access to their database is usually not one of those things.

The problem is security. Opening access to your LabVIEW program would create a security hole through which someone not as kind and wholesome as you, could wiggle. The solution is to let the web server mediate the data transfer — and provide the security. To explore this technique, we will consider two simple examples: one that inserts data and one that performs a query. In both cases, I will present a LabVIEW VI, and some simple PHP code that it will be accessing.

However, remember that the point of this lesson is the LabVIEW side of the operations. The PHP I present simply exists so you can get a general idea of how the overall process works. I make no claims that the PHP code bears any similarity to code running on your server, or even that it’s very good. In fact, the server you access might not even use PHP, substituting instead some other server-side tool such as Perl. But regardless of what happens on the other end of the wire, the LabVIEW code doesn’t need to change.

As a quick aside, if you don’t have a friendly HTTP server handy and you want to try some of this stuff out, there is a good option available. An organization called Apache Friends produces an all-in-one package that will install a serviceable server on just about any Windows, Linux or Apple computer you may have sitting around not being used. Note that you do not want to actually put a server that you create in this way on the internet. This package is intended for training and experimentation so they give short shrift to things like security.

The following two examples will be working with a simple database table with the following definition:

CREATE TABLE temp_reading (
   sample_dttm  TIMESTAMP PRIMARY KEY,
   temperature  FLOAT

Note that although the table has two columns we will only be interacting directly with the temperature column. The sample_dttm column is configured in the DBMS such that if you don’t provide a value, the server will automatically generate one for the current time. I wanted to highlight this feature because the handling of date/time values is one of the most un-standardized parts of the SQL standard. So it will simplify your life considerably if you can get the DBMS to handle the inserting or updating of time data for you.

Writing to the Database

The first (and often the most common) thing a backend application has to do is insert data into the database, but to do that you need to be able to send data to the server along with the URL you are wanting to access. Now HTTP’s GET method which we played with a bit last week can pass parameters, but it has a couple big limitations: First, is the matter of security.

The way the GET method passes data is by appending it to the URL it is getting. For example, say you have a web page that is expecting two numeric parameters called “x” and “y”. The URL sent to the server would be in the form of:


Consequently the data is visible to even the most casual observer. Ironically though, the biggest issue is not the values that are being sent. The biggest security hole is that it exposes the names of the variables that make the page work. If a person with nefarious intent know the names of the variables, they can potentially discover a lot about how the website works by simply passing different values and seeing how the page responds.

A second problem with the GET method’s way of passing data is that it limits the amount of data you can sent. According to the underlying internet standards, URLs are limited to a maximum of 2048 characters.

To deal with both of these issues, the HTTP protocol defines a method called POST that works much like GET, but without exposing the internal operation or limiting the size of the data it can pass. Here is an example of a simple LabVIEW VI for writing a temperature reading to the database.

Insert to database with PHP

This code is pretty similar to the other HTTP client functions that we have looked at before, but there are a few differences. To begin with, it is using the POST we just talked about. Next, it is formatting and passing a parameter to the method. Likewise, after the HTTP call it verifies that the text returned to it is the success message. But where do those values (the parameter name and the success message) come from? As it so happens I didn’t just pull them out of thin air.

Check out the URL the code is calling and you’ll note that it is not a web page, but PHP file. In fact, what we are doing is asking the server to run a short PHP program for us. And here is the program:

	if($dbc = mysqli_connect("localhost","myUserID","myPassword","myDatabase")) {
    	// Check connection
		if (mysqli_connect_errno()) {
			// There was a connection error, so bail with the error data
			echo "Failed to connect to MySQL: " . mysqli_connect_error();
		} else {
			// Create the command string...
			$insert = "INSERT INTO temp_reading (temperature) VALUES (" . $_POST["tempData"] . ")" ;
			// Execute the insert
				// The insert generated an error so bail with the error data
				die('SQL Error: ' . mysqli_error($dbc));
			// Close the connection and return the success message
			echo "1 Record Added";

Now the first thing that you need to know about PHP code is that whenever you see the keyword echo being used, some text is being sent back to the client that is calling the PHP. The first thing this little program does it try to connect to the database. If the connection fails the code echoes back to the clients a message indicating that fact, and telling it why the connection failed.

If the connection is successful, the code next assembles a generic SQL statement to insert a new temperature into the database. Note in particular the phrase $_POST["tempData"]. This statement tells the PHP engine to look through the data that the HTTP method provided and insert into the string it’s building the value associated with the parameter named tempData. This is where we get the name of the parameter we have to use to communicate the new temperature value. Finally, the code executes the insert that it assembled and echoes back to the client either an error, or a success message.

Although this code would work “as is” in many situations, it does have one very large problem. There is an extra communications layer in the process that can hide or create errors. Consider that a good response tells you that the process definitely worked. Likewise receiving one of the PHP generated errors tells you that the process definitely did not work. A network error, however, is ambiguous. You don’t know if the process failed because the transmission to the server was interrupted; or if the process worked but you don’t know about it because the response from the server to you got trashed.

PHP Mediated Database Reads

The other thing that backend applications have to do is fetch information (often configuration data) from the database. To demonstrate the process go back to the GET method:

Fetch from database with PHP

As with the first example, the target URL is for a small PHP program, but this time the goal is to read a time ordered list of all the saved temperatures and return them to the client. However, the data needs to be returned in such a way that it will be readable from any client — not just one written in LabVIEW. In the web development world the first and most common standard, is to return datasets like this in an XML string. This standard is very complete and rigorous. Unfortunately, it tends to produce strings that are very large compared to the data payload they are carrying.

Due to the size of XML output, and the complexity of parsing it, a new standard is gaining traction. Called JSON, for JavaScript Object Notation, this standard is really just the way that the JavaScript programming language defines complex data structures. It is also easy for human beings to read and understand, and unlike XML, features a terseness that makes it very easy to generate and parse.

Although, there are standard libraries for generating JSON, I have chosen in this PHP program to generate the string myself:

    if($dbc = mysqli_connect("localhost","myUserID","myPassword","myDatabase")) {

    	// Check connection
		if (mysqli_connect_errno()) {
			echo "Failed to connect to MySQL: " . mysqli_connect_error();
		} else {
			// Clear the output string and build the query
			$json_string = "";
			$query = "SELECT temperature FROM temp_reading ORDER BY sample_dttm ASC";
			// Execute the query and store the resulting data set in a variable (called $result)
			$result = mysqli_query($dbc,$query);
			// Parse the rows in $result and insert the values into a JSON string
			while($row = mysqli_fetch_array($result)) {
				$json_string = $json_string . $row['temperature'] . ",";
			// Replace the last comma with the closing square bracket and send the
			// to the waiting client
			$json_string = "[" . substr_replace($json_string, "]", -1);
			echo $json_string;

As in the first PHP program, the code starts by opening a database connection and defining a small chunk of SQL — this time a query. The resulting recordset is inserted into a data structure that the code can access on a row by row basis. A while loop processes this data structure, appending the data into a comma-delimited list enclosed in square brackets, which the JSON notation for an array. This string is echoed to the client where a built-in LabVIEW function turns it into a 1D array of floats.

This technique works equally well for most complex datatypes so you should keep it in mind even if you are not working on web project. I once had an application where the client wanted to write the GUI in (God help them) C# but do all the DAQ and control in LabVIEW-generated DLLs. In this situation the C# developer had a variety of structs containing configuration data that he wanted to send me. He, likewise, wanted to receive data from me in another struct. The easy solution was to use JSON. He had a standard library that would turn his structs into JSON strings, and LabVIEW would turn those strings directly into LabVIEW clusters. Then on the return trip my LabVIEW data structure (a cluster containing a couple of arrays) could be turned, via JSON, directly into a C# struct. Very, very cool.

The Big Tease

So what is in store for next time? Well something I have been noticing is that people seem to be looking for new ways to let users interact with their applications. With that in mind I thought I might take some time to look at some nonstandard user interfaces.

For my first foray into this area I was thinking about a user interface where there are several screens showing different data, but by clicking a button you can pop the current screen out into a floating window. Of course, when you close the floating window, it will go back into being a screen the you can display in the main GUI. Should be fun.

Until Next Time…


Pay Attention to Units

The title for this post was a common quote from a favorite teacher of mine in high school. Mr Holt taught chemistry and physics and it seemed like I spent most of my life in his classroom — and I guess I did because in four years of high school I took 3 years each of chemistry and physics. Being a small school in a small Missouri town with a small budget we spent a lot of time (especially in physics) working on what would today be called, “mathematical modelling”. Back then, we just called it, “figuring out what would likely happen if we did something that we can’t really try because there isn’t enough money available to actually run the experiment”.

It was in those classes that I began to develop an understanding of, and appreciation for, all the pieces of contextual information that surrounds numbers. For example, if you have the number 4 floating around in a computer the most basic thing you can define about that number is what that 4 is counting or quantifying. Is it the mass of something (4 pounds)? Is it the cost of something in England (also, interestingly, 4 pounds)? Or is it a measure of how much your college roommate could drink before passing out (4 beers)?

In my work today, where I sometimes still have budgetary constraints, the need to remember that I have to, “…pay attention to the units…” remains. The difference is that now I have a powerful ally in the form of the facility that LabVIEW provides for assigning units to floating point numbers.

LabVIEW Units Aren’t Smarter Than You Are

Before we get into the meat of the discussion, however, we need to deal.with a few concerns and consideration that developers — even some very senior developers — have about using units. While there are a few behaviors that are clearly bugs, there are many more that people think are bugs but really aren’t. We should, of course, not be surprised by this situation. Anything related to floating-point will exhibit similar issues. For example, there used to be claims that LabVIEW didn’t do math correctly because the results of some calculations were different from what you got from Excel. Such claims ignored the fact that at the time Excel did all math in single-precision, while LabVIEW has always used double-precision math.

Similarly, a lot of the upset over “bugs” in units handling comes from an incomplete understanding of the entire problem. The simple truth is that there are a lot of things we do unconsciously when handling values with units so when we try to automate those operations, things can get messy. For example, we realize that sometimes when we square a number with units we want to square the unit, so meters become meters2. But sometimes we just want to square the number. Does it after all really make sense to convert volts into volts2? In implementing the units functionality, NI had to deal with similar ambiguity in something approaching a systematic manner so they made some decisions and then documented those decisions. So we may disagree with those decisions, but disagreement is not the same thing as a bug.

Another complaint that you sometimes hear is that using units complicates simple math. People will say things like, “…all I want to do is increment a number and LabVIEW won’t let me.” Ok, let’s look at that problem. You say you want to add 1 to a value, but 1 what? Even assuming a given type of unit, like length, how is LabVIEW supposed to know what you want to do? Do you want to add 1 meter, 1 furlong, 1 light-year?

Finally, there can be issues caused by shortcuts we take in our everyday thinking and communications, that are (to put it kindly) imprecise. For example, there are two units that can be, and often are, referred to as “PSI”. However, one is a force and one is a pressure, so mathematically they are very different beasts. In the end, a good way of summarizing a lot of this discussion is that using explicit units requires us to be more precise and careful in our thinking — which is always a good thing any way.

How Units Work

The basic idea behind LabVIEW’s implementation of units is that it draws a sharp distinction between how a number is stored and how it is expressed. The first step in making this distinction is deciding what kind of unit the developer selected and then expressing the value they entered in the “base unit” for the type of unit being used. For example, lengths are converted meters, temperatures are converted to degrees Kelvin, and time is converted to seconds and it is this converted value that exists on the wire. Likewise, when this wire encounters an indicator, LabVIEW converts the value on the wire to the units specified in the indicator before display. It is this basic functionality that makes simple math on timestamps possible.

pi time

In this snippet, each data source (the constants) converts the number it contains into seconds using its associated time unit. For example, 3 hours is converted into 10,800 seconds (60 x 60 x 3), 14 minutes is converted into 840 seconds (14 x 60) and the last input is left as is because it is already in seconds. Because all the numbers are now expressed the same unit, we can simply add them together and then add the result to the timestamp — which is also expressed in seconds. By the way, this is one of my favorite places to use units because it does away with so many magic numbers like 86,400 (the number of seconds in a day).

In terms of selecting the units to use, we have already seen one technique. On a control or constant you can show the units label and then enter or select the unit you want. To select the units, right-click on the units label and select the Build Unit String… option from the popup menu. The resulting dialog allows you to browse the standard units that LabVIEW supports and select or build the desired units.

However, if you have an existing number to which you want to apply a unit, you need to use a Convert Unit node. It looks a lot like the Expression Node but instead of typing simple math expressions into it, you enter unit strings.

unit conversion nodes

This snippet is one that I sometimes use when specifying timeouts. Note that a Convert Unit node is used to both apply are remove units.

When applying units, the unit string tells LabVIEW how to interpret the unitless number. It does not, as some suppose, specify the specific unit that value will have. For example, you may tell LabVIEW to interpret a particular number as feet, but internally the value will still be stored (like all lengths) in meters. When removing units, the node’s unit string tells LabVIEW how to express the internal value. In the above snippet, time is always carried as a floating point number of seconds, so the node driving the output causes LabVIEW to express the number of seconds as milliseconds.

Dealing with Temperatures

Another place I like to use units is with temperatures, but let’s be honest things can seem to get a bit strange here — or it can seem strange until you understand what is going one. The root of this apparent strangeness lies in the fact that there are absolute temperatures (like the room is 72° Fahrenheit) and there are relative temperatures (like the temperature went up 5° Fahrenheit in the last hour). Moreover, as we think about and manipulate temperatures we unconsciously, and automatically, keep track of the difference between the two. Unfortunately, when using a computer nothing happens unconsciously or automatically. Things happen because we tell them to.

The first thing NI had to decide when implementing this functionality was what base unit to use for temperature, and they picked Kelvin because it is metric and 0° K is the coldest something can get. Being metric is important because it means that the magnitude of change in 1° K is the same as for 1° C. By the way, did you notice what your brain just did? It went from thinking about absolute temperatures to thinking about relative temperatures — without thinking about it. This is what computers can’t do. Computers need some sort of indication or hint to tell them how to think about things, and in the context of temperature, this hint needs to provide a means for drawing a distinction between the units used for absolute temperature and units for relative temperatures. To see this issue in action, let’s revisit the previous time example but recast it to use temperature unit:

bad temperature calculation

Here we are taking a temperature value 20° C and incrementing it by 5° C. Unfortunately, when you run this example, the result is 298.15° C. “But,” you protest, “I thought that converting everything to the same units allowed you to perform any math you wanted with impunity. Where did this wild result come from. Surely this is a bug!”

Actually, it’s not. The only “problem” is that LabVIEW did exactly what the programmer told it to do, and not what they wanted it to do. LabVIEW added together two absolute temperature values and came up with an answer they didn’t expect. While its true that adding absolute temperatures probably doesn’t make any logical sense — how is LabVIEW supposed to know that? The real error is with the way the developer framed the problem and not with LabVIEW or its handling of units. What the developer obviously intended was to add a relative temperature value to the absolute temperature. To do that, you have to change the units of the increment input.

The way that LabVIEW identifies a relative temperature is to change the unit label from degC or degF, to Cdeg or Fdeg. By the way, now that you have the unit definitions correct, you can freely mix and match temperature scales. For example, you can add a 5° F increment to a 23° C room temperature and LabVIEW will happily give you the right answer (25.7778° C).

good temperature calculation

More Calculations

Other math operations can sometimes be confusing as well. For example, we have already discussed why you can’t simply increment or decrement numbers with units, but multiply and divide operation do not have the same restriction. You can multiply or divide a number with units by one without units with no problems. This apparent inconsistency makes sense when you remember that these two math operations used in this way simply scale the value that is there, regardless of how it is stored internally. Also NI has decides that many operations only operate on the numeric value. For example, if you square 5 meters using the square function, you get 25 meters. If you want to operate on the unit, you have to use the multiply function which will generate any unit squared.

But moving beyond these issues, there are many operations that using units will simplify, and even prevent you from making some kinds of logical errors. For instance, dividing a distance by a time will automatically generate a velocity unit (meters/sec by default). In the same way dividing a velocity by time will calculate an acceleration. And these conversions work the other way too: multiply a velocity by time and you get a distance (by default, meters).

The only negative here is that some of the unit labels can look a bit odd at first glance — though they are mathematically correct. For example, meters per second is shown as s^-1 m. However, despite the strangeness of this representation, you can alter the unit displayed by changing either of the base units shown. If you wanted feet per second, the unit string would be s^-1 ft, or miles per hour would be h^-1 mi. By the way, if you want to hide these potentially confusing representations from your users, the units string is a writable property so you could create a popup menu on a control that shows prettier labels.

unit selector

Note that he enumeration making the selection from the array has three values: Miles per Hour, Kilometers per Hour and Meters per Second.

Carrying on further, the units understands Ohm’s Law. Divide a voltage by a resistance and you get amps. Multiplying ohms and amps returns volts, and the calculations even know that the reciprocal of a frequency is time. All and all, this is some very cool stuff here that can significantly improve your code — and perhaps prevent you from making some silly mistakes. But as with all things new, work into it slowly, and thoroughly double-check the results when you try something different.

The Big Tease

There seems to be rumor running around that I don’t like queues. Well that is not true. They have a couple features that make them uniquely qualified for certain kinds of operations. My next couple posts will demonstrate a really good place use a queue — a data processing engine for manipulating data that will dynamically resize itself on the fly to provide more or less processing bandwidth.

Until Next Time…


Validating Data Input

Last week we covered the basics of creating a dialog box that doesn’t block the execution of the application that called it. This time I want to expand on that discussion to look at dialog boxes that allow the user to enter data – which in turn brings up the topic of data validation.

What We’ll Build

To demonstrate the various ideas that I want to cover, I have created a dialog box that shows some typical situations. As you can see from this screen shot of the front panel we will look at two ways of validating numeric data, as well as a couple more techniques for dealing with strings.

Number and String Validation FP

Remember, the one thing that is foundational to this discussion is the concept that the Ok button doesn’t get enabled until all the data is valid.

Building the Infrastructure

As you can see from the screen shot below, the basic infrastructure for determining when to enable the Ok consists of a cluster containing one Boolean value (I call them validation flags) for each item that needs to be validated, and for which the user could make an invalid entry. The second part of that definition is important because it’s perfectly feasible for there to be parameters that have no invalid values. Moreover, a couple of the techniques I will show you can actually prevent the user from entering incorrect data so they don’t require explicit validation.

Validation Flag Cluster

Note also that I always use a cluster to store these validation flags because, in a cluster, you can give each value a meaningful name. Consequently, the validation cluster in our example contains two flags named, “Numeric 1” and “String 1”. The other two values will be the ones where the user won’t be able to enter incorrect data.

The key idea behind this cluster of validation tags is that each value needing validation has its own flag. Consequently, the various tests can’t interfere with each other. All you have to do is wait until all the flags show that their associated values are good and you can enable the Ok button. To be consistent with the error logic that LabVIEW uses, a true value indicates a parameter that is in error, while a false indicates no error. To determine the enabled state of the button I created a polymorphic VI that accepts either a single Boolean or an array of Booleans, and outputs the enumeration that the Disabled control property expects. Any true input results in a Disabled and Grayed Out output value.

Getting Down to Specifics

To see how all this fit together, we’ll start with the two values where users can enter incorrect values.

Validating Numeric 1

We’ll start with this input because it is the easiest case. The technique is basically to take the NewVal value from the event data and compare it to the constraints to which it has to conform. In this case, upper and lower limits.

Numeric 1 Validation

If the new data, is not in the desired range, the code writes a true to the input’s validation flag. If there is in range, it writes a false. Once the flag is updated, the cluster is converted to an array and the result drives the polymorphic VI I discussed earlier to determine the Ok button’s new state.

Validating Numeric 2

This input is the first of the “no error possible” solutions but is, frankly not the way I prefer to deal with simple limits — and you’ll see why in a few moments. Note that in this example, the code implementing this method actually exits outside the event loop in the form of two property nodes.

Numeric 2 - no error

The first of these nodes define the control’s behavior if an out-of-range value is entered. To ban invalid data, we want the control to coerce the value to the closest valid number. The second node defines what the high and low limits are. The API for this functionality also allows you to control the minimum amount a value can change. For example, you could define an input that is settable between 1 and 10 in increments of 0.1.

But why isn’t this one of my preferred techniques? This method can ensure that the input is technically valid, but it can’t guarantee that it is what the user wanted. If this field is one of many on the screen, it is too easy for the user to miss it and fail to enter the number that they want. The result is a value that is valid, but nevertheless incorrect. Allowing the presence of invalid data on the screen forces the user to correct the field’s contents.

The best place to use this approach is with data fields where the default value is correct 90% or more of the time. Also, the ranges can also be set at any time — not just during initialization.

Validating String 1

This field is the other one that needs to be programmatically validated, and as you can see it’s not very different from the Numeric 1 example. The test for this string is that it can’t be empty and it can’t be more than 15 characters long.

String 1 Validation

But even this simple pair of constraints can provide us with some food for thought. For example, you can and should tell your users what the expected criteria are for each input. However, telling a user that the string can’t be more than 15 characters long isn’t going to as helpful as you might think it would be. The problem is that most people can’t look at a string that long and tell how many characters are in it. They have to count the letters — which is a pain.

A better approach is to simply manipulate the input such that it will only accept 15 characters. Now I’m not going to show how to do that. Instead I will leave that (as they say in fancy college textbooks) as an exercise for the reader. But to get “the reader” off on the right foot, let’s look at what I’m doing with the other string input.

Validating String 2

For this example I am drawing on an experience where a customer wanted a numeric input that was blank until the operator typed a number into it. The problem, of course, is that numeric inputs always have a number in them. The solution is to create a string control that operates like a numeric input. And to accomplish that, we need to look at each keystroke as the operator is making it and only allow those keys that are valid. The alchemy that allows such magic is the Key Down? filter event.

But you also need to know a bit about how a PC keyboard works. Some of the keys have ASCII values associated with them, like 0x35 for the character “5” or 0x2E for the period. LabVIEW returns the ASCII value of keys in the Char ;event data node. However, some keystrokes return an ASCII code of 0. These are control keys that do things and don’t represent printable characters. Here I’m talking about things like the delete or arrow keys. When you see an ASCII value of 0, the VKey event data value, which is an enumeration, will tell you what key was pressed.

Our event handler will need to manage both. So let’s first consider the ASCII codes we need to handle. Clearly, the 10 digits will need to be accepted (0x30 through 0x39) so when any of those values are returned we will set the Discard? node to false.

ASCI Digit Enables

Likewise if we are to accept floating point numbers we need to be able to type a period as well — but we can only have one so we need a tiny bit more logic.

ASCI Period Enable

So we’re done right? Not really, no. To begin with, you’ll notice that there are two other ASCII codes in the same case as the numeric digits: tab and backspace. They are there because while your users are entering information, they still need to be able to navigate the screen, so these two codes are also needed — and several more besides that fall in the area of control keys. Check out the code in the repository…

Summing up…

So that’s all I have for now. I do however want to point out that the last technique I showed you is very powerful and I have used it many time to “synthesize” custom inputs that don’t natively exist in LabVIEW, like an IP address entry field. The modified code is in SVN at: application/Tags/Release 5

and 2

Check it out and if you have any questions, post ’em.

Until next time…