Web Interface

If the socket fits, wear it…

Posted on by mlportersr

One of this blog’s recurring themes is the importance of modularity as an expression of the age-old tactic of “divide and conquer”. What is perhaps new (or at least daunting) to some readers is the idea of spreading tasks across not just separate processes on the same computer, but across multiple networked computers. Of course if this strategy is to be successful, the key is communications and to that end we have been examining ways of incorporating remote access capabilities into out testbed application.

Last time out, we implemented the first interface for remote applications to monitor and control our application. That interface took the form of a custom TCP protocol that used packets of JSON data to carry messages over a vanilla TCP connection. I started there because it provides a simplified mechanism for exploring some of the issues concerning basic code structure. Although this interface worked well, and in fact would prove adequate for a wide variety of applications, it did exhibit one big issue. To wit, clients had to be written in a specific way in order to use it. This fact is a problem for many applications because users are growing increasingly reticent about installing special software. They want to know why they need to load special code to do a job? The way they see it, their PCs (and cell phones for that matter) come with a bunch of networking software preloaded on them – and they have a valid point! Why should they have to install something new?

A complete answer to that question is far beyond the scope of this post, but we can spend a few useful moments considering one small niche of the overall problem, and a standardized solution to that problem. Specifically, how can we leverage some of those networking tools (read: browsers) to support remote access to our testbed application? As we have discussed before, the web environment provides ample tools for creating some really nice interfaces. The real sticking point is how that “really nice” interface can communicate with the testbed application. You may recall that a while back we considered one technique that I characterized as a “drop box” solution. The idea was to take advantage of the database underlying a web application by using it to mediate the communications. In other words, the LabVIEW application writes new data to the database and the web application reads and displays the data from the database – hence the “drop box” appellation.

While we might be able to force-fit this approach into providing a control capability, it would impose a couple big problems: First, it would mean that the local application would have to be constantly polling a remote database to see if there have been any changes. Second, it would be really, Really, REALLY slow. We need something faster. We need something more interactive. We need WebSockets.

What are WebSockets?

Simply put, the name WebSockets refers to a message-based protocol that was standardized in 2011 as RFC 6455. The protocol that the standard defines is low-overhead, full-duplex and content agnostic, meaning that it can carry data of any type – even JSON-encoded text data (hint, hint).

An interesting aspect of this protocol is that its default port for establishing a connection is port 80 – the same as the default port for HTTP. While this built-in conflict might be confusing, it actually makes sense. You see when a client initiates an HTTP connection, the first thing it does is pass to the server a number of headers that provide information on the requested connection. One of those headers allows the client to request an Upgrade connection. The original purpose of this header was to allow the client to request an upgraded connection with, for example, enhanced security. However, in recent years it has become a mechanism to allow multiple protocols to listen to the same port.

The way the process works is simple: The client initiates a normal HTTP connection to the server but sets the request headers to indicate that it is requesting a specific non-HTTP protocol. In the case of a request for the WebSockets protocol, the upgrade value is websocket. The server responds to this request with a return code of 101 (Switching Protocols). From that point on, all further communications are made using the WebSockets protocol. It is important to note that while this initial handshake leads some to assume that WebSockets in some ways dependent upon, or rides on top of the HTTP protocol, such is not the case. Aside from the initial connection handshake, the WebSockets protocol is a distinct process that shares nothing with HTTP. Consequently, while the most common application of the technique might be web-based client-server operation, the WebSockets protocol is equally well-suited for peer-to-peer messaging. The only limitation is that one of the two peers needs to be able to respond correctly to the initial handshake.

It is also worth understanding why the basic idea of using Port 80 for the initial connection is significant. A conversation on Stackoverflow gives a pretty good explanation of several issues, but for me the major advantage of using port 80 is that it avoids IT-induced complications. Many corporate IT departments will lock down ports that they don’t recognize. While there are some that try to lock down port 80, it is much less common. Before continuing on, if you’re interested, you also can find the details of the initial handshake here.

The LabVIEW Connection

Ok, so it sounds like WebSockets could definitely have a place in our communications toolbox, but how are we going to take advantage of it from LabVIEW? The answer to that question lies in the work of LabVIEW CLA Sam Sharp. He has developed a set of “pure G” VIs that allows you to implement either side of the connection. Because these are written in nothing but G, there are no DLLs involved so they can run equally well on any supported LabVIEW platform. Making the deal even sweeter, he has documented his code, created a tutorial on them, released his VIs for anyone to use, and all the compensation he requests is “…it would be great if you credit me…”. So, Sam, may you have a million click-throughs.

The following discussion is written assuming Sam’s VIs which I have converted to LabVIEW 2015. One quick note, if you don’t or can’t use the VIPM, you can still use the *.vip file, all you have to do is change the “v” to a “z” and you are good to go. As a first taste of how these VIs work, let’s look (like we did with the TCP example last time) at an over-simplified example to get a sense of the overall logical flow.

The Simplist WebSockets Server

For our purposes here, the testbed application will be the “server” so our code starts by listening for a connection attempt on the default Port 80. When it receives a connection, a reference to that connection is passed to a VI (DoHandshake.vi) that implements the initial handshake to activate the WebSockets protocol. Note that a key part of this process is the passing of a couple of “magic strings” between the client and server to validate the connection and protocol selection.

With the handshake completed and both ends of the connection satisfied that the WebSockets protocol is going to be used, the following subVI (Read.vi) reads a data packet from the client that, in our application, represents a data or control request. Next comes the subVI (Write.vi) that writes a response back to the client. Finally the code calls a subVI (Close.vi) that sends a WebSockets command to close the connection, and then closes the TCP connection reference that LabVIEW uses.

Building the Interface

To build this bare logic into something usable, the structure of the server task is essentially identical to that of the TCP process we built last time. In fact, the only difference between the two is ports to which they are listening, and the specific reentrant handlers that they launch in response to a TCP connection. So let’s concentrate on that alternate process. During initialization, the handler calls the subVI that implements the initial handshake.

Handler Initialization

In addition to the connection reference, this routine also outputs a string that is the URI that was used to establish the connection. Although we don’t need it for our application, it could be used to pass additional information to the server. Once initialization is complete the main event loop starts, but unlike the TCP handler we wrote earlier, it is not based around a state-machine structure.

Main Event Loop

While we could have broken up the process into separate states, the fact that Sam has provided excellent subVIs implementing the read and write functionality makes such a structure feel a bit contrived – or at least to me it does. When the timeout event fires, the code waits for 500 msecs for the first user data coming from the connection. If the read times-out, the loop waits for another 500 msec and then tries again. This polling technique is important because it allows other things (like the system shutdown event) to interrupt the process. Likewise, because we are waiting for a response that is, at least potentially, coming from a remote computer the polling allows us to wait as long as necessary for the response.

When the request data does arrive, the JSON data string is processed by a pair of subVIs that we originally created for the TCP protocol handler. They create the appropriate Remote Access Commands object and pass it on to the dynamic dispatch VI (Process Command.vi) that executes the command and returns the response. The response data is next flattened to a JSON string and written to the connection. Because the current implementation assumes a single request/response cycle per connection, the code closes the WebSockets connection and the TCP connection reference. However, it would be easy to visualize a structure that would not close the connection, but rather repeat one of the data read commands at a timed interval to create a remote “live” interface.

In terms of the errors that can occur during this process, the code has to correctly respond to two specific error codes. First is error code 56, a built-in LabVIEW error that flags a network operation timeout. Because this is the error that is generated if server hasn’t yet received the client’s request, the code basically ignores it. Second is error code 6066, which is a WebSockets-specific error defined in RFC 6455 to flag the situation where the remote client closes a WebSockets connection. Our code responds by closing the TCP connection reference and stopping the loop.

Testing our Work

Now that we have our new server up and running we need to be able to test its operation. However, rather than creating another LabVIEW application to act as the test platform, I built it into a web application. The interface consists of a main screen that provides a pop-up menu for selecting what you want to do and 5 other screens, each of which focus on a specific control action. As these things go, I guess it’s not a great web application, but it is serviceable enough for our purposes. If you need a great application, talk to Sam Sharp – that’s what his company does.

The HTML and CSS

As I have preached many times before, one of the things that makes web development powerful is the strict “division of labor” between its various components: the HTML defines the content, the CSS specifies how the content should look, JavaScript implements client-side interactivity and a variety of languages (including JavaScript!) providing server-side programmability. So lets start with a quick look at the HTML that defines my web interface, and CSS that makes it look good in spite of me… In order to provide some context for the following discussion, here is what the main screen looks like:

Main Screen

It has a title, a header and a pop-up menu from which you can select what you want to do. As a demonstration of the effect that CSS can have, here’s the part of the HTML that creates the pop-up menu.

<button class="btn btn-default dropdown-toggle" type="button" data-toggle="dropdown">Available Actions<span class="caret"></span></button>
<ul class="dropdown-menu">
  <li><a href="ReadGraphData.html">Read Graph Data</a></li>
  <li><a href="ReadGraphImage.html">Read Graph Image</a></li>
  <li class="divider"></li>
  <li><a href="SetAcquisitionRate.html">Set Acquisition Rate</a></li>
  <li><a href="SetDataBufferDepth.html">Set Data Buffer Depth</a></li>
  <li><a href="SetTCParameters.html">Set TC Parameters</a></li>
</ul>

You’ll notice that pop-up menu is constructed from two separate elements: A button and an unordered list – normally a set of bullet points – where each item in the list is defined as an anchor with a link to one of the other pages. However, as the picture shows, when this code runs we don’t see a button and a set of bullet points, we see one pop-up menu. How can this be? The magic lies in CSS that dramatically changes the appearance of these elements to give them the appearance of a menu. Likewise, some custom JavaScript makes the visually manipulate elements work like a menu. What is very cool, however, is that the resources making this transformation possible are part of a standard package, called Twitter Bootstrap, that is free for anyone to use. In a similar vein, let’s look at the page that displays a plot of data acquired from the testbed application:

Graph Screen - Blank

At the top of the screen there’s a small form where the user enters information defining the task to be performed, and a button to initiate the operation that the user is requesting. Below that form, is a blank area where the software will draw the graph of the acquired data. Let’s look at two specific bits of HTML, first the code that builds the data entry form…

<form>
  <fieldset class="input-box">
    <legend>View Graph Data</legend>
    <input type="text" class="str-input" id="ipAddr" value="localhost">  Host</input><br>
    <input type="number" class="num-input" id="portNum" value="80">  Port Number</input><br>
    <select id = "targetPlugin">
      <option value = "Sine Source">Sine Source</option>
      <option value = "Ramp Source">Ramp Source</option>
      <option value = "Hen House TC">Hen House TC</option>
      <option value = "Dog House TC">Dog House TC</option>
      <option value = "Out House TC">Out House TC</option>
    </select><label>  Select Target for Action</label><br>
    <input type="button" id="just-submit-button" value="Send Command">
  </fieldset>
</form>

…and now the code that defines the graph:

<div id="container" style="min-width: 310px; height: 400px; margin: 0 auto"></div>

But, something seems to be missing. The first snippet will create data-entry fields and a button, but what happens when the button is clicked? Apparently, nothing. Likewise, the consider the graphing element. We can see how large the area is to be, but where is the data coming from? And where are the graphing operations? To answer those questions, we need to look elsewhere.

The JavaScript

The power behind much of the web in general – and our application in particular – is the interpreted language JavaScript. In addition to being able to access all resources on your computer, JavaScript can interact directly with web pages and their underlying structures. For folks that like to split hairs, JavaScript is “object-based” because it does support the concept of object, but it is not “object-oriented” because it doesn’t explicitly support classes.

More important for what we are going to be doing is that it supports the concept of “callbacks” (read: User Defined Events). In other words, you can tell JavaScript to automatically performs functions when certain events occur. For example, our JavaScript code is going to be interacting with the web page that loaded it, we need to be sure that the page is fully loaded before that program starts. In order to accomplish that goal, the JavaScript file associated with the page includes this structure:

$(window).load(function() {
	...  // a lot of stuff goes here
});

This code creates a callback for the .load() event. The parameter passed to the .load() event is a reference to the function that JavaScript will run when the event fires. As is common in JavaScript, the code declares the function in line so everything between the opening and closing curly brackets will be executed when the event fires. So after declaring a few variables the code includes this:

$("#just-submit-button").click(function(){
  //The code here retrieves all of the input data and formats the request.
  target = $("#targetPlugin").val();
  remAddr = $("#ipAddr").val();
  remPort = $("#portNum").val();
  jsonData = '\"Read Graph Data\":' + JSON.stringify({"Target":target}); 

  // the websocket logic
  wc_connect(remAddr, remPort, parseData);
  wc_send(jsonData);
});

So the first thing the code does when the page finishes loading is register another callback, but this one defines what JavaScript will do when the user clicks the button in the form. The first three lines read the values of the form data entry fields, and the fourth assembles that data into the JSON string that will be sent to the server. The last two lines are the interface to the WebSockets logic. The first of these lines establishes the connection to the server, while the other one sends the command. But what about the response? Shouldn’t there be a line with a command like wc_receive? You really should be expecting this by now: Inside the wc_connect command the code registers another callback to handle the response.

The event (called onmessage) that is tied to this callback fires when a message is received from the server. The code implementing the callback resides in the file websockets.js (in case you’re curious) and its job is to read the JSON response data packet, check for errors, parse the data and generate the output – the graph. The only question now is, “How does it know how to parse the data and generate the graph?” And the answer is (all together now): “There’s another callback!” See the third parameter of wc_connect, the one named parseData? That value is actually a reference to a function contained in the JavaScript code for this particular page, and is an example of how JavaScript implements a “plugin architecture”. So here is how the data parser for this page starts…

var parseData = function(rawData){
  var plotData = JSON.parse(rawData);
  // trim decimal places
  plotData.forEach(function(element, index, array){
    plotData[index] =  Number(element.toFixed(3));
  });

At this point in the process, the data portion of the response is still a string, so to make processing the data easier, we first parse it to convert it into a JSON object. In the case of this particular response, the resulting object is the array of numbers expressed as strings. Really long strings. You see when LabVIEW encodes a number as a JSON string it includes far more digits of precision than are really needed, so forEach element in the array, I convert the value to a number with 3 decimal places. Here’s the rest of the code:

  // logic for drawing the graph
  $('#container').highcharts({
    title: { text: 'Recent Data', align: 'center' },
    subtitle: { text: 'System: '+remAddr+':'+remPort, align: 'center' },
    xAxis: { title: { text: 'Samples' }, tickInterval: 1 },
    yAxis: { title: { text: 'Amplitude' }, gridLineColor: "#D8D8D8" },
    tooltip: { headerFormat: '<small>Sample: {point.key}</small><br>' },
    series: [{ turboThreshold: 0, name: target, data: plotData, lineWidth: 1, marker:{enabled: false}, color: '#000000' }]
  });
}

This is the code that does the plotting, and as we shall see in a moment, this small amount of code produces a beautiful and highly functional chart that displays the values of individual points in a tooltip when you hover over them with the mouse and even provides a pop-up menu that allows you to save the plot image in a variety of image formats. This functionality is possible thanks to a plotting library called Highcharts that uses the structure defined in the HTML as a placeholder for what is going to draw. I have used this library before in demonstrations because in my experience it is stable, easy to use, and very well-documented. I also like the fact that regardless of what kind of plot I am trying to create they have a demo online that gets me about 95% of the way to my goal. Please note that this library is a commercial product, but they make it available for free for “non-Commercial” applications – however even for commercial usage, the one-time license fee is really pretty reasonable. Finally, even though it doesn’t appear that they actively police their licensing with things like crippled versions or the like, if you are using this on a professional project, pay the people. They have certainly earned their bread.

Testing the Pages

So at last we have our server in place and some test web pages (and supporting code) created. We need to consider how to run the web client. Here you have three options: First, you could just double-click the top-level file in Windows Explorer and Windows will dutifully open the file in your browser and everything will work as it should. Second, if you have access to an existing web server you can copy the dozen or so files to it and test it from there. Third, you could create a small temporary server strictly for testing. If you choose that path, a good option is a server called Express.js. As it name implies, it is written in JavaScript, which means it runs under the Node.JS execution engine. You can set one up sufficient to test our current code in about 10 minutes – including the time required to download the code.

The overall test process is similar to what we did to test the custom TCP server last time. The only significant change is the interface. First, test things that should work to make sure they do. Second, test the things that shouldn’t work and make sure they don’t. Here are examples of what you can expect to see on the graphing and image-fetch screens:

Graph Screen

Image Screen

Testbed App – Release 20
Toolbox – Release 17
WebSockets Client – Release 1

Big Tease

So what’s next? We have looked at access via a custom TCP interface and the standard WebSockets interface. How about next time, we look at how to do embed this connectivity in a C++ program using a DLL?

Until Next Time…
Mike…

Building a Web Backend in LabVIEW

Posted on by mlportersr

When building a web-based application, one of the central goals is often to maximize return on investment (ROI) by creating an interface that is truly cross-platform. One of the consequences of this approach is that if you are using a LabVIEW-based program to do so-called “backend” tasks like data acquisition and hardware control, the interface to that program will likely be hidden from the user. You see the thing is, people like consistent user interfaces and while LabVIEW front panels can do a pretty good job as a conventional application, put that same front panel in a web page and it sticks out like the proverbial sore thumb.

Plus simply embedding a LabVIEW front panel in a web pages breaks one of the basic tenets of good web design by blurring the distinction between content and appearance. Moreover, it makes you — the LabVIEW developer — at least partially responsible for the appearance of a website. Trust me, that is a situation nobody likes. You are angered by what you see as a seemingly endless stream of requests for what to you are “pointless” interface tweaks, “So big deal! The control is 3 pixels too far to the left, what difference does that make?” And for their part, the full-time web developers are rankled by what they experience as a lack of control over an application for which they are ultimately responsible.

If you think about it, though, this situation is just another example of why basic computer science concepts like “low coupling” are, well, basic computer science concepts. Distinguishing between data collection and data presentation is just another form of modularity, and we all agree that modularity is A Very Good Thing, right?

It’s All About the (Data)Base

So if the concept that is really in play here is modularity, then our immediate concern needs to be the structure of the interface between that part of the system which is our backend responsibility, and that which is the “web guy’s” customer facing web-based GUI.

Recalling what we have learned about the overall structure of a web application, we know that the basic way most websites work is that PHP (or some other server-side programming tool) dynamically builds the web pages that people see based on data that they extract from a database. Consequently, as LabVIEW developer trying to make data available to the web, the basic question then is really, “How do I get my data into their database?”

The first option is to approach communications with the website’s database as we would with any other database. Although we have already talked at length about how to access databases in general, there is a problem. While most website admins are wonderful people who will share with you many, many things — direct access to their database is usually not one of those things.

The problem is security. Opening access to your LabVIEW program would create a security hole through which someone not as kind and wholesome as you, could wiggle. The solution is to let the web server mediate the data transfer — and provide the security. To explore this technique, we will consider two simple examples: one that inserts data and one that performs a query. In both cases, I will present a LabVIEW VI, and some simple PHP code that it will be accessing.

However, remember that the point of this lesson is the LabVIEW side of the operations. The PHP I present simply exists so you can get a general idea of how the overall process works. I make no claims that the PHP code bears any similarity to code running on your server, or even that it’s very good. In fact, the server you access might not even use PHP, substituting instead some other server-side tool such as Perl. But regardless of what happens on the other end of the wire, the LabVIEW code doesn’t need to change.

As a quick aside, if you don’t have a friendly HTTP server handy and you want to try some of this stuff out, there is a good option available. An organization called Apache Friends produces an all-in-one package that will install a serviceable server on just about any Windows, Linux or Apple computer you may have sitting around not being used. Note that you do not want to actually put a server that you create in this way on the internet. This package is intended for training and experimentation so they give short shrift to things like security.

The following two examples will be working with a simple database table with the following definition:

CREATE TABLE temp_reading (
   sample_dttm  TIMESTAMP PRIMARY KEY,
   temperature  FLOAT
   )
;

Note that although the table has two columns we will only be interacting directly with the temperature column. The sample_dttm column is configured in the DBMS such that if you don’t provide a value, the server will automatically generate one for the current time. I wanted to highlight this feature because the handling of date/time values is one of the most un-standardized parts of the SQL standard. So it will simplify your life considerably if you can get the DBMS to handle the inserting or updating of time data for you.

Writing to the Database

The first (and often the most common) thing a backend application has to do is insert data into the database, but to do that you need to be able to send data to the server along with the URL you are wanting to access. Now HTTP’s GET method which we played with a bit last week can pass parameters, but it has a couple big limitations: First, is the matter of security.

The way the GET method passes data is by appending it to the URL it is getting. For example, say you have a web page that is expecting two numeric parameters called “x” and “y”. The URL sent to the server would be in the form of:

http://www.mysite.html?x=3.14;y=1953

Consequently the data is visible to even the most casual observer. Ironically though, the biggest issue is not the values that are being sent. The biggest security hole is that it exposes the names of the variables that make the page work. If a person with nefarious intent know the names of the variables, they can potentially discover a lot about how the website works by simply passing different values and seeing how the page responds.

A second problem with the GET method’s way of passing data is that it limits the amount of data you can sent. According to the underlying internet standards, URLs are limited to a maximum of 2048 characters.

To deal with both of these issues, the HTTP protocol defines a method called POST that works much like GET, but without exposing the internal operation or limiting the size of the data it can pass. Here is an example of a simple LabVIEW VI for writing a temperature reading to the database.

Insert to database with PHP

This code is pretty similar to the other HTTP client functions that we have looked at before, but there are a few differences. To begin with, it is using the POST we just talked about. Next, it is formatting and passing a parameter to the method. Likewise, after the HTTP call it verifies that the text returned to it is the success message. But where do those values (the parameter name and the success message) come from? As it so happens I didn’t just pull them out of thin air.

Check out the URL the code is calling and you’ll note that it is not a web page, but PHP file. In fact, what we are doing is asking the server to run a short PHP program for us. And here is the program:

<?php
	if($dbc = mysqli_connect("localhost","myUserID","myPassword","myDatabase")) {
    	// Check connection
		if (mysqli_connect_errno()) {
			// There was a connection error, so bail with the error data
			echo "Failed to connect to MySQL: " . mysqli_connect_error();
		} else {
			// Create the command string...
			$insert = "INSERT INTO temp_reading (temperature) VALUES (" . $_POST["tempData"] . ")" ;
			
			// Execute the insert
			if(!mysqli_query($dbc,$insert)){
				// The insert generated an error so bail with the error data
				die('SQL Error: ' . mysqli_error($dbc));
			}
			
			// Close the connection and return the success message
			mysqli_close($dbc);
			echo "1 Record Added";
		}
	}
?>

Now the first thing that you need to know about PHP code is that whenever you see the keyword echo being used, some text is being sent back to the client that is calling the PHP. The first thing this little program does it try to connect to the database. If the connection fails the code echoes back to the clients a message indicating that fact, and telling it why the connection failed.

If the connection is successful, the code next assembles a generic SQL statement to insert a new temperature into the database. Note in particular the phrase $_POST["tempData"]. This statement tells the PHP engine to look through the data that the HTTP method provided and insert into the string it’s building the value associated with the parameter named tempData. This is where we get the name of the parameter we have to use to communicate the new temperature value. Finally, the code executes the insert that it assembled and echoes back to the client either an error, or a success message.

Although this code would work “as is” in many situations, it does have one very large problem. There is an extra communications layer in the process that can hide or create errors. Consider that a good response tells you that the process definitely worked. Likewise receiving one of the PHP generated errors tells you that the process definitely did not work. A network error, however, is ambiguous. You don’t know if the process failed because the transmission to the server was interrupted; or if the process worked but you don’t know about it because the response from the server to you got trashed.

PHP Mediated Database Reads

The other thing that backend applications have to do is fetch information (often configuration data) from the database. To demonstrate the process go back to the GET method:

Fetch from database with PHP

As with the first example, the target URL is for a small PHP program, but this time the goal is to read a time ordered list of all the saved temperatures and return them to the client. However, the data needs to be returned in such a way that it will be readable from any client — not just one written in LabVIEW. In the web development world the first and most common standard, is to return datasets like this in an XML string. This standard is very complete and rigorous. Unfortunately, it tends to produce strings that are very large compared to the data payload they are carrying.

Due to the size of XML output, and the complexity of parsing it, a new standard is gaining traction. Called JSON, for JavaScript Object Notation, this standard is really just the way that the JavaScript programming language defines complex data structures. It is also easy for human beings to read and understand, and unlike XML, features a terseness that makes it very easy to generate and parse.

Although, there are standard libraries for generating JSON, I have chosen in this PHP program to generate the string myself:

<?php
    if($dbc = mysqli_connect("localhost","myUserID","myPassword","myDatabase")) {

    	// Check connection
		if (mysqli_connect_errno()) {
			echo "Failed to connect to MySQL: " . mysqli_connect_error();
		} else {
			// Clear the output string and build the query
			$json_string = "";
			$query = "SELECT temperature FROM temp_reading ORDER BY sample_dttm ASC";
			
			// Execute the query and store the resulting data set in a variable (called $result)
			$result = mysqli_query($dbc,$query);
			
			// Parse the rows in $result and insert the values into a JSON string
			while($row = mysqli_fetch_array($result)) {
				$json_string = $json_string . $row['temperature'] . ",";
			}
			
			// Replace the last comma with the closing square bracket and send the
			// to the waiting client
			$json_string = "[" . substr_replace($json_string, "]", -1);
			echo $json_string;
		}
	}
	mysqli_close($dbc);
?>

As in the first PHP program, the code starts by opening a database connection and defining a small chunk of SQL — this time a query. The resulting recordset is inserted into a data structure that the code can access on a row by row basis. A while loop processes this data structure, appending the data into a comma-delimited list enclosed in square brackets, which the JSON notation for an array. This string is echoed to the client where a built-in LabVIEW function turns it into a 1D array of floats.

This technique works equally well for most complex datatypes so you should keep it in mind even if you are not working on web project. I once had an application where the client wanted to write the GUI in (God help them) C# but do all the DAQ and control in LabVIEW-generated DLLs. In this situation the C# developer had a variety of structs containing configuration data that he wanted to send me. He, likewise, wanted to receive data from me in another struct. The easy solution was to use JSON. He had a standard library that would turn his structs into JSON strings, and LabVIEW would turn those strings directly into LabVIEW clusters. Then on the return trip my LabVIEW data structure (a cluster containing a couple of arrays) could be turned, via JSON, directly into a C# struct. Very, very cool.

The Big Tease

So what is in store for next time? Well something I have been noticing is that people seem to be looking for new ways to let users interact with their applications. With that in mind I thought I might take some time to look at some nonstandard user interfaces.

For my first foray into this area I was thinking about a user interface where there are several screens showing different data, but by clicking a button you can pop the current screen out into a floating window. Of course, when you close the floating window, it will go back into being a screen the you can display in the main GUI. Should be fun.

Until Next Time…

Mike…

Understanding Web I/O

Posted on by mlportersr

A question that seems to be popping up with increased urgency is how to incorporate LabVIEW into an environment that is largely web-based. As you would expect, in a technology that is developing as fast as the web, it can be hard to keep track of where things are right now. In fact at least one major tech publisher (O’Reilly Media) is diving full throttle into electronic publishing because with the lead time inherent in conventional publishing, web development books tend to be out of date before they can even be printed.

This is important to us because we don’t have the time or money to waste on technological dead ends. For example, I wouldn’t recommend you buy stock in companies manufacturing compact fluorescent lightbulbs. I mean who really wants to have to call in a hazmat team to clean up the mercury contamination when one breaks? The “curly light bulb” may be all the rage right now, but it is a technological dead end — to be replaced by LEDs. In the same way, I believe that many of the technologies that are in use today to access online resources are also technological dead ends.

If your company is online right now or wants to offer some service online, the standard way of doing that is with an “app”. Or rather, I should say, several apps. You will need an app for Android, one iOS, one for Windows Mobile, and perhaps others as well. But to get them you will need separate at least three separate development team because none of those operating systems are compatible with one another — or for that matter even very similar. This approach is simply not sustainable, and is another technological dead end. The solution is to build a system based on the platform independent standards that drive the web. So we need to learn where LabVIEW fits into this picture.

The Web Landscape

The first thing to understand is that the web as a collection of static HTML pages is gone. The web of today is a dynamic environment where the web pages you view (including this one) are built on the fly and in some cases may only exist for the few milliseconds it takes for the server to transmit it to you. While the underlying technology that accomplishes this magic is fascinating, the good news is that we don’t really need to know very much about it to accomplish our goals. Instead it is adequate for our purpose to know who the “main players” are from our (rather limited) point of view and how they interact.

One of the key developments in the past few years has been the understanding that each piece of the overall web environment needs to have a clearly defined role and that the various components don’t step outside their designated roles. To see what I mean, let’s consider a few of the basic pieces that you will encounter working with LabVIEW:

HTML — The Content

Tn the beginning the need was simple, Researchers at CERN needed a way to use this new “internet” to share research results with colleagues. But they wanted more than just a way to ship text file around. They wanted documents that could include pictures and graph, and could be linked to one another. In short, they wanted (to borrow a buzzword from an even earlier time) hyper-text.

The approach that they used to reach for this goal was simple: take bare text documents and embed in them tags (which were also text) that could tell a custom designed program called a “browser” how to treat the text contained in the tags. The thing to remember is that this idea was hardly a new one — so-called “markup languages” had been used for decades in the mainframe environment to implement word processors and report generators.

Even today, HTML (hyper text markup language) documents are just text files with embedded tags. This point is an important one to remember because it means that while browsers have one way of interpreting tags, other programs — even LabVIEW, or programs you write in LabVIEW — can read the same files and parse them to extract the data, the same as they would any other text file.

CSS — How Things Should Look

As it was originally created, HTML mushed together a lot of different things. Some tags defined the structure of the document, some were links and some simply defined what the text should look like. But this approach soon began to present some limitations. For example, if the logic defining what a particular bit of text looks like is embedded in the document, everybody has to see the same thing no matter how big their browser window might be or what the physical characteristics of their screen might be.

The solution to this problem with as easy. All you had to do is define a separate syntax for defining the content’s appearance, and then take all that logic and move it to a separate file with a .css file extension called a style-sheet. Now two people could view the same HTML document and would see it formatted to best fit their system because their browsers were using style sheets that were optimized for their system hardware.

As a practical example of this flexibility in action, consider the very webpage you are reading right now. Regardless of whether you are looking at it on a monitor on your desk, or a smartphone in the palm of your hand, everybody gets to see a nicely styled layout that fits their device — and I only had to generate the content once.

PHP — Server-Side Programmability

But even with that styling flexibility, there is still a very large problem. If all you have at your disposal is HTML, then each web page becomes a separate program that somebody has to write. Imagine if you will how much effort it would take to maintain a site such as Amazon if every on of its thousands of products needed its own handcrafted and programmed web page. The solution is to automate the creation of web pages using a programming language that can automatically assemble pages from predefined common components as they are needed.

One of the most popular language for this kind of work is PHP — though there are many others. It works by inserting itself into the HTML process in one of two ways. First, there are special HTML tags that cause the server to momentarily stop sending the data from the HTML file and run a PHP function. The purpose of this function would be to programmatically build and echo to the waiting browser some portion of the page, like a standardized menu structure or footer.

The second way PHP may appear is that some web servers (like Apache, for instance) will automatically call a predefined file called index.php if you access the site without specifying a specific page. Consequently, while it might look to you like you are accessing a webpage, you are really running a PHP program that builds the initial screen that you see on the fly. In addition, if you watch your URL bar while browsing you will occasionally notice that you are accessing pages with the same php file extension.

Database — Data Storage

If you have PHP building web pages dynamically, a logical question to ask is where it gets the data that it uses to build the dynamic pages that we see in our browsers? Not surprisingly, PHP uses a database, and it does so for the same reason that we use databases when building a test application: because it’s better to reconfigure than it is to recode. In fact, one of the major functions of PHP programs is to query a local database and pass the result to the browser for display. You will also find PHP programs storing to a database the information that you provide when filling out forms to do things like ordering pizza online or registering for a blog.

The type of databases that are used can vary widely though the most common is MySQL. The important thing to remember is that the specific database that is used to fill this particular niche in the overall web ecosystem is largely irrelevant.

Getting back to LabVIEW

Now that we understand how some of the major pieces fit together, we can begin to explore how our LabVIEW programs fit into this picture. Since the point of all this structure is to manage test files being sent to browsers, it shouldn’t be too much of a surprise to learn that one of the major ways that LabVIEW programs fit in is by pretending to be browsers. In support of that “deception” LabVIEW comes with a set of HTTP client VIs that support all the major HTTP methods as well as routines for dealing with access control, and setting session parameters that you are wanting to use. As an example of how the interface works, I have created a VI which illustrates about the simplest transaction possible:

simple get method

The code starts by opening an HTTP session, It then uses the VI that implements the HTTP GET method to fetch a URL and display the contents in a string indicator. Finally, it closes the connection. As you will see, the tricky part of this operation isn’t so much how you do it, but rather what do you do with the data you get back. For example, if you run this VI with the default value in the url control you get something that looks like this:

raw web results

Now you have to admit that the data in the Response indicator would seem to fit the definition that most people have for “unintelligible garbage”. But wait a minute, look at the first 6 bytes of the file. Many proprietary file formats will put a short identifier in the first few bytes of the file. For example, if you look at a Zip archive, the first three characters are “ZIP”. Applying this understanding to the data we just read from the internet, “GIF89a” is how files are identified that meet the 1989 Graphics Interchange Format standard. In other words, it’s a gif format graphic. To see the image all you have to do it provide a way to save the data to a file by adding a bit of logic to the block diagram.

gif saver

You see the url is the link that the LabVIEW forum uses to fetch one of the predefined avatars, in this case one called “fatbot”. But there are more serious things that you can do also. For example, by parsing the HTML that you get back from reading an actual webpage, you can obtain data that you might not otherwise be able to access. However, if you plan to do such data harvesting, be sure that you have the permission to do so. Otherwise the practice could lead to serious ethical/karmic/legal issues.

So why would someone need to do this sort of data collection as part of a professional test system? One possible scenario would be that the application you are needing to test is inaccessible, and all you can see is the data that it publishes online. Or perhaps, the web interface is the thing that you are wanting to test.

The big risk, of course, is that if you don’t have control over the page that you’re accessing, you can have your code suddenly stop working because the owner changed their webpage. Ideally, you should be able to negotiate changes with the site owner, but if not you need to remember that your number 1 mitigation strategy is good design. While you might not be able to keep changes from breaking your code, you can at least limit the damage to perhaps 1 or 2 VIs.

Assuming that you can negotiate with the folks that are working on the webpage, one of the things that you can request is that they surround the data that you need with special tags like so: (Imagine this is in the middle of a block of text on the web page)

… the test was performed by <lvData id="testOp">John Doe</lvData> using test fixture <lvData id="testLoc">5</lvData> …

This snippet defines a unique HTML tag lvData that allows the LabVIEW code that is parsing the content to quickly find the values it want, while the id="…" clauses allow the LabVIEW application identify the label to associate with each value. Moreover this creation of an ad hoc tag won’t impact the viewing of the page with a standard browser because fundamental to the operation of all browsers is the simple rule: If you see a tag you don’t recognize, ignore it. Here is code that you could use to locate and parse these tags.

parse lvData

This idea could also be expanded by embedding the custom tags in HTML comments. Like this:

<!-- <lvData id="testOp">John Doe</lvData> <lvData id="testLoc">5</lvData> -->

In this implementation, the comment tag would hide the custom tags from view when the page is being displayed in a browser, but they would still be available to a LabVIEW application parsing the page as text. Using this technique you could hide data inside any page — perhaps even this one.

The Big Tease

This should be enough for now. Next time we’ll look at how to use LabVIEW as a data source for a web-based application. We’ll check out the HTTP POST method, and even look at a little PHP code and some JSON data.

Until Next Time…

Mike…