Building a Testbed Database

As we continue our exploration of using databases to store data in your applications, the next thing we need to do is consider what the basic structure off a database should look like and how to construct a simple one that would be suitable for our Testbed application. Considering what we have assembled so far, we can immediately see two existing areas where a database would be useful:

  • Lookup for processes to launch: Currently this information is being stored in the INI file.
  • Recording errors that occur: Currently errors are being logged to a test file that is stored in the same directory as the application.

In both cases, moving this data into a database will offer a number of benefits in terms of configurability and reliability. But in addition to these two existing opportunities, there is a third value to implement that is new.

  • The default sample period: In the existing code, this value is defined by the default value of a control on the front panel of the Display Data process. This technique works, and in many case is a perfectly fine solution, but there are also cases where your customers will want to be able to change a value without you needing to modify the code. So we’ll make that change too.

As we go through the following discussion, an important point to remember is that project development is — in addition to everything else — a process of ongoing refinement. As you work through a project you are always going to be learning more about what it is that you are trying to accomplish and you need to be willing to incorporate those “lessons-learned” in your code. Of course this point assumes that your code is modularized such that this sort of “mid-course correction” can be implemented without doing violence to what you already have in place.

In order to break this work up into bite-sized pieces, this post will address the structure of the database itself. The VIs needed to implement this configurability will be addressed in the next installment.

Why be Normal(ized)?

According to Wikipedia, normalization (in the database sense) is “…the process of organizing the fields and tables of a relational database to minimize redundancy.” Basically, this means that a given piece of information, like a person’s name, should only be stored in one place in the database. Normalization is important because it is one of the major ways that databases ensure consistency in the data they contain. For example, if an operator’s name is only stored in one place, you don’t have to worry after the fact if “Robert Porter” and “Robert Portor” are really the same person.

Beyond this basic explanation, there is a lot of detail that we don’t need to discuss right now. In fact, there are whole books written on the topic of normalization. For your further reading, I have included links to two very good ones are at the end on the post. Our goal right now is to simply cover the basics so as you can understand what a real database developer puts together for you. Moreover, if the need should ever arise for you to create a database for yourself, this information certainly won’t make you an expert, but it will help you avoid a few major pitfalls.

The first step in creating a database is called Data Modelling. Data modelling is a process of looking at your data in order to understand its inherent structure. Ideally, data modelling should be something that you do early in your design process, regardless of whether you are going to be using a database. If you think about it, this point is particularly true when doing LabVIEW development. After all, LabVIEW’s core paradigm is one of dataflow, so how can you expect to properly define an application’s dataflows if you don’t understand the data’s inherent structure? Naturally therefore, when starting work on this testbed, I did some basic data modelling. Let’s go over what I discovered.

Supporting Many Applications

When studying the data that an application needs to handle, a good place to start is with a list of the data items that the code will need to complete the intended tasks. Some of this information will be common to all the tasks the code will be supporting, and some will be unique. At first glance, it might not seem like there is any information that the various functions have in common, but if we think ahead a bit, we see that there is one. To this point, we have been talking about the program that we are creating, as if it is the only one we are ever going to build and install on a user’s computer. But is this really a reasonable assumption?

In a way, this is sort of like talking about a computer that will only have one program installed on it. In addition, we have talked before about the advantages of seeing an “application” as the aggregation of the independent behaviors of many different processes that operate more or less independent of one another. It seems to me unlikely that the assumption we have been making will continue to hold up in the future.

In the world governed by INI files, this reality is addressed by simply having a separate INI file for each program. Databases accomplish the same task by adding a field that identifies which application is associated with (or in database-speak, related to) each data record. But how should these relationships be recorded in the database? Well, we could just include a field in each table called something like application_name, but this solution has two problems. First, remembering what we said about normalization, this approach would produce a lot of redundant data and a lot of opportunities for things to be misspelled, or mislabeled. Second, since we would be using that value to qualify the queries we perform in order to extract just the data related to one particular application, those string fields will end up being search terms and indexing or searching strings is inherently very inefficient.

Both problems are solved by creating a separate table that uniquely identifies each application. Primarily, it would hold the application name, but could contain other information such as version number, who wrote it, and when it was modified. In this table, each application will have one record and each record has a unique number associated with it called the Primary Key. Any other table that needs to relate to its data to a specific application can simply include a field that holds this number — and numbers are very fast to search and index. Here is the SQL code needed to create this type of table, sometimes called a header table:

CREATE TABLE appl (
    id       AUTOINCREMENT PRIMARY KEY,
    label    TEXT(40) WITH COMPRESSION,
    ver_nbr  TEXT(10) WITH COMPRESSION,
    dev_name TEXT(50) WITH COMPRESSION
  )
;

The first line contains the command (CREATE TABLE) and specifies the new table’s name (appl). Everything between the parentheses is a comma-delimited list of the columns that the table will contain. Each column definition consists of a column name and a definition of the column’s contents. Note that column names can contain spaces, but it complicates the syntax, so I don’t use them.

The first column is named id and the column definition presents the Jet DBMS syntax for identifying a column that you want to be an automatically generated integer primary key. Other DBMS also have mechanisms for accomplishing the same thing, but the syntax varies. The next three columns are defined as variable-length text fields of differing sizes. Note that defined in this way, the indicated count specifies the maximum number of characters that the field can contain, but the DBMS only stores the actual number of characters you put in it. Finally, the WITH COMPRESSION keywords are needed because a few versions ago, Jet was converted to storing UNICODE characters, which are 2-bytes long. The effect of this change is that ASCII strings started taking up twice as much memory as was needed. The WITH COMPRESSION keywords tell Jet to store the strings as 1-byte values.

One more point. The last field (dev_name) is intended to contain the name of the developer that last modified the application. But, wouldn’t what we said earlier about normalization apply to this information as well? Yes it would. To be formally correct, there should be another table of developers to which this table should relate. However, I put this in here to highlight the important point that in database design — as with LabVIEW programming — there trade-offs and so we need to ask ourselves whether the added normalization provides sufficient benefit to justify the added complication of another table. In this case, the answer I came up with was, “No”.

Processes to Launch

When we consider the information needed to launch the startup processes with the desired user feedback, we find that there are three things we need — plus of course a reference to the application identified in the table we just created. This is what we have.

  1. appl_id: This is the numeric reference to the table identifying the applications.
  2. label: This is the string that will be displayed to the user while the process is being launched. Before it was derived from the name of the VI. This approach is more flexible.
  3. item_path: As when this data was stored in the INI file, this string is a relative path to the VI to be launched.
  4. item_mode: This string indicates the mode used to launch the process VI. It is associated with an enumeration in LabVIEW that (for now at least) simply indicates whether the VI is reentrant or non-reentrant.
  5. launch_order: This is an integer column we will use to sort the query results so the process VIs are launched in the right order.

So with this information in hand we are ready to define our next table, right? Well, not quite. We need to consider another assumption that we have made. To this point, the code we have created only uses VIs that are dynamically loaded in one place: at program startup. Will this assumption always be true? Simple logic would say that dynamic loading is a very powerful technique, so we will very likely be wanting to use it in the future. So let’s create another header table that will store a string indicating the condition where the VI will be launched. Right now the table will only have a single value in it, “Startup”.

Turning all this discussion into SQL commands to create the needed tables, this code creates the new header table:

CREATE TABLE launch_cond (
    id                AUTOINCREMENT PRIMARY KEY,
    launch_condition  TEXT(128) WITH COMPRESSION
  )
;

…then this code inserts the one record we need now…

INSERT INTO launch_cond (launch_condition) VALUES ('Startup');

…and finally we can create the table for holding the launch items:

CREATE TABLE launch_item (
    id              AUTOINCREMENT PRIMARY KEY,
    appl_id         INTEGER NOT NULL,
    launch_cond_id  INTEGER NOT NULL,
    label           TEXT(40) WITH COMPRESSION,
    item_path       TEXT(128) WITH COMPRESSION,
    launch_mode     TEXT(40) WITH COMPRESSION,
    launch_order    INTEGER,
    CONSTRAINT launch_cond_FK FOREIGN KEY (launch_cond_id) REFERENCES launch_cond(id),
    CONSTRAINT launch_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id)
  )
;

This table contains all the same stuff we saw in the other table creation commands, but with a couple additions. The last two items in the column list aren’t columns. Rather, they create the relational links between this table and the two header tables by defining a pair of foreign key constraints. The name of the constraints are important because they will be used in error messages associated with the constraint. The constraint definitions themselves are straight-forward, specifying a column that references a specified column in a specified table.

Storing Errors

The foregoing case demonstrates how to deal with tables that store configuration data. Now we turn our attention to a table that the application writes to while running. Specifically, we will look at the table for storing errors that occur during program execution. Here are the pieces of information we know we will need going in:

  1. appl_id: This is the numeric reference to the table identifying the applications. In this case it is the application that generated the associated event.
  2. evt_dttm: This field holds timestamp values indicating when the events occurred.
  3. evt_type_id: In addition to knowing when an event occurred, you also need to capture what type of event it was. Starting off we can imagine three types of events (errors, warnings and notes) but there could be more, so we’ll store the type in another header table and reference it here.
  4. evt_code: This integer column hold the error code from the LabVIEW error cluster.
  5. evt_source: Another text column, this field holds the Source string from the LabVIEW error cluster.

First comes the new header table and its three records. Note that these event types correlate to the three values of a LabVIEW enumeration (Event Types.ctl).

CREATE TABLE event_type (
    id     AUTOINCREMENT PRIMARY KEY,
    label  TEXT(40) WITH COMPRESSION
  )
;

INSERT INTO event_type (label) VALUES ('Error');
INSERT INTO event_type (label) VALUES ('Warning');
INSERT INTO event_type (label) VALUES ('Note');

And then the table for storing the events…

CREATE TABLE event (
    id             AUTOINCREMENT PRIMARY KEY,
    appl_id        INTEGER NOT NULL,
    event_type_id  INTEGER NOT NULL,
    evt_dttm       DATETIME DEFAULT now() NOT NULL,
    evt_code       INTEGER,
    evt_source     MEMO WITH COMPRESSION,
    CONSTRAINT event_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id),
    CONSTRAINT event_event_type_FK FOREIGN KEY (event_type_id) REFERENCES event_type(id)
  )
;

The first new idea on display here is in the definition of the evt_dttm field. In addition to stating the data type (DATETIME), it also specifies that the field cannot be empty (NOT NULL) and tells Jet what value to use if an insert statement does not contain a time value: now(). This built-in Jet constant returns the current date and time. You also see that I introduced a new datatype MEMO to hold the textual description of the error. Given that it uses the same WITH COMPRESSION keywords as we use with the TEXT, you might assume that it is another way of storing textual data — and you’d be right. The difference between the two is that while both are variable length fields, a TEXT field can only hold a maximum of 255 characters. By contrast a MEMO field can (in theory at least) hold strings as long as 2.14-Gbytes.

The Default Sample Period

Finally there are always simple, unstructured setup values, like the default sample period, so let’s set up a table for them too. Note that I picked column names that reflect the simple organization of an INI file with sections, keys and values.

CREATE TABLE misc_setting (
    id         AUTOINCREMENT PRIMARY KEY,
    appl_id    INTEGER NOT NULL,
    p_section  TEXT(50),
    p_key      TEXT(50) WITH COMPRESSION,
    p_value    TEXT(255) WITH COMPRESSION,
    CONSTRAINT miscsettings_appl_FK FOREIGN KEY (appl_id) REFERENCES appl(id)
  )
;

INSERT INTO misc_setting (appl_id, p_section, p_key, p_value)
   SELECT id, 'Data Acquisition','Sample Period','1000'
     FROM appl
    WHERE label = 'Testbed'
;

The syntax used for inserting the record we need is a bit different because it has to look up the value that will go into the appl_id field using something called a subquery. The syntax for the Jet implementation of this type of operation is rather obscure and is in no way compliant with the SQL standard, but this is what it looks like. Unfortunately, it is not the only place where Jet runs counter to the standard. For example, getting the event table to automatically insert the event timestamp value using by using a default value of now() sidesteps the problem of formatting time values, which is also very non-standard.

A Few More Things

So there we have it: the structure for our simple database — or as much of it as we need right now. But you may be wondering what you are supposed to do with all that SQL code? I’m glad you asked. I have created a simple utility that executes the code to create the database, and you can find it (and the complete SQL source code file) here:

http://svn.notatamelion.com/blogProject/local database builder/Tags/Release 1

When you run the utility, it builds the new database file in the directory where the source code file is located. While I’m linking to things, it can be helpful sometimes to simply open a database file and look at its contents. If you have Access installed on your computer you can look at the resulting database file immediately, otherwise I can recommend a small, lightweight utility called Database .Net. It doesn’t require an installer, supports a bunch of different DBMS, and does not impact a computer’s registry so can run from anywhere. I like to keep it on a USB thumb-drive in case I need to take a quick look at the contents of a database.

Finally, here are a couple more links for books that I recommend for learning more about the issues we rushed through in this post. I really like The Practical SQL Handbook because it covers all the important points in a readable, entertaining way. A dry college text, this is not. Designing Quality Databases With IDEF1X Information Models, on the other hand, is a college text that goes into a tremendous amount of detail on the topic of data modelling and a common technique for documenting what you find — which of course makes it very through. This book is not a quick read, but it is worth the effort.

All we have left to cover now is the LabVIEW side of the job, which we’ll get into with the next post.

Until next time …

Mike…

Managing Data — the Easy Way

As I work on this blog there are times when I have to put certain topics on hold because the infrastructure doesn’t yet exist to allow me to effectively cover the topic. Well, this is one of those times. What I want to do is start getting into some topics like advanced user interfaces. However, these capabilities presume the existence of a data management capability that our testbed does not yet possess. To begin filling in that blank, the next few posts are going to start looking at techniques for utilizing databases as a data storage mechanism for LabVIEW-based applications.

But why databases? Wouldn’t it be easier to store all our configuration or test data in text files? When considering the storage of any sort of data the first thought for many developers is often to just, “…throw it in a text file…”. Therefore, this question needs to be the first thing that we discuss.

The Ubiquitous INI File

The INI file, as a method for storing configuration data, is a standard that has been around for a long time. Its main advantage is that it is a simple human-readable format that anyone with a text editor can manipulate. However, INI files also have a significant downside, and its number one problem is security. Simply put, INI files are inherently insecure because they use a simple human-readable format that anyone with a text editor can manipulate. Anybody who knows how to use Notepad can get in and play around with your program’s configuration, so unless you are very careful in terms of data validation, this openness can become an open door to program instability and failure.

A second issue is that while INI files can be subdivided into sections, and each section can contain key name and value pairs; beyond that simplistic formatting they are largely unstructured. This limitation shouldn’t be a surprise, after all the format was developed at a time when programs were very simple and had equally simple configuration requirements. However, with many modern applications one of the big challenges that you have to face when using INI files is that it is difficult, if not impossible, to store configuration data that exhibits internal relationships.

For example, say you have 2 or 3 basic types of widgets that you’re testing and each type of widget comes in 2 or 3 variants or models. In this scenario, some of the test parameters will be common for all widgets of a specific type while others are specific to a particular model. If you want to capture this data in a text file, you have two basic options. First you can put everything in one big table — and so run the risk of errors resulting from redundant data. Second, you can store it in a more normalized form and try to maintain the relationships manually — which is a pain in the neck, and just as error prone.

Text Data Files?

OK, so text files aren’t the best for storing configuration data. What about test data? Unfortunately, you have all the same problems — in spades — plus a few new ones: For example, consider Application Dependency.

Application dependency means that the data format or structure is specific to the program that created the file. Basically, the limitation is that outside programs have to know the structure of your file ahead of time or it won’t be able to effectively read your data. Depending upon the complexity of your application, the information required to read the data may expose more about the inner workings of your code than you really feel comfortable publishing.

Another problem is numeric precision. You can’t save numbers in a text file, just string representations of the numbers. Consequently, numeric precision is limited to whatever the program used when the file was created. So if you think that all you need when saving the data is 3 decimal places, and then find out later that you really need 4, you’re pretty much hosed since there is no way to recreate the precision that was thrown way when the data was saved.

Finally, data in a text file usually has no, or at least inadequate, context. Everybody worries about the accuracy of their test results, but context is just as important. Context is the information that tells you how to interpret the data you have before you. Context includes things like who ran a test, when it was run, and how the test was configured. Context also tells you things like what unit of measure to use in reading numeric data, or when the instruments were calibrated.

The Case for a Database

Due to all the foregoing issues, my clear preference is to use databases to both manage configuration data and store test results. However some LabVIEW developers refuse to consider databases in a misguided effort to avoid complication. Out of a dearth of real information they raise objections unencumbered by facts.

My position is that when you take into consideration the total problem of managing data, databases are actually the easiest solution. Of course that doesn’t mean that there won’t be things for you to learn. The truth is that there will be things to learn regardless of the way you approach data management. My point is that with databases there is less for you to learn due to the outside resources that you can leverage along the way. For one simple (but huge) example, consider that you could figure out on your own the correct ways to archive and backup all your test data — or you could put the data into a database and let the corporate IT folks, who do this sort of thing for a living, handle your data as well.

So let’s get started with a few basic concepts.

What is a DBMS?

One common point of confusion is the term DBMS, which stands for DataBase Management System. A DBMS is software that provides a standardized interface for creating, maintaining and using databases. In a sense, the relationship between a DBMS and a database is exactly the same as the relationship between a word-processor, like Microsoft Word, and a letter home to your Mom. Just as a word-processor is a program for creating textual documents, so a DBMS can be seen as a program for creating databases. One big difference though, is that while word-processors are programs that you have to explicitly start before you can use them, a DBMS will typically run in the background as a Windows service. For example, if you are reading this post from a Windows-based computer, there is at least one DBMS (called Jet) running on your computer right now.

What is a database?

A database is a structured collection of data. In that definition, one word is particularly important: “structured”. One of the central insights that lead to the development of databases was that data has structure. Early on, people recognized that some pieces of information are logically related to other pieces and that these relationships are just as important as the data itself. The relationships in the data are how we store the data context we discussed earlier.

By the way, as an aside, people who talk about an “Access Database” are wrong on two counts since Access is neither a database or a DBMS. Rather is it an application development environment for creating applications that access databases. By default, Access utilizes the Jet DBMS that is built into Widows, but it can access most others as well.

How do you communicate with a database?

It wasn’t that long ago that communicating with a database from LabVIEW was painful. The first product I ever sold was an add-on for the early Windows version of LabVIEW that was built around a code interface node that I had to write using the Watcom C compiler. Actually, looking back on it, “painful” is an understatement…

In any case, things are now dramatically different. On the database side the creation of standards such as ODBC and later ADO (also called OLE DB) provided standardized cross-vendor interfaces. For their part, National Instruments began providing ways of accessing those interfaces from within LabVIEW (starting with Version 5). Today accessing databases through ActiveX or .net interfaces is a breeze. To demonstrate this point, I’ll present a package of drivers that I have developed and posted on the user forum several years ago.

Getting Connected…

The VIs we will now discuss are the core routines that you will need to interact with any DBMS that supports an ADO interface — which is basically all of them. The only common DBMS that doesn’t support ADO is SQLite. Instead, it has its own DLL that you have to access directly. Still, if you want a very lightweight database engine that will run on nearly anything (including some real-time hosts) it is a good choice and there are driver packages available through the forum.

Getting back to our standard interface, the following six routines provide all the support most applications will ever need. One thing to notice is that with the exception of one subVI that requires added code to work around a bug in certain version of SQL Server, most of the code is very simple, which is as it should be.

To start that exploration, we’ll look at the routine that is the logical starting place for any database interaction — and incidentally is the only routine that cares what database you are using.

ADO Database Drivers.lvlib:Acquire Connection.vi

The start of any interaction with a database, naturally involves establishing a connection to the DBMS that is managing it, and then identifying which specific database you want to use. This VI calls the ADO method that performs that operation.

Open Connection.vi

You’ll notice that the ActiveX method only has one required input: the Connection String. This string is a semicolon-delimited list of input parameters. Although the exact parameters that are required depends on the DBMS you are accessing, there is one required parameter, Provider. It tells the ADO functionality what driver to use to access the DBMS. This is what a typical connection string would look like for connecting to a so-called Access database.

Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\mydatabase.mdb;User Id=;Password=;

Note that in this case the Data Source consists of a path to a specific file. For another DBMS this same parameter might point to an IP address, a logical name the driver defines or the name of a Windows service. But what if you don’t know what connection string to use? Well you can ask the DBMS vendor — or you can check: www.ConnectionStrings.com. Yes, there is an entire website dedicated to listing connection strings, and it covers nearly every ADO-compatible DBMS on the planet.

ADO Database Drivers.lvlib:Execute SQL Command.vi

OK, you are connected to your DBMS and you have a reference to your database. The next question is what do you want to do now? One answer to that question would be to send a command to the DBMS telling it to do something like create a new table in which you can store data, or add a new record to an existing table. This VI meets that need by sending to the DBMS command strings consisting of statements written in a language called SQL.

Execute SQL Command.vi

The core of this routine is the connection Execute method. Because this method could be used to execute a command that returns data, the 0x80 input optimizes operation by telling the method to not expect or handle any returned data. The method, instead, returns a count of the number of rows that the last command impacted.

ADO Database Drivers.lvlib:Create and Read Recordset.vi

While sending commands is important, the most common thing to do with a database connection is to use it to read data from the database. To optimize operation when you have multiple users concurrently accessing the same database, ADO creates something called a recordset. A recordset is a memory-resident copy of the requested data that you can access as needed without requiring further interaction with the database itself.

Create and Read Recordset

The three subVIs shown create a recordset using a source string consisting of an SQL query, reads the recordset’s contents, and then closes the recordset to free its memory. For details of how these subVIs work, checkout the comments on their block diagrams.

ADO Database Drivers.lvlib:Release Connection.vi

Once you are finished using a connection, you need to tell ADO that you are done with it.

Close Connection.vi

Note that although the method called is named Close, it doesn’t really close anything. Thanks to built-in functionality called connection pooling the connection isn’t really closed, rather Window just marks the connection as not being used and puts it into a pool of available connections. Consequently, the next time there is a request for a connection to the same database, Windows doesn’t have to go to the trouble of opening a new connection, it can just pull from the pool a reference to a connection that isn’t currently in use.

ADO Database Drivers.lvlib:Start Transaction.vi

A topic that database folks justifiably spend a lot of time talking about is data integrity. One way that a DBMS supports data integrity is by ensuring that all operations are “atomic”, i.e. indivisible. In simple terms, this means that for a given operation either all the changes to the database are successful, or none of them are. This constraint is easy to enforce when inserting or modifying a single record, but what about the (common) case where a single logical update entails adding or modifying several individual records?

To handle this situation, DBMS allows you to define a transaction that turns several database operations into a single atomic operation. This VI marks the start of a transaction.

Start Transaction.vi

To mark the other end of the transaction, you have to perform either a Commit (make all the changes permanent) or a Rollback (undo all the changes since the transaction’s start).

ADO Database Drivers.lvlib:Rollback Transaction on Error.vi

This VI combines the Commit and Rollback operations into a single routine. But how does it know which to do? Simple, it looks at the error cluster. If there is no error it commits the transaction…

Commit Transaction on no Error

…but if there is an error it will rollback the transaction.

Rollback Transaction on Error

What’s Next

Clearly the foregoing six VIs are not a complete implementation of all that is useful in the ADO interface — for instance, they totally ignore “BLOBs”. However I have found that they address 95% of what I do on a daily basis, and here is a link to the new version of the toolbox that includes the drivers.

Toolbox Release 3

But let’s be honest, having an appropriate set of drivers is in some ways the easy part of the job. You still need a database to access and you need to know at least a bit about the SQL language. This is where what I said earlier I said about leveraging the time and talents of other people come into play. Your corporate or institutional IT will often assist you in setting up a database — many times, in fact, they will insist on doing it for you. Likewise, with an interface that is based on command strings written in SQL, you will find a wealth of talent willing to lend a hand, perhaps in-house but certainly online. Still, having said all that, it’s good for you to understand at least a bit about how these other aspects operate. Therefore, next time we’ll create a small local database (using Jet) and start implementing the data management for our testbed application.

Until next time…

Mike…

When are UDEs not the Right Answer?

As we have seen, UDEs can be a powerful way of passing data between processes, but they are no Silver Bullet, no Panacea, no Balm of Gilead, no … well, you get the point. Given that we aren’t going to do something silly like only use one method for passing all our data, we need to ask an important question: When should you consider the other techniques?

Knowing What is Important

The reason people choose one thing over another (or at least why people should choose one thing over another) is that they are looking for better performance — however you might define that word. But before we can say one technique is better than another, we need to understand the key features of each technique. With that knowledge in hand we can then apply those techniques in ways where the features work to our advantage, but we avoid problematic side-effects. So, since our present concern is data passing, let’s consider the issue I call, immediacy.

There are times when something needs to happen as quickly as possible after an input value changes. In the hardware world this sort of condition is called an edge trigger and as a system architect working with them you are often primarily interested in when the change occurred. Of course an event can carry data, but the key distinguishing factor for this sort of communications is timing. These are the sorts of signals for which UDEs are an excellent choice because UDEs are very good at telling remote processes that something has changed.

On the other hand there are signals where functionally it doesn’t matter really when they changed. All the code really cares about is what their value is the next time they are used. These are the signals that you could pass with a UDE, but to do so would be wasteful of the receiving process’ time. Remember that when an event fires any process registered to receive that event has to stop what was doing to handle the event and then return to what it was doing before it was interrupted — even though it fundamentally doesn’t care when the value changes. Although these diversions might be small, they do take some time and so can effect loop timing. Moreover, if you have something running in a Timeout event a UDE (or any other type of event) can drastically effect the timing of when the next timeout occurs. Remember that a timeout, of say 1000 msec, does not mean that the code in it will execute every 1000 msec. Rather, the event triggers when the time since the last occurrence of any event exceeds 1000 msec.

In fact, in this situation, if you have a UDE that fires every 500 msec, the timeout will never occur. To summarize this point, a good way to think about this is that an event actually carries two pieces of information: data and timing. What we need sometimes, though, is just the data.

Just the Data

The good news is that LabVIEW offers a variety of options for passing just the data. The oldest (and still very useful) way is with a Functional Global Variable (FGV), also sometimes called a LabVIEW 2 style global. A FGV uses an uninitialized shift register to hold data in memory. More recent additions to our arsenal include feedback nodes, data value references (DVRs) and shared variables.

Your choice between these options should be driven by memory and performance concerns. A FGV is easy to create and very flexible, but if the data being buffered is large, they can suffer performance issues due to copying of memory buffers. A DVR on the other hand, is a little trickier to use because there is a reference you have to pass around, but is very efficient. With large datasets i will often split the difference and combine the two: I will use the DVR to store the data, but use a FGV to buffer and store the DVR reference. Although it is sort of overkill for the data that I will transferring, I’ll show you the technique in the following example.

What We’ll Build

Right now in the testbed application the delay between sampled is hardcoded to 1000 msec. So to demonstrate the passing of non-time-critical data, let’s modify our code to allow the UI process to vary the delay between data samples.

The first thing we want to do is create the buffer we will use to transfer the data. So in the project directory create a subdirectory named _buffers and inside it create a subdirectory named Sample Rate. Finally inside that subdirectory create a LabVIEW library named Sample Rate.lvlib and two more subdirectories called _subVIs and _typedefs. If you have been following this blog from the beginning, this arrangement should look familiar as it is very similar to what we did for organizing UDEs — and we are repeating it here for all the same reasons (unique name space, access control for subVIs, etc.).

The Buffer

The first real code will be the buffer and this is what it looks like:

DVT-Based Data Cache

As promised, it has the basic structure of a FGV, but the global data is the DVR reference. Note also that the DVR’s datatype is a cluster (typedef’d of course) that contains a single integer. The typedef is saved in the typedef directory with the name Data.ctl and the buffer is saved in the subVI directory with the name Buffer.vi. The two subdirectories have been added to the library and the access scope for the subVI directory is set to Private.

Reading and Writing the Buffer

With the buffer itself created we need to add to the library a pair of publicly accessible VIs for accessing it — and here they are. First the read:

Read Write DVR Data Cache

…and now the write:

Write DVR Data Cache

Note that if anything about the data or how it is stored would ever need to change in the future, this one library would be the only place in the code that would be impacted.

Modifying the Testbed

All we have to do now is modify the testbed to add in the logic that we just created, but thanks to the infrastructure that we have created, that will be an easy task. In fact it will take exactly three changes — total modification time, less than 5 minutes.

  1. Add an I32 control to the front panel of the display process and create a value change event for it. In the event handler for the value change, wire the NewVal event data node to the input of the buffer write VI.

    Sample Period Value Change Event

  2. Setup the initialization by adding a property node to fire the value change event using the default control value.

    Initialize Sample Period Event

  3. Add the buffer read to the acquisition VI as shown.

    Modified Acquisition Loop Timing

That’s all there is to it. Another good point for this approach is that because the DVR’s datatype is defined as a typedef, you can make changes to the data in the buffer without recreating everything. In addition, because the data is a cluster, the existing code won’t break if you add another value to the cluster. Go ahead and give it a try. Open the typedef and add another control to the cluster. After you have saved the change note that the the existing code did not break.

Check here for the updated project:
http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 6

Oh yes, one more thing. Remember how we created a zip archive holding the basic template files for a UDE? It might be a good idea to do the same thing for this structure. Until next time…

Mike…

Building a LabVIEW Toolbox

In some ways, reusable LabVIEW code is sort of like motherhood and apple pie. Everybody agrees with the ideal, but sometimes the reality doesn’t quite measure up to the hype. While nobody will openly talk down code reusability, it often times ends up as one of those things we’ll get to, “one day”. No one questions that reusable code simplifies work and improves the quality of finished applications, but how do you create this toolbox in the first place? Well, dear readers, is what we are going to deal with this time: Where to look to get the VIs for the toolbox, and where to put them so they are available.

Every Toolbox Needs a Home

So let’s start with where we are going to store our reusable VIs, and the first thing to observe is that there are a couple basic requirements that any potential location has to be able to meet. To begin with, it should be a set location that doesn’t need to change from one project to the next. This is why I recommend the conventions that I presented a couple of weeks ago: Setting up your workspace as I suggested creates a directory structure that doesn’t need to change from one project to the next. Likewise, each functional area of your toolbox should, have its own subdirectory that contains a LabVIEW library with the same name as the directory where it is located. All VIs associated with a given functional area should be in that area’s directory and linked to its library. This organization creates a unique namespace that can help clarify what a given VI does and reduce the likelihood of naming conflicts.

Directory-Structure

The second requirement for a reuse code storage location is that it has to be easy to find during development. While having the toolbox located just a couple clicks off the root of a hard drive is a step in the right direction, there’s a lot more we can do. For example, on the function menu that you get when you right-click in free space on the block diagram, there is a palette selection called User Library and in theory it is the place where your toolbox would go. The only problem is that by default it points to the user.lib subdirectory in the LabVIEW installation directory. This directory association is a problem because with each release, Windows is getting more restrictive about being able to modify the contents of program installation directories. Luckily, that default association can be changed that the new association will automatically update to show new files that you add to the toolbox.

To proceed, with just LabVIEW open go to the Advanced submenu on the Tools menu, and select the Edit Palette Set… option. In the Functions palette window, you should see User Libraries icon. Right click on it and you will see an item with a check mark next to it called Synchronize With Directory — click on it once to deselect it and then select it a second time to reselect it. When you reselect it, LabVIEW will open a dialog box asking you to select the directory that you want to associate with the icon. Navigate to the directory where you are going to be creating your toolbox (by convention, C:/Workspace/Toolbox) and click the Select Folder Button. At this point, the icon will be linked to your new toolbox directory, plus as you add items to your toolbox, they will automatically show-up in the menu the next time you restart LabVIEW.

While we are here, there is one other palette modification you might want to make, and that is to add a copy of the User Libraries menu to the Programming palette. Because that palette is often open by default, it makes it handy to have the link to your toolbox there as well. To add it, click on the Programming icon to open the palette, and then right-click in any open space. From the resulting menu, go to the Insert submenu and select Subpalette… . The first thing you will see is the Insert Subpalette dialog box. From the available options, select Link to a directory and click the OK button. In the resulting dialog box, navigate to the directory where LabVIEW is installed, and then open the user.lib subdirectory. Finally, click the Select Folder Button.

To save your work and return to the LabVIEW development environment, click the Save Changes button in the Edit Controls and Functions Palette Set window.

Where to get the VIs

Now that we have a place for the VIs to go, let’s get some code to put there. The first place to look is with your existing code — in our case, the test bed application. However, don’t start first thing looking for large involved chunks of code to reuse. That level of reuse is important, and it will come, but let’s get our feet wet looking for small simple pieces. Remember that small things you use a lot provide greater time savings than large complex VIs that you reuse once or twice. So what do you look for to make reusable routines?

Here’s a few things to consider:

  1. Things you do a lot
    A good case in point here is the delay that follows the launching logic in testbed.vi. I have a routine in my toolbox that wraps a conditional structure around a wait, and it is probably one of the most commonly used routines in my toolbox. It’s very simple, but it seems that I am constantly running into situations where I need either the error clusters to create a data dependency that defines exactly when the delay runs; or the error case to allow me to bypass long errors.However, you will likely want to modify this snippet before reusing it — otherwise all you will have is a reusable 2 second delay. Before turning the code into subVI, bring the constant outside the case structure so the delay time becomes a parameter that is passed into the routine. Carrying this idea a step further, turning this code into a subVI can also be an opportunity to recreate the API to better-fit your needs. For instance you may want to be able to wait until a particular clock time, or you may want to be able to specify the delay is something other than milliseconds. In a future post we’ll revisit this subVI talking about using LabVIEW units to simplify code.
    For another example of how I used the repackaging for reuse to tweak the API to make it more to my liking, check-out the changes I made to the logic for closing the launcher front panel. As it was, it would always close the windows regardless of what you were doing, or wanting to do. However, there are times during development when it would be better to leave it open. To address this need, I modified the code so it has an additional input that allows you to specify when you want it to close the front panel. The enumeration has two values: Always and Not in Development.
  2. Low-level functions that should have error clusters, but don’t
    Repackaging the Wait (ms) function is a good example of this point too, but I also have a whole family of routines that put wrappers around things like the built-in one- and two-button dialog box functions. Remember, just because a given function might not be able to generate an error, it doesn’t mean that the routine might not reasonably need to be able to respond to an error.If an earlier stage of a process has failed, do you really want to keep prompting the user to do things? To do so confuses the problem by hiding (from the user’s perspective) where the error occurred. If you don’t think the user’s perceptions matter, remember that a confused user can’t give you good feedback, and efficient troubleshooting and debugging is dependent upon good feedback.
  3. Functions that, while not difficult, took you a while to figure out
    Often times you will run into a situation where it takes you a while to figure out how to do something rather simple, like using .NET calls to perform a network ping more efficiently than using the system executive function. Such code is perfect to turn into reusable code. Just remember to give a little thought to what should be input parameters to make the code flexible.
  4. Functions that implement an agreed-upon convention
    I spend a lot of time talking about conventions and the efficiency they can bring to a development process. An easy way of implementing some conventions is to embody them in code. For example, you can create a VI that builds the standardized directory structure you use, or one that builds paths to the application INI file based on the conventions that your organization uses.
    Note that this point can also serve as your avenue for reuse of larger sections of code. For example, you may to want collect and report errors is a certain way, or store test results in a particular format or in a particular location. Developing a standalone process that implements these techniques can significantly simplify the work needed to standardize on the techniques across all your projects.

Obviously, there are a lot of other places to look for reusable code, but this will get you started. Just don’t be surprised when we come back to this in the future.

A testbed and a toolbox

So where does all this work leave us? Going through what we have built so far, I have implemented a toolbox that currently has two VIs in it, and then used those VIs in the testbed. By the way, while I was there, it occurred to me that many of the projects I will be creating will use the same basic structures for stopping the application and reporting errors, so I also moved the UDEs associated with those operations to a section in the toolbox called Standard UDEs. You can find the modified testbed code in our subversion repository at:

http://svn.notatamelion.com/blogProject/testbed application/Tags/Release 3

while the toolbox is at:

http://svn.notatamelion.com/blogProject/toolbox/Tags/Release 1

Be assured that going forward we will be adding more code to our toolbox, but for now this will do. In terms of using the toolbox, you’ll note also that I did not include the reuse libraries in the project. To do so can create its own kind of cross linking problems, and is unnecessary. All you need to do is simply use the VI’s you need and let LabVIEW maintain the libraries as included in the Dependencies section of the project.

Oh yes, one more thing for you to think about: If I am going to be using the same error collection technique in the future, why didn’t I make the Exception Handler.vi a part of the reuse library? And why is it named “Exception Handler” anyway, it’s just for errors, right?

Until next time…

Mike…

PS: Happy Birthday to me… Yes, it is official: I am older than dirt.

Making UDEs Easy

For the very first post on this blog, I presented a modest proposal for an alternative implementation of the LabVIEW version of the producer/consumer design pattern. I also said that we would be back to talk about it a bit more — and here we are!

The point of the original post was to present the modified design pattern in a form similar to that used for the version of the pattern that ships with LabVIEW. The problem is that while it demonstrates the interaction between the two loops, the structure cannot be expanded very far before you start running into obstacles. For example, it’s not uncommon to have the producer and consumer loops in separate VIs. Likewise, as with a person I recently dealt with on the forums, you might want to have more than one producer loop passing data to the same consumer. In either case, explicitly passing around a reference complicates matters because you have to come up with ways for all the separate processes to get the references they need.

The way around this conundrum lies in the concept of information hiding and the related process of encapsulation.

Moving On

The idea behind information hiding is that you want to hide from a function any information that it doesn’t need to do its job. Hiding information in this sense makes code more robust because what a routine doesn’t know about it can’t break. Encapsulation is an excellent way if implementing information hiding.

In the case of our design pattern, the information that is complicating things is the detailed logic of how the user event is implemented, and the resulting event reference. What we need is a way to hide the event logic, while retaining the functionality. We can accomplish this goal by encapsulating the data passing logic in a set of VIs that hide the messy details about how they do their job.

Standardizing User-Defined Events

The remainder of this post will present a technique that I have used for several years to implement UDEs. The code is not particularly difficult to build, but if you are a registered subscriber the code can be downloaded from the site’s Subversion SCC server.

The first thing we need to do is think a bit and come up with a list of things that a calling program would legitimately need to do with an event — and it’s actually a pretty short list.

  1. Register to Receive the Event
  2. Generate the Event
  3. Destroy the Event When the Process Using it Stops

This list tells us what VIs the calling VI will need. However, there are a couple more objects that those VIs will be needed internally. One is a VI that will generate and store the event reference, the other is a type definition defining the event data.

Finally, if we are going to be creating 4 VIs and a typedef for each event in a project, we are going to need some way of keeping things organized. So let’s define a few conventions for ourselves.

Convention Number 1

To make it easy to identify what event VI performs a given function, let’s standardize the names. Thus, any VI that creates an event registration will be called Register for Event.vi. The other two event interface VI will, likewise, have simple, descriptive names: Generate Event.vi and Destroy Event.vi. Finally, the VI that gets the event reference for the interface VIs, shall be called Get Event Reference.vi and the typedef that defines the event data will be Event Data.ctl.

But doesn’t LabVIEW require unique VI names? Yes, you are quite right. LabVIEW does indeed require unique VI names. So you can’t have a dozen VIs all named Generate Event.vi. Thus we define:

Convention Number 2

All 5 files associated with an event shall be associated with a LabVIEW library that is named the same as the event. This action solves the VI Name problem because LabVIEW creates a fully-qualified VI name by concatenating the library name and the VI file name. For example, the name of the VI that generates the Pass Data event would have the name:
Pass Data.lvlib:Generate Event.vi
While the VI Name of the VI that generates the Stop Application event would be:
Stop Application.lvlib:Generate Event.vi

The result also reads pretty nice. Though, it still doesn’t help the OS which will not allow two files with the same name to coexist in the same directory. So we need:

Convention Number 3

The event library file, as well as the 5 files associated with the event, will reside in a directory with the same name as the event — but without the lvlib file extension. Hence Pass Data.lvlib, and the 5 files associated with it would reside in the Pass Data directory, while Stop Application.lvlib and its 5 files would be found in the directory Stop Application.

So do you have to follow these conventions? No of course not, but as conventions go, they make a lot of sense logically. So why not just use them and save your creative energies for other things…

The UDE Files

So now that we have places for our event VIs to be saved, and we don’t have to worry about what to name them, what do the VIs themselves look like? As I mentioned before, you can grab a working copy from our Subversion SCC server. The repository resides at:

http://svn.NotaTameLion.com/blogProject/ude_templates

To get started, you can simply click on the link and the Subversion web client will let you grab copies of the necessary files. You’ll notice that when you get to the SCC directory, it only has two files in it: UDE.zip and readme.pdf. The reason for the zip file is that I am going to be using the files inside the archive as templates and don’t want to get them accidentally linked to a project. The readme file explains how to use the templates, and you should go through that material on your own. What I want to cover here is how the templates work.

Get Event Reference.vi

This VI’s purpose is to create, and store for reuse, a reference to the event we are creating. Given that description, you shouldn’t be too surprised to see that it is built around the structure of a functional global variable, or FGV. However, instead of using an input from the caller to determine whether it needs to create a reference, it tests the reference in the shift-register and if it is invalid, creates a reference. If the reference is valid, it passes out the one already in the shift-register.

If you consider the constant that is defining the event datatype, you observe two things. First, you’ll see that it is a scalar variant. For events that essentially operate like triggers and so don’t have any real data to pass, this configuration works fine. Second, there is a little black rectangle in the corner of the constant indicating that it is a typedef (Event Data.ctl). This designation is important because it significantly simplifies code modification.

If the constant were not a typedef, the datatype of the event reference would be a scalar variant and any change to it would mean that the output indicator would have to be recreated. However, with the constant as a typedef, the datatype of the event is the type definition. Consequently you can modify the typedef any way you want and every place the event reference is used will automatically track the change.

Register for Event.vi

This VI is used wherever a VI needs to be able to respond to the associated event. Due to the way events operate, multiple VIs can, and often will, register to receive the same event. As you look at the template block diagram, however, you’ll something is missing: the registration output. The reason for this omission lies in how LabVIEW names events.

When LabVIEW creates an event reference it naturally needs to generate a name for the event. This name is used in event structures to identify the specific particular event handler that will be responding to an event. To obtain the name that it will use, LabVIEW looks for a label associated with the event reference wire. In this case, the event reference is coming from a subVI, so LabVIEW uses the label of the subVI indicator as the event name. Unfortunately, if the name of this indicator changes after the registration reference indictor is created, the name change does not get propagated. Consequently, this indicator can only be created after you have renamed the output of the Get Event Reference.vi subVI to the name that you wish the event to have.

The event naming process doesn’t stop with the event reference name. The label given to the registration reference can also become important. If you bundle multiple references together before connecting them to an event structure’s input dynamic event terminal, the registration indicator is added to the front of the event name. This fact has two implications:

  1. You should keep the labels short
  2. You can use these labels to further identify the event

You could, for example, identify events that are just used as triggers with the label trig. Alternately, you could use this prefix to identify the subsystem that is generating the event like daq or gui.

Generate Event.vi

The logic of this template is pretty straight-forward. The only noteworthy thing about it is that the event data (right now a simple variant) is a control on the front panel. I coded it this way to save a couple steps if I need to modify it for an event that is passing data. Changing the typedef will modify the front panel control, so all I have to do is attach it to a terminal on the connector pane.

Destroy Event

Again, this is very simple code. Note that this VI only destroys the event reference. If it has been registered somewhere, that registration will need to be destroyed separately.

Putting it all Together

So how would all this fit into our design pattern? The instructions in the readme file give the step-by-step procedure, but here is the result.

image

As explained in the instructions, I intend to use this example as a testbed of sorts to demonstrate some things, so I also modified the event data to be a numeric, and changed the display into a chart. You’ll also notice that the wire carrying the event reference is no longer needed. With the two loops thus disconnected from each other logically, it would be child’s play to restructure this logic to have multiple producer loops, or to have the producer loop(s) and the consumer loop(s) in separate VIs.

By the way, there’s no reason you can’t have multiple consumer loops too. You might find a situation where, for example, the data is acquired once a second, but the consumer loops takes a second and a half to do the necessary processing. The solution? Multiple consumer loops.

However, there is still one teeny-weensy problem. If you now have an application that consists of several parallel processes running in memory, how do you get them all launched in the proper order?

Until next time…

Mike…

Don’t re-code when you can re-configure

A common problem that you see being discussed on the LabVIEW user forums comes from people who hard-code configuration data in their programs. Often they will be complaining about things like how long it takes to maintain all the different versions of their application, when in truth the only difference between “versions” is the contents of a few constants on a block diagram. Ultimately, these folks may realize that putting configuration data is constants is not good, but they may feel like they are skilled enough, or don’t have enough time, to do anything better.

i suppose that the fundamental trouble is that it is way too easy to drop down a constant and type in some configuration data. Unfortunately, you can end up in a situation where you are constantly having to go back and manipulate or manage this “easy” solution. The good news however, is that it is not hard to come up with a really good way of handling configuration data.

Data Abstraction

The first step to really fixing the problem is to draw a distinction between two things that we typically hold as being the same. Namely, what a particular parameter represents or means, and the parameter’s value. What we need to do is develop a mechanism for obtaining parameter’s value based on what that value represents. Or to say it differently, we need to replace the constant containing a set parameter value with a VI that can return a value based on the parameter’s logical identity. I want to be able to say, for example, “What is the name of the directory where I should store the data files?” and have this VI return the parameter’s current value. Assuming that the configured value is stored outside the code itself, you can change that value to anything you want, anytime you want, without modifying a bit of code.

In terms of where to store this configuration that is outside the code, there are several options:

  • A configuration file on disk
  • Local or networked database
  • The Windows registry

Each of these options has a place but at this point the important point is that you have it somewhere outside the code. Once you have accomplished that, changing the data’s location from say an INI file to a database is comparatively trivial. With that point in mind, here is the code that I use to read data from the application’s ini file. I will put this code is a VI named for the parameter it is reading and use it in place of the constants that I would otherwise use. To make it easy to reuse, you might want to put it in a VI template file. The remainder of this post will concentrate on how the code works.

ini-based parameter reader

The first thing you’ll notice is that it takes the basic structure of a self-initializing functional global variable, or FGV. Remember that the point of this VI is to replace a constant, but you don’t want to have the code re-reading the configuration file every time that it is called. Using a FGV structure allows the code to automatically load the required data the first time it’s needed, and thereafter use the buffered value. Note that if the file ini file operations fail for whatever reason, the logic will also attempt to reread the file each time it is called. Feel free to remove this logic if you desire, but it can be useful.

Next, consider how I derive the path to the ini file. This logic is based on (and simplified by) the conventions that I follow for naming things. All the files associated with a project like my “Ultimate Data Logger” will reside in the directory:

c:/workspace/projects/Ultimate Data Logger

Likewise, the name of the project file itself will be Ultimate Data Logger.lvproj, the name of the top-level VI will be Ultimate Data Logger.vi, the name of the executable that is eventually built will be Ultimate Data Logger.exe and it will be installed in the directory Ultimate Data Logger on the target machine. Moreover, due to the way that Windows works, the name of the ini file associated with that executable must be Ultimate Data Logger.ini. Consequently, simply knowing the name of the application directory will tell me the name of the ini file. All I need to do is add the “.ini” file extension.

When using this logic, you will need to define the desired data value’s logical identity. In the context of this VI, that identity is defined by the Section and Key values on the block diagram. In this definition, the Section value should be seen as a broad category that includes several Keys or individual values. For example, you might have a section “file paths” that includes all the paths that your application needs so in that section you would find keys like “data file destination” or “import directory”.

Obviously, you will also need to change the datatype of the data being obtained from the ini file. This job is easy because the low-level configuration file VIs are all polymorphic and so will adjust to the datatype you wire to them. A couple of recommendations I would offer is to not store paths as paths, but rather convert them to strings and then save the string. Either way will work, but saving paths as strings produces a result that is more human-readable in the ini file. Likewise, when you are working with strings, there is an optional input on the VI that tells the function to save and read the data as “raw” strings. Always wire a true Boolean constant to these inputs.

Finally, note that the code includes the logic to create the section and key in the ini file, if they do not already exist. This is something that I have found useful, but if you don’t feel free to remove it as you see fit.

This basic structure works well for parameters that have single values, but what about if you have a parameter that is associated with a several values, like a cluster? That’s a situation we will deal with — real soon.

Until next time…

Mike…

Conventional Wisdom

The dictionary refers to a “convention” as a set or standardized way of doing something. In LabVIEW programming, conventions are an effective organizational tool that provides an avenue for solving many common problems. In fact there are far more opportunities for defining useful conventions than there is room in this post, so to get you thinking we’ll just cover some of the key areas.

Directory Conventions

The topic of directory structure is a good place to start this discussion because this area is often neglected by developers. As you might expect their primary focus is getting their code running — not where to store the code on disk. After all, one place is as good as any other, right?/p>

Uh, not so much, no… Storing files just anywhere can cause or facilitate a variety of problems. For example, one of the most common ones is that you can end up with multiple VIs sporting the same file name at various locations on disk. The best case scenario is that your application gets cross-linking to the wrong version of a file. The worst case situation is that your code gets cross-linked to a completely different file that just happens to have the same name. And if you’re really unlucky, the wrong file will have the same connector pane and IO as the one you want, so your code won’t even show up as being broken — it just stops working.

More problems can also arise when you decide to try to reuse the code on a new project or backup your work. If the files are spread all over your disk, how can you be absolutely sure you got everything? And the right version of everything? Of course, I have seen the opposite problem too. A developer puts EVERYTHING in one directory. Now you have a directory containing hundreds of files, so how do you find anything?

The correct answer is to go back to first principles and put a little effort into designing your development environment. So if you think about it, there basically two kinds of code you need to store: code that is specific to a particular project, and code that you are planning on reusing. Likewise, you want to minimize the code “changes” that are associated with a VI having been moved on disk. Consequently, I like to start at the root of a hard drive (doesn’t matter which one) and create a folder called Workspace.

image

This directory could actually be named anything, the point is that it will be the home of all my LabVIEW code. Inside this directory I create two sub-folders. The Toolbox sub-folder contains all my code that is intended for reuse. To keep it organized, its contents are further broken up into sub-folders by function type. In addition, I make it easy to get to my reusable code, by point the user.lib menu link for the LabVIEW function palette to this toolbox directory.

The other sub-folder inside my workspace directory is called Projects. It contains sub-folders for the various projects I am working on. Because I sometimes work for multiple clients, I also have my projects segregated by the year that I got the contract. If you work for a single company but support multiple departments, you might want to have separate folders for the various departments.

Please note that this directory structure is one that has worked well for me over the years but it isn’t written in stone. The main thing is that you be consistent so you (and people who in the future might be supporting your code) can find what is needed.

Naming Conventions

The next area where having some set conventions can pay big dividends is in the area of naming things. If you are a part of a larger organization, this is also an area where you should consider having a meeting to discuss the issue because it can have long-term effects of the way you and your group operate.

Online you will find a lot of recommendations as to how to name all sorts of things from entire projects to individual controls and indicators. The thing to remember is that some of these suggestions originated years ago and so are no longer appropriate — or necessary. For example, some people still advise you to prefix the name of a VI with the name of the project in which the VI is used. This technique, however, is counter-productive because it complicates the reuse of the VI. In addition, if you are making appropriate use of LabVIEW libraries (*.lvlib) it is unnecessary because the name of the library creates a namespace that, for LabVIEW’s internal use, is prepended to the VI’s file name.

Say for instance you have a library called Stop Application.lvlib and let’s further say that the library references a VI called Generate Event.vi. Now without the library, this VI name would be a problem because it is too non-specific. It is easy to imagine dozens of VIs that could have that same name. But because the VI is associated with a library, as far as LabVIEW is concerned, the name of that VI is Stop Application.lvlib:Generate Event.vi. By the way, you’ll notice that this composite name reads pretty nice for human beings too. The only remaining issue is that your OS doesn’t know about LabVIEW’s naming conventions, which is why I always put libraries (and the files they reference) in separate directories with the same name as the library, but minus the lvlib part.

Another common place where a naming convention can be helpful is in showing fixed relationships that LabVIEW doesn’t otherwise illustrate. A good use case for this type of naming convention would be with LabVIEW classes. Currently (as of LV2014) you can name a class anything you want but LabVIEW doesn’t organize the class lists in the project file to illustrate inheritance relationships. Until NI decides to fill this gaping hole, you can work around it by implementing a naming convention that (when sorted by name) shows the inheritance relationships. Starting with a top-level parent class, like database.lvclass, you build a hierarchical name for the subclasses by prepending the base name and an underscore character to the subclass name. In this case you might have two subclasses: one for ADO connections and one for SQLite (which uses its own dll). So you would create classes with the names database_ado.lvclass and database_sqlite.lvclass. Now within ADO-compatible databases there is a lot of commonality, but there can be differences so to cover those cases you might want to further define subclasses like database_ado_access.lvclass, database_ado_sqlserver.lvclass and database_ado_oracle.lvclass. Obviously this technique has limitations — like it has to be maintained manually, and you can end up with very long file names — but until NI addresses the basic issue, it is better than nothing.

Two other important conventions are that you should never use the same name to refer to two things that are structurally or different. Likewise, you should never use two different names to refer to the same object.

Style Conventions

At NIWeek this year, I had an interesting meeting. I had a young developer come up to me and asked me whether he was right in assuming that I had written a particular application. I said I had, but asked how he knew. He replied that he had worked on some code of mine on an earlier contract and said he recognized my coding style. For him it was an advantage because when making a modification he knew where to start looking.

Basically, style conventions define a standardized way of addressing certain common situations. For example, I make use of units for floating-point numbers to the greatest degree that I can because it simplifies code. One common situation that illustrates this point is the need to perform math on time values. If you want to generate a time that is exactly 1 day in the future you have a few options:

image

This works, but it assumes that the maintainer is going to recognize that 86,400 is the number of seconds in a day.

image

This is also a possibility and, thanks to constant folding, is just as efficient as the previous example. However it takes up an awful lot of space on the diagram and it essentially hardcodes a 1 day decrement — though that might not be clear at first glance either.

image

Personally, I like this solution because it literally expresses the math involved: Take the current time expressed in days, subtract one day from it and express the result in seconds. Remember that the point of good style is to make it clear what the code is logically doing. Great style makes it clear what you were thinking.

So that’s enough for now to introduce you to the concept of conventions. But be assured, it will come up again.

Until next time…

Mike…