NMEA Data Logging Utilities

This project presents a couple of utilities you can run to log the data processed by your NMEA on-board station. It is hosted on Source Forge.

This document intends to provide access to the software to non developer persons, who would not necessary be interested in working on the source code.

All its components have been developed in Java®, to preserve the possibility to run on several platforms - in has currently been tested on Windows and Linux - and written keeping in mind that this would have to run on small laptops with limited resources, in term of memory and CPU.

The project is made out of three distinct generic pieces

The path is simple:
  1. You read the data on board when sailing, and you store them somewhere on your hard disc.
  2. Once you're back at the dock - or later - you turn the data you've logged above into a more generic format, like XML or ASCII Delimited.
  3. You analyze the data transformed above with the analyzer of your choice, like Open Office, Microsoft Excel, Lotus Notes, or any other tool - including something you could write - as long as it supports the format you've turned your data into during the previous step. XML is a very generic one. If necessary, it can be turned into an Acrobat PDF document.

Note for non-Java programmers

Java is a language the runs virtually on any platform. Its specifications have been elaborated by Sun.
In case you just want or need to run the software this document is about, you just need two things: Once you have downloaded the Java piece, you have to install it. And once it is installed, you can run the commands described in this document.
If you feel that those explanations are not sufficient, please let me know (by email, at olivier@lediouris.net), and I'll add more .

The Logging Software

 Top

This piece is the corner stone of all the process. Without data logged during the sailing, the story stops here.
It does two main things:

  1. It reads the NMEA ouput through the serial port of the computer
  2. It can log those data on the hard disc of the computer.
Also, it can read data without logging them. This is what you would do to make sure the data are read correctly from the station, like when you are still at the dock for example, and not interested yet in logging them.
Reading the NMEA data is done as we said through the serial port of the laptop. The serial port access in Java is provided by a free package from Sun, called commapi, which you have to download, and make sure it fits your system. Windows and Linux are both different, at least from the serial port access point of view. The logging software makes extensive use of the API (Application Programming Interface) provided by this package.
Storing the data currently happens to use flat files - the simplest ever on any system. We've tried to use databases, but they're actually providing features we don't need, as we only intend to write on the storage device. Indexing the data does not make any sense during the logging process. Plus using flat files consumes way less resources than any database mechanism you can think of.
For a given day of logging, a directory is going to be created on your system, and populated with the data file as long as the logging goes on. The name of this directory is created after the current date. If your logging is done on April the 27th, 2004, then the directory will be called "2004-04-27". It will be pretty easy to identify...
In this directory, files named data_000001, data_000002, etc, will be created as the logging goes on. Those are text files, you can edit them, although that would not tell you much. Anyhow, if you do so, you will see that they contain bulk NMEA data, unparsed, once again to save resources during the logging proceess. Those files contain up to 100 records. After that, another file is created. After our experience with a B&G Hydra 2000 station, as we log only 5 channels (GLL, HDM, MWV, VHW, and DBT), it takes between 1.5 and 2 minutes to fill up one file. And after the pattern we use for the file name, we can generate up to ten millions files. In the worst case, it represents 15 millions of minutes. Should be OK for one day!

This is what the user interface looks like:

Minimal Logger

Notice the box you have to check to enable the logging. This box can be checked, un-checked, re-checked, re-un-checked at any time.

Download this piece

You first need to download and install Java, as well as the CommAPI package, both from Sun.
Once you're done, you can download the component, for Windows, or for Linux, and see below what to do with it.

Windows users

@echo off
set path=C:\commapi;%PATH%
set classpath=C:\commapi\comm.jar;minimalLogger.jar;%CLASSPATH%
java  ui.launcher.MinimalLogger

Linux users

export PATH=/somewhere/commapi;$PATH
export CLASSPATH=/somewhere/commapi/comm.jar;minimalLogger.jar;$CLASSPATH
java  ui.launcher.MinimalLogger

The Transformer

 Top

The transformer is the piece to run once the logging part is completed. It will turn the NMEA strings logged into the data files into something less specific, that can be understood by generic techniques or tools. It actually provides two output formats:

The GUI looks like it:

Data Transformer

Notice the following points:
  1. You have to provide the name of the directory containing the files you've logged
  2. You have to provide the name of the file to generate
The two "Browse" buttons are here to help you the right resources.
Then when you click the "Proceed" button, the data files are read and parsed, to produce the requested output.
If you've chosen the Spreadsheet output, you will generate a file looking like this: 

27-Mar-2004 07:59:44:;0.8;359;3.2
27-Mar-2004 07:59:46:;22.0;0.79;352;3.1;37.80838166666666;-122.43738333333334;31
27-Mar-2004 07:59:47:;21.899999618530273;0.79;352;3.1;37.808393333333335;-122.43736666666668;31
27-Mar-2004 07:59:48:;21.899999618530273;0.77;355;3.1;37.808393333333335;-122.43736666666668;31
27-Mar-2004 07:59:49:;21.799999237060547;0.77;355;3.1;37.80840499999999;-122.43735000000001;31
27-Mar-2004 07:59:50:;21.799999237060547;0.77;354;3.0;37.80840499999999;-122.43735000000001;31
27-Mar-2004 07:59:51:;21.700000762939453;0.77;354;3.0;37.80842666666667;-122.43731666666666;31
27-Mar-2004 07:59:53:;21.700000762939453;0.78;354;2.8;37.80842666666667;-122.43731666666666;31
27-Mar-2004 07:59:55:;22.100000381469727;0.79;348;2.6;37.808440000000004;-122.4373;30
...
XML would be like: 
      
<NMEALogging>
   <log date="27-Mar-2004 07:59:44:" rnk="1">
      <nmea type="$IIVHW">$IIVHW,,,031,M,0.80,N,,*6E</nmea>
      <nmea type="$IIMWV">$IIMWV,359,R,3.2,N,A*2D</nmea>
   </log>
   <log date="27-Mar-2004 07:59:46:" rnk="2">
      <nmea type="$IIDBT">$IIDBT,022.0,f,06.7,M,03.7,F*14</nmea>
      <nmea type="$IIVHW">$IIVHW,,,031,M,0.79,N,,*68</nmea>
      <nmea type="$IIMWV">$IIMWV,352,R,3.1,N,A*25</nmea>
      <nmea type="$IIGLL">$IIGLL,3748.5029,N,12226.243,W,,A*19</nmea>
      <nmea type="$IIHDM">$IIHDM,031,M*3E</nmea>
   </log>
   <log date="27-Mar-2004 07:59:47:" rnk="3">
      <nmea type="$IIDBT">$IIDBT,021.9,f,06.7,M,03.7,F*1E</nmea>
      <nmea type="$IIVHW">$IIVHW,,,031,M,0.79,N,,*68</nmea>
      <nmea type="$IIMWV">$IIMWV,352,R,3.1,N,A*25</nmea>
      <nmea type="$IIGLL">$IIGLL,3748.5036,N,12226.242,W,,A*16</nmea>
      <nmea type="$IIHDM">$IIHDM,031,M*3E</nmea>
   </log>
   <log date="27-Mar-2004 07:59:48:" rnk="4">
     ...
The next section will describe what to do with such data.

Download this piece

You first need to download and install Java, from Sun, as mentionned above. You've probably done this already.
Once you're done, you can download the component, for Windows, or for Linux, and see below what to do with it.

Windows users

@echo off
@echo +----------------------+
@echo ! Data File Generation !
@echo +----------------------+
set PATH=.\jre\bin;%PATH%
java -classpath transformer.jar parsenmeadata.BackFromSailing

Linux users

echo +----------------------+
echo ! Data File Generation !
echo +----------------------+
java -classpath transformer.jar parsenmeadata.BackFromSailing

The Analyzer

 Top

Spreadsheet

The easiest would be to use the 'Spreadsheet' generation, and then to import the generated file in your Spreadsheet program. I have tried successfully MS Excel, as well as OpenOffice. You would have a rendering like this:

Spreadsheet rendering

Notice that it only displays Apparent Wind data. We would be more interested in the True Wind data.
That is pretty easy to do, you just need to enter the right formula:

Spreadsheet rendering, with true wind data

In the spreadsheet above, the formulas on line 5 are:
For TWS: =SQRT((E5*E5) +(C5*C5) - 2*C5*E5*COS(D5* PI()/180))
For TWA: =IF(J5<>0,ATAN(E5*SIN(D5*PI()/180)/(E5*COS(D5*PI()/180)-C5))*(180/PI()), 0)
For TWD: =IF(L5>360, L5-360, IF(L5<0,360+L5,L5))
Notice that there is a hidden column, L, its formula is:
=IF(J5<>0,H5+$C$1+K5, 0)
This being done, the spreadsheet will give you enough flexibility to draw any diagram you want!

XML

This one is the most generic format you can think of, XML parsers can be found for virtually any programmation language. We've used Java.
It involves a bit of programmation, but it can pay off!
We are currently working on such a UI:

DashBoard

... More to come.

Download

We do not provide any download for this section.

Feedback

If you have any feedback, question, suggestion, please drop me an email at olivier@lediouris.net.

© 2004, The Don Pedro Project