Nortek to Matlab

The latest version of bin2mat is now available.
It will convert binary files from
-Aquadopp (*.aqd)
-Aquadopp Profiler (*.prf)
-Vector (*.vec)
into Matlab mat-files (*.mat).

It is a dll that can be called from Matlab just like any other matlab-function. Download from Software

Best regards,
Kristoffer, Nortek\n\n

 

Kristoffer,

This code makes the transition to MatLab a bit easier.  Howerver, the code is somewhat inelegant - it enlarges the .vec file by a factor of 10, which is non-trivial for a 78 Mb recorder.  780 Mb is a bit much even for today's hard disk and RAM - my dual processor, 2 Gb RAM machine even chokes on these files sometimes.  Especially since  a significant portion of the data is replicated more than once in the .mat file, e.g. the same velocity, amplitude and correlation data in both burst and direction formats.  

A fairly easy and useful revision would be to allow an input argument of the name of the variable(s) desired, and just that variable(s) returned as output.

Regards,

Matt

Hello Matt,

Unfortunately Kristoffer is not available to handle your question. The increase in file size of a factor of 10 is a real problem for large data files.  I have looked into this and identified the problem to be a result of two issues.

1) The first has to do with the fact that the MatLab format operates with doubles. This means 8 Bytes, whereas the Nortek data format is either 1 or 2 Bytes.  The end result of saving the Vector data under the MatLab format means the data file will be a minimum of 6 times greater  " />'> .

2) The second issue is the fact that there is some reduntant data being saved.  This acounts for the other increase in file size.

The solution, as you mention, would be to make the funtion accept an arguement list of just what you wanted so that the end file size would be much smaller (but probably never smaller than the original raw file size due to issue 1).

Instead of climbing into unfamiliar code for this project, I suggest the following alternative.  First use the Vector software to do a file conversion.  This will output several different files.  Choose the one that suits your needs (output listed in *.hdr file), and load this ASCII file into MatLab using one of the MatLab functions like DLMREAD (may be issues if you have version 5.2 or lower!.  This way you can load just the data as you want.  Especially if you use a range arguement list (see the help on this function).

I realize this does not provide the ease of Bin2Mat but it will probably get the data into a working MatLab array within 15 minutes, whereas waiting for me put additions into someone elses code would take ... well at a factor of at least 10   .

I hope this gets you on track and I welcome any other comments you may have.

regards,
Torstein

Torstein,

Thanks for the workaround suggestion.  For the files which I've already converted w/bin2mat, I can also use load.m to specify only the variable I need instead of loading the entire .mat file.

Hopefully, a future update to bin2mat will enable selective extraction from the .vec files.

Regards,

Matt

One more note on bin2mat - this executable converts .vec to .mat quickly, but names two variables, ST?dnum and ST?time, with an extra character (I've seen ., $ and " in place of the ?).  MatLab then chokes on this character when a load command tries to load the entire .mat file, i.e. load('vector1.mat'.  But these two variables can still be accessed via a specified load command, i.e. time=load('vector1.mat','ST.time';

One more issue with bin2mat:

There seems to be discrepencies in how bin2mat handles assigning samples to their bursts.  

1) The number of columns in the VB matrices does not match the number of bursts which the Vectors were programmed to collect.  The difference is not constant, but unique to each .vec file.

2) The variable VTburst has step changes in the burst number at irregular spacing, e.g.

>> find(diff(VTburst(1:5e4))==1)'
ans =
       4800
       9600
      14404
      19209
      24011
      28817
      33617
      38417
      43217
      48020

3) The number of colums in the VB matrices does not agree with the VTburst count, e.g. for the example in #2, the number of columns in VB matrices is 4795.

Hello Matt,

Your more specific questions really demonstrate one thing to me and that is that you know far more than me (or probably anyone else here at Nortek) about the bin2mat tool  

Unfortunately, without the creator available, I do not see the necessary improvements getting done in the near future.

If there is anyone who would like to see this move forward, then I offer the source code.  The source code is C++.  Simply give a shout and I will email it or post it on our ftp site.

regards,
Torstein

How does the .vec file format differ from the .adv file format
generated by the CollectV software?  Will the bin2mat utility convert files generated by my NDV based system?

Jim Kirby

Hello Jim,

Both are binary files, but the data structures are different. If you would like to have a description of the data formats please send a request for the Paradopp interface manual to inquiry@nortek.no.

bin2mat does not convert NDV files.

Best regards,
Oistein

Hopefully this is the right thread for this....

Would it be beneficial to have a thread/place devoted to sharing of Matlab scripts for all aspects Nortek instruments?

I've written a couple useful scripts for ASCII import, which perhaps others could use.

-Tyson Hilmer

Hello Tyson,

Good idea, please feel free to start a new "Matlab" thread and share your scripts.

Best regards,
Oistein

Has the mex file for bin2mat been recompiled for MATLAB7? It doesn't appear to be working on my PC laptop anymore. Thoughts?

 

Kurt

Hello Kurt,

Which data type (instrument) are you trying to convert to Matlab.  As you can see from earlier postings the Bin2Mat is not exactly bug free and maintanance on this stopped several years ago when the person moved on to other things.  The code is available if you wish to roll up your sleaves

 

You would probably be better off if you used Nick Mortimer's (CSIRO in Australia) code.  He basically started from square one and fixed the problems of memory consumption.  He has posted the code on the forum elsewhere (I will get back to you on that if you can not find it with the search (use "nick")).  He may also have a new an improved version.  This was first written for the Vector and expanding on to other instruments, and this is why I was asking which instrument you were using.

kind regards,

Torstein

 

Hi Torstein,

I'm using Aquadopp single point and Profilers (and AWAC's as well but not for this). I did download Nick's mex file, but haven't had a chance to try it yet. If you have the souce code for bin2mat, I may try recompiling it for mac. Otherwise, I can always just have people use the data conversion program to export ascii data and deal with it that way. We generally don't have massive datafiles with the Aquadopp. Thanks

 

Kurt

Kurt,

I agree that if the data files are small and managable like the ASCII files from Aquadopps and Profilers, then it is probably easier to import these in matlab.  The binary converter tools are better suited for data files from the Vector, AWAC and HR Profiler which tend to get cumbersome due to the higher sampling rates.

Attached is what I could dig up on the Bin2Mat C code.  Sorry I can't be more helpful with this.  Again, I think Nick's converter is will serve you better.

 

kind regards,

Torstein

Do you have a "bin2mat" program to process Continental Profiler data in Matlab?

Thanks,
Eric

Eric,

Don't think that the Continental data format was included at the time Bin2Mat was made. 

-Torstein