Matlab - Parsing Data Files With Header Content

Another Parsing Example

Consider the following data format:

!Agilent Technologies
!Agilent 
!Date: Thursday, November 23, 2006 05:48:58
!Correction: S11 S21 S12 S22 
!S2P File: Measurements: S11, S21, S12, S22:
# Hz S  dB   R 50
2820000000 -40 0 -40 -178.796 -40 -178.796 -40 0
2820225000 -40 0 -40 -177.592 -40 -177.592 -40 0
2820450000 -40 0 -40 -176.388 -40 -176.388 -40 0
2820675000 -40 0 -40 -175.1839 -40 -175.1839 -40 0
2820900000 -40 0 -40 -173.9799 -40 -173.9799 -40 0
2821125000 -40 0 -40 -172.7759 -40 -172.7759 -40 0
.
.
.
.
3179775000 -40 0 -40 -53.5786 -40 -53.5786 -40 0
3180000000 -40 0 -40 -52.3746 -40 -52.3746 -40 0
!End of file
!Agilent Exiting

You can use the following code to parse the data:

function [parsedData]= importMyData(name)
%this function parses data files that have some header content
%and some ending content
 
%the input to this function is the file name of the data file.  
%If the data file is not in the current Matlab directory
%you must include the entire directory path.  
%opens the file    
fid = fopen(name);
 
%reads line one by one into a cell
a= 1;
line = 0;
while line ~= -1
    line = fgetl(fid);
    data{a} = line;
    a = a +1;
end
data(end) = []; %get rid of the last line
 
fclose(fid);
 
%this for loop determines where the numerical data starts
for p=1:length(data)
    if (~isempty(str2num(data{p}))) break; end
end
 
%this loop saves the numerical data into parsedData
%until the numerical data stops
for x=p:length(data)
    if(isempty(str2num(data{x}))) break; end
    temp = str2num(data{x});
    parsedData(x-p+1,:) = temp; 
 
end

End of tutorial.

Pages: 1 2 3

7 Responses to “Matlab - Parsing Data Files With Header Content”

on 25 Oct 2007 at 8:12 pm 1Quan Quach

Also, if you don’t know how many rows your numerical data set consists of, you can use a while loop instead of a for loop.

r = 1
while 1
readin = fgetl(id);
if readin == -1 break; end
data(r,:) = str2num(readin);
r = r+1
end

on 24 Jan 2008 at 2:44 am 2arya

I can only say

You Made My day

thanx a lot

on 15 Apr 2008 at 12:33 pm 3misha

Hi,
your matlab help is super useful for me. thank you.

Q: i have a file that has a row of text every 1000 lines of numeric data. looks like that:
text
int1
int2
…
int1000
text
int1
int2
…
int1000
text

etc etc.

if there a way to read the last line of text (it has useful information) and get numeric data. well, i’m sure there is a way, i’m wondering if you know how to do it.
thank you

on 15 Apr 2008 at 12:36 pm 4misha

for the previous post-
i forgot to mention a few things.
1. the last line of text is not at the very end of the file
2. the original file is a .txt file

on 15 Apr 2008 at 5:33 pm 5Daniel Sutoyo

Misha,

the examples in the tutorial should lay out the framework you need… Since you already know there are 1000 numeric lines, this makes it straightforward

nblocks = # of txt + 1000 lines (if you don’t know this, change this to outer loop to while loop, and make it true when there is no empty lines read in)

for j=1:nblocks

1. call fgetl to read in text ( I don’t know how many lines or how your text are)
2. Then call 1000 times to read in numeric data

end

You can switch the order around if necessary. If you want your code to be more dynamic, you can always add ‘if’ statements to check if the content are text headers or numerical numbers.

on 21 Apr 2008 at 11:50 pm 6behzad

thanks for the previous postings…very helpful.
however, i have some data in a .txt file. i don’t know when the data row finish (i.e. i don’t know which row is the last row!) and i have some lines of text in between every (for instance) 10 or 20 rows of data. could you please help me with that?

cheerZ
behzad

on 27 Jun 2008 at 10:20 am 7Joshua

behzad–

I’m doing something very similar, where the # of lines of numbers between headers is variable. What I do is use str2double to check if the current line is data or a string. I also put in state variables to keep track of where to put the data, ignore whitespace and empty lines, etc., but this is the core of it.

while ~feof(fid) % go until EOF
line = fgetl(fid);
if isnan(str2double(line)) % it’s a text header
disp(line); % do something with it
else % else, it’s data
disp(str2double(line)); % do something
end
end

Or if you need to input matrices, not just doubles and ints, use str2num:

while ~feof(fid) % go until EOF
line = fgetl(fid);
[x status] = str2num(line);
if ~status % it’s the text header
disp(line); % do something with it
else % else, it’s data
disp(x); % do something
end
end

Cheers!

blinkdagger