Files
University of Dublin
Trinity College
Data placed on secondary storage is collected into files
The layout of the data in the file will have an enormous
impact on how long it takes to find a particular piece of
data
File Basics
We don’t want to have to look through the whole file if
we are on looking for one small piece of data
[email protected]
We want to find the right piece of data with as few
accesses to the file as possible
File Basics
Logical Files
2
Opening and Creating Files in Java
How do we OPEN and CREATE files in Java?
output = new DataOutputStream(new FileOutputStream("client.dat"));
Program
Operating
System
In this code:
• Files are opened by creating objects of stream classes
FileInputStream and FileOutputStream
• If the file does not exist a file is created with that name
• A DataOutputStream object named output is created for the
client.dat file. The argument “client.dat” is passed to the
FileOutputStream constructor that opens the file. This establishes a
“line of communication”
Logical
Files
Physical
Files
Other
Hardware
Devices
File Basics
3
File Basics
Closing Files in Java
When you close a file, you can use the logical filename
for another file
Closing a file that has been used for output also
ensures that everything has been written to disk:
Reading Files in Java
Before reading a file it must be opened
input = new DataInputStream(new
FileInputStream("client.dat"));
To read the file you must specify what type of data is to
be read. The piece of data is usually assigned to a
previously declared variable, e.g.
• More efficient to write data in blocks than in bytes
• Therefore operating system store data for input/output in a buffer
• Closing a file ensures the buffer is flushed to disk
In Java, a FileOutputStream object can be closed by
close(). For example output.close()
File Basics
4
account = input.readInt( );
name = input.readUTF( );
balance = input.readDouble();
5
File Basics
6
1
Writing Files in Java
Detecting End of File (EOF)
As we read from a file the operating system keeps track of our
location using a read/write pointer
Before writing, a file must be opened:
output = new DataOutputStream( new
FileOutputStream("client.dat"));
In Java if an EOF marker is reached while reading the
EndofFileException is thrown and file closed
This is coded as follows –
The file is written by copying values to it:
try {
account = input.readInt( );
...
}
catch ( EOFException eof ) {
input.close( );
}
output.writeInt( accountNumber );
output.writeUTF(firstNameField.getText());
output.writeDouble(d.doubleValue());
File Basics
7
File Basics
8
Stream File
Consider the following information –
Stream File
wilde> od -A x -xc lotr.txt
000000 4c6f 7264 206f 6620 7468 6520 5269 6e67
L
o
r
d
o
f
t
h
000010 732c 2031 3839 2c20 4661 6e74 6173 790a
s
,
1
8
9
,
F
a
000020
• Lord of the Rings, 189, Fantasy
• Star Wars, 210, Sci-Fi
e
n
t
R
i
n
g
a
s
y
\n
When this information is written to a (stream) file, we
lose the organisation of the units of data
od is a program which enables the user to dump file
information in octal and other formats.
When the file is accessed by a program its information
is seen only as a string of characters
The output above is shown as hexidecimal shorts.
Solution : Use records with field structures
File Basics
9
File Basics
Field
What is a Field?
Movie Class as Fields
Writing out the Movie Class as fields to a stream file
using Binary, length based and fixed length encodings
of the fields, looks like
• A Fixed or variable number of bytes that form a data value
• E.g. Movie Name such as Star Wars
There are many ways of adding structure to files to
maintain the identity of fields, for example
• Choose a special character/delimiter that will not appear as a
legitimate character within a field and then insert that character into
the file after writing each field… called delimited-text field
• Use a fixed length for each field (the size depending on field in
question) and pad out when length of actual data value is less than
the fixed length… called fixed-length field
• Write the length of the value (in bytes) followed by the value in
exactly that number of bytes… called length-based field
• Write the name of the field and then value both represented as
delimited-text fields… called identified field
File Basics
10
11
public class Movie {
….
public void write(DataOutputStream out)
throws IO Exception {
//write object as length based and fixed length encodings
out.writeShort(name.length());
out.writeBytes(name);
out.writeInt(id);
out.writeShort(genre.length());
out.writeBytes(genre);
}
};
File Basics
12
2
Class to support length-based field
Class FieldOps {
// a class that supports static operations on length-based fields
public static String readLength(DataInput in)
throws IOException {
//read string from 2-byte length and bytes
int len = in.readShort();
byte [] bytes = new byte[len];
in.readFully(bytes);
return new String(bytes);
}
More complex than write, as first needs to read length
and then create array of that many bytes
Public class Movie { ….
public void read(DataInput in)
throws IO Exception {
//read object as length-based and fixed-length fields
name = FieldOps.readLength(in);
public static void writeLength (DataOutput out, String str)
throws IOException{
//wrtie string as 2-byte length plus bytes
out.writeShort(str.length());
out.writeBytes(str);
}
};
File Basics
Reading Movie Fields
id = in.readInt();
genre = FieldOps.readLength(in);
}
};
13
File Basics
Record
What is a Record?
• Fixed or variable length collections of fields
• E.g. Movie Record such as Star Wars, 210, SciFi
A record is a set of fields that belong together when the
file is viewed in terms of a higher level organisation.
A record is another level of organisation that we impose
on the data to preserve meaning
Records do not necessarily exist in the file in any
physical sense
But they are an important logical notion
File Basics
15
Movie Class as a Record
Writing out the Movie Class as a record to a stream file using Binary,
length based and fixed length encodings of the record, looks like
Byte Array Streams support use of stream operations on a memory buffer
Public class Movie { ….
public void writeRecord(DataOutput out)
throws IO Exception {
//write the record into a byte stream
ByteArrayOutputStream byteStream
= new ByteArrayOutputStream();
DataOutputStream recStream
= new DataOutputStream(byteStream);
write(recStream); // movie.write the movie object into ByteStream
//write the length and the record into the output stream
byte [] buffer = byteStream.toByteArray();
out.writeShort(buffer.length());
out.write(buffer);
}
File Basics };
16
Movie Class as a Record
Reading the movie record into a movie class
14
Records to Blocks mapping
Records of a file must be allocated to disk blocks
public class Movie { ….
public void readRecord(DataInput in)
throws IO Exception {
int len = in.readShort();
byte [ ] buffer = new byte[len];
in.readFully(buffer);
//turn the byte array buffer into an input stream
DataInputStream recStream
=new DataInputStream(new ByteArrayInputStream(buffer));
read(recStream); // read the Movie object using movie.read
}
};
• Recall, a block is the unit of data transfer between disk and
memory
Blocking Factor (bfr) for a file
• Number of records that can fit in a block when block size > record
size
• bfr = round_down(B/R)
– B is block size in bytes
– R is actual record size in bytes for fixed length records but is the average record
size in bytes for variable length records
• Unused portion = B – (bfr*R) bytes
File Basics
17
File Basics
18
3
Spanned/Unspanned Organisation
What happens in case R > B?
• Spanned organisation
– End of first block points to rest of record in an unused portion of another block
• Unspanned organisation
– Each record starts at a new block and uses next block, with unused portions not
used
Files of Records
• A file is a sequence of records, where each record is a collection of
data values (fields or data items)
• A file descriptor (or file header) includes information that describes
the file, such as the field names and their data types, and the
addresses of the file blocks on disk
• Records are stored on disk blocks.
• The blocking factor for a file is the (average) number of file records
stored in a disk block
• The physical disk blocks that are allocated to hold the records of a
file can be contiguous, linked, or indexed.
• In a file of fixed-length records, all records have the same format.
– Usually, unspanned blocking is used with such files
• Files of variable-length records require additional information to be
stored in each record, such as separator characters and field types.
– Usually spanned blocking is used with such files
File Basics
19
File Basics
File Operations
OPEN: Readies the file for access, and associates a
pointer that refers to a current file record at each point
in time
20
File Operations
READ: Reads the current file record into a program
variable
INSERT: Inserts a new record into the file, and makes it
the current file record
FIND: Searches for the first file record that satisfies a
certain condition, and makes it the current file record
FINDNEXT: Searches for the next file record (from the
current record) that satisfies a certain condition, and
makes it the current file record
DELETE: Removes the current file record from the file,
usually by marking the record to indicate that it is no
longer valid
MODIFY: Changes the values of some fields of the
current file record
File Basics
21
File Basics
File Operations
CLOSE: Terminates access to the file
22
Review
Operating Systems provide a logical view of physical
files to Programs
Files in Java
REORGANISE: Reorganises the file records. For
example, the records marked deleted are physically
removed from the file or a new organisation of the file
records is created.
• Opening, reading, writing, EOF, closing
Stream Files
Fixed and Variable Fields
• Delimited-text, fixed-length, length-based, identified
READ_ORDERED: Reads the file blocks in order of a
specific field of the file
Records (and their relationship to Blocks)
• bfr, spanned/unspanned
Files of Records
File Operations
File Basics
23
File Basics
24
4