Technical Women
Today
-
Just files.
-
File metadata.
-
UNIX file system semantics.
-
Hierarchical file systems.
ASST3.1 Checkpoint
-
If you have not started
ASST3.1, you’re behind. -
If your coremap allocator is mostly working, you’re OK.
ASST3.1 is due Friday at 5PM. No extensions.
Disks: Questions?
Disk Parts
-
Platter: a circular flat disk on which magnetic data is stored. Constructed of a rigid non-magnetic material coated with a very thin layer of magnetic material. Can have data written on both sides.
-
Spindle: the drive shaft on which multiple platters are mounted and spun between 4200–15,000 RPM.
-
Head: the actuator that reads and writes data onto the magnetic surface of the platters while rotating at tens of nanometers over the platter surface.
Disk Locations
-
Track: think of a lane on a race track running around the platter.
-
Sector: resembles a slice of pie cut out of a single platter.
-
Cylinder: imagine the intersection between a cylinder and the set of platters. Composed of a set of vertically-aligned tracks on all platters.
Spinning Disks Are Different
Spinning disks are fundamentally different from the other system components we have discussed so far.
-
Difference in kind: disks move!
-
Difference in degree: disks are slow!
-
Difference in integration: disks are devices, and less tightly-coupled to the abstractions built on top of them.
Sources of Slowness
Reading or writing from the disk requires a series of steps, each of which is a potential source of latency.
-
Issue the command. The operating system has to tell the device what to do, the command has to cross the device interconnect (IDE, SATA, etc.), and the drive has to select which head to use.
-
Seek time. The drive has to move the heads to the appropriate track.
-
Settle time. The heads have to stabilize on the (very narrow) track.
-
Rotation time. The platters have to rotate to the position where the data is located.
-
Transfer time. The data has to be read and transmitted back across the interconnect into system memory.
The I/O Crisis
Disks are getting bigger but not faster.
Disks: Questions?
File Systems To The Rescue
-
Requires reading and writing entire 512-byte blocks.
-
No notion of files, directories, etc.
File systems take this limited block-level device and create the file abstraction almost entirely in software.
-
Compared to the CPU and memory that we have studied previously more of the file abstraction is implemented in software.
-
This explains the plethora of available file systems: ext2,3 and 4, reiserfs, NTFS, jfs, lfs, xfs, etc.
-
This is probably why many systems people have a soft spot for file systems even if they seem a bit outdated these days.
What About Flash?
No moving parts! Great! We can eliminate a lot of the complexity of modern file systems. Yippee!
-
Have to erase an entire large chunk before we can rewrite it.
-
And it wears out faster that magnetic drives, and can wear unevenly if we are not careful.
Sigh… things are sounding complicated again.
Clarifying the Concept of a File
Most of us are familiar with files, but the semantics of file have a variety of sources what are worth separating:
-
Just a file: the minimum it takes to be a file.
-
About a file: what other useful information do most file systems typically store about files?
-
Files and processes: what additional properties does the UNIX file system interface introduce to allow user processes to manipulate files?
-
Files together: given multiple files, how do we organize them in a useful way?
Just a File: The Minimum
-
Reliably store data. (Duh.)
-
Be located! Usually via a name.
Basic File Expectations
-
file contents should not change unexpectedly.
-
file contents should change when requested and as requested.
Basic File Expectations
These requirements seem simple but many file systems do not meet them!
03 Mar 2012: Bug Report–Serious file system corruption and data loss caused to other NTFS drives by Windows 8 CP
Basic File Expectations
Failures such as power outages and sudden ejects make file system design difficult and exposed tradeoffs between durability and performance.
-
Memory: fast, transient. Disk: slow, stable.
About a File: File Metadata
-
When was the file created, last accessed, or last modified?
-
Who is allowed to what to the file—read, write, rename, change other attributes, etc.
-
Other file attributes?
Where to Store File Metadata?
An MP3 file contains audio data. But it also has attributes such as:
-
title
-
artist
-
date
-
In the file itself.
-
In another file.
-
In attributes associated with the file and maintained by the file system.
Where to Store File Metadata?
-
Example: MP3 ID3 tag, a data container stored within an MP3 file in a prescribed format.
-
Pro: travels along with the file from computer to computer.
-
Con: requires all programs that access the file to understand the format of the embedded metadata.
Where to Store File Metadata?
-
Example: iTunes database.
-
Pro: can be maintained separately by each application.
-
Con: does not move with the file and the separate file must be kept in sync when the files it stores information about change.
Where to Store File Metadata?
-
Example: attributes have been supported by a variety of file systems including prominently by BFS, the BeOS file system.
-
Pro: maintained by the file system so can be queried and queried quickly.
-
Con: does not move with the file, and creates compatibility problems with other file systems.
Processes and Files: UNIX Semantics
Many file systems provide an interface for establishing a relationship between a process and a file.
-
"I have the file open. I am using this file."
-
"I am finished using the file and will close it now."
-
Can improve performance if the OS knows what files are actively being used by using caching or read-ahead.
-
The file system may provide guarantees to processes based on this relationship, such as exclusive access.
-
Some file systems, particularly networked file systems, don’t even bother to establish these relationships. (What happens if a networked client opens a file exclusively and then dies?)
File Location: UNIX Semantics
UNIX semantics simplify reads and writes to files by storing the file position for processes.
-
This is a convenience, not a requirement: processes could be required to provide a position with every read and write.
UNIX File Interface
-
open("foo"): "I’d like to use the file named foo." -
close("foo"): "I’m finished with foo."
-
read(2): "I’d like to perform a read from file handle 2 at the current position." -
write(2): "I’d like to perform a write from file handle 2 at the current position."
-
lseek(2, 100): "Please move my saved position for file handle 2 to position 100.