Документация к PFS3 — ZXNet «real.speccy»

From Pavel Fedin To All 1 January 2000

В связи с поднявшимся интересом кидаю сюда выдержку из guide с описанием
структуры PFS-диска.

--- cut ---
This file forms part of PFS3 (Professional File System 3)
Written by Michiel Pelt
Copyright 1999 by GREat Effects Development
$Id: Proginfo 1.7 1999/06/30 18:38:08 Michiel Exp Michiel $

13 May 1999

PFS3 disk structure
-------------------

This document describes the blocks and structures used by PFS. Only the
new harddisk version will be described here. For the floppy version you
can refer to previous releases.

*********************
* the bootblocks
*********************

The first two blocks are the bootblocks, as required by AmigaDOS. The
first longword of the first bootblock contains the dostype: PFS\1. The
rest is empty. So the first bootblock looks like this:

___________________
| |
000 | disktype |
|___________________|
004 | |
| |
| . |
| . |
200 |___________________|

********************
* the rootblock
********************

Directly behind the bootblocks you can find the rootblock, on block 2.
This is the most important block of a PFS disk. PFS keeps this block
cached at all times. It looks like this:

With the disktype field PFS3 recognizes that the disk is a PFS disk. It is
the same as what can be found in the bootblock.

Options is a 32 bit field which specifies which PFS3 extensions are enabled.
The table below states the bit number, the symbolic name, the release in
which the option was introduced and its meaning.

bit | name | rel | function
----|----------------------|-----|-----------------------------------------
0 | MODE_HARDDISK | 4.1 | harddisk mode if 1; currently always 1
1 | MODE_SPLITTED_ANODES | 4.1 | anodes splitted in block/offset if 1
2 | MODE_DIR_EXTENSION | 4.1 | enables directory-entry extension
3 | MODE_DELDIR | 4.1 | deldir enabled if 1
4 | MODE_SIZEFIELD | 4.1 | the disksize field (offset 0x54) is valid
5 | MODE_EXTENSION | 4.1 | the rootblock extension is enabled
6 | MODE_DATESTAMP | 4.1 | block datestamps enabled if 1
7 | MODE_SUPERINDEX | 4.2 | extension that makes disks >5G possible
8 | MODE_SUPERDELDIR | 5.0 | extension that makes larger deldir possible
9 | MODE_EXTROVING | 5.0 | reserved roving pointer is per block (was
32)
10 | MODE_LONGFN | 5.0 | support for long filenames

PFS3 has, by default, all options enabled. One exception to this is
MODE_SUPERINDEX, which is only enabled for disks larger than 5G. If
MODE_SUPERINDEX is enabled, the list of bitmap index blocks is larger
and covers offset 060 to 200 of the rootblock. The anodeindex blocks
are referenced by a new blocktype: the superblock which can be reached
from the rootblock extension.

PFS3 checks the options when the volume is mounted. If MODE_DELDIR is
cleared, PFS3 will automatically create a delete directory and set
MODE_DELDIR. The rootblock extension is also automatically created if not
present.

If future versions of PFS3 extend the diskstructure this will be reflected by
a new option bit.

All 'not_used' fields are reserved for future use and must be set to 0.

First and last reserved indicate the 'reserved area'. All disk
information, like directories, allocation data etc are stored in this
reserved area. The rest of the disk is available for data. Currently
firstreserved always is 2, so the reserved area starts with the rootblock.
'Reserved free' indicates the number of reserved blocks that are
unoccupied.

The 'datestamp' is an update counter. When the disk is formatted, all blocks
will have a datestamp of 0. Every time the disk is updated (goes from one
state to the next using atomic commit), the datestamp is increased by one.
Using the datestamp all blocks in the reserved area can be ordered in time.
This is usefull for repair tools enabling easier and more reliable recovery
of a broken volume.

'Blksize' indicates the size of reserved blocks in bytes. Currently this
is always 1024 (1K), independent of the actual blocksize. The actual
blocksize (as specified in the mountlist) can be either 512 or 1024 byte.
So if blocks are 512 byte, which is the normal case, each reserved block
takes two blocks on disk. BTW: the before mentioned first and last
reserved indicate real blocknumbers, not reserved block numbers. However,
'reserved free' does refer to reserved blocks. It indicates the number of
reserved blocks that can be allocated before the reserved area is full.

Blocks free indicates the number of free blocks. It is equal to the total
number of blocks on the disk - the number of reserved blocks - 2 - the
number of blocks taken by data. PFS always keeps a small part of the data
area free. This is indicated by 'always free'.

The roving pointer is used for allocation. If you change any bitmap blocks
it is wise to set this pointer to zero. This will never hurt.

********************
* Block ID
********************

The following blocks each have a 2 byte block id field. This indicates the
blocktype. Currenly used IDs are:

DB -- dirblock
AB -- anodeblock
IB -- anode index block
BM -- bitmap block
MI -- bitmap index block
DD -- deldir block
EX -- rootblock extension
SB -- super block

********************
* Bitmap blocks
********************

To keep track of which blocks are occupied and which are not, PFS uses
bitmap blocks. Such a block looks like this:

___________________
| | |
000 | id | x |
|_________|_________|
| |
004 | datestamp |
008 | sequence number |
|___________________|
00c | |
| the bitmap |
| |
400 |___________________|

Each block has room for 253 longwords allocating 8096 blocks. Each bit
represents a block and a bit is set if it is available, and clear when it
is taken.

There are two bitmaps, one for the reserved area and one for the data area.
For the data area several bitmap blocks are needed. They are numbered with
the sequence number field, starting with 0. The first bit of the first
bitmap block represents the first data block (the block directly behind the
last reserved block)

The bitmap for the reserved area is directly behind the rootblock. The
rootblock cluster field in the rootblock (rblkclt) indicates the number of
blocks occupied by the rootblock together with the reserved bitmap. The
reserved bitmap has the same header as the data bitmap, but has a different
size. The rootblock cluster is chosen such, that it can contain the
rootblock itself and the entire reserved bitmap, and it is rounded up to a
multiple of 1024 byte blocks.

The first bit of the reserved bitmap represents the first reserved block
(usually 2). Each bit represents a 1024 byte reserved block.

********************
* Index blocks
********************

PFS refers to bitmap and anode blocks by using index blocks. These blocks
look like this:

___________________
| | |
000 | id | x |
|_________|_________|
| |
004 | datestamp |
008 | sequence number |
|___________________|
00c | |
| 253 indices |
| |
400 |___________________|

Each block contains references to 253 anode or bitmap blocks. For large
disks several index blocks will be needed. They are numbered with the
sequence number, starting with 0.

There is a maximum of 5 bitmap index blocks and 99 anode index blocks. The
rootblock contains the pointers to these blocks.

An anode or bitmap block of a certain sequence number doesn't exist if the
corresponding field in the index is zero. Holes in the index are allowed
(e.g. it is possible for anode block 5 to exist, while 4 doesn't).

********************
* Anode blocks
********************

Files and directories take space on the disk. PFS keeps track of which
objects takes which blocks by means of so called anodes. Anodes are stored
in anode blocks, which look like this:

___________________
| | |
000 | id | x |
|_________|_________|
| |
004 | datestamp |
|___________________|
| |
008 | sequence number |
|___________________|
| |
00c | reserved |
|___________________|
010 | |
| 84 anodes |
| |
400 |___________________|

The way anodes are numbered depends on the MODE_SPLITTED_ANODES:

MODE_SPLITTED_ANODES disabled:
The anodes are numbered starting with 0. The first 84 are in anodeblock 0,
the next 84 in anodeblock 1 etc. Each object (file or directory) is
identified by such an anode number.

MODE_SPLITTED_ANODES enabled:
The first 16 bit of the anode indicate the anodeblock by sequence number.
The following 16 bit indicate the offset within the anodeblock. So anode
0x00010004 is the fifth anode in the second anodeblock.

An anode looks like this:

___________________
| |
000 | clustersize |
004 | blocknr |
008 | next |
|___________________|

The clustersize indicates how many blocks are allocated by this anode. The
blocknr is the first block allocated. So an unfragmented file can always
be allocated by a single anode. A fragmented file, however, needs one
anode for each fragment. For that purpose is the next field. It indicates
the anode number of the next fragment. If this is the last fragment, the
next field will be zero.

The clustersize and blocknr in an anode always refer to real block numbers,
even if the blocks are in the reserved area.

If all fields of the anode are zero, it is considered free and can be
reused.

It is possible to associate an anode with a file without allocation any
blocks. This can be done by setting the clustersize and next fields to 0
and the blocknr field to -1.

The first six anodes have a special meaning:

anode name use
----------------------------------------------
0 ANODE_EOF should always be 0
1 not used, set to 0
2 not used, set to 0
3 not used, set to 0
4 ANODE_BADBLOCKS Bad block list. Not used by PFS3. Repair
tools can chain bad blocks to this anode
5 ANODE_ROOTDIR The anode of the root directory. You can find
the root directory by looking at this anode

All anodes from 6 and higher are used for directory and file allocation.

********************
* dirblocks
********************

Directories have an anode number, just like files. Each anode points to
one directory block of 1024 byte (so, with 512 byte blocks, the clustersize
field of an anode belonging to a directory is always 2). If 1024 byte is
not enough for the directory, a new block with a new anode is allocated and
chained to the first. Two anodes belonging to the same directory are never
joined to form one (by increasing the clustersize).

A dirblock looks like this.

___________________
| | |
000 | id | x |
|_________|_________|
004 | |
| datestamp |
|___________________|
| |
008 | reserved |
|___________________|
| |
00c | anodenr |
010 | parent |
|___________________|
014 | |
| directory entries |
400 |___________________|

The anodenr (anode number) refers to the anode belonging to the directory,
not the anode belonging to the block. So it always points to the head of
the anode chain, and is the same for all blocks of the same directory.

The parent is the anode number of the parent of this directory, or 0 if
this is the rootdirectory (which doesn't have a parent).

A directory entry looks like this:

___________________
| | |
000 | next | type |
|_________|_________|___________________
| |
002 | anode number |
006 | size |
|_______________________________________|
| |
00a | creation day |
00c | creation minute |
00e | creation tick |
|___________________|
| | |
010 | prot | nlength |
|_________|_________|
| |
012 | name |
| |
|___________________|
012 + | |
nlength | flength |
|_________|_________
013 + | |
nlength | filenote |
| |
|___________________|

|___________________|

The next field indicates the length of this directory entry in bytes. This
must always be even.

The type is the type of direntry, just like FFS: ST_FILE, ST_USERDIR etc.
See include file for details.

The anode number is the head of the anode chain for this object. The size
is the filesize in bytes. The number of blocks in the anode chain must
correspond to the number of blocks needed to store 'size' bytes.

The creation day, minute and tick is the DOS timestamp of the object. The
protection are the lower 8 file protection bits, like FFS.

The name and the filenote are pascal strings (length followed by that
number of characters). They are placed directly behind each other and are
byte aligned. If the file has no filenote, flength will be zero.

Each directory entry is padded if that is needed to word align the next
directory entry.

The end of the directory block is indicated by appending a zero at the end
of the last entry. The directory ends with the last entry in the last
block, where the blocks are chained by anodes.

********************
* dirextension
********************

If MODE_DIREXTENSION is enabled some optional extra WORD size fields are
added after the word align pad. The last extension field is a WORD size
bitfield that indicates which fields are present.

Currently the extra fields include:

___________________
| |
| link (high) | Anodenr of link
|___________________|
| |
| link (low) |
|___________________|
| |
| uid | User ID
|___________________|
| |
| gid | Group ID
|___________________|
| |
| prot (high) | 32 bit protection field
|___________________|
| |
| prot (low) |
|___________________|
| |
| virtualsize (high)| 32 bit virtual size (for rollover files)
|___________________|
| |
| virtualsize (low) |
|___________________|
| |
| rollpointer (high)| 32 bit rollpointer (for rollover files)
|___________________|
| |
| rollpointer (low) |
|___________________|
| |
| flagfield | Only field that always is present
|___________________|

'Link' is for links, described later. The prot field is combined with the
8 bit protection field direntry->protection (OR operation) to form a 32 bit
protection value. If the upper 24 bit are zero, this prot field will not
be used. The filesystem only checks direntry->protection when accessing
files.

The following code fragments show how to extract and make extrafields. Used
macros and types can be found in 'pfs.h'.

void GetExtraFields (struct direntry *direntry, struct extrafields
*extrafields)
{
UWORD *extra = (UWORD *)extrafields;
UWORD *fields = (UWORD *)(((UBYTE *)direntry)+direntry->next);
UWORD flags, i;

flags = *(--fields);
for (i=0; i < sizeof(struct extrafields)/2; i++, flags>>=1)
*(extra++) = (flags&1) ? *(--fields) : NULL;
}

static void AddExtraFields (struct direntry *direntry, struct extrafields
*extra)
{
UWORD offset, *dirext;
UWORD array[16], i=0, j=0;
UWORD flags = 0, orvalue;
UWORD *fields = (UWORD *)extra;

offset = (sizeof(struct direntry) + (direntry->nlength) +
*FILENOTE(direntry))&0xfffe;
dirext = (UWORD *)((UBYTE *)(direntry) + (UBYTE)offset);

orvalue = 1;
/* fill packed field array */
for (i=0; i < sizeof(struct extrafields)/2; i++)
{
if (*fields)
{
array[j++] = *fields++;
flags |= orvalue;
}
else
{
fields++;
}

orvalue <<= 1;
}

/* add fields to direntry */
i=j;
while (i)
*dirext++ = array[--i];
*dirext++ = flags;

direntry->next = offset + 2*j + 2;
}

BTW: if MODE_DIREXTENSION is enabled, the flags field must always be
present, even if none of the other fields are used. An unused field is
assumed to be zero.

********************
* hard links
********************

The main object (the object that is linked to) has type ST_FILE (ST_DIR)
like an ordinary file (dir). Its anodenr points to the object data, as
usual. The link field in the directory extension points to the head of the
LINK LIST.

The LINK LIST is mapped on an anode list. The 'link' is nothing more than
an anode number. The fields have been redefined, however:

___________________
| |
000 | object dir |
004 | link dir |
008 | next |
|___________________|

Each entry in the LINK LIST represents a link (the object itself doesn't
have a LINK LIST node).

The 'object dir' is the anode number of the directory where the object can
be found. The 'link dir' is the anodenr of the directory where the link
can be found. 'Next' is the anodenr of the next LIST NODE or 0 for end of
list.

A link has type ST_LINKFILE (ST_LINKDIR). Its anode number points to its
LINK LIST element. The link entry in the directory extension is the
anode number of the object.

A link can be resolved as follows:
╖ Get LINK LIST element (anodenr in directory)
╖ Get object dir (object dir field)
╖ Locate object by looking for anodenr as stated in the directory extension
link field.

BTW: the anode number as stated in the directory is an unique directory
entry identifier, even for links.

********************
* soft links
********************

A softlink is a file of length strlen(target) and type ST_SOFTLINK. The
block containing the name of the target can be located by reading the link
as a file (anode.blocknr).

********************
* deldir
********************

The deldir is a hidden directory that contains the 31 most recently deleted
and overwritten files. It can be accessed from the root by its name
'.deldir'.

The delete directory looks like this:

___________________
| | |
000 | id | x |
|_________|_________|
| |
004 | datestamp |
|___________________|
| |
010 | roving | (<4.3) current delete position
012 | uid | (<4.3) deldir user id
014 | gid | (<4.3) deldir group id
016 | protection | (<4.3) deldir protection (LW)
01a | creationday | (<4.3)
01c | creationminute | (<4.3)
01e | creationtick | (<4.3)
|___________________|
020 | |
| |
| 31 deldirentries |
| |
400 |___________________|

The deldir contains 31 slots containing deldirentries. These slots are
filled in a round-robin manner, using 'roving' as position pointer.

A deldirentry looks like this:

___________________
| |
000 | anodenr |
004 | size | size in bytes
008 | creationday | datestamp (copied from original file)
00a | creationminute |
00c | creationtick |
|___________________|
00e | |
| filename | filename, truncated to 17 characters
020 |___________________|

'filename' is a pascal string; length followed by 17 characters.

When a file is deleted or overwritten it is added to the deldir in the slot
indicated by 'roving'. Only files are put in the deldir, no directories or
links. The diskspace taken by these files is freed at the moment the
original file is deleted, so an entry in the deldir can become invalid at
any time data is written to the disk. When the deldir is scanned (e.g. by
calling "dir pfs:.deldir") PFS will check the vadility of each entry and
will return only the valid entries. The scan routine adds the deldir slot
number to each entry (e.g. foo@003) to make sure each entry has an unique
name. Files in the deldir should also be accessed by that name.

The anodechain taken by a deldirentry is not freed until the deldirentry is
reused.

********************
* rootblock extension
********************

AFS version 16.4 introduced the rootblock extension. This is a reserved
area block that contains general information about the disk that didn't fit
in the rootblock. The block is referenced by the 'extension' field in the
rootblock. Its presence is indicated by the MODE_EXTENSION bit in the
rootblock options field. PFS3 automatically creates the rootblock
extension on older disks, so every disks that is operated on PFS3 should
have an extension block.

The rootblock extension looks like this:

___________________
| | |
000 | id | x |
|_________|_________|
| |
004 | ext. options | flags for (future) options. Currently 0.
008 | datestamp |
00c | pfsversion | PFS revision id.
|___________________|
| |
010 | rootdate day, min | the root directory datestamp
|___________________|
| |
014 | tick |
|_________|_________
| |
016 | voldate day, min | the volume datestamp
|___________________|
| |
01a | tick |
|_________|_________
| |
01c | postponed - |
| operation |
|___________________|
| |
02c | reserved roving | reserved area roving pointer
|___________________|
| |
030 | roving bit | roving pointer bitnumber (normal area)
|___________________|
| |
032 | current anode seq | roving pointer for anode allocation
|___________________|
| | |
034 | dd_rov | dd_size | roving pointer and size of deldir
|_________|_________|
| |
038 | fnsize | filename size (18 +)
|_________|_________
| |
03a | x |
|___________________|
| |
040 | superindex | MODE_SUPERINDEX only: upto 16 superblocks
| |
| ╖ |
| ╖ |
|___________________|
| | |
080 | dd_uid | dd_gid | uid and gid of delete directory
|_________|_________|
| |
084 | dd_protection | protection of delete directory
|___________________|
| |
088 | deldir day, min | deldir datestamp
|___________________|
| | |
08c | tick | x |
|_________|_________|
| |
090 | deldir | upto 32 deldir blocks
| |
| ╖ |
| ╖ |
|___________________|
110 | . |
| . |
| . |
400 |___________________|

PFS stores the current revision id (version number in high word, revision
number in low word, e.g. $00100004 for version 16.4) in the 'pfsversion' field
when formatting a disk. This way the oldest PFS revision that accessed the
volume can be identified. The two new dates, rootdate and volumedate,
complement the creationdate in the rootblock. The three datestamps are used as
follows:

Volume creation date (rootblock): Only set by format and by
ACTION_SERIALIZE_DISK.

Volume date (extension): Zerod by format. Touched each update.

Root date (extension): Root directory datestamp.

The postponed operation is a 4 longword field used internally by PFS3. It is to
be considered an error if these fields are non-zero on an inhibited PFS3
volume.

The superindex is for MODE_SUPERINDEX only. The longwords in the superindex are
references to superblocks. A superblock is an indexblock which indexes
anodeindex
blocks.

********************
* rollover files
********************

Rollover files are files with a filesystem enforced maximum size. When the file
is appended to it will be trimmed to keep the size under the maximum.

Rollover files are identified by the filetype ST_ROLLOVERFILE (== -16). The
size field in the directory specifies how much diskspace is allocated for the
rollover file. The virtualsize field in the extrafields indicates how much of
the allocated diskspace currently is in use. This will initially be 0 and grow
until it is equal to the allocated diskspace (extrafields->vitualsize ==
direntry->size). The virtual size can never be larger than the allocated size.
The rollpointer field in the extrafields specifies the location in the
allocated area where the rollover file starts. Rollover files are read
circular, starting and ending at the rollover pointer.

Example rollover file with 2048 allocated bytes:

0 real offsets 2047
______________________________________________________________
| | | |
| L|N|F |
|____________________________|_|_______________________________|

virtual offsets 2046 0

<-------------------------------------------------------------->
allocated area
2048 bytes

L = last byte of file
N = end of file
F = first byte of file

The usable size of rollover files always is a multiple of the blocksize
minus one.
--- cut ---

Всего хорошего, Pavel Fedin.

From Oleg Golenkoff To All 4 January 2000

Hу хай тебе, Pavel !
Как-то Pavel Fedin настучал на клаве к All:

PF> В связи с поднявшимся интересом кидаю сюда выдержку из guide с описанием
PF> структуры PFS-диска.

[...slipus longus....]

По этому поводу есть пару вопросов:

-Диск отформатированный в PFS3 имеет стандартную скорость? т.е. его можно будет
прочитать на ZX или PC ?

-Hа Ammy этот стандарт поддерживается? я имеею ввиду если вставить диск
отформатированный в PFS3 - WB его сразу откроет или придется опять моунтить
что-то ?

-Можно ли скинуть образ такого диска? если да, можно ли его мылом закинуть..

C ya in nE> ASCENDANCY HQ: ascen...@mail.ru http://www.ascendancy.homepad.com

From Pavel Fedin To All 5 January 2000

Пpивeт всем! Пoмнитcя, 04 Jan 00 Oleg Golenkoff взял дa и зaпуcтил в мeня
вот этим:

OG> -Диск отформатированный в PFS3 имеет стандартную скорость? т.е. его
OG> можно будет прочитать на ZX или PC ?

Во-первых, амижные диски не читаются на ZX/PC не из-за скорости.
Во-вторых, физическая разметка дорожки (именно в ней дело - пробелов между
секторами нет) к файловой системе не относится никак.

OG> -Hа Ammy этот стандарт поддерживается? я имеею ввиду если вставить диск
OG> отформатированный в PFS3 - WB его сразу откроет или придется опять
OG> моунтить что-то ?

Придется монтировать. Хотя, дискетки в PFS3 никто не держит. Смысла особого
нет. В основном ее юзают на винчестерах, поскольку она там монтируется при
запуске системы специальным образом.

OG> -Можно ли скинуть образ такого диска? если да, можно ли его мылом
OG> закинуть..

У меня нет ни одной дискетки в PFS3.

Всего хорошего, Pavel Fedin.

From Kirill Frolov To All 5 January 2000

Сомневаюсь... Hасчёт PC точно могу сказать -- нельзя.

Kirill Frolov. [ZX]

From Oleg Golenkoff To All 8 January 2000

C ya in nE> ASCENDANCY HQ: ascen...@mail.ru http://www.ascendancy.homepad.com

From Pavel Fedin To All 11 January 2000

PF>> между секторами нет) к файловой системе не относится никак.

OG> т.е.? можно подробнее плиз? про структуру диска?

Hет межсекторных пробелов. Hа ВГ93 и подобных они используются для
синхронизации, а на Amiga синхронизировать нечего, там контроллер диск сразу
дорожками читает и пишет. Таким образом, получается больше секторов на
дорожке. Деление на сектора сохранено исключительно в целях повышения
устойчивости информации (каждый сектор имеет свою CRC).

Всего хорошего, Pavel Fedin.