Applications
It is rather strange that there are no screen storage formats apart from the basic ones. Apparently, this is because all other platforms had several screen output modes (high, medium, and low screen resolutions), while the Spectrum traditionally had only one. We should develop not only formats for storing screen files, but other formats as well. Ideally, there should be a single service for standardizing formats used on ZX. In general, this could be, say, a link on all ZX resources to a standardization page, where there would be a brief and detailed description (or, if necessary, a ZIP file to download) of the most common file formats adopted for use on ZX. Then any programmer could adjust their format to make it compatible with another one, or create their own, confident that any fellow programmer would understand and be able to use it.
I VERY much ask everyone who wants ZX to continue to discuss this topic with at least two other Speccy users.
To avoid making unsupported claims, here are some specifics:
*.S,*.SCR— traditionally, these are screen files, 6912 or 6144 bytes long. This format should be extended to cover files of any size and any format—that is, any file with the SCR extension should ALWAYS be treated as a screen file. The thing is, these files may be packed in a format different from those familiar to everyone. For this purpose, provide a signature, for example"SCRext"— the first 6 bytes.@.C,@.COD— Magick files, traditional for TR-DOS.
I should make it clear right away that I do not consider TR-DOS to be the only system on the Speccy that exists today. However, the system of symbolic names and extensions is a commonly used construct, so I think we should consider these files not only in the context of TR-DOS (although, as a hands-on user, that is the one I have in mind), but also in relation to other DOSes, OSes, and file systems.
*.B,*.BAS— the good old BASIC.*.T,*.TXT— text files. They may be packed.*.M,*.MOD— sound module files.*.P,*.PAK— files that are simply packed, without any specific purpose (they may contain data, dumps, etc.).*.Z,*.ZIP— archives made with ZXZIP.
Formats
The first bytes of the files are specified here:
Packed HRUST1.X
Offset Length Description
+0 2 "HR" signature
+2 2 Original file length
+4 2 Packed file length
+6 w[+4] Packed file
Unpacker
V>>> The unpacker source code follows.
Packed HRUST2.1
offset length
+0 3 "hr2" identifier
+3 1 "1" reserved.
if bit 7=1, then the file was
simply "saved".
+4 2 original file length
+6 2 packed file length
+8 w[+6] packed file.
Depacker
V>>> The depacker source code follows.
Packed pictures with unpackers
As a rule, they contain instructions of this kind:
1) CALL <NN> — call a RET instruction in ROM. Sometimes a call
to RET <condition> is used (there are also calls to instructions
such as Pop <RP>, Mov HL,<RP>, Jp (HL)); flags are set beforehand
as needed — the current address is pushed onto the stack.
2) DEC SP DEC SP POP <RP> — the value is then taken from the stack.
3) LD <RP>,#4000 — write to the register pair indicating where to
unpack. Depackers usually do not use an intermediate buffer —
in my experience, I have seen very few that do. On the other hand,
the address may differ from #4000, so it is much more correct to
check whether it falls within the range #4000-#5AFF or #C000-#DAFF.
4) LD <RP>,#1B00 or LD <RP>,#1800 — the unpacked length. This
beginning is used by all depackers because they need to determine
the address at which the loaded depacker file is located. The
drawback of this construction is that it depends on interrupts
(i.e. an interrupt may occur and destroy data on the stack), and
it depends on the ROM type — what if the address doesn’t work?
I have also seen this construction:
1) Halt
2) Dec SP Dec SP Pop <RP>
3) etc.
A possible option:
1) Ld HL,#E9E1 — instructions Pop HL Jp (HL)
2) Ld (<SCR_ADR>),HL — write to the area where the picture will go;
it will be changed anyway
3) Call <SCR_ADR>
The last two options are optimal, since they do not depend on which ROM is used. The penultimate one (based on Halt) loses an interrupt, so it is less preferable.
Packed without a depacker
This is the most correct format in terms of size optimization, although, on the other hand, it requires the viewer used to browse the files to “know” the format.
Laser Compact 5.X
offset length
+0 5 "LCMP5" — identifier
+5 2 packed length, excluding the header (the
comment is included in the header)
+7 1 construction type
+8 1 comment block length
+9 NN comment
Depacker
V>>> The depacker source code follows.
IS-DOS
These files have no identifier, but can be identified indirectly: a sufficiently fast algorithm (about 1–2 interrupts) is enough, and if the result corresponds to a size of #1B00, then that’s what it is.
V>>> The algorithm source code follows.
Applications
Fragment 3/11.
Why only two formats—is there really nothing more?
Strangely enough, no.
I deliberately looked through a large number of picture packers, and can state that there are no other widespread formats whose packed pictures, when saved without an unpacker, are accompanied by an identifier. Or which could be recognized fairly easily (as in the case of the iS-DOS format).
I have listed several specific formats and how they should be identified. Unfortunately, as noted above, most packer authors don’t think about adding ID records to their archives, so often the only way to tell one archive from another is simply knowing that this particular archive is one thing and that one is something else. If such an archive arrives without any explanation, then, obviously, there’s nothing you can do with it.
A special personal invitation to all authors of viewers, shells, and other programs: please do what you can to help fill in this table. Often, the authors of such viewers reinvent the wheel every time and spend their time not programming, but poking around in someone else’s program. If, in addition to descriptions of the formats, the algorithms for using them and the unpacker itself for each format are included, that will allow the ZX community to work on making these formats operate more efficiently and, ultimately, improving the software. The more information is added there, the more efficiently the ZX standardization service will work, and consequently, the more efficiently application software on the ZX will work.
That is, for example, if some Vasily Pupkin wants to write a program for viewing pictures, he can simply download these sources, compile them, and there’ll be a new picture-viewing program by Vasily Pupikin. Looking at the wider computer world, you could say that the most promising areas of software development for other machines (not Spectrum, that is) work in exactly this way.
In general, I see a great future for DMA USC. No other device on the Speccy has offered so many possibilities. Being able to do full-screen multicolor at the base clock, play AY music, and run a MOD file at the same time is far from all this device can do. Most importantly, it has a programmable interrupt system, which will make it possible to write efficient operating systems, and that will make it possible to write proper programs. Hardware interrupts will allow multitasking operating systems to be written: an interrupt arrives, the processor saves data on the stack, analyzes what needs to be done and how, then launches the necessary application using a priority table. In general, the whole problem is the lack of proper operating systems. For now, the only limitations are the limited spread of DMA USC and its poor support (or, more accurately, no support) in ZX emulators. Although, of course, I’m not suggesting that you throw out your General Sound, Covox, and so on, install DMA USC, and forget about everything else like a bad dream.
Still, it isn’t all that straightforward. There have been examples of programmers (good programmers) writing their own BIOS and installing it in place of the 48K or 128K ROM. Scorpion also set a good example: BIOS access went through restart calls, which really does make a programmer’s job easier. What’s more, restart calls worked the same way across different versions of the Scorpion BIOS, while compatibility with SOS was also preserved. That said, whether you have a Scorpion or a Baltic-128, you have the right to expect your system (meaning the OS) to work well. So, on the one hand, we need to write hardware-independent operating systems; on the other, they need to detect and use the sufficiently varied resources of your ZX. As mentioned above, memory-related problems can be considered solved: we can write a driver (fast enough) that detects the type of memory, and the process will be invisible to the end user. In general, any OS should have several levels of operation—for programmers, users, drivers, programs, and so on. If you recall (and if you didn’t know about it, just read up on it) the seven-layer OSI model, it becomes clear that all the services of any OS can be assigned to one of the layers of this model (taking it, of course, as a model for an OS)—drivers belong to the physical layer, and so on. Thus, any OS should be written sequentially, from the lower layers up, implementing all the necessary sets of functions for users and programmers along the way. In general, ISO has recommendations for OS design and operation, so it would probably be interesting for all developers to read them. The standards can be found…
Applications
Fragment 5/11.
In the boundless web, they should be available there, though not necessarily in Russian.
I really liked the ZX-windows system (I think the last version was 1.6). It implemented almost all the features a proper OS should have, including allocating memory for a program. At the same time, writing a program for such an OS comes with a bit of a headache: you can no longer directly address a page and write to it, or load through #3D13 or your own DOS-loader for TR-DOS. All input/output operations and memory manipulations are handled by the OS, which is, in principle, very good in itself from the point of view of developing such a system and software for it. Once written, programs won’t have to be broken if something in the system changes (memory allocation, etc.).
In general, the idea of a multitasking system on the Spectrum has been around for a long time, but nobody has the guts (or maybe the time/desire/energy/etc.) to make it happen. Those writing the system (nice word, isn’t it?) face several problems:
- Compatibility with other (old/new) products.
There are two ways to go about it:
- Forget about everything old; only new stuff written for the system will be supported by the OS. I don’t think this approach needs any comment.
- Try to preserve compatibility with other (old) programs. It seems to me there’s a way to do this: every OS has compatibility modes (classic mode, or the more appealing-sounding Legacy Mode) and full-featured modes. When you need to, say, run an old program, you can switch the system into compatibility mode. This should happen automatically when the program is launched: only a small section of the whole system remains in the basic 48 KB (256 bytes, for example), enough to restore the system. The system itself, just in case, saves its state and copies itself into the upper memory banks (obviously, a 48 KB machine has no such alternative). With relatively little modification, this makes all old programs compatible with this particular OS. It also allows those programs to use the OS’s broad range of services—save/load/print, etc. When one of these services is called, the system temporarily switches the memory banks to itself and performs the required function, then restores all the memory banks to their original state. To be specific: I’m playing Sim City and I absolutely need Save Game. In this case, I have no idea about the file system or what medium I’m writing to (virtual disk, floppy disk, hard drive), but I do know that there are specific restarts—used...
Using them, I save my state and do Quit. The system is quickly restored from memory, and I don’t need to load anything again. Of course, when switching to Legacy mode I lose almost all the features (except the basic system functions); these may include multitasking and, as a consequence, background music, for example. But then I can restore everything without any problems (unlike in iS-DOS, where you have to load everything again when launching a large program), and I don’t lose the ability to run old programs either—there’ll always be at least a small stretch of memory where the OS recovery system can be placed.
Interface
It should be fully customizable. Here we need to move forward by introducing OOP into the interface core. The use of OOP has already proved itself on all platforms (except the Spectrum :)))).
Multitasking
There’s a lot that can be said here. The problem of multitasking is rooted not so much in the power of the processor as in the capabilities of the peripherals.
- First, it’s how many memory pages the computer can offer the system—this needs some explanation.
The thing is, multitasking is possible only when tasks are almost completely parallelized; this applies to both data and programs. Because of this, it would of course be very nice if all of the computer’s memory could completely remap its base addresses (since full absolute addressing isn’t possible). As the Spectrum doesn’t offer this in its STANDARD configurations, even 128KB machines have problems: programs written for a multitasking OS have to reside in separate pages. So the more available pages, the better. Implementing applications for multitasking OSes on 48KB seems completely unfeasible to me when using this method.
Another approach: applications are written so that they have no fixed load addresses—which means no absolute jumps or calls. This last approach would remove the limitation on the number of free memory banks required, allowing applications to run even on 48KB, but it makes them more labor-intensive to write. Still, solutions can be found here too: for example, when an application is launched, the system scans a table at the beginning of the file which specifies which memory cells require address correction. The system then changes all the NECESSARY addresses to their current ones (i.e. where the program has been loaded), and only after that launches the program. For example, cell 75 requires address correction—the OS looks and finds the number 12 there, adds the load address 12345 to it, gets 12357, and writes it back. The result is a Call 12357 instruction. This approach will save a considerable amount of memory. At the same time, the system itself…
Applications
The address translation table is stored only in the application header on the medium, and requires no memory while the application is running.
-
The process of implementing multitasking itself. I’d like to clarify this topic. In fact, a processor can NEVER be multitasking. Only a system with several processors can truly be considered multitasking. The principle used is pseudo-multitasking: the processor is periodically switched in hardware to tasks whose priority requires it. Obviously, the Z80 and all its peripherals provide no hardware support for this kind of multitasking, so it will have to be introduced somehow through software.
Ideally, on each interrupt the processor hands control to just one designated application. Naturally, there must be a priority table: some things will require frequent execution, others only occasional execution. I don’t think tying this to standard ZX interrupts is practical—1/50 of a second is too long an interval for running one program. It could probably be used, but only if several tasks are included in each interrupt. This is exactly why I said it would be very promising to use the USC DMA, with its adjustable interrupt system.
On the other hand, we need to account for the standard configuration. This can be done as follows: each task has—or declares when loaded—its own priority.
- Highest priority—every frame. This could be a clock, control of working elements (cursor, keyboard polling, etc.), a background AY player, or something else. The procedure must run faster than the interrupt; up to 10,000 processor cycles maximum is recommended.
- High priority—every second frame.
- Medium priority—every fourth frame.
- Low priority—every eighth frame, or perhaps simply using all idle cycles for this type of task (after the main tasks have run, of course).
- There must be exceptions: drivers may authorize disabling interrupts and exclusive use of resources.
Applications
By the way, polling the keyboard can actually be a low-priority task.
Within each priority there should be at least four priority groups. After completing its work, each task must return control to the task-priority coordinator.
If an interrupt occurs while a task is running (interrupts are ALWAYS enabled), the interrupt handler can save the task’s state and “cycle through” all the higher-priority tasks before returning control to it—or return control to it immediately, depending on its priority. But if an interrupt occurs while the coordinator program is running, control is returned to it immediately (so to speak, “no questions asked” :)))).
By the way, priority scheduling also lets you use programs that haven’t been adapted for a multitasking system: they’ll simply run at the lowest priority.
The task coordinator must have its own stack. A separate stack area is allocated for each task. Each task also gets 24 bytes for all the registers + the stack (pointer value) + the return address (there’s no need to save the refresh or interrupt registers).
In reality, despite how complicated it may seem, and despite the apparent fact that this algorithm won’t let each task have a genuinely independent priority, it’s not all that complicated and really will work. Even with a 1/50 interrupt rate :)))).
If you don’t believe it, simulate every possible situation—it’ll work beautifully in most cases.
Besides, during startup the OS can check: if it can’t vary the interrupt frequency, it uses the standard timer. With a system like this (i.e. a multitasking one), your processor’s speed will matter: the faster it is, the less slowdown you’ll notice. So software will give the hardware guys an incentive to “overclock” the system.
Such a system will be able to automatically detect programs that don’t follow the multitasking rules (returning control to the coordinator) and automatically impose penalties: push them down to the lowest priority, issue warnings, and so on.
Applications
File system
Obviously, none of the Spectrum OSes implement anything remotely like what we want to see. I think we need to make our own file system—perhaps based on ТР-ДОС, perhaps on ИС-ДОС, or something else.
The main requirements for the file system:
- Full-fledged file descriptors, with support for file fragmentation. I think this is clear: we need to get everything we can out of Spectrum disks, rather than tie ourselves to an inferior system, no matter how widespread it may be.
- A mandatory directory system.
- All numbers in files must be 4 bytes, to avoid any misunderstandings.
- Accordingly, the number of entries in a directory can be 2^32.
- Directories can be fragmented, just like files.
- Optional: a permissions/ownership system.
- Number of devices: 1 byte, i.e. 256 devices.
- Each device has its own identifier—one for floppies, another for hard drives, and some other one for other removable media. The identifier is a character string, a zero byte, and the device type.
- Each device must have its own dedicated descriptor in the system, describing all its controllable parameters.
- Optional: the ability to create nonexistent directories—that is, to link a directory on one device to a directory on another.
A sophisticated file system will solve many problems.
Accounting for regional specifics
An essential feature, in my opinion. No user should be disadvantaged in this respect, whether they’re a Tatar like me or an Uzbek like my father. And we don’t have to reinvent the wheel: we can use the same approaches as other systems, such as code pages, Unicode, etc.
Do you see the possibilities this opens up? No one will stop you from playing music, typesetting text, browsing files on disk, and so on. To the point where you can sit there playing games and genuinely not give a damn about all of MS Windows :))))
You could even make something like a CPU-load analyzer: count all the cycles the processor spends in the coordinator while waiting for the next interrupt.
And what about sitting in your favorite, say, Asm, periodically switching over to the documentation, and listening to music at the same time?
I think all the possibilities I’ve outlined are real for the Spectrum, but it will take not just one team of programmers working on this, but EVERYONE.
The situation is completely baffling. Apparently, all Speccy programmers have some kind of illness—“low-level masochism”: nobody tries, or even wants to try, writing high-level languages. Three people could write a C compiler/translator in half a month—and that would provide a foundation for writing Pascal, BASIC, and so on (assuming, of course, further development). Once we have programming languages, we can start thinking about writing a proper OS. Someone will ask: “Well, if you’re so smart, why don’t you write one yourself?”
Answer: in my city (Naberezhnye Chelny), I’m the only Speccy programmer of this kind—or at least, I don’t know any other Speccy programmers. Second, I don’t have all that much time. We have a holiday break right now, so I’ve sat down to work on this document; normally, I don’t have the strength or the time to work on something like this. Third, EVERYONE should be involved in this, not just one particular team.
Besides, creating high-level programming languages could make it possible to develop a fairly challenging field like OOP. Moving over to OOP later would let us forget Assembly like a bad dream and proclaim to the whole world (including those damn PC users) that the Spectrum can support ANY programming method.
While typing this conclusion, I happened to come across “Reflections on GUI on the Spectrum” by Mr. Norinchak.
Well, in reply to him, I can say that with OOP languages, there’ll be no need to think about creating complex GUI libraries—they’ll simply be implemented using the programming language itself. We’ll be able to create object interfaces, classes, families, program events, and not worry about where to get the resources. The environment (the OS or compiler/translator) will take care of that.
Of course (I can’t deny it), some programmers might consider such a language wasteful. But it depends how you look at it: do I, as an OOP programmer, spend 2 minutes writing a program to calculate a fractal, or do I spend 2 days hacking away at the same thing in Assembly?
Sure, it’ll run ten times faster in Assembly—but who needs it to run fast? And a project like this isn’t the whole story... What about genuinely complex programs? Or the interface? This is where interface flexibility comes in, too: I write TextBox172.Text = "Real OOP" to display something, say, and don’t clutter my head with the details. And, to be honest, many of those who consider themselves true Speccy users don’t work (and often don’t even play) on a Speccy. The Unreal Spec emulator itself was written in an OOP language. And much more easily.
it will, say, be easier for a developer of a Speccy program or game to use certain objects, interfaces, properties, methods, and so on from a networking library than to clutter his head with where he has a modem or (maybe there’ll be a network card, huh?). He’ll just use them, that’s all. After all, you can’t write everything in asm. The source code alone could end up taking up more than a TR-DOS disk packed to the brim... And speed is only essential for basic programs (drivers, translators, compilers...). No, I’m not denying the importance of asm for the ZX, but there’s no need to get fixated on it....
Share your thoughts about the article