Systems - NedoLang: acceleration (part 5).

NedoLang: Acceleration
 ...as well as democratization and openness
Alone Coder

   There were naive hopes of laying the full sa─
mocompilation in one minute on Z80@3.5 MHz. 
It is clear that you can copy each 
module separately (assembly only) 
together), but there is no time stamp in TR-DOS─ 
no, but there’s no time to write a new OS yet. 
   By the way, the timestamp should be very
accurate - personally, I often run recomputer─ 
lation, accuracy of 2 seconds is not enough. K 
besides, she always has problems when transferring 
from computer to computer. And the fields "number" 
I didn’t see the file version" in any file─ 
howl system. As, by the way, the possibilities under─ 
paste symbols at the beginning of the file so that when 
editing the file in the development environment it 
was always synchronized with the file system─ 
topic (like two pieces: before the cursor and after). 
   Moreover, I didn’t want to rewrite the compile─
Thor in assembler, although now it’s already about─ 
one hundred - manually optimize what is output─ 
there is a compiler. But I wouldn’t spoil it like that─ 
shaft on ARM. 

   First of all, it was necessary to speed up
label search and lexer. This is what scientists write 
about the speed of lexers: 
 "A study by Waite[1373] found 41% of
total translation time was spent in a
handcraftedlexer (with little code
optimization performed by the translator).
An automatically produced lexer, the lex
tool was used, consumed 3 to 5 as much
time."(cbook1_0.pdf, p. 232)
 That is, a normal lexer takes up 41%
compile time, and if it automatically 
generated from the syntax description, then 
generally a pipe. 

 03/27/2017:
 - made the parameter transfer via bay─
sic-variablea$.
 Standard "Command-line friendly" in
TR-DOS (see ACEdit) cannot be used with 
code files - afterCODE does not receive─
to enter additional parameters even with
separator, or : (with none at all),
even if you specify 2 numeric parameters.
The only separator that works is
OxOd, but it cannot be entered from the keyboard.
 You can RUN "filename blablabla"CODE, but
how to track the address of an additional tex─
that, especially when calling from BASIC?
  - accelerated the compiler, tokenizer, assembly─
ler and exporter on Speccy, generally faster─
several times:
┌────────────────────────── ──────────────┐
compile.c (here and below there is always the same 
same copy) compiles in 151 seconds (no delays) 
then all compiler modules will be 
151 s * 152k/48k = 478 seconds 
   optimized tag search:
now compiles in 44.86 seconds (no delays) 
   replaced the hash function:
now compiles in 43.72 seconds (no delays) 
   cleaned the readchar:
42.68 - inaccurate, because did not reset FRAMES 
   removed waseol:
42.08 
   optimized comparison in readcomment:
41.08 
   online readcomment:
41.08 (i.e. no winnings) 
   replacing _tword[_FIRST] with *(PCHAR)_tword
40.96 
   simplified the logic after readcomment:
40.68 
   removed _tword[_FIRST]==_DIERESIS:
40.66 
   removed _docomment: 40.34
   isnum according to the table: 40.18
   isalpha according to the table: 39.86
   isalphanum according to the table: 38.18
   isalphanum inline in readconcattword:
37.56 
   accelerated strjoin: 36.02
   strcopy via asmolib: 35.5
   strpush via strcopy: 34.88
   strpop via strcopy: 33.86
   removed the terminator in strjoin: 33.74
   replaced strjoin with strcopy where possible:
32.44 
   in read inline strclose: 32.12
   in compile inline strclose: 32.12
(i.e. there is no gain in speed, only insize) 
   strjoin via asmolib: 12/31
   I accelerated it further (by 4t in the cycle, now 67t):
31.08/31.2 
   I also accelerated it (cpir + ldir):
31/31.08/31.02/31.1/31.16 (the longer we press 
NumLock in the emulator, the longer the time) 
   bypass readcomment logic: 30.8
   I also accelerated isnum: 30.8
   isnum online: 30.78
   removed isalpha, and isalphanum through the table─
tsu bool everywhere inline: 30.06 

readfin time (bd75) = usually 450 t 
(including call) * 49870 b = 22441500 t = 
6.4 s (let 7) - accelerated to usually 95 t 
(including call) 
fwrite time (62bf) = typically 243 t 
(including call) * 61452 b = 14932836 t = 
4.3 s (let it be 5) 
Let the total time for file operations be 12 s 

ASMA tokenizer from exporter 52.98 s 
(we need to speed up string comparisons - a) via 
hash, b) allow commands only with tab and 
check a couple of characters) 
   after removing _docomment: 53.26
   bypass readcomment logic: 53.3
   isnum online: 53.08
   removed isalpha, and isalphanum via tab─
face bool everywhere inline: 52.58 
   checking commands requires _asmspcsize!=0:
44.18 
   Let's sort command checking by frequency:
31.32 

export to tok.f from exporter: 15.38 

assembly time on tok.f from 
exporter (55424 bytes (3715 lines, 578 
labels) vs 203175 (13015 rows, 1645 
labels) from the compiler): 
33 seconds for 2 passes 
   after optimizing reading and writing semi─
took 19.86 seconds (3640 lines) 
total 183 lines per second 
   replacementhash functions in tag search: 18.7
└────────────────────────── ──────────────┘

 03/28/2017 - accelerated by half more times─
tora:
┌────────────────────────── ──────────────┐
 sorted checks in cast: 29.68
   sorted the checks into push, pop,
comparisons, removed char and bool in comparisons 
more or less (excluded comparison error 
signed char): 30 
   sorted the checks in calls and log.
operations, removed char and int from the log. 
operations: 30.06/29.86 
   I bypassed unnecessary operations in eat_cmd:
29.66 
   tried the quick exit from eat_expr and
etc. by ')' - no win (removed) 
   remade eat_expr, etc. in procedures:
29.68 
   sorted operations in eat_expr, etc.:
29.62 
   in eat_expr, etc. instead of && put
if..if..if: 29.54 
   removed unnecessary comparison in let: 29.4
   in eat_sumval, eat_mulval removed dbl and
did if..if: 29.52 (removed ^^ at the same time) 
   speeded up number format checking:
29.34 
   in readchar I made if..if: 29.02
   in readchar assignment wasdieresis in
different places: 28.64 
   in readchar assignment wasescape in
different places: 28.36 
   in readchar I made comparisons on the top
level per character: 27.4 
   in readchar(rdch) fast output: 26.18
(memory for tags and 
the size has grown104 bytes) 
   in rdch online stradd: 25.22
   removed noskip: 25
   fixed condition optimization in
eat_repeat: 24.66 
   simplified hash function: 23.68
   search for tags via strcp (asm),
adding labels via strcopy: 22.78 
   returned _docomment: 22.84
total 70 lines per second (1590 lines) 

ASMA tokenizer from exporter 52.98 s 
(we need to speed up string comparisons - a) via 
hash, b) allow commands only with tab and 
check a couple of characters) 
   after removing _docomment: 53.26
   bypass readcomment logic: 53.3
   isnum online: 53.08
   removed isalpha, and isalphanum through
table bool everywhere inline: 52.58 
   checking commands requires _asmspcsize!=0:
44.18 
   Let's sort command checking by frequency:
31.32 
   after all optimizations read: 23.62
   strcp in asmolib: 22.4
   returned _docomment: 21.88
total 160 lines per second (3518 lines) 

export to tok.f from exporter: 15.2 
   via writestr: 7.38
total 500 lines per second (3518 lines) 

assembly time on tok.f from 
exporter with new exporter: 18.02 
   asmwordpopvalue via fwrite: 17.6
   asmbytepopvalue via writefout: 17.94
   _nvalues as byte: 17.7
   inline asmpopvalue, without checking: 17.44
   Fix condition optimization in eat_repeat:
16.82 
   simplified hash function: 15.82
   search for tags via strcp (asm): 14.74
   adding labels via strcopy: 14.24
total 250 lines per second (3518 lines)└────────────────────────── ──────────────┘

 03/29/2017:
 - more small accelerations:
compiler - 82 lines per second (was 70) 
tokenizer - 178 lines per second (was 160) 
assembler - 292 lines per second (was 250) 
  - made nedotrd so he could cut
large files according to the follower standard─
nary TR-DOS files (not through the extension, but
via the low byte start), and so that 3-sim─
the free extension did not get to disk.
Come up with a different cutting standard to
did not touch either the extension or start? isn't it
what's the name? Or switch the second character
extensions are tolerable - support in lib 3-
character extension?

 03/30/2017:
 - very small accelerations:
compiler - 88 lines per second (was 82) 
tokenizer - 209 lines per second (was 178) 

 03/31/2017:
 - automated NedoLang auto-assembly on
Speccy: parameters (more than one) are transmitted 
via REM in BASIC. BASIC size on
limit. For this I also had to do
recovery after RUN "" CODE (glitch in
TR-DOS, described by Fedin).
  - self-compilation time for the entire projectcompile:
 comp: 41.68
   tok: 46.04 - 45.92
   asm: 54.28 - 54.04
total: 142 
this is from 4814 lines on the shortcut (141447 
bytes) and 998 lines of assembly language (16117 
byte). That is, the overall build speed is 41 
lines per second (1.1 KB/s). 
  - for the entire projectasm:
 comp: 22.38 (many comments in fmttg)
   tok: 23.68
   asm: 37.22
total: 83.28 
  - for the entire projecttok:
 comp: 28.10 (many comments in fmttg)
   tok: 29.40
   asm: 37.66
total: 95.16 
  - bugfixesDIVLONG and MULLONG

 04/03/2017:
 - slight acceleration of self-assembly of the entire
systems:
   for the compile project - 138 (was 142)
   for the asm project - 79 (was 83.28)
   for the tok project - 93 (was 95.16)

 04/04/2017:
 ;;  didn't work after the line that
ends with; - resetwaseols
 corrected the command after the label in one
line

 04/05/2017: wrote to minidem in Nedo─
Lang. Isometric engine first wrote─in C++ Builder, then transferred to Nedo─
Lang. 

 04/06/2017:
 - added the library ptЗplay and inserted
to minidem (now called NedoGift
and several more people participate in it,
initiator -Sand/MAYhEM ).
 - added the nedodefb utility, while assembling─
ler does not supportincbin (include too -
he simply assembled a chain of files,
specified in the parameters).
  - addedNEG.
 - corrected the command format_FMTCBCMDIDX.
 - correctedI,R.
 - fixed recursion in expressions in that─
kenizator.
  - added check) to if.

 04/07/2017:
 - slight acceleration of self-assembly (compile
for 4 seconds, asm for 1 second). Got up
the issue of reducing the number of structures of the type
/*...*/ .How to make the size of the declaration array─
linear constant (C does not know the size of the globe─
ball array from const)? Constants ge─
they don't bother with the point. How to calculate one con─
I'll stan out ofanother? Even if you support─
reading the parentheses, then the constants are generated with
dot.
   Is it easier to do #define without parameters?
But then you need to do it inreadconcattword
(or onlyreadtword?) reading from the buffer
lines instead of a file.

 04/08/2017:
 - slight acceleration of self-assembly (compile
for 7 seconds, asm for 4 seconds).
   If you rearrange the points with non-points, then
will intersect with labels in lib (although
you can also have them with dots), with switch marks
(and how to glue a dot to them?), there will be more
problem with sticking and unsticking namespace─
sy, also write the constant twice - in
#define and inconst ! 

   It's easier to implement#define at le level─
xera.

 04/09/2017:

  - added a music analyzer to NedoGift,
simultaneously translated procedures from Nedo─
Lang in NedoAsm for speed. Before that I 
Billiard wrote this method in EvoSDK.


 A
 B
 B
 G
 D
 E
 F


 04/10/2017:

  - I also accelerated self-assembly due to computer─
translation of constants directly into the command (comparisons
oraddition) and acceleration of mass addressing─
vov (addition instead of shift):

for the compile project - 122.30 (was 126.62) 
for the asm project - 70.26 (was 74) 
for the tok project - 82.68 (was 93) 

The total assembly speed was 41 lines 
per second. 

   How to speed up further:

a)simplify the compiler
 b)speed up or remove the tokenizer
c)make a single-pass ASM?
 d)improve code generation? it's not a fact that
this will speed up the build 
e)remove specific procedures in
startup asmom (e.g.rdtword, genautonum)
e)add a hintREGISTERfor
code generator 

 04/11-13/2017: finished NedoGift, then
13-15.04 made the final touches from co-authors─
torov and sent it to the congratulatory recipient (comradeLVD ),
then17-20.04 I prepared a release with fixes for
reviewsLVD.