Kestrel-3

Check-in [a6c0c403cb]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:refining some of the storage protocol requirements before my laptop loses power
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256:a6c0c403cbda2b84834913808449757cca077f00568a7f7ecb221be1900f3935
User & Date: kc5tja 2019-09-15 06:24:23
Context
2019-09-15
23:57
Refined some more SSP procedures. But, I am finding that complexity is growing beyond my ability to keep it in my head, and am running into limitations when it comes to distinguishing commands intended for the controller or for the units. Leaf check-in: c6e384b2d2 user: kc5tja tags: trunk
06:24
refining some of the storage protocol requirements before my laptop loses power check-in: a6c0c403cb user: kc5tja tags: trunk
2019-09-11
03:25
Simplify forth auto-load block logic check-in: 46379ac38b user: kc5tja tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to REPORT.org.

298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
...
719
720
721
722
723
724
725






















































































































726
727
728
729
730
731
732
...
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
...
834
835
836
837
838
839
840

































































841
842
843
844
845
846
847
  POSTCONDITION: 3 <= formatting offset <= len(formatting buffer)
  END
#+END_SRC

When booting into the Forth environment, the following sequence of
code can be performed.  Note that the decision of whether or not to
auto-load was made when the user selected the appropriate menu
selection in the boot screen.  Any errors that happen as a result of
attempting to ~LOAD~ the auto-start block will be reported like any
other error that LOAD can generate.

#+BEGIN_SRC
  TO boot into Forth environment DO
O   IF user wants to perform auto-boot sequence DO
O     LOAD Forth auto-start block.
    ELSE
O     Report that auto-start was skipped on user request.
................................................................................
     1      0   131072  2228223  RAID Storage
     2      0  2228224  2229663  Floppy
     2      1  2229664  2231103  Floppy
  OK
_
#+END_SRC























































































































*** Discovering Available Bootable Volumes
When Forth starts up, it must find out what storage devices exist and
which ones can potentially be booted from.  Using this information, it
can populate the initial boot menu, which lets the operator select how
they want to use their computer from that moment forward.

#+BEGIN_SRC
................................................................................
In addition, each unit with a medium inserted can potentially have
more than one partition as well.  We know that partitions can come and
go at any time as well, as anyone who has used a partitioning tool
like ~fdisk~ can attest.

#+BEGIN_SRC
  TO Discover attached devices and volumes DO
U   Ask emulator for its complete unit list.
U   IF emulator responded THEN
O     DO WHILE not at end of unit list
U       Create a unit descriptor for the unit.
O       IF unit is mounted THEN
U         Create a mount-list descriptor.
        END
      END
................................................................................
software will generally only have one partition on them.  Thus, as a
simplification, I've decided to not bother parsing out partitions
during this step.  Thus, each inserted /medium/ is mapped to a single
/volume/ in the Forth environment.

This behavior can be changed in the future with no known impact to
software not explicitly designed to manage storage devices.


































































* Data Structures and Variables
** Global Data
| Description                                   | Type                                                 |
|-----------------------------------------------+------------------------------------------------------|
| Flag that determines if OK prompt is visible. | BOOLEAN                                              |
| Mount list                                    | Pointer to volume descriptor                         |







|
|
|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
...
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
...
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
...
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
  POSTCONDITION: 3 <= formatting offset <= len(formatting buffer)
  END
#+END_SRC

When booting into the Forth environment, the following sequence of
code can be performed.  Note that the decision of whether or not to
auto-load was made when the user selected the appropriate menu
selection in the boot screen.  Any errors (exceptions) that happen as
a result of attempting to ~LOAD~ the auto-start block will be reported
like any other error that LOAD can generate.

#+BEGIN_SRC
  TO boot into Forth environment DO
O   IF user wants to perform auto-boot sequence DO
O     LOAD Forth auto-start block.
    ELSE
O     Report that auto-start was skipped on user request.
................................................................................
     1      0   131072  2228223  RAID Storage
     2      0  2228224  2229663  Floppy
     2      1  2229664  2231103  Floppy
  OK
_
#+END_SRC

**** Guiding Principles Behind the Storage System Design
Anyone who wishes to contribute a peripheral for the Kestrel-3 should
be able to do so without concern for or consent from any special
interest group.  Further, they should be free to use whatever
technology is currently available to them, and that they feel
comfortable or affordable to use.  For example, I'm a big fan of FPGAs
(obviously); however, someone else might prefer microcontroller
technology instead.  Like with all engineering decisions, which you
choose for a particular project will depend on how much you're willing
to spend developing the software and/or hardware for your needs.

Thus, the Kestrel-3 must reasonably expect to interoperate with
peripherals of all varieties, at data rates ranging from kilobits per
second to tens of megabits per second.  As the Kestrel-3's core design
evolves, we can expect this ceiling to be raised as well; there's no
conceivable reason why we couldn't support multi-gigabit speeds on
peripherals, given a suitable I/O channel design.

Most of today's so-called "open" I/O standards are encumbered in some
fashion.  If you're familiar with the design and implementation of PC
peripherals in any way, you'll recognize many of these traits.

- Some standards are open for independent implementation, but legally
  requires an individual contributor to pay impossibly dear sums of
  money /on a periodic basis/ to maintain "vendor IDs" and other
  tokens which enables them to make peripherals that play nicely with
  others in the ecosystem (example: RapidIO).

- Some standards require individual contributors to invest in
  expensive and/or poor-quality parts or tools from limited sources to
  make a peripheral (example: USB).  It's almost as if collusion were
  happening.

- Still other standards require skills that maybe not everyone has or
  /wants/ to acquire just to play (example: parallel PCI, with their
  ingenius, yet no less odd-ball, signaling requirements).

- Other standards make liberal use of entrapment techniques.  For
  example, it's a trivial matter to physically wire up an SD slot;
  however, the latest protocols are behind closed doors until you
  become a paying member of some special interest group.

- Further, despite /all/ of these different affronts, not all of which
  are mutually exclusive, we often find that the authors of some
  peripheral standard are lax when it comes to compliance testing for
  their own products, undermining the very value of the specification
  in the first place.  (Example: SD protocol, again.  So, you not only
  have to pay for the privilege of receiving a copy of the spec, it's
  unlikely to help you in your compatibility testing anyway.)

The Kestrel Computer Project has always intended to rely, as much as
possible, not just on libre software, but also upon libre hardware
technologies.  This implies that the preferred approach to talking to
the outside world should involve /libre I/O standards/ as well.  The
most desirable characteristics of a libre I/O standard include, but
may not be limited to, the following:

- Libre licensed. :: Anyone can learn about the hardware, software,
     and protocols necessary to create peripherals, but also how to
     diagnose problems when things go wrong, etc.

     Correspondingly, this also implies that such a protocol should
     strive to not have any patent or trademark encumberences.

- Strive for implementation simplicity. :: Given the choice between
     two competing technologies that solves a particular problem, the
     libre protocol should always opt for the simpler to implement
     solution.

     When judging implementation difficulty, be sure to use
     whole-system thinking.  It is a trivial matter, for example, to
     implement an SD card interface in hardware; however, the software
     driver stack is anything but an exercise in simplicity.  You need
     to manage SD card protocol versions, voltage levels, whether it's
     byte- or block-addressed, and a litany of card-specific and/or
     manufacturer-specific spec violations in commercially available
     SD cards.  I have learned the hard way that supporting any
     variant of the SD protocol is more difficult than it's worth.  I
     find it /much/ easier to support IDE/ATA storage devices in
     comparison.

- Strive to abstract. :: Where possible, a protocol should facilitate
     entire /classes/ of devices with comparable levels of support, or
     to at least degrade gracefully where required features are
     missing.  As implementors, we're not working in the industry
     24-hours a day, so we /must/ maximally leverage the time we spend
     maintaining these protocols and building compatible devices.

     For example, in this section, I discuss an I/O channel and
     supporting protocol which can handle read-only and read-write;
     removable and fixed; and, physical and virtual storage media of a
     wide variety of sizes.

- Strive for decentralization. :: To the greatest extent possible, the
     protocol and its implementations must strive to be without any
     central authority, except perhaps for compatibility compliance.
     For example, if a libre protocol is built to support
     auto-configuration facilities, there should be no /need/ for a
     centralized registry of vendor IDs.  Put another way, such a
     centralized registry may be convenient to have (especially for
     resolving who made an unknown peripheral); however, it should not
     be a /requirement/ to register with such a body to build
     compatible devices.

- Try as hard as possible to not keep gates. :: This is hard, because
     hardware is, by its nature, /inflexible/; once you adopt a
     technology, it becomes a barrier to entry for those not familiar
     with it.  Worse, unless the designer is very careful, it's
     frequently difficult to separate attributes of an I/O channel
     from that of the protocol that sits on top of it.

     At some point, if you want a device that speaks to the Kestrel at
     multi-megabit-per-second speeds, you will /have/ to invest in
     FPGA technology or better.  If you're happy with slower speeds, a
     simple UART should suffice.  If your expansion mechanism could,
     in theory, be isolated from from the physical attributes of a
     particular I/O channel, then it /should/ be.

*** Discovering Available Bootable Volumes
When Forth starts up, it must find out what storage devices exist and
which ones can potentially be booted from.  Using this information, it
can populate the initial boot menu, which lets the operator select how
they want to use their computer from that moment forward.

#+BEGIN_SRC
................................................................................
In addition, each unit with a medium inserted can potentially have
more than one partition as well.  We know that partitions can come and
go at any time as well, as anyone who has used a partitioning tool
like ~fdisk~ can attest.

#+BEGIN_SRC
  TO Discover attached devices and volumes DO
R   Ask emulator for its complete unit list.
U   IF emulator responded THEN
O     DO WHILE not at end of unit list
U       Create a unit descriptor for the unit.
O       IF unit is mounted THEN
U         Create a mount-list descriptor.
        END
      END
................................................................................
software will generally only have one partition on them.  Thus, as a
simplification, I've decided to not bother parsing out partitions
during this step.  Thus, each inserted /medium/ is mapped to a single
/volume/ in the Forth environment.

This behavior can be changed in the future with no known impact to
software not explicitly designed to manage storage devices.

**** Secondary Storage Port Protocol
All secondary storage peripherals are attached via the Secondary
Storage Port, a 3-wire, 115.2kbps UART interface at 3.3V wired through
a single 1x6 PMOD connector.  Because it's so simple to implement in
hardware, some additional software complexity is required to
facilitate things like flow-control.  I make every effort I can to
keep the total system complexity at a manageable level.

The two sources of inspiration for this design come from the Commodore
64/128/VIC-20 serial bus design, and the Atari 8-bit SIO bus protocol.

When booting, the Kestrel must learn how many storage units are
available, which ones have media installed on them, and their physical
characteristics.  This will provide the required information needed to
construct the boot menu.

#+BEGIN_SRC
  TO Ask emulator for its complete unit list DO
R   Send command (SSP_LIST_UNITS).
O   IF timed out or command rejected THEN RETURN END
U   Receive unit descriptor quantity.
O   IF timed out THEN RETURN END
O   FOR each unit pending DO
U     Process one unit descriptor.
    END
  END
#+END_SRC

All devices are required to respond with a command accepted or
rejected indication, which tells the client two things: (1) did the
server receive the command, and (2) is the command a valid or invalid
command.  In either case, the Kestrel knows that the link is alive and
working well.

If the link is not working well, then the device will never receive
the command, or the Kestrel will never receive the acknowledgement.
In either case, eventually, the Kestrel will time out.

The Kestrel will wait for up to one second for a command acceptance
acknowledgement.  Since intermittent noise can cause corrupted
commands or missing acknowledgements, we try to invoke the command up
to some finite number of times (nominally set to 3 times).  If we
can't get a command through after so many tries, then we give up with
a timeout condition.

#+BEGIN_SRC
  TO Send command (x) DO
O   Begin with the first transmission attempt.
O   WHILE we have transmission attempts remaining DO
U     Start interval timer for 1 second.
U     Wait for sufficient transmission credits or for timeout.
O     IF NOT timeout THEN
U       Send the command bytes for (x).
U       Restart interval timer for 1 second.
U       Wait for command acceptance response or for timeout.
O       IF NOT timeout THEN
O         RETURN command acceptance result.
        END
      END
O     Count transmission attempt.
    END
O   RETURN timeout.
  END
#+END_SRC

* Data Structures and Variables
** Global Data
| Description                                   | Type                                                 |
|-----------------------------------------------+------------------------------------------------------|
| Flag that determines if OK prompt is visible. | BOOLEAN                                              |
| Mount list                                    | Pointer to volume descriptor                         |