Skip to content

ROADMAP.MD

Latest commit

 

History

History
336 lines (297 loc) · 24.3 KB

ROADMAP.MD

File metadata and controls

336 lines (297 loc) · 24.3 KB

Roadmap: Sysclone - Milestone 1 (QBasic / Nibbles)

Phase 1: Foundation and Tooling (Monadic Parsers)

The objective is to implement the syntactic analysis engine in pure JavaScript.

  • 1.1. Implement the core Monadic Parser:
    • Create the base structure (Parser(state) -> { result, rest, error }).
    • Implement fundamental combinators: sequence, sequenceObj, choice, many, optional, lazy.
    • Implement transformation combinators: map (for AST generation).
    • Optimize token parsers (using strict str matching over regex for static symbols/keywords to boost performance).
  • 1.2. Create base Lexers (Primitives):
    • Line-Based Architecture: Create the strict eos (End Of Statement) token to rigorously handle line breaks \n, colons :, and empty lines.
    • Horizontal spaces (ws, optWs) and comments (ignored until line break).
    • Numbers (integers and floats, e.g., 100, .5).
    • Strings ("...").
    • Case-insensitive identifiers (variables, function names, including type suffixes like $, %).
  • 1.3. Custom Test Framework (Quality Harness):
    • Implement test orchestrator with performance KPIs (execution time per test) and global pass/fail summaries.
    • Add V8 stack trace extraction to accurately map syntax/parse errors to specific source files during CI.

Phase 2: QBasic Grammar and AST Generation

Parse the Nibbles source code to build a typed Abstract Syntax Tree (AST).

  • 2.1. Parse Declarations and Types:
    • Handle DEFINT A-Z.
    • Handle structure creation with TYPE ... END TYPE.
    • Handle array and global variable declarations DIM [SHARED] array(x TO y) AS type.
    • Handle constants CONST name = value.
  • 2.2. Parse Control Flow:
    • Conditions: IF ... THEN ... ELSE ... END IF.
    • Loops: FOR ... TO ... STEP ... NEXT, DO ... LOOP UNTIL.
    • To do (if necessary): WHILE ... WEND, SELECT CASE ... END SELECT.
  • 2.3. Parse Subroutines and Functions:
    • DECLARE SUB / FUNCTION.
    • Definitions SUB name (args) ... END SUB.
  • 2.4. Parse Operators and Expressions:
    • Mathematical operators (+, -, *, /, MOD, INT()).
    • Logical operators (AND, OR, NOT, >, <, <>).
    • Complex access: array calls or type properties (arena(r, c).sister).

Phase 3: Runtime Environment

Create the state machine (Virtual Memory) that will store the program's state.

  • 3.1. Memory and Scope Management:
    • Create a chained Environment manager to store global and local variables.
    • [CRITICAL] Enforce MS-DOS "Flat Scoping" to prevent JS-style lexical bleeding, isolating the Main Module strictly from Subroutine scopes by restricting hierarchical traversal to the Tier 1 Global Environment.
  • 3.2. QBasic Array Abstraction:
    • Implement the QArray class to flatten multi-dimensional arrays with custom bounds (e.g., 1 TO 50).

Phase 4: Web Artifact Emulation (DOS/VGA System)

Bridge the gap between specific QBasic DOS commands and browser APIs.

  • 4.1. VGA Text Mode Video Engine (80x25):
    • Implement Pixel-Perfect rendering via bitwise manipulation of an ImageData buffer on a <canvas>.
    • Use the authentic DOSBox bitmapped ROM (Code Page 437) for absolute block fidelity (CHR$ 219, 220, 223).
    • Wire commands: COLOR (text, background), CLS, LOCATE, PRINT.
  • 4.2. I/O Engine (Keyboard & Time):
    • Wire INKEY$ to JavaScript keydown/keyup events with a FIFO keystroke buffer.
    • Wire TIMER to local time (seconds since midnight).
    • Add interactive console commands like LINE INPUT.
  • 4.3. Basic Audio Engine (PC Speaker):
    • Wire the PLAY command (e.g., "MBT160O1L8CDEDCDL4ECC") using the Web Audio API (Macro Music Language).
  • 4.4. Motherboard and BIOS (Segmented Memory):
    • Purist wiring of a 1MB RAM space (Segment:Offset) for DEF SEG, PEEK, and POKE.
    • Simulation of the BIOS Data Area (Equipment Word) and implementation of Memory-Mapped I/O (keyboard buffer flush hack via address &H41A).

Phase 5: Dynamic Evaluation (Virtual CPU)

Connect the AST to the hardware via a virtual processor.

  • 5.1. Generator-Based Evaluation Engine (Anti-Freeze):
    • Transform the Evaluator into a synchronous iterator (function* and yield*) to avoid Promise (async/await) hell and guarantee performance.
  • 5.2. Parse Hardware Instructions:
    • Add instruction parsers (statements.js) for hardware: CLS, PRINT, LOCATE, COLOR, POKE, ASSIGN.
  • 5.3. CPU Loop (Runner):
    • Implement "Instruction Cycle" (Ticks) pacing limited by requestAnimationFrame (e.g., 5000 cycles / frame) allowing total control over execution speed.

Phase 6: Advanced QBasic Features (Pre-Nibbles)

Implement complex structures necessary for game logic.

  • 6.1. Arrays and Types (UDT) at Runtime:
    • Teach the Evaluator to handle TYPE_DECL nodes (register the structure's "blueprint").
    • Teach the Evaluator to instantiate DIM (and dynamically resize via REDIM), handling both array structures and single scalar variable declarations (DIM x AS type).
    • Implement deep cloning for UDTs to prevent JavaScript object reference mutation during SWAP and ASSIGN.
  • 6.2. Subroutines (SUB and FUNCTION):
    • Handle SUB definition registration.
    • Implement the CALL mechanism: creating a sub-scope (new memory "bubble") and passing arguments.
    • [CRITICAL] Implement Pass-by-Reference for arguments.
    • [CRITICAL] Handle FUNCTION return values (assignment to the function's name).
  • 6.3. Jumps and Advanced Control Flow:
    • Implement GOSUB ... RETURN (heavily used in Nibbles for screen/music mini-routines).
    • [CRITICAL] Refactor AST to allow Single-Line IFs (IF ... THEN statement without END IF).
    • [CRITICAL] Add WHILE ... WEND loop structure.
    • Add support for conditions at the beginning of DO loops (DO WHILE/UNTIL cond ... LOOP).
  • 6.4. Missing Operators and Literals:
    • Add integer division \ with operand rounding, and MOD operator.
    • Add logical operators AND, OR, NOT.
    • Add exponentiation ^ and bitwise operators XOR, EQV, IMP.
    • Add support for octal format (&O) and scientific notation (E, D).

Phase 7: Trial by Fire - Nibbles.bas (Update)

The final boss: Assemble everything to run the original game, correcting critical architectural flaws.

  • 7.1. Source File Integration:
    • Fetch text loading of the original NIBBLES.BAS file.
    • Full end-to-to parsing (requires prior patching of the eos regex against ReDoS).
  • 7.2. Debugging and Structural Corrections:
    • Scope Correction: Isolate LABEL tags by scope (Main vs Sub/Function) in evaluator.js to avoid collisions.
    • Memory-Mapped VRAM: Link POKE instructions from the &HB800 segment (Memory) to VGA engine cells.
  • 7.3. Engine Tweaks (Pre-JIT):
    • VGA Optimization (Memory): Implement rendering via Uint32Array for fast buffer manipulation and pure Dependency Injection (DOM decoupling).
    • VGA Optimization (CPU): Implement a "Dirty Rectangles" system to update only modified areas and secure framerate.
    • CPU Lightening: Optimize Generator Tick frequency (per-statement instead of per-node) and implement Hybrid Runner (Retro/Turbo UI).
  • 7.4. Play Nibbles!

Phase 7.5: Architecture Refactoring (The Web Shell)

Decouple the Virtual Machine core from the Web interface to ensure a clean architecture.

  • 7.5.1. Entry Point Renaming:
    • Rename main.js to webvm.js to explicitly designate it as the browser orchestrator.
  • 7.5.2. UI Logic Extraction:
    • Create src/ui.js to encapsulate all DOM manipulations (Sliders, Checkboxes, Event Listeners).
    • Establish a clean event-driven communication layer between ui.js and webvm.js.

Phase 7.6: The WebVM Developer Experience (DX)

Finalize the user experience with dynamic loading, emulator-standard controls, and source inspection.

  • 7.6.1. Dynamic ROM Loader & Build System:
    • Implement VM reset capabilities (Clear RAM, Reset Environment, Cancel AnimationFrame) in webvm.js.
    • Create tools/build_catalog.js with private draft filtering to statically generate the ROM index.
    • Wire the #rom-selector to fetch the catalog and execute new .bas files on the fly.
  • 7.6.2. Fullscreen & Pixel-Perfect Scaling:
    • Implement the HTML5 Fullscreen API.
    • Add CSS logic to maintain the original 4:3 aspect ratio and image-rendering: pixelated during viewport resizes.
  • 7.6.3. Split-Screen Pro IDE Layout:
    • Redesign the interface into a modern workspace (Editor on the left, Execution on the right).
    • Display the raw, original .bas source code currently being executed.
    • Upgrade the CPU speed visualizer to reflect realistic retro hardware frequencies (MHz).
  • 7.6.4. VM Execution Controls (State Management):
    • Implement a Play / Pause toggle to freeze/resume the Virtual CPU requestAnimationFrame loop.
    • Implement Restart (halt execution, clear VRAM, and reboot the current ROM).
  • 7.6.5. Canvas Screenshot Tool:
    • Add a "Screenshot" button using native canvas.toBlob.
    • Ensure the output is a clean, lossless PNG preserving the DOS pixel ratio.
  • 7.6.6. Native Syntax Highlighting (Dog Fooding):
    • Refactor lexers.js to explicitly yield COMMENT and WHITESPACE tokens instead of silently swallowing them in eos. (Included strict regex exclusion of \r\n in comments to fix UI line break rendering).
    • Build a lightweight Tokenizer that processes the loaded source code through our Monadic Lexers.
    • Render the token stream as colored HTML in the UI without external dependencies.
  • 7.6.7. Source File Injection:
    • Allow users to import or drag-and-drop local .BAS files directly into the WebVM for instant execution.
  • 7.6.8. Execution Context Tracking:
    • Implement a visual tracker in the IDE that highlights the exact line of code currently being executed when the VM is paused.
  • 7.6.9. Zero-Backend Media Studio:
    • Implement a Web Worker-based 24FPS GIF recorder to capture pixel-perfect XOR graphics without blocking the CPU loop.

Roadmap: Sysclone - Milestone 2 (Gorillas & Mandelbrot)

Phase 8: I/O Encoding & Robust Loading

Ensure safe and authentic loading of legacy source files.

  • 8.1. CP437 / Unicode Translation Layer:
    • Create a bidirectional mapping table (toCP437 / fromCP437).
    • Update main.js to load standard UTF-8 raw files and transpile them into CP437 bytes, ensuring rigorous preservation of legacy control bytes.
    • Implement debugging and CLI helpers (convert_bas.js) for legacy file restoration.

Phase 9: Advanced QBasic Semantics (Gorillas Prep)

Implement complex language features required by advanced Microsoft demos.

  • 9.1. Static Data Structures:
    • Implement DATA, READ, and RESTORE with a global data pointer in the Environment.
    • Handle unquoted strings and empty entries in DATA.
  • 9.2. Exception Handling:
    • Implement ON ERROR GOTO, RESUME, and RESUME NEXT to catch runtime hardware/software faults.
  • 9.3. Implicit Typing & Macros:
    • Support DEFINT, DEFSNG, etc., for default variable typing.
    • Parse and evaluate single-line macros: DEF FNname(args) = expr.
    • Support multiple CONST declarations and STATIC subroutine scoping.
    • Support fixed-length string allocation (AS STRING * N).
    • Support implicit function calls strictly by base name (without type suffix).
    • Implement memory aliasing (VariableVault) to natively link explicitly typed variables (e.g., AS STRING) to their MS-DOS suffixed counterparts ($).
    • Support empty parentheses () in array declarations for dynamic or shared arrays.
    • Unify AST generation between DIM/SHARED statements and Subroutine parameter lists.
  • 9.4. Math, Strings & Built-ins:
    • Abstract all pure native functions into a builtins.js standard library.
    • Implement strict PRINT USING formatting engine (variable-length string interception & and decimal precision ###.###).
    • Implement math: INT, FIX, ABS, SQR, RND, SIN, COS, TAN, ATN, EXP, LOG, CINT.
    • Implement strings: CHR$, ASC, STR$, HEX$, VAL (including Hex &H and Octal &O prefixes), LEN, MID$, LEFT$, RIGHT$, UCASE$, LTRIM$, RTRIM$, INSTR, STRING$, SPC.
    • Add memory/variable utilities: SWAP, ERASE.
    • Add hardware intercepts: TAB, INPUT$.
  • 9.5. The "Truth Vector" Compatibility Framework (Automated DOSBox): A rigorous, automated infrastructure to prove the VirtualCPU's exact historical accuracy against MS-DOS originals without polluting the repository.
    • Design a language-agnostic format (e.g., JSON/YAML) to strictly document QBasic/MS-DOS quirks and edge cases.
    • Isolation & Clean Repository: Move truth vectors out of examples/ into a dedicated sandbox (tools/truth_vectors/). Implement strict .gitignore rules (engines/) to keep proprietary MS-DOS binaries (QB45/TP7) out of version control.
    • Orchestration Node.js (CLI Runner): Develop a headless DOSBox controller (runner.js) that dynamically mounts workspaces, triggers native MS-DOS compilers/interpreters via batch scripts, and captures standard output (output.txt) for direct comparison.
    • Contributor DX: Write a comprehensive README.md detailing the local DOSBox setup and binary sourcing (Archive.org) to make truth vector validation reproducible for anyone.
    • Build a generator that compiles these vectors into a native .BAS self-testing suite.
    • Build a companion generator that translates these verified vectors into JavaScript unit tests for the Sysclone engine.
    • Generate comprehensive Markdown documentation automatically from the Truth Vectors to serve as a definitive guide to QBasic behavior.
  • 9.6. Exhaustive Truth Vector Implementation (Instruction by Instruction): Systematically port and document every QBasic instruction and built-in function to the Truth Vector JSON format. Each entry must detail standard behavior (Black Box) and historical MS-DOS quirks (White Box).
    • Suite 1: Strings & Type Casting (builtins_string.json):
      • STR$, HEX$, VAL (Hex/Octal).
      • MID$, LEFT$, RIGHT$, LEN, UCASE$, LCASE$, LTRIM$, RTRIM$, SPACE$, STRING$.
      • CHR$, ASC, INSTR.
    • Suite 2: Math & Floating-Point Unit (builtins_math.json):
      • INT, FIX, CINT, \, /, MOD.
      • ABS, SQR, EXP, LOG, RND.
      • Trigonometry (SIN, COS, TAN, ATN).
    • Suite 3: Memory, Scoping, & Structures (memory_scoping.json):
      • Explicit Aliasing (DIM X AS STRING to X$ mapping).
      • Implicit Typing (DEFINT, DEFSNG).
      • Fixed-Length Strings (AS STRING * N).
      • STATIC & SHARED (Tier 1 global routing vs localized shadowing).
      • SWAP, ERASE.
    • Suite 4: Control Flow & Jumps (control_flow.json):
      • FOR ... NEXT, EXIT FOR (Iterator mutation and overshoot).
      • DO WHILE/UNTIL ... LOOP (Pre-test vs Post-test Truthiness).
      • WHILE ... WEND (Legacy fallback handling).
      • SELECT CASE (TO ranges, IS relational operators, no fallthrough).
      • GOSUB ... RETURN (Mini-callstack and spatial scoping).
      • Single-Line IF: Statement chaining via colons (:) and ELSE binding.
    • Suite 5: Procedures & Functions (procedures.json):
      • CALL / SUB: Pass-by-Reference vs Forced Pass-by-Value via parenthesis.
      • FUNCTION: Return value assignment via function name.
    • Suite 6: Data & Error Handling (data_errors.json):
      • DATA, READ, RESTORE: Global data pointer manipulation and unquoted string parsing.
      • ON ERROR GOTO, RESUME [NEXT]: Hardware stubbing verification.
    • Suite 7: I/O, Formatting & Terminal (io_terminal.json):
      • PRINT: Spacing rules for positive/negative numbers and ghost commas.
      • PRINT USING: Variable-length string interception (&) and decimal precision (###.##).
      • LOCATE: Ghost parameter handling (LOCATE ,,0) and out-of-bounds protection.
    • Suite 8: Graphics & VRAM - Silent Probing (graphics_vram.json):
      • WINDOW: Cartesian inversion and viewport clipping logic.
      • PSET, PRESET: Bounds checking and default background fallback.
      • LINE: Bresenham accuracy, B/BF flags, and STEP relativity.
      • CIRCLE: Aspect ratio normalization, negative aspect ellipses, and 0-crossing arcs.
      • PAINT: Boundary collision tracking.
      • GET, PUT: Little-Endian header packing, XOR blending, and EGA/VGA translation.

Phase 10: Advanced Graphics Engine (Gorillas & Mandelbrot)

Expand the engine capabilities to support computationally and graphically intensive classics.

  • 10.1. Viewports & Coordinate Systems:
    • Support WINDOW to map logical coordinates to physical screen pixels natively in the VGA router.
    • Support STEP relative coordinate tracking (Graphic Cursor) across all geometry primitives.
    • Support VIEW PRINT to restrict text rendering and scrolling to specific screen rows.
  • 10.2. Palettes & Screen Modes:
    • Implement generic setPalette and default hardware palettes (CGA, EGA, MCGA/VGA).
    • Implement high-resolution simplified SCREEN 9 (EGA 640x350), SCREEN 12 (VGA 640x480), alongside SCREEN 13 (VGA 320x200).
    • Load mode-specific, dynamic hardware typography (DOSBox 8x8 font for VGA, authentic 8x14 font for EGA block alignment).
    • Implement elegant, dynamic viewport resizing and CSS scaling to seamlessly transition between native hardware resolutions (e.g., SCREEN 0 Text Mode, SCREEN 12 VGA 640x480, SCREEN 13 VGA 320x200) without breaking the DOM layout or pixel aspect ratio.
    • Add PALETTE and PALETTE USING dynamic mapping support.
  • 10.3. Geometry Primitives (SCREEN 7, 8, 9, 13):
    • Implement PSET, PRESET, and POINT (Pixel Write/Read).
    • Add geometry rendering: LINE (Bresenham, Box, Box Fill), including relative origin support LINE - (x, y).
    • Support STEP (Relative coordinates) dynamically across all geometric primitives.
    • Implement advanced CIRCLE logic: accurately emulating QBasic's default screen aspect ratios, undocumented negative aspect inversions, arc drawing, and 4-connected Midpoint anti-aliasing to prevent Moire holes in concentric loops.
    • Add algorithmic drawing: PAINT (Iterative Stack-based Flood Fill algorithm).
  • 10.4. Hardware Blitting (GET / PUT):
    • Emulate legacy QBasic sprite array formats (Little-Endian Header + Pixel data).
    • Implement generic GET and PUT with bitwise operations (PSET, XOR, AND, OR, PRESET).
    • Implement intelligent formatting logic: processing EGA Planar bit-packing (Bitplanes) vs VGA Linear chunk packing seamlessly based on the active hardware driver.

Phase 11: Hardware Interrupts & Advanced Input

  • 11.1. Advanced Keyboard Controller:
    • Upgrade state tracking with a strict 16-byte FIFO buffer.
    • Implement deep anti-ghosting safeties (filtering e.repeat held-down keys) to prevent input flooding and game logic corruption (e.g., Gorillas 0-velocity bugs).
  • 11.2. Time & Audio Engine:
    • Wire SLEEP and SOUND to trigger a SYS_DELAY hardware interrupt, yielding execution control asynchronously to the Orchestrator without freezing the browser UI.
    • Implement preliminary BEEP instruction.
    • Wire the full PLAY command to the Web Audio API (AudioContext).

Phase 12: The Grand Playtest

  • 12.1. Rosetta Code Mandelbrot:
    • Execute math-heavy visual rendering (WINDOW, PSET, MCGA).
  • 12.2. Gorillas.bas:
    • Full execution (projectile math, explosive PAINT rendering, XOR sprite blitting, sound effects).
  • 12.3. SortDemo.bas & Reversi.bas:
    • 100% Parsing achieved.
    • Full execution of board logic (Reversi) and preliminary sorting algorithms (SortDemo).

Phase 12.3: Universal Syntax Linter (The AST Gatekeeper)

Before splitting the architecture, establish a strict diagnostic layer. The Linter must catch syntax errors and logic flaws directly from the AST, protecting the VirtualCPU from runtime crashes.

  • 12.3.1. Core Linter Engine:
    • Create tools/linter.js.
    • Implement an AST traversal mechanism to silently dry-run .bas programs.
  • 12.3.2. Lexical & Semantic Validation:
    • Reserved Keyword Protection: Flag variables explicitly named after reserved QBasic Native Functions or Tokens (e.g., VAL = 5, LEN = 10).
    • Label Integrity: Detect and report duplicated jump labels (EndTest: used twice) and "Label Not Found" jumps.
    • Block Matching: Verify structural integrity (e.g., FOR without NEXT, DO without LOOP).

Phase 12.5: Multi-Language Architecture Refactoring (The Engine Split)

Before introducing Turbo Pascal or JIT compilation, the monolithic engine must be strictly decoupled. The goal is to separate the universal components from language-specific execution rules and memory behaviors.

  • 12.5.1. Environment Decoupling (Memory & Scoping):
    • Extract the core memory interactions (RAM mapping, Data Bank skeleton) into an abstract BaseEnvironment.
    • Create QBasicEnvironment (inheriting from Base): Isolate the 3-Tier Flat Scoping, VariableVault aliasing, and DEFINT implicit typing rules.
    • Create PascalEnvironment (inheriting from Base): Implement standard Lexical Scoping (this.parent traversal) and prepare a mock Heap Allocator for typed pointers.
  • 12.5.2. Evaluator Specialization (Monolithic Pattern):
    • Rename Evaluator to QBasicEvaluator and move to src/runtime/qbasic/ to explicitly reflect its language-specific nature.
    • Create PascalEvaluator (Stub) to handle strict lexical and procedural execution.
  • 12.5.3. UI & Bootstrapping (Dynamic Strategy):
    • Create the QBasicTokenizer extending BaseTokenizer.
    • Create the PascalTokenizer extending BaseTokenizer to handle TP keywords and syntax (BEGIN, END, ;).
    • Update src/ui.js and src/webvm.js to dynamically inject the correct Tokenizer, Environment, and Evaluator based on the loaded file extension (.bas vs .pas).

Roadmap: Sysclone - Milestone 3 (JIT Compilation)

Phase 13: JIT Evolution and Advanced Optimizations

Transform the interpreter into a true Just-In-Time compiler to fulfill the project's vision.

  • 13.1. Strict Memory Typing:
    • Replace standard JS arrays in QArray with Typed Arrays (Int16Array, Float32Array) based on the defined QBasic type.
  • 13.2. AST to JS Compilation (Level 1 JIT):
    • Create an AST visitor that generates pure JavaScript code instead of evaluating it dynamically.
  • 13.3. WebAssembly Compilation (Level 2 JIT):
    • Translate the intermediate code (IR) into WASM binary for near-native performance.

Roadmap: Sysclone - Milestone 4 (Headless & CLI)

Phase 14: Headless Execution (Node.js Integration)

Bring the Sysclone engine to the backend, allowing QBasic scripts to be executed or compiled as standalone CLI applications.

  • 14.1. Core CLI Orchestrator:
    • Create tools/nodevm.js (or a compiler CLI) to load, parse, and execute .bas files directly from the terminal.
    • Implement command-line argument handling (e.g., node tools/nodevm.js script.bas).
  • 14.2. Headless Hardware Abstraction (HAL):
    • Implement ConsoleVGA: A headless display driver that maps basic PRINT statements to process.stdout and ignores graphical commands (PSET, WINDOW).
    • Implement NodeIO: A headless input driver mapping INKEY$ and INPUT to process.stdin.
  • 14.3. Backend Math & Logic:
    • Validate the execution of heavy computational scripts (e.g., algorithmic math, text-based data processing) without browser overhead.