Watson and the FORTRAN Typing Dictionary

Inspiration from IBM's Projects

The names Watson and FORTRAN in Morinaga OS pay homage to IBM's legacy in computing:

• Watson: Named after IBM's founder Thomas J. Watson and their AI platform, symbolizing intelligence and continuous monitoring.
• FORTRAN: Derived from "Formula Translation," IBM's pioneering programming language, representing computational efficiency.

Introduction to Stenography

Stenography is a method of rapid data entry using chorded key presses, allowing users to input words or commands with fewer keystrokes. Originally used in court reporting, stenography increases typing speed and efficiency.

FORTRAN: Enhancing Computer Control

In Morinaga OS, the FORTRAN typing dictionary extends stenography beyond text input to control the computer with greater accuracy. By mapping chorded inputs to commands and actions, users can:

• Launch applications.
• Execute complex commands.
• Navigate the system efficiently.

Implementing the Typing Dictionary

The FORTRAN dictionary is a JSON file containing chord-to-action mappings:

{
  "STKPWHRAO*EU": "open_terminal",
  "PH-FPLT": "launch_browser",
  "TKEUD": "close_window",
  "KPWR": "volume_up",
  "KPWH": "volume_down"
}

The steno engine listens for chord inputs and executes corresponding actions. Integration with Watson allows these actions to be context-sensitive, adapting to the active application.

Benefits of Stenographic Control

• Speed: Perform actions faster than traditional keyboard shortcuts.
• Efficiency: Reduce repetitive strain by minimizing keystrokes.
• Customization: Tailor the dictionary to individual workflows.

Synchronizing Application State

Overview

Watson is a continuously running background process in both the Linux and Android environments. It monitors the active window and tracks changes in application states to synchronize them between the two systems.

Implementation Details

Monitoring Active Windows

Under X11, Watson uses the Xlib library to detect window focus changes:

// Simplified example in C
#include <X11/Xlib.h>

void monitor_windows() {
    Display *display = XOpenDisplay(NULL);
    Window root = DefaultRootWindow(display);
    XEvent event;

    XSelectInput(display, root, FocusChangeMask);

    while(1) {
        XNextEvent(display, &event);
        if(event.type == FocusIn) {
            Window focused_window = event.xfocus.window;
            // Get window properties and synchronize
            synchronize_state(focused_window);
        }
    }
}

Synchronizing State with Android

Watson communicates with its Android counterpart using the 9P protocol:

1. Establish a 9P connection over a network socket or shared memory.
2. Serialize application state data.
3. Send and receive updates to keep both environments in sync.

Audio Management with PulseMixer and MPV Sinks

For audio synchronization, Watson uses pulsemixer and MPV's sinks to control audio streams:

# Adjust volume
pulsemixer --set-volume 80

# Redirect audio output
mpv --audio-device='pulse/alsa_output.pci-0000_00_1f.3.analog-stereo' audiofile.mp3

This ensures consistent audio behavior across both environments.

Challenges and Solutions

Latency: To minimize latency in synchronization, implement efficient event handling and data transmission methods.

Data Consistency: Use transaction mechanisms to ensure that state changes are applied atomically.