Jonathan Blow on Code as Data

Here are detailed notes from the transcript:

Detailed Notes: Code as Data, Reality-Based Design, and Compile-Time Execution

Main Topics Discussed:

1. The Concept of "Code as Data": Debunking the literal interpretation and explaining its practical significance, especially in the context of Lisp and macros.
2. "Reality-Based Design": A philosophy for building systems informed by practical experience and daily challenges.
3. Introduction to a Large-Scale Project: A commercial-quality video game built using the speaker's custom systems language.
4. Unique Compilation Architecture: Focusing on the design goal of program understanding through compile-time execution and "workspaces."

Key Points and Arguments:

• "Code as Data" Nuance:
  • The literal meaning (Turing machine abstraction, early computers interpreting data as process) is not what Lisp users are excited about; nobody was ever confused by this literal interpretation.
  • For Lisp enthusiasts, "code as data" truly means the ability to manipulate a higher-level version of data using powerful macros.
  • For a systems language, this implies very high levels of viewing your code as data.
• Project Design Philosophy:
  • The speaker's current project is an example of "reality-based design."
  • This design approach is directly informed by past building experiences and the practical, day-to-day problems encountered in software development.
• Motivation for Architecture:
  • The project's unique and "crazy" compilation architecture evolved from a fundamental question: "What are you going to do that's different from other systems languages?"
  • The core answer and goal was to "understand my program better."
  • This understanding is achieved through extensive compile-time execution.
• The Role of Workspaces:
  • Workspaces are presented as a key architectural component to facilitate program understanding.
  • They are conceptualized as separate containers (like Unix containers or module spaces) within the programming language.
  • These workspaces allow the compilation of separate programs that can still "see each other," enabling a layered or interconnected view of the code.
  • This system becomes active as soon as anything is compiled.

Important Facts or Data Mentioned:

• Project Type: A commercial-quality video game.
• Project Size: Approximately 300,000 lines of code (excluding whitespace and comments).
• Language Efficiency: The 300,000 lines are considered equivalent to a "substantially larger C++ program" due to the higher-level nature of the speaker's language.
• Game Features Demonstrated:
  • Player character movement and interaction.
  • Puzzle-solving mechanics (e.g., pulling objects).
  • Monster interactions/attacks.
  • "Undo" functionality for gameplay actions.
  • Extensive game mechanics.
• Developer Tooling:
  • Includes a built-in editor for scene manipulation (moving, scaling, copy-pasting objects, undoing editor actions).
  • All UI elements and widgets (e.g., text inputs for entity properties) are written in the same programming language.

Conclusions or Recommendations (Implied from Design Philosophy):

• Focus on Program Understanding: A primary driver for systems language design should be to enable developers to better understand their programs.
• Leverage Compile-Time Execution: Extensive compile-time execution can be a powerful tool for achieving deeper program understanding.
• Innovative Architectural Components (Workspaces): Designing structured environments like "workspaces" can facilitate the compile-time analysis and interaction needed for improved program understanding, allowing interconnected yet distinct code segments.
• Reality-Informed Development: Software design should be grounded in practical experience and address real-world problems faced by developers.