Finding Integration Win-Wins with Patch

[PullJosh]

Patch is a Python-for-Scratchers coding environment that shares many of Leopard's goals and challenges. I've been meeting with the BX Coding team (particularly Elliot) to share what we're working on and what we've learned, and we've been thinking a lot about where we can find win-win opportunities for collaboration.

The purpose of this issue is to discuss how the projects are similar, how they are different, and where there are opportunities for collaboration.

What is Leopard? Patch?

Leopard

What:

1. Leopard: A JavaScript game engine that provides a one-to-one mapping between Scratch blocks and JavaScript methods
2. sb-edit: An automatic Scratch-to-JavaScript translator that converts .sb3 files to JavaScript code using the Leopard game engine library
3. Project editor (alpha): A static website code editor that thoughtfully combines the user experience of VSCode with the user experience of the Scratch project editor
4. Social website (later this summer): A social website, analogous to Scratch, which allows users to share static websites (which will generally–but not always–be Leopard games)

Why: To help Scratchers transition to JavaScript & web development.

How: By thoughtfully blending the concepts and conventions from each world.

• The Leopard library ("game engine") provides a one-to-one analogue for each Scratch block. When a Scratch block or tool corresponds nicely to an equivalent tool in JavaScript, we map it directly.
• The Leopard editor provides a visual layout that is similar to Scratch while operating on a directory-based static file system that allows users to build any static website they want.
• The Leopard social website will provide a Scratch-like project sharing experience, extended to meet the needs of a slightly more mature audience (but not drastically different) and incorporating features specifically for sharing static web pages and sites.

Patch

The Patch folks can paint a more accurate picture, but I would describe Patch in the following way:

What:

1. Patch editor: A Scratch-like program that enables students to build projects by typing Python code rather than dragging and dropping blocks.
2. Patch curriculum: Scratch and Patch lessons and experiences designed to be used by teachers in classrooms.
3. Classroom features (later this summer?): More features built into the editor and website that allow teachers to design classroom coding experiences for students–including customized editing experiences that add or remove "magic" to provide on-ramps for students learning to code.
4. Non-profit organization: The BXCoding organization teaches coding camps to Ohio students and acts as a source of development funding/resources.

Why: To make computer science education more accessible for both students and teachers.

How: I don't have a great "how" because I'm not on the team. But I will note that they are doing a very good job of focusing on curriculum development and teaching in the real world, not just building a tech tool.

Similarities & Differences

We are both building Scratch -> [language] transition tools. This leads to a lot of inherent similarities.

• We are both building runtimes that emulate Scratch in some way
• We are both building editors that are designed to look like Scratch but allow for text-based code editing
• We are both aiming to help primarily young people who know Scratch develop skills with other languages

Leopard focuses on JavaScript. Patch focuses on Python. But the other differences are more subtle:

• Leopard provides a custom runtime that behaves the same as Scratch. Patch is wiring into the real scratch-vm.
• Leopard takes a zero-magic approach, generating static HTML/CSS/JS that can be run with zero specialized tooling. Patch uses a bit of behind-the-scenes magic to execute Python code in the browser in a way that is attached to the Scratch VM.
• The Leopard editor is, at its core, designed to be un-special. It should rely on standard tooling that developers are familiar with. (In the future, we would hope to support lots of the IDE features that you get in, for example, VSCode.) The Patch editor aims to be customizable so that teachers and curriculum designers can create tailored experiences for students. (For example, you might want specialized autocomplete that only reveals certain functions, etc.)
• The Leopard library/editor/website/approach to everything is designed with at-home enthusiasts in mind. (I always imagine myself from a few years ago as the user.) Patch is designed to be used by teachers and students in classrooms as part of a CS curriculum.
• Leopard has no registered organization and no monetary support. (We're broke, except out of my own pocket. I've been looking at setting up a Patreon, but that hasn't happened yet.) Patch lives under the BX Coding umbrella and has resources and support from the university.

Searching for Win-Wins

I've been chatting with the Patch/BXCoding team, particularly Elliot, and Elliot brought up the idea of combining efforts and finding development win-wins between Leopard and Patch where we can.

There are essentially three layers of infrastructure that both projects are independently building at the moment:

1. The code runtime. For Leopard, this means the Leopard game engine/library. For Patch, this means the code that wires up PyScript (Python in browser) to the Scratch VM.
2. The code editor. For Leopard, this is currently a more traditional static html/css/js editor (VSCode-inspired), just with the option to view the file tree in a Scratch-like layout. For Patch, this is a more customized Python editor.
3. The social/sharing features. For Leopard, this is a Scratch-like website for sharing html/css/js projects (most likely Leopard projects, but there's no limit!) For Patch, this is a website for sharing Patch projects.

At each layer, there are big philosophical and technical differences, but also lots that is the same.

The question for this discussion: Are there win-win opportunities where we can collaborate or share code/knowledge/infrastructure between Leopard and Patch?

[PullJosh]

To get us started on searching for win-wins, here are a few notes that Elliot and I discussed at each layer:

Runtime

At the runtime level, Patch development has been challenging because fighting with the scratch-vm is hard. It is old, big, and was never designed to be used for this purpose. It would be lovely if they could completely replace the Scratch VM with the Leopard runtime instead.

• Patch would benefit from no longer having to wrestle with scratch-vm. They could instead use a lightweight runtime that is perfect for their needs.
• Leopard would benefit from having additional programmers who have an incentive to make the Leopard runtime better. Ideally we would see upstream bug-fixes and improvements.

Having an additional use case for the Leopard library would push and pull on its design in interesting ways. It's not clear to me what the pros and cons of these new tensions would be. It's possible that expanding the ways Leopard is being used and deployed would encourage more accessible, reusable APIs that are better-designed for more use cases. It's also possible that these new use cases could distract from or compete with the existing set of Leopard design constraints. Personally, I'm inclined to believe that this would be much more of a net gain than a loss, but I don't have much reasoning–just my gut sense, which is not necessarily well-trained.

Code Editor

This one is interesting. On the one hand, we are both building the same thing: A text-based code editor with JavaScript conventions.

But we also have different goals. In particular, I think that Leopard being no-magic/its-just-web-development drives us towards mostly just neatly integrating existing tooling as-is. (For example, I've plopped in the Monaco text editor which powers VSCode and it is giving tons of benefits for free. No additional setup required.) On the other hand, Patch wants to design more custom-tailored (potentially magical) experiences, and ideally expose this customization functionality to users like teachers who might want to create custom experiences for their kids. This lends itself to choosing technology that provides less out of the box but is more tweakable, like the CodeMirror editor.

One narrow slice I could see us easily collaborating on is costume and sound editing. Those things are nice to have integrated into the editor and would likely be the same between Leopard and Patch.

In my mind, it's still a very open question about where we can find win-wins in the building of our code editors.

Project Sharing/Community

There is lots to be explored here, but so far the only thing Elliot and I have touched on together is the fact that both of us will have to do content moderation (at the very least filtering out nsfw content) and there's room for sharing our learning or possibly even our infrastructure there.

I think there's lots more room for discussion on this. I could honestly imagine the win-win opportunities here being a total dud or the most important thing we do.

[ElliotRoe]

Hello! Elliot from aforementioned the Patch & BX Coding team here. Figured I pop in here with some more details on Patch.

Patch's Goals

Make a student's transition from Scratch as easy, intuitive and fun as possible.

Because of this goal, we fall much closer to the ✨magic✨ end of the spectrum of conversion as @PullJosh mentioned. An example of this is events. Leopard's method for event handling is to use a user-created trigger object to designate scripts that should run during certain event (e.g. new Trigger(Trigger.GREEN_FLAG, this.myScript). Patch, on the other hand, simply just provides a drop down for users at the top of each sprite's script to select which event trigger's it. Each option has it's trade offs and neither is particularly better than the other, but they are definitely better suited for different types/skilled of users.

Make teaching CS in-schools easier

Because we use Patch in BX Coding's summer camps, it has an associated curriculum. It starts kids in Scratch, then jumps to Patch and finally Python. We want to continue to build out Patch's educator features (in-editor explanations, clearer/custom error messages, lesson & curriculum builder) to begin allowing for use in in-schools environments. Critically, we want to build an editor that allows educators to decide how much ✨magic✨ they want to show to their students and also share this configuration with other educators. This is very similar to what exists in Replit right now, but--from what I can tell--they've shifted away from educational use cases. And, no other platform that I've found so far (could definitely be wrong about this) easily allows the sharing of editor configurations.

If you're interested in reading more about our actual implementation, our repo is here (the documentation is admittedly a little out of date) and our live MVP is here.

Will be following up with more thoughts on win-wins and potential challenges a little later, but in the meantime wanted to follow up with the above info!

[towerofnix]

Hello! Totally not experienced with Patch before (although it looks awesome), but we've helped code Leopard and sb-edit. We have a little technical perspective!

Runtime

Leopard provides bare utilities for a totally separate runtime than scratch-vm. However, compatibility with Scratch has been a huge goal from the start, since a lot of its magic comes from showing that your very own Scratch projects really will work when coded with plain old JavaScript! That magic would fizzle without good compatibility 😅

It sounds like it would be awesome if Patch could use Leopard as the internal "framework" which deals with all the nitty gritty of actually running the projects. But, if we're standing in for an existing implementation based on scratch-vm, we need to consider two different questions, IMO:

1. How much does Leopard still not support scratch-vm behavior? Would this have an impact on Patch?

• The simplest answer is to see issue Support most Scratch blocks #18. We currently don't support the Music extension or in-player drag modes. But the most glaring omission is "stop all" and "stop other scripts in sprite".
  • This might or might not really be a problem for Patch — I've only really read over the component diagram, but I get the sense that the Patch VM may be responsible almost only for controlling scratch-render and maybe handling user input. If execution is handled almost 100% by the Patch Worker, then I think Leopard is best fit to slot in for Patch VM, not replace Patch Worker. This means we aren't (re: Patch) concerned about supporting special control flow management blocks, because that execution state is and will remain handled by Patch Worker.
  • If that's right, then lacking support for the Music extension would comparatively be a bigger deal!
  • We still need to make sure we support canceling certain long-lasting primitives that Leopard provides (as in "stop all"), though - "play sound until done", possibly glide blocks if an analogue isn't internally coded in Patch Worker, etc.
• Leopard provides its own rendering engine (dating to 2020 and periodically revisited since) and does not interface with scratch-render. This is mainly a good thing, because it's both an analogue of scratch-render and rather cohesive code. Some notable differences, still:
  • It doesn't yet support fonts in Scratch costume SVGs (Support using different fonts #107). Possibly low impact if Patch doesn't involve literally converting Scratch projects to Python, but good to be aware of.
  • It's been used for most of as long as Leopard is around, so we consider it mature, but obviously Leopard has a lot fewer sum users and bug reporters than Scratch does.
  • @adroitwhiz, who authored basically all of the renderer, has a ton of experience with scratch-render and rendering in general, and has filed and maintained PRs like Handle subpixel viewboxes for SVG skins scratchfoundation/scratch-render#594. We can't speak for her, but there's the possibility that her improvements for scratch-render could show in Leopard sooner than scratch-render.
• I'm not sure where Leopard's compatibility is at regarding making sure two copies of a trigger don't run at once, edge-triggered hat blocks work like they do in Scratch, etc. I think these are things we've accounted for but we'd have to review. It could have an impact on Patch, since obviously those triggers (which Leopard would remain repsonsible for) should line up with Scratch behavior.

2. Is Leopard a good fit as a "just slot me in" internal project runtime? Could it be? Does it meet Patch's needs?

• We're comparing with scratch-vm, which has been an uphill battle. Leopard is a much simpler set of internal systems and isn't centuries old and untouched, so ostensibly it has an advantage by those merits already. But we need to consider its external interfaces, too.
• Philosophically, I think Leopard should serve as a good "slot me in" library. If it doesn't yet, it's no static code beast: it's a big cat, and agile! We should be able to alter its interfaces and assumptions to better fit in with use cases besides projects generated by sb-edit (like Patch).
• If Patch Worker handles / continues to handle most of project execution and control flow, we need to make sure that:
  • ...we have a well-defined interface for running primitive blocks which take time, e.g. play sound until done or wait-seconds. (I.e, does this use promises or generators, and can it work either way?)
  • ...our interface for user input triggers is sufficient for Patch to make use of. (I.e, can Patch readily listen for events like "when this sprite clicked" and use that to activate its threads?)
  • ...our behavior for not duplcating a still-active trigger (assuming that's present, see above) is either something Patch can understand and readily interface with, or just something that Patch Worker will handle all on its own.

We can't follow most of these dots up with relevant detail yet, because we don't have a good understanding of exactly which behavior Patch Worker is responsible for and is ready to take over from scratch-vm (and thus from Leopard). Generally, the more Patch takes responsibility for its own execution state, the more useful I think Leopard would be to it as a runtime library. I think Leopard should be really quite simple and provide useful interfaces for making things happen, like a ready-to-go game engine with convenient primitives, regardless if you're using Leopard's built-in control flow (with JS generators/yield, etc) or just hooking into its primitives from your own control flow (as it sounds like Patch would).

I'd love to hear about what challenges Patch has run into when interfacing with scratch-vm, and how the team personally feels about the new(?) framework, with most execution functionality stripped out of scratch-vm and moved into Patch Worker. Does that feel like the right direction for Patch's runtime? Are there things you wish could be easier or nicer than they have been, or still are, with scratch-vm? I think sharing some of that would help identify how well Leopard could work as a library that helps Patch!

Editor

One narrow slice I could see us easily collaborating on is costume and sound editing. Those things are nice to have integrated into the editor and would likely be the same between Leopard and Patch.

Totally agreed. I don't know what direction you're feeling (@PullJosh) for these two components, but it would obviously be great to have them be nice and compartmentalized for easy integration into both editors, and I think there's a lot of room for making something basically accessible/tailored to any Scratcher while not being a 1:1 port of scratch-paint or scratch-gui audio editing components — if you're leaning towards DIYing paint and audio editor interfaces.

This is just based on vibes, but I don't think there is a lot of reason to directly share technologies for the editor as a whole between Leopard and Patch. They have very different goals and would each probably benefit from having a focused team who gets to work with just their own code — not worrying about making sure they're figuring out the perfect prototypical interface to make something work for both Leopard and Patch, stepping on each other's feet, etc. I love the idea of a rather magical and highly-teacher-customizable editing experience, but that doesn't seem to be in the scope of what Leopard is focused on. These two editors also just have very different user audiences, so letting the editors be tailored on a very basic level to both seems fine.

IMO the far bigger challenge is user playtesting and interface design, not technical implementation, so unless the editors share a lot of technical overlap, better to have the editor technical teams work independently of each other. Web-based IDEs are a pretty mature area of software at this point so if those teams have differing opinions on which libraries to use / how to integrate them, better that they can figure the right way out for themselves. (There can still totally be sharing of knowledge and experience here! I just think the editor codebases should be separately written and not directly related to each other.)

Project Sharing/Community

Not a lot of room for us to comment on this, sorry! It totally depends on what Patch's and Leopard's goals both are. I do think there's a lot more room for technical overlap between the Patch and Leopard websites / website tooling, though! It seems like it could be super reasonable to share not just moderation but technical efforts here, if they have a significant amount of feature / functionality overlap.

[PullJosh]

@towerofnix Elliot can comment on this better than I can, but my understanding is that it would be best for Patch to handle all control flow of blocks. Essentially, Patch would own the triggers situation and would simply call things like your_sprite.move() when the Python move() function is called.

I'm wondering about your opinion on how this would affect Leopard. I honestly don't think it would require very many changes. You listed a few: exposing events like when a sprite is clicked, providing better tools for long-running blocks (maybe async versions in addition to their generator counterparts?), etc. I'm thinking these changes would be almost entirely positive for Leopard. You?

(Oh, also! When users write a Leopard project, the expectation is that each sprite they create will be a subclass of Sprite. That way, the instance can have its own methods representing its scripts. In the case of Patch, where everything would be puppeteered by Patch code anyway, do you think it would make sense to just make every sprite be an instance of the base Sprite class instead? With costumes/sounds/etc defined manually by having Patch set my_sprite.costumes = [...] rather than doing it in a constructor? I'm having a difficult time wrapping my head around this, but I think it would work?)

I don't know what direction you're feeling (@PullJosh) for [image and audio editors]

I'm not sure either. Part of me is tempted to just embed the Scratch costume editor directly, but another part of me is very much not. Working with Scratch's libraries seems like a bit of a pain. Either way, I personally think that image/audio editing is not Leopard's core competency and it seems like the perfect thing to outsource to an existing component that exists somewhere on npm.

IMO the far bigger challenge is user playtesting and interface design, not technical implementation, so unless the editors share a lot of technical overlap, better to have the editor technical teams work independently of each other.

The more I reflect on it, the more I'm inclined to agree. The editor design seems like the place where Leopard and Patch differ the most philosophically, and it seems like there are meaningful differences in the tech stack (Monaco vs CodeMirror, for example) that make sense for each project. This may be a place to share ideas more than code.

[ElliotRoe]

Scratch VM Challenges

I'd love to hear about what challenges Patch has run into when interfacing with scratch-vm, and how the team personally feels about the new(?) framework, with most execution functionality stripped out of scratch-vm and moved into Patch Worker. Does that feel like the right direction for Patch's runtime? Are there things you wish could be easier or nicer than they have been, or still are, with scratch-vm? I think sharing some of that would help identify how well Leopard could work as a library that helps Patch!

The main positive of the Scratch-vm is it allowed us to bootstrap a reasonable MVP relatively quickly, which it has succeeded in that use. However, most of our struggles developing with the Scratch-vm is because of it's age and bloat. We were able to slim down the code base. There was a huge amount of functionality that was ripped out when we forked the repo and made the Patch-vm (mostly execution & threading stuff). But, the remaining code base is left oddly disjointed, dense, and still old! It doesn't have any typing, it does not use any of the new async/await js syntax sugar for async code, and still includes much more functionality than we actually need. As we were working towards our MVP last summer, we couldn't justify the dev time that it would have taken to clean up some of the tech debt left behind. So, for example, there's a package (I believe minilog) that scratch-vm depends on that has not been updated for 7 years. We couldn't get working with Vite (there were also a number of other older packages that were causing issues) so we had to just go with using webpack instead, which, while not the biggest deal, characterizes a lot of our struggles. Additionally, It just costs a huge amount of time to get new devs onboarded to the codebase. In summary, a lot of the appeal of the Leopard VM would be to simply improve our dev experience with Patch.

Patch Execution

There is one final decision that needs to be made in terms of Patch's runtime is: how do we handle forever loops? Right now, we just convert this into a regular python while true loop. However, this destroys our performance if multiple loops are running at the same time. Leopard deals with this by making the forever loop just a trigger, and I think we might head a similar way ourselves (which makes Patch even more compatible with Leopard VM?).

...we have a well-defined interface for running primitive blocks which take time, e.g. play sound until done or wait-seconds. (I.e, does this use promises or generators, and can it work either way?)

Currently the Patch worker does not handle this, because of our goal to make the block -> function conversion exactly 1:1. We hide a lot of the syntax magic that does this right now, but from the worker POV, it's just awaiting a promise and then resumes execution of the python code afterward its resolution. Other than this, Patch handles all runtime execution.

...our interface for user input triggers is sufficient for Patch to make use of. (I.e, can Patch readily listen for events like "when this sprite clicked" and use that to activate its threads?)

I believe so, we're wired into something analogous right now through the Patch VM

...our behavior for not duplicating a still-active trigger (assuming that's present, see above) is either something Patch can understand and readily interface with, or just something that Patch Worker will handle all on its own.

I'm not sure what you are referring to specifically here. Could you link some code or an example? Most likely, Patch worker couldn't handle it though.

Supported Blocks

1. How much does Leopard still not support scratch-vm behavior? Would this have an impact on Patch?

In terms of support, from what I can tell, you guys are ahead of us (other than the stop all block). I don't think this would be a huge issue for us.

Editor

I am also inclined to agree that right now there are no compelling arguments to collaborate on editors. However, I agree that

... there's a lot more room for technical overlap between the Patch and Leopard websites / website tooling, though! It seems like it could be super reasonable to share not just moderation but technical efforts here, if they have a significant amount of feature / functionality overlap.

For example, it would be cool to have a single platform in which Leopard or Patch projects can be created. In terms of resources, I'd have to get a cost estimate from @PullJosh, but BX Coding would most likely be happy to cover hosting/deployment costs--especially if it's on AWS as we have a fair bit of credit on the platform.

Patch & Leopard Interoperability

I still can't let go how easy it was to integrate the pyscript library into Leopard. There has to be a way to easily to convert from Patch's simplified Python to a raw Python version that works out of the box within pyscript tags in Leopard.

[towerofnix]

Scratch VM challenges (@ElliotRoe)

Sorry to hear about the troubles there—and thank you for sharing that! It totally makes sense that the age and overall lack of "catching up with the times" would pose a lot of difficulties and inconveniences. We occasionally see signs of movement and progress on the Scratch dev team's part in bringing these things up to date, but they clearly have additional responsibilities. It'll only be done when it's done, and that's not now. Definitely makes sense that bringing it up to date yourselves just isn't in scope for Patch's efforts (it really is a big and slightly gruesome project).

Patch execution (@ElliotRoe)

Could you explain in some more detail why while True: poses an issue for your forever loops? Leopard uses almost exactly the same approach for its "forever" block:

while (true) {
  /* contents of the forever block go here */

  yield;
}

I'm especially curious why using one forever loop is OK, but multiple tanks your performance.

The main principle to understand with Scratch multithreading is that it's not really multithreaded: within one "tick", "step", or "cycle" of the VM, scripts always execute in a determinate order, and only one is literally running at a time. If you are using multiple real threads (i.e. multiple web workers), then the analogy is to only pass control to one thread at a time, and always observe all results of the thread's execution until it reaches a yield point. At that yield point, the worker is no longer doing anything, and waiting for itself to be started up again—this is an idle state on the CPU, i.e. awaiting a promise/event/socket message, and shouldn't affect performance more than any other await / idle worker.

(This is also why Scratch's interpreter is literally single-threaded: if you're just going to observe all the results of execution immediately after the thread is done, and certainly never have two threads actually running simultaneously, then there's no reason to internally represent threads with separate literal threads i.e. workers. Of course, Patch could have its own reasons for doing this, e.g. compartmentalized state management for Python instances. But real-world multithreading is never necessary just to represent the behavior of Scratch.)

...our behavior for not duplicating a still-active trigger (assuming that's present, see above) is either something Patch can understand and readily interface with, or just something that Patch Worker will handle all on its own.

I'm not sure what you are referring to specifically here. Could you link some code or an example? Most likely, Patch worker couldn't handle it though.

Yeah, that was a sort of obtuse description on our part! What I meant is that Scratch never runs two copies of the same script (under a specific hat block) at once. In general, if you have a script which runs "when space key pressed", then it will run all the way to completion, even if you try to "interrupt" or "duplicate" it by pressing the space bar while it's still running. It just ignores the second time you press the space bar.

This is something that whatever controls which hat blocks get started needs to be aware of and responsible for.

For example, if Leopard tells Patch "the user just pressed space, trigger all when space pressed hat blocks" then Patch is the one who needs to say, "OK, OK, but no, not the one that is already running. I'll start any any when-space-pressed hat blocks that are not running though."

On the other hand, if Patch has already explained to Leopard what scripts exist, and Leopard can ask Patch which of those scripts are running, then Leopard can give Patch a more specific instruction: "OK, OK, I see that some of your scripts are still running in response to the space key. So, since the user did just press space, please start these specific other scripts, which I know are when-space-key-pressed, and which I also see are not currently running."

The right approach depends on what level of responsibility Patch wants to take for project execution. Also, work is absolutely needed either way on Leopard's part, since we don't have 100% compatibility on this mark yet (for example in Scratch, "when key pressed" can't be interrupted, but "when this sprite clicked" can be interrupted, and we appear to treat them the same right now).

Leopard, the Library™ (@PullJosh)

I'm wondering about your opinion on how this would affect Leopard. I honestly don't think it would require very many changes. You listed a few: exposing events like when a sprite is clicked, providing better tools for long-running blocks (maybe async versions in addition to their generator counterparts?), etc. I'm thinking these changes would be almost entirely positive for Leopard. You?

Oh, totally. We love this direction for Leopard and having a specific project to make sure we are able to support and be ready for is a great motivator and really helps us find direction for what Leopard's interfaces should be like. And do we think those interfaces should exist at all!? Yes, absolutely!

IMO the project leopard has its greatest potential in serving not just as the target for sb-edit's toLeopard.ts, but as a general, easy to interface with, very friendly interface for dealing with the hard parts of emulating Scratch's behavior. My personal vibes are:

• Scratch is a kind of weird programming language. Leopard should give you, someone interfacing with it, tools to experience and hook into as much of Scratch's behavior "for free" as you want. This is the biggest draw for working with Leopard as a library.
• Different projects stride different strokes in just what counts as "the hard part".
  • For example, some projects don't want to take care of all the nitty gritty execution order details, and are happy to reflect exactly Scratch's behavior in this regard—they want Leopard to take care of it.
  • Some projects want to create a totally new execution environment and are happy to deal with each and every aspect of project execution. But they still want primitives like "set audio effect" and graphic effects and movement and they really don't want to worry about Scratch's weird-as-balogna implementation of atan, so please can they just get an awesome assortment of functions that explain and perform the high-level behavior of Scratch, please.
  • Some projects are somewhere in between. They want some features that Leopard provides and will take care of other parts, or just don't find those other parts applicable for their use case.
• I think Leopard should be good at two things:
  1. Providing clearly bounded "levels" or "systems" (or just "modules") of behavior, which allow choosing just how much or little you want to use from Leopard, and how much you want to take responsibility for. Particularly in terms of project execution, Leopard should have easy to use and flexible interfaces, so that you aren't getting "half the experience" or dealing with a bunch of crufty communication because you didn't use exactly the combination or bundle of behavior that we had in mind.
  2. Documenting and describing those levels/systems/modules effectively, and relating them explicitly to Scratch's functionality, so that people using Leopard are informed and have the knowledge to decide which parts of Leopard are suitable for their project (and which aren't). Because Leopard is a reflection of Scratch, a knowledge of how Scratch works is fundamentally necessary to make informed decisions, but we can provide that information in both a summary, high-level overview, and as more detailed explanations, so that you understand what Leopard will expect and what to consider when interfacing with it.
• I think Leopard can do a great job at being exactly what Patch is looking for, not by forming itself exactly to expectations or assumptions that Patch holds, but by being a practical and educational tool (or set of tools) that helps Patch and other projects like it mirror Scratch in just the ways that they want to.
  • I think this is something that currently no other project even comes close to, and a valuable niche for Leopard to fill! Forkphorus and Turbowarp are both neat runtime engines for Scratch but they are singly suited for their own purposes, not made to be an extendable or decomposable environment. Scratch VM is the canon and de facto implementation of Scratch, but it's headache-inducing to build on and to tear down; scratch-render and scratch-audio desire to be good modular components, but face the same lack of maintenance and only have a single real-world consumer (Scratch!). sb-edit is a library for processing Scratch blocks and projects (ostensibly) in some rather generic ways, but runtime behavior is totally outside of its scope.
  • Specifically, I think Leopard, the library as we've described it above, would be a great spiritual counterpart to sb-edit. The primary practical difference, on a personal level, is that we don't know much about making a good block editing interface (and don't have a real-world consumer to do interesting things with it, guiding its creation) — whereas with Leopard, we have a ton of experience observing and understanding Scratch, and would love to turn that into a practically and educationally consumable library. This already applies for parts of Leopard like its rendering system, and of course the gritty details about implementation compatibility for all the more subtly complex blocks. I think it absolutely should go for the low- to medium-level execution and sequencing of projects too, and that all our systems should get a good, hard look about exposing and documenting nice external interfaces.

Not to blow the air out of our own discussion, but I don't know how much effort is needed on these parts. Leopard's greatest focus has always been compatibility with Scratch. We're already almost all the way there, in that regard, but figuring out how to expose it in useful ways (and at differently useful levels of responsibility) may be challenging. We have some thoughts on practical ways to go about this, but in summary, we're pretty sure it's well approachable and a very worthwhile direction for Leopard itself, that it could be useful to Patch as long as Patch is ready to understand what Scratch's behavior is, what Leopard can provide, and incorporate the appropriate parts of Leopard into its own structure by combining the knowledge Leopard provides with the sense of scope of what Patch wants to (or needs to) be responsible for handling.

In the case of Patch, where everything would be puppeteered by Patch code anyway, do you think it would make sense to just make every sprite be an instance of the base Sprite class instead? With costumes/sounds/etc defined manually by having Patch set my_sprite.costumes = [...] rather than doing it in a constructor? I'm having a difficult time wrapping my head around this, but I think it would work?

Totally! There is no reason Patch can't interface with Leopard (as it's written today, even) in this regard. The only thing that would need special care, AFAIK, is the createClone function, which currently constructs a new instance of the same Sprite subclass. Since nothing would be specially in the constructor ("the same Sprite subclass" would just mean Sprite), we'd need to make sure that createClone itself is setting everything that the subclass constructor might currently be responsible for. Other than that, I don't see any issues with directly constructing the base Sprite class at all.

[PullJosh]

@towerofnix I like where your head is at.

IMO the project leopard has its greatest potential in serving not just as the target for sb-edit's toLeopard.ts, but as a general, easy to interface with, very friendly interface for dealing with the hard parts of emulating Scratch's behavior.

I really like this framing.

As far as breaking Leopard down into modules that represent the different parts of Scratch's behavior that library users might want to hook into, it seems like in all of our discussions things are clearly breaking down into two separate pieces:

1. The representation of the sprites and stage and all of their associated state (position, graphic effects, etc), costumes, sounds, and so on. As well as some whole-project state like pen lines, the "ask and wait" situation, etc. Basically, everything you would need to know in order to fully render the project to the stage.
2. The control-flow of executing Scratch scripts. Basically, everything that is currently done using generator functions in Leopard.

It feels like Leopard could be improved by somehow pulling these two pieces apart so that they can be used independently.