How to implement Cursorless tutorials?

[pokey]

Per diataxis, we've decided to double down on investing in tutorials. However, it's not clear exactly what machinery to use

[pokey]

One possibility is to use VSCode walkthroughs

Pros

• No need to install anything else
• Could use for Cursorless keyboard

Cons

• You can’t display any dynamic content, so we couldn’t use the user’s custom spoken forms
• It also is forced to show up in one unified walkthrough area shared by all extensions, so could very easily get buried
• It just generally seems like a fairly limited and not super well supported api

[pokey]

We could use the talon_hud walkthrough machinery

Pros

• Fairly fully featured, eg supports:
  • skipping backward / forward steps
  • running action to establish necessary prereqs, eg putting hats in the right place
  • nice gui for putting explanatory text while clearly indicating command they need to say
  • automatically advancing to next step when command is issued
  • displays progress bar
• Nice floating GUI so that it can be positioned exactly where we want it, and user can drag it
• Probably the least amount of work to get something that is fully interactive
• We could leverage it for teaching users things other than Cursorless commands, eg the other useful commands that form the core “terse” grammar that we come up with later. Tho to be fair we could probably do that with other approaches as well

Cons

• Spoken form must match exactly, so requires us to finish implementing custom spoken forms in VSCode
• Couldn't use for Cursorless keyboard
• Requires user to install the talon_hud
• Not sure we can easily force Cursorless tutorial to appear on Cursorless install, but we can look into that

Questions

• Can we have a voice command that directly starts a particular tutorial?
• Can we get a tutorial to show on Cursorless install?
• Could we figure out a way for them not to have to install talon_hud? Would prob need to move large amounts of talon_hud machinery into community.
  • Or maybe could make it easier for users to install several user file sets, eg have the code snippet they copy install all three
  • Maybe punt and assume that Talon package manager will make that easier later

[pokey]

This would be my leaning, because it is far and away the best ratio of functionality to time investment. In fact, the time investment specific to this approach is fairly close to zero. So I'd be inclined to get a working version using the talon hud, and then if forcing users to install hud becomes too much of an issue, we should be able to leverage most of the stuff we've built when using a different approach

[merriam]

All the methods look like there needs to be a way to force a mark for Talon. That is, something like set_mark(line=2, word="brillig", mark='b'), so that any text and hints would follow the tutorial. Is there a way to do this?

[pokey]

Yeah that part is already working; see #2131. At this point we're just trying to figure out which UI infrastructure to use

[merriam]

Thank you. I'll look into it.

[pokey]

We could use a VSCode webview

Pros

• Complete flexibility
• No need to install anything
• Could work with keyboard
• Could display immediately on launch
• Could easily launch with voice command
• Could put in sidebar, editor, etc

Cons

• Lots of work, esp if we want things like showing progress, skipping steps, etc
• Doesn't float, so not quite as nice as talon_hud from display perspective

[pokey]

This is what we did in the end. Update your cursorless talon files then say "Cursorless tutorial" to try it out!

[pokey]

We could use a VSCode tree view

Pros

• No need to install anything
• Could use for Cursorless keyboard
• Could display immediately on launch
• Could easily launch with voice command
• Fairly easy to implement, esp if we just want basic functionality. Skipping steps / indicating progress is a bit more work, but still fairly easy

Cons

• Steps are forced to fit on a single line, ie no wrapping
• Forced to be off to the side

[pokey]

We could implement it using just a bunch of notification dialogues in VSCode, but I think they disappear pretty quickly, and it's not obvious how to make skipping steps work, etc

[josharian]

We could implement it similar to the cursorless katas, shipped with the extension (like the cheatsheet), with special spoken forms to open said katas. So:

• the website would say: "With VSCode open, say "cursorless tutorial start"
• the tutorial start page would explain something small, and then instruct them to say "cursorless tutorial next"

In addition to (or instead of?) "cursorless tutorial start/next/previous", we could also support jumping to a specific page, like "cursorless tutorial hats".

[pokey]

yeah I think all those voice commands make sense regardless of which UI pattern we go with

wrt the specific interface you're suggesting, are you proposing to have instruction text in one editor, and the text you're operating on in opposing split? And then we just replace the instruction editor with the next step each time they advance? I guess that's simple enough. We could even indicate progress with just a simple (7/10) at the top of the editor

[josharian]

I was originally thinking of having the user operate in a single file, as in my example videos. Maybe instructions above a dividing line, playground below the line? And from there, yes, replacing the instruction editor as they advance / hop around. But I don't feel strongly about any of that. The key here is that this approach is (a) very little work and (b) probably has good flow.

[pokey]

We might be able to repurpose the VSCode quick input UI element. Definitely a hack, but has a way of indicating progress, and floats at the top. Tho It will be displaced if they open another quickpick and then we'd need to figure out how to reinstate it

[merriam]

We could implement it as straight VSCode with an Audio (or video)

The sound from video can directly control CursorLess. This means a properly installed CursorLess might have a video saying, "Next we want to get rid of the word Hello. We can use the command Change or the command Chuck. Change moves the cursor, so let's do that. Wake up. Focus Code. Change Blue Harp. Say Goodbye. Go to sleep. Notice that...."

Pros:

• No need to install anything if starting with empty files
• No need for special authoring tools

Cons:

• Does not work with headphones

[merriam]

Oh, also still unclear how to force marks from video to apply to code.

[pokey]

Neat idea. One big drawback is that it doesn't support user customizations. Eg if they've changed their alphabet or Cursorless commands it will break

Also, I worry about accuracy being good enough

[merriam]

OK, I'm taking a quick pass at this associated with a cheatsheet that incorporates tutorials.

Yes, there is no way around making a tutorial if the configuration is no longer compatible. That is, if "ill" is set to the new word for "i", but "sit" still works, the tutorial will work. If you make an incompatible configuration, then a tutorial should not work.

I'm still not understanding how I could fix the marks in a VSCode snippet. That is, I want a way to make a five line demonstration snippet, such that printf("Hello, world\n"); will have consistent marks for printf, Hello, and world. Take way, I can say "take edge". You did point to a large pull request, but I would like a better hint.

[pokey]

Sorry, not sure I'm following. Fwiw I'm actively working on the approach proposed in #2220 (comment)

[merriam]

More is probably better. I've been putting together a new cheatsheet, which I grant is oriented towards my issues. Two influences on style are the ancient RGruet Python sheet (http://rgruet.free.fr/PQR27/PQR2.7.html) for compactness and easily browsable by eye status and the Explanix sheet (https://zaynetro.com/explainix) for its easy detail pop down. I'm playing with it as one of several projects. One aspect is having very short tutorial segments showing how to use specific commands.

The technical issue I had was reproducibility. I would love to have a "Voice button" that would start with "Focus Code", bring up a snippet with known anchors, and then run the mini-tutorial live on a system using voice from the speaker. This would leave the system up in a disposable sample, encouraging a user to play with that command before moving on.

In the meantime, I'll try to put in some documentation updates that unobjectionable.

[pokey]

huh interesting. would be cool to discuss in one of our discord meet-ups if you get the chance! come find us in cursorless channel of talon slack workspace for more info

[merriam]

Ok. Care to just post the discord link? Or do you need me to install slack to read a message?

[pokey]

No worries here's a discord link, and here's the calendar (times in UTC). Sorry I'm just slightly paranoid about posting Discord links; I'm just sure the bots are waiting to move in 🤖⚔️. This one expires after 7 days / 10 joins so should do the trick while assuaging my paranoia. But do lmk if it expires before you get a chance to use it and I'll post another np. Looking forward to connecting!

[pokey]

Btw explainix is super cool; thanks for the pointer. Would love to explore the possibility of applying similar ideas to Cursorless