How to write good prompts - Matuschak

Prompt design is ... design. :: task

If prompt design is task design, how do you know the prompt works? :: That task changes you in some useful way.

What is "testing effect"? :: Learning produced by trying to remember knowledge.

What is the difference between a retrieval practice and a school exam? :: Retrieval practice tests to produce learning, not assess it.

Writing retrieval practice prompts is like ... written text. :: translating

How is writing spaced repetition prompts similar to language translation? :: They try to reconstruct the original idea.

Good retrieval practice prompts should be focused, precise, consistent, tractable, and effortful.

How do you ensure a retrieval practice prompt is focused? ... :: Make it about one detail.

When is a retrieval practice prompt consistent? :: When you always give the same answer.

If you fail to produce consistent answers during retrieval practice prompts, information you forget is ... :: inhibited

When is a retrieval practice prompt tractable? :: When you can always answer it correctly.

What can you add in a prompt to help ensure that your retrieval practice prompt is always tractable? :: A cue

What should a cue avoid to do in a retrieval practice prompt? :: Give the answer away.

The most important thing to remember to ensure that your prompts are focused, precise, consistent, tractable, and effortful. :: Make them tightly-scoped.

Two skills required to write effective retrieval practice prompts.

1. Determining what knowledge to reinforce.

?
Writing prompts to reinforce that knowledge.

Two skills required to write effective retrieval practice prompts.
1.
2. Writing questions to reinforce that knowledge.
?
Determining what knowledge to reinforce.

What prompt can be paired with a factual prompt to make them more meaningful? :: Explanation prompt

When a retrieval prompt is precise, it causes you to retrieve an exact information.

Why keep the items in a list prompt in the same order? :: You learn the list's visual shape, which helps with recall.

Instead of using clozes to remember a list, you can... :: Write an explanation prompt for each component of the list.

If a knowledge can't be connected to other memories, how can you still leverage elaborative encoding? :: Create a mnemonic device.

What is the emotional effect that mnemonic devices should produce? :: Vivid.

Give at least two ways of making a mnemonic device vivid. :: e.g., using visuals, meaningful personal experiences, humor, and disgust

Where should you put mnemonic devices in a retrieval writing prompt? :: In the answer (inside parentheses).

Why should you put mnemonic devices inside parentheses when using them in prompt answers? :: They're not part of the answer but simply help remember it.

In lieu of creating a mnemonic device, what can you do to leverage elaborative encoding in your retrieval practice prompts? :: Use images in the questions and answers.

What is the rule of thumb when determining the amount of prompts to write? :: Write more prompts than feels natural.

If you're already familiar about a certain subject, you should write fewer prompts about it.

Two ways of writing retrieval prompts for a procedure.
1.
2. Keywords
?
List

Two ways of writing retrieval prompts for a procedure.

1. List

?
Keywords

Steps for writing retrieval prompts for a procedure as a list.
1.
2. Cut words to emphasize keywords.
?
Use clozes.

How should we write list-based retrieval prompts for a procedure?

1. Use clozes.

?
Cut unnecessary words to emphasize keywords.

If procedures branch and become sufficiently complex, what should you include in your prompts? :: A flowchart.

Use a keyword-based approach in writing retrieval prompts about a procedure if this is important for you. :: Precision

Use a list-based approach in writing retrieval prompts about a procedure if all you need is :: an outline

What must you know to move between the steps of a procedure? :: Conditions/heuristics for when to stop doing one thing and start doing another.

To internalize a concept, write prompts that help you understand its components and relationships.

The five lenses in understanding concepts are:
1.
2. Similarities and differences
3. Parts and wholes
4. Causes and effects
5. Significance and implications
?
Attributes and tendencies

The five lenses in understanding concepts are:

1. Attributes and tendencies
3. Parts and wholes
4. Causes and effects
5. Significance and implications
   ?
   Similarities and differences

The five lenses in understanding concepts are:

1. Attributes and tendencies
2. Similarities and differences
4. Causes and effects
5. Significance and implications
   ?
   Parts and wholes

The five lenses in understanding concepts are:

1. Attributes and tendencies
2. Similarities and differences
3. Parts and wholes
5. Significance and implications
   ?
   Causes and effects

The five lenses in understanding concepts are:

1. Attributes and tendencies
2. Similarities and differences
3. Parts and wholes
4. Causes and effects

?
Significance and implications

To write prompts about a concept’s attributes and tendencies, what should you identify about its instances? :: What’s always / sometimes / never true.

To write conceptual prompts using the ‘parts and wholes’ lens, what kind of diagram is helpful to visualize? :: Venn diagram

When I'm thinking about how a concept matters to me personally, I'm considering its ... :: significance and implications.

A closed list is like an equation, while an open list is like a tag.

Why does an open list behave like a tag? :: It can be attached to many examples.

Three useful prompt types for open lists:
1.
2. About tag itself
3. Link tag to instances
?
Link instance to tag

Three useful prompt types for open lists:

1. Link instance to tag
3. Link tag to instances
   ?
   About tag itself

Three useful prompt types for open lists:

1. Link instance to tag
2. About tag itself

?
Link tag to instances

When writing retrieval prompts, how can you link a tag to its instances? :: Write a prompt that asks you to generate examples of that tag.

Appendage to an example-generating prompt that turns it into a creative prompt. :: (give an answer you haven’t given before)

How do creative prompts differ from usual retrieval practice prompts? :: They help you generate new answers, not recall.

What’s the common problem with writing only a ‘give an example’-type prompt for an open list? :: You’ll usually end up remembering only one or two examples.

What does a creative prompt reinforce? :: Whatever knowledge you consistently use when generating an answer.

Explain the generation effect. :: We remember information better when we generated it ourselves.

Define the Baader-Meinhof Phenomenon. :: New ideas are more salient.

How do you leverage the Baader-Meinhof Phenomenon when learning new ideas? :: Write retrieval prompts about new ideas to extend their salience.

What can you do to increase the chance of remembering a new idea when it is most useful? :: Write prompts about contexts where the idea can be most useful.

Why write prompts about your own ideas? :: To help you muse on them until they grow.

Keep new ideas salient until they become useful.

Why shouldn't you write comprehensive retrieval prompts about everything you read? :: You'll burn out and waste your time.

Why is it difficult to write good prompts on your first exposure to a new subject? :: You don't know a lot about the new subject.

Give one example of a thing you don't know about new ideas you're exposed to.
?

• Which detail is objectively important
• Which detail is difficult to remember
• The idea's essence
• The idea's relevance to you

Since you don't know a lot about a new idea, what approach should you take when writing prompts about it? :: An iterative approach.

Number of prompts to aim per article read to prevent writing retrieval prompts from feeling like a chore. :: 5–10

When engaging with a challenging or unfamiliar piece of text, when is the best time to write prompts? :: After reading, highlighting, and taking notes

When engaging with a challenging or unfamiliar piece of text, what details should you write prompts about initially? :: Basic details you can build on

When engaging with a book, when is the best time to write prompts? :: After reading, highlighting, or taking notes about a chapter or major section

When you're trying to thoroughly internalize a rich text, how many prompt-writing passes should you do? :: Multiple

What's the best guide in deciding which piece of knowledge to write a prompt about? :: Curiosity

If you notice a feeling of completionism when writing prompts about a text, remind yourself that :: you can always write more prompts later.

In retrieval practice, what is a "false positive"? :: Producing a correct answer without knowing the information you intend to know.

This happens when you memorize the shape of a question and its answer but not understand the knowledge involved. :: Pattern matching

How do you discourage pattern matching in retrieval practice? :: Write short and simple questions.

How could you rephrase a binary retrieval prompt? :: As more open-ended.

In retrieval practice, what is a "false negative"? :: You know the information you intend to know but fail to generate a correct answer.

How do you avoid false negatives in retrieval practice? :: Add enough context.

Retrieval practice prompts must unambiguously exclude alternative correct answers.

How do you identify when to revise a prompt? :: When you notice an internal "sigh."

Per Andy Matuschak, how does one revise a prompt in relation to others? :: Holistically.

Per Andy Matuschak, what is the most important thing to optimize in your spaced repetition practice? :: Your emotional connection to your sessions and their contents.

Per Andy Matuschak, what are the two things you can do when you don't care about a prompt anymore?
1.
2. Delete it.
?
Revise it to reflect your original motivation.

Per Andy Matuschak, what are the two things you can do when you don't care about a prompt anymore?

1. Revise it to reflect your original motivation.

?
Delete it.