Deep Learning

Someone asked the following question on Quora:


Learning: Do grad school students remember everything they were taught in college all the time?

Note – Question specific to Physics. Answers from all other disciplines welcome.Do they remember the derivation to something they learn as a freshman? If someone comes up and asks them a question relating to any topic in Physics would they always know the answer? I asked Sigurd Wenner the same question a long time ago and he replied – “No, you don’t need to remember all human knowledge on physics. If I need the formula for the Doppler effect, I look it up on Wikipedia. If you can derive it by yourself, it’s great, because then you won’t even need to look it up, but that’s usually faster. Remembering formulas is only good for one thing: passing exams. If I were to take an exam in a course I had 5 years ago, I’d probably do bad. But if I were asked to tell about some phenomenon, I would have a lot of relevant things to say, and that’s what matters. Understanding stays with you. Things that are for remembering, you just look up. For example, I don’t know the integrals of all the trigonometric functions, but I know how to find out.” Somehow I feel uncomfortable about it. Shannon Sequeira asked me how to derive the equation of motion of a simple pendulum earlier today and it took me 15 minutes to come up with the answer (with help from Wikipedia and Resnick-Halliday). Shamefully I just completed a course in Oscillations and Waves 6 weeks ago and I scored 100% in theory. I feel immensely ashamed of myself. Am I not learning Physics the way it’s meant to be?


And Mark Eichenlaub gave a beautiful answer:


You arrive at lecture and sit perched on the edge of your seat, notebook open to a clean page and freshly-sharpened pencil in hand. You follow every word the professor says. Well, maybe you zone out a few times in the middle, but who doesn’t? Besides, you’re copying everything down and can review it later.

That weekend, you diligently read the textbook. Maybe you skip a few parts since it’s a busy week, but you definitely study the chapter summary and read all the examples. You do the homework problems, even starting three days early. When you’re stuck, you go to office hours and ask the TA for help until they show you how to do it.

Before the exam, you study your notes and the published homework solutions. You try the practice exam, and it seems the pieces are finally falling into place. You can solve most of the problems and remember most of the formulas and derivations! At last you take the final, referencing the single allowed sheet of notes you prepared at length the night before. You get almost every question right, or at least partial credit, and take home a well-deserved A.

Three months later, you can hardly remember what the class was all about. What’s going on? Why did you forget so much? Are you the only one? Should you have memorized more and worked even harder?

The answer is no. A student who memorizes the entire physics curriculum is no more a physicist than one who memorizes the dictionary is a writer. Studying physics is about building skills, specifically the skills of modeling novel situations and solving difficult problems. The results in your textbook are just the raw material. You’re a builder. Don’t spend all your time collecting more materials. Collect a few, then build things. Here’s how.

The Cathedral and the Stones

While delivering his famous set of freshman lectures on physics, Richard Feynman held a few special review sessions. In the first of these, he discussed the problem of trying to memorize all the physics you’ve learned:

It will not do to memorize the formulas, and to say to yourself, “I know all the formulas; all I gotta do is figure out how to put ’em in the problem!”

Now, you may succeed with this for a while, and the more you work on memorizing the formulas, the longer you’ll go on with this method – but it doesn’t work in the end.

You might say, “I’m not gonna believe him, because I’ve always been successful: that’s the way I’ve always done it; I’m always gonna do it that way.”

You are not always going to do it that way: you’re going to flunk – not this year, not next year, but eventually, when you get your job, or something – you’re going to lose along the line somewhere, because physics is an enormously extended thing: there are millions of formulas! It’s impossible to remember all the formulas – it’s impossible!

And the great thing that you’re ignoring, the powerful machine that you’re not using, is this: suppose Figure 1 – 19 is a map of all the physics formulas, all the relations in physics. (It should have more than two dimensions, but let’s suppose it’s like that.)

Now, suppose that something happened to your mind, that somehow all the material in some region was erased, and there was a little spot of missing goo in there. The relations of nature are so nice that it is possible, by logic, to “triangulate” from what is known to what’s in the hole. (See Fig. 1-20.)

And you can re-create the things that you’ve forgotten perpetually – if you don’t forget too much, and if you know enough. In other words, there comes a time – which you haven’t quite got to, yet – where you’ll know so many things that as you forget them, you can reconstruct them from the pieces that you can still remember. It is therefore of first-rate importance that you know how to “triangulate” – that is, to know how to figure something out from what you already know. It is absolutely necessary. You might say, “Ah, I don’t care; I’m a goodmemorizer! In fact, I took a course in memory!”

That still doesn’t work! Because the real utility of physicists – both to discover new laws of nature, and to develop new things in industry, and so on – is not to talk about what’s already known, but to do something new – and so they triangulate out from the known things: they make a “triangulation” that no one has ever made before. (See Fig. 1-21.)

In order to learn how to do that, you’ve got to forget the memorizing of formulas, and to try to learn to understand the interrelationships of nature. That’s very much more difficult at the beginning, but it’s the only successful way.

Feynman’s advice is a common theme in learning. Beginners want to memorize the details, while experts want to communicate a gestalt.

Foreign language students talk about how many words they’ve memorized, but teachers see this as the most trivial component of fluency. Novice musicians try to get the notes and rhythms right, while experts want to find their own interpretation of the piece’s aesthetic. Math students want to memorize theorems while mathematicians seek a way of thinking instead. History students see lists of dates and facts while professors see personality, context, and narrative. In each case, the beginner is too overwhelmed by details to see the whole. They look at a cathedral and see a pile of 100,000 stones.

One particularly clear description of the difference between the experts’ and beginners’ minds comes from George Miller’s 1956 study “The magical number seven, plus or minus two.” Miller presented chess boards to both master-level chess players and to novices. He found that the masters could memorize an entire board in just five seconds, whereas the novices were hopeless, getting just a few pieces. However, this was only true when the participants were memorizing positions from real chess games. When Miller instead scattered the pieces at random, he found the masters’ advantage disappeared. They, like the novices, could only remember a small portion of what they’d seen.

The reason is that master-level chess players have “chunked” chess information. They no longer have to remember where each pawn is; they can instead remember where the weak point in the structure lies. Once they know that, the rest is inevitable and easily reconstructed.

I played some chess in high school, never making it to a high level. At a tournament, I met a master who told me about how every square on the chess board was meaningful to him. Whereas, when writing down my move, I would have to count the rows and columns to figure out where I had put my knight (“A-B-C, 1-2-3-4, knight to C4”) he would know instantaneously because the target square felt like C4, with all the attendant chess knowledge about control of the center or protection of the king that a knight on C4 entails.

To see this same principle working in yourself right now, memorize the following. You have two seconds:



Easy, right? Well, it would be if you were literate in Chinese. (And assuming Google Translate did a good job with the phrase, “first the peanut butter, then the jelly”.)

You can remember the equivalent English phrase no problem, but probably don’t remember the Chinese characters at all (unless you know Chinese, of course). This is because you automatically process English to an extreme level. Your brain transforms the various loops and lines and spaces displayed on your screen into letters, then words, then a familiar sandwich-related maxim, all without any effort. It’s only this highest-level abstraction that you remember. Using it, you could reproduce the detail of the phrase “first the peanut butter, then the jelly” fairly accurately, but you would likely forget something like whether I capitalized the first letter or whether the font had serifs.

Remembering an equally-long list of randomly-chosen English words would be harder, a random list of letters harder still, and the seemingly-random characters of Chinese almost impossible without great effort. At each step, we lose more and more ability to abstract the raw data with our installed cognitive firmware, and this makes it harder and harder to extract meaning.

That is why you have such a hard time memorizing equations and derivations from your physics classes. They aren’t yet meaningful to you. They don’t fit into a grand framework you’ve constructed. So after you turn in the final, they all start slipping away.

Don’t worry. Those details will become more memorable with time. In tutoring beginning students, I used to be surprised at how bad their memories were. We would work a problem in basic physics over the course of 20 minutes. The next time we met, I’d ask them about it as review. Personally, I could remember what the problem was, what the answer was, how to solve it, and even details such as the minor mistakes the student made along the way and the similar problems to which we’d compared it last week. Often, I found that the student remembered none of this – not even what the problem was asking! What had happened was, while I had been thinking about how this problem fit into their understanding of physics and wondering what their mistakes told me about which concepts they were still shaky on, they had been stressed out by what the sine of thirty degrees is and the difference between “centrifugal” and “centripetal”.

Imagine an athlete trying to play soccer, but just yesterday they learned about things like “running” and “kicking”. They’d be so distracted by making sure they moved their legs in the right order that they’d have no concept of making a feint, much less things like how the movement pattern of their midfielder was opening a hole in the opponent’s defense. The result is that the player does poorly and the coach gets frustrated.

Much of a technical education works this way. You are trying to understand continuum mechanics when Newton’s Laws are still not cemented in your mind, or quantum mechanics when you still haven’t grasped linear algebra. Inevitably, you’ll need to learn subjects more than once – the first time to grapple with the details, the second to see through to what’s going on beyond.

Once you start to see the big picture, you’ll find the details become meaningful and you’ll manipulate and remember them more easily. Randall Knight’s Five Easy Lessons describes research on expert vs. novice problem solvers. Both groups were given the same physics problems and asked to narrate their thoughts aloud in stream-of-consciousness while they solved them (or failed to do so). Knight cites the following summary from Reif and Heller (1982)

Observations by Larkin and Reif and ourselves indicate that experts rapidly redescribe the problems presented to them, often use qualitative arguments to plan solutions before elaborating them in greater mathematical detail, and make many decisions by first exploring their consequences. Furthermore, the underlying knowledge of such experts appears to be tightly structured in hierarchical fashion.

By contrast, novice students commonly encounter difficulties because they fail to describe problems adequately. They usually do little prior planning or qualitative description. Instead of proceeding by successive refinements, they try to assemble solutions by stringing together miscellaneous mathematical formulas from their repertoire. Furthermore, their underlying knowledge consists largely of a loosely connected collection of such formulas.

Experts see the cathedral first, then the stones. Novices grab desperately at every stone in sight and hope one of them is worth at least partial credit.

In another experiment, subjects were given a bunch of physics problems and asked to invent categories for the problems, then put the problems in whatever category they belonged. Knight writes:

Experts sort the problems into relatively few categories, such as “Problems that can be solved by using Newton’s second law” or “Problems that can be solved using conservation of energy.” Novices, on the other hand, make a much larger number of categories, such as “inclined plane problems” and “pulley problems” and “collision problems.” That is, novices see primarily surface features of a problem, not the underlying physical principles.


The “Aha!” Feeling

It is clear that your job as a student is to slowly build up the mental structures that experts have. As you do, details will get easier. Eventually, many details will become effortless. But how do you get there?

In the Mathoverflow question I linked about memorizing theorems, Timothy Gowers wrote:

“As far as possible, you should turn yourself into the kind of person who does not have to remember the theorem in question. To get to that stage, the best way I know is simply to attempt to prove the theorem yourself. If you’ve tried sufficiently hard at that and got stuck, then have a quick look at the proof — just enough to find out what the point is that you are missing. That should give you an Aha! feeling that will make the step far easier to remember in the future than if you had just passively read it.”

Feynman approached the same question:

“The problem of how to deduce new things from old, and how to solve problems, is really very difficult to teach, and I don’t really know how to do it. I don’t know how to tell you something that will transform you from a person who can’tanalyze new situations or solve problems, to a person who can. In the case of the mathematics, I can transform you from somebody who can’t differentiate to somebody who can, by giving you all the rules. But in the case of the physics, I can’t transform you from somebody who can’t to somebody who can, so I don’t know what to do.

Because I intuitively understand what’s going on physically, I find it difficult to communicate: I can only do it by showing you examples. Therefore, the rest of this lecture, as well as the next one, will consist of doing a whole lot of little examples – of applications, of phenomena in the physical world or in the industrial world, of applications of physics in different places – to show you how what you already know will permit you to understand or to analyze what’s going on. Only from examples will you be able to catch on.”

This sounds horribly inefficient to me. Feynman and Gowers both reached the highest level of achievement in their domains, and both are renowned as superb communicators. Despite this, neither has any better advice than “do it a lot and eventually expertise will just sort of happen.” Mathematicians and physicists talk about the qualities of “mathematical maturity” and “physical insight”. They’re essential to moving past the most basic level, but it seems that no one knows quite where they come from.


Circular Reasoning

There are certainly attempts to be more systematic than Feynman or Gowers, but before we get to that, let’s take a case study. I recall that as a college freshman, I knew that the formula for the acceleration of a ball orbiting in a circle was a = v^2/r. I wanted to know why, so I drew a picture:

I imagined a small ball starting on the right side of the circle, heading upwards where the blue velocity vector v_1 is drawn. The ball moves around the circle, goes counter-clockwise over the top and then heads downwards on the left hand side, where the red velocity v_2 is. The ball’s velocity changed, which means it accelerated. The acceleration is

a = \frac{\Delta v}{\Delta t}

\Delta v is clearly 2v, and \Delta t is the time it takes to go half way around the circle, which is \frac{\text{distance}}{\text{speed}} = \frac{\pi r}{v}. Hence, the acceleration is

a = \frac{2v}{\pi r/v} = \frac{2 v^2}{\pi r} \approx 0.64 \frac{v^2}{r}

This isn’t quite right. The answer is supposed to be v^2/r. Somehow there is an extra factor of 2/\pi floating around.

If you already understand calculus, this is a silly and obvious mistake. But for me it took quite some time – weeks, I think – until I understood that I had found the average acceleration, but the formula I was trying to derive was the instantaneous acceleration.

The way I broke out of this mental rut was to think about the case where the ball has gone one quarter of the way around, like this:

Then the same approach gives

 a = \frac{\Delta v}{\Delta t} = \frac{2\sqrt{2}v^2}{\pi r} \approx 0.90 \frac{v^2}{r},

which is closer to the right value. If you try it when the ball goes 1/8 the way around, you get

a = \frac{4 \sqrt{2 - \sqrt{2}}v^2}{\pi r} \approx 0.97 \frac{v^2}{r}

and you’re getting the idea that what you have to do is take the limit as the ball goes an infinitesimal fraction of the way around. (By the way, if I had been clever, maybe I’d have discovered Viète’s formula this way, or something like it. I only recognized this now because I remembered encountering Viete’s formula. So memory certainly has its place in allowing you to make connections. It’s just not as central as beginners typically believe.)

How do you do that “infinitesimal fraction of the way around” thing? Well, if the ball travels an angle \theta around the circle, we can draw the before and after velocities as


 \Delta v = 2 \sin (\theta/2) v

which in the limit \theta \to 0 becomes

 \Delta v = \theta v


 a = \frac{\Delta v}{\Delta t} = \frac{\theta v}{\theta r/v} = \frac{v^2}{r}

But all of this took a long time to come together in my mind, assembling gradually, but in discrete chunks with each small epiphany. As I walk through it now, I can see there are many concepts involved, and in fact if you’re a beginning student it’s likely that the argument isn’t clear because I skipped some steps.

The main idea in that argument is calculus – we’re looking at an infinitesimal displacement of the ball. To understand the entire argument, though, we also need to do a fair amount of geometry, develop the idea of sliding velocity vectors around in space so they originate at the same point, introduce the concept of an arbitrary angle of rotation \theta, find the time it takes to rotate by that angle for a given r and v, use the small-angle approximation of the sine function, and maybe a couple other things I’m not seeing.

That’s a lot of mental exercise. It’s no wonder that working all this out for yourself is both harder and more effective than reading it in the book. Just reading it, you’ll skip over or fail to appreciate how much goes into the derivation. The next time you try to understand something, you want those previously-mastered ideas about geometry and calculus already there in your mind, ready to be called up. They won’t be if you let a book do all the work.

Today, I can solve this problem in other ways. For example, I could write down the rectangular coordinates and differentiate, describe the motion in the complex plane as r e^{i\omega t} and differentiate that, or transform to a rotating reference frame and note the centrifugal force on the stationary ball and conclude it must be accelerating in an inertial frame. A cute one is to write down the position and velocity vectors by intuition, and notice that going from position to velocity you rotate 90 degrees and multiply the length by v/r. To go from velocity to acceleration is mathematically identical, so rotating another 90 degrees and multiply by v/r again we obtain the answer.

I can argue from dimensional analysis that the only way to get something with units of acceleration is v^2/r, or heuristically point out that if you increase the velocity, the velocity vectors get bigger, but we also go from one to the next in less time, so the acceleration ought to scale with v^2, etc.

I also see aspects of the problem that I didn’t back then, such as that this isn’t really a physics problem. There are no physical laws involved. It would become a physics problem if we included that the ball is circling due to gravitational forces and used Newton’s gravitational law, for example, but as it stands this problem is just a little math.

So yes, I can easily memorize this result and provide a derivation for it. I can do that for most of the undergrad physics curriculum, including the pendulum and Doppler formulas you mentioned, and I think I could ace, or at least beat the class average, on the final in any undergraduate physics course at my university without extra preparation. But I can do that because I built up a general understanding of physics, not because I remember huge lists of equations and techniques.


How to Chunk It

I can do these things now because of years’ of accumulated experience. Somehow, my mind built chunks for thinking about elementary physics the same way chess players do for chess. I’ve taught classes, worked advanced problems, listened to people, discussed with people, tutored, written about physics on the internet, etc. It’s a hodgepodge of activities and approaches, and there’s no way for me to tease from my own experience what was most important to the learning process. Fortunately, people from various fields have made contributions to understanding how we create the cognitive machinery of expertise. Here is a quick hit list.

George Pólya’s How to Solve It examines the problem-solving process as a series of stages, and suggests the student ask themselves specific questions like, “Is it clear that there enough information to solve the problem?”

Scott H Young, Cal Newport, and many others give specific advice on study skills: how to take notes, how to diagram out the connections between ideas, how to test your knowledge, how to fit what you’re learning into the larger scheme of things, etc.

When you do need to memorize things, spaced repetition software like Anki takes an algorithmic, research-backed approach to helping you remember facts with the minimum of time and effort.

K. Anders Ericsson has tried to find the key factors that make some forms of practice better than others – things like getting feedback as you go and having clear goals. He refined these into the concept of Deliberate Practice. He also believes there is no shortcut. Even if you practice effectively, it usually takes around 10,000 hours of hard work to reach the highest levels in complex fields like physics or music.

Chunking and assigning meaning are your mind’s ways of dealing with the information overload of the minutiae that inevitably pop up in any field. Another approach, though, is to try to expand your mind’s ability to handle those minutiae. If you can push your “magical number” from seven to ten, you’ll be able to remember and understand more of your physics work because it takes a bit longer to fill your cognitive buffer. Dual N-Back exercises are the most popular method of working on this. Nootropic drugs may also provide benefits to some people. Low-hanging fruit first, though. If you aren’t sleeping 8-9 hours a day, getting a few hours of exercise a week, and eating healthy food for most meals, you’re probably giving up some of your mind’s potential power already. (There is individual variation, though.)

Howard Gardner is one champion of the idea of multiple intelligences, or different learning types. When working on electric fields, for example, Gardner might advise you to study Maxwell’s equations, draw pictures of vector fields and intuit their curls, get up and use your body, pointing your arms around to indicate electric field vectors, write or speak about what you’re studying, learn with a friend or tutor, or maybe even create musical mnemonics to help you study, depending on where your personal strengths lie. Certainly, all students should build facility with drawing sketches, plotting functions, manipulating equations, visualizing dynamics, and writing and speaking about the material.

Psychologist Carol Dweck’s research studies the effect of your attitude towards learning on how much you learn, finding, for example, that children praised for their hard work are likely to press on further and learn more when given tough problems, whereas children praised for their intelligence are more likely to give up.

Productivity guru David Allen helps people organize their lives and defeat procrastination with specific techniques, such as dividing complicated tasks into small, specific “next actions” and deciding when to do them, then organizing them in a planner system.

Mihály Csíkszentmihályi believes that people operate best in a state of “flow”, where they are so focused on the task they find it enjoyable and engrossing to the point they’re innately motivated to continue. He emphasizes, for example, that the task needs to be the right level of difficulty – not too hard and not too easy – to find the flow state. (Some people think this state doesn’t jibe with deliberate practice; others contend it’s possible to achieve both simultaneously.)

Taken together, this yields enough practical advice to chew on for months or years. To summarize, when you are learning something new:

  • Try to figure it out for yourself
  • If you get stuck, take a peek at your textbook to get the main idea
  • Teach the idea to someone else
  • Once you’ve learned something, repeat the entire reasoning behind it for yourself, working through each detail
  • Ask yourself Pólya’s questions when you’re stuck
  • Use Young and Newport’s techniques to map out the ideas of your class and relate them to your prior knowledge
  • Make Anki decks and review them a few minutes a day to retain what you’ve learned
  • Make sure your study sessions include all the principles of deliberate practice, especially feedback, challenge, and attention
  • Build an image of yourself as someone motivated by learning and proud of having worked hard and effectively rather than as someone proud of being smart or renowned.
  • Find a organizational system that lets you handle all the details of life smoothly and efficiently.
  • Search for the flow state, notice when you enter it, and put yourself in position to find flow more and more often.
  • Work on different subjects, reviewing both advanced and basic material. They will eventually all form together in your mind, and you’re likely to have to take at least two passes at any subject before you understand it well.
  • Take care of your physical health.

This list does not include reading every page of the textbook or solving every problem at the end of the chapter. Those things aren’t necessarily bad, but they can easily become rote. Building the material up for yourself while dipping into reference materials for hints is likely to be more effective and more engaging, once you learn to do it. It is a slow, difficult process. It can be frustrating, sitting there wracking your brain and feeling incredibly stupid for not understanding something you know you’re supposed to have down. And strangely, once you have it figured out, it will probably seem completely obvious! That’s your reward. Once the thing is obvious, you’ve chunked it, and you can move on. (Though you still need to review with spaced repetition.) This is the opposite of the usual pattern of sitting in lectures and feeling you understand everything quite clearly, only to find it all evaporated the next day, or acing a final only to find your knowledge is all gone the next month.

That, I believe, summarizes the practical knowledge and advice about the learning process. Memorizing equations and derivations is difficult and ineffective because they are just the details. You can only handle a few details before your mind gets swamped. To cope, train yourself to the point where you process equations and physical reasoning automatically. This will free your conscious effort up to take in the big picture and see what the subject is all about.


It Just Gets In The Way, You See

Somehow, I’ve developed a “this is calculus” instinct, so that if I see the problem about acceleration in circular motion, or any other problem about rates of change, I know that it’s talking about a limit of some kind. Where does this instinct exist in my brain? What form does it take? How does it get called up at the right time?

George Lakoff believes that almost everything we understand is via metaphor. Any sort of abstract concept is understood by linking it to concrete concepts we’ve previously understood. For example, in Where Mathematics Comes From, Lakoff and coauthor Rafael Nuñez argue that we think of the mathematical concept of a “set” as a sort of box or container with things stacked in it. We reason about sets using our intuition about boxes, then later go back and support our conclusions with the technical details. Learning to reason about sets, then, is learning to think about the box metaphor and translate it back and forth into the formal language of axioms and theorems. This seems to fit with the introspective reports of many mathematicians, who say they build intuitive or visual models of their mathematics when finding results, then add in the deltas and epsilons at the end.

This may be why we so often see beginning students asking things like, “but what is the electron, really?” If they were told it is just a tiny little ball, that would work, because it’s a very easy metaphor. But instead, they’re told it’s not a ball, not a particle, not a wave, not spinning even though it has spin, etc. In fact, they’re told to dismiss all prior concepts entirely! This is something Lakoff believes is simply impossible. No wonder students are bobbing in an ocean of confused thought bubbles, with nothing but mixed metaphors to grasp at until the last straw evaporates, across the board.

Linguists like Steven Pinker believe that the language we use tells us how our mind works. Physicists certainly do have a specialized lexicon, and the ability to use it correctly correlates pretty well to general physics intuition, in my experience. In his review of Pinker’s The Stuff of Thought, Douglas Hofstadter summarizes:

“Pinker shows, for example, how subtle features of English verbs reveal hidden operations of the human mind. Consider such contrasting sentences as “The farmer loaded hay into the wagon” and “The farmer loaded the wagon with hay.” In this pair, the verb “load” has two different kinds of objects: the stuff that gets moved and the place it goes. Also, in the first sentence, the destination is the object of one preposition; in the second, the stuff is the object of another. Pinker sees these “alternations” as constituting a “microclass” of verbs acting this way, such as “spray” (“spray water on the roses” versus “spray the roses with water”). Where does this observation lead him? To the idea that we sometimes frame events in terms of motion in physical space (moving hay; moving water) and sometimes in terms of motion in state-space (wagon becoming full; roses becoming wet).”

Moreover, there are verbs that refuse such alternations: for instance, “pour.” We can say “I poured water into the glass” but not “I poured the glass with water.” What accounts for this curious difference between “load” and “pour”? Pinker claims that pouring merely lets a liquid move under gravity’s influence, whereas loading is motion determined by the human agent. “Pour” and “load” thus belong to different microclasses, and these microclasses reveal how we construe events. “[W]e have discovered a new layer of concepts that the mind uses to organize mundane experience: concepts about substance, space, time, and force,” Pinker writes. ” . . . [S]ome philosophers consider [these concepts] to be the very scaffolding that organizes mental life. . . . But we’ve stumbled upon these great categories of cognition . . . by trying to make sense of a small phenomenon in language acquisition.”

If correct, then in order to think about physics the way an expert does, we should learn to speak the way experts do. If we try to solve physics problems using the words “load” and “pour”, we may be carrying around a bunch of distracting anthropocentric baggage. If we don’t recognize that, we’ll get stuck, saying the problem “doesn’t make sense”, when really it’s our linguistically-instilled expectations that are wrong. To combat this, it may be just as helpful to gain facility with the language of physics as with its equations.

Five Easy Lessons provides a clear example of such difficulties: the case study of “force”. As I type this, my laptop is sitting on a desk which exerts an upward force on it. Few beginning students believe this is really a force, even after they’ve been browbeaten into drawing arrows for the “normal force” on exam diagrams.

The problem is in the way we use “force”:

“The robber forced the door open.”
“Your apology sounded forced.”
“…the force of the explosion…”
“…the force of righteousness…”
“I’m being forced to take physics even though I’ll never use it.”

Literally or figuratively, we think of “force” as implying not only motion, but intent or purpose, and also control. Force is for people pushing on things, or maybe for cars and projectiles. These things are using energy and will run down if left alone. But the desk under my laptop? It’s just sitting there, totally passive. How could it be “exerting a force” when it doesn’t even get tired? Needing some sort of rationalization for why the laptop doesn’t fall, beginners say that it’s not that the desk exerts a force on the laptop, the desk just provides something for the laptop to sit on. Or if something falls on the desk, the desk didn’t exert a force to stop it. It just got in the way is all. Why doesn’t the professor understand this obvious difference? A desk exerting a force? Come on…

Five Easy Lessons describes how students only overcome this difficulty after seeing a classroom demonstration where, using a laser pointer and a mirror laid on the desk top, the professor demonstrates how when a heavy cinder block is laid on the desk, the surface responds by bending out of its natural shape, exerting force on the cinder block like a compressed spring would.

You may need to find many such visualizations before you can reconcile your colloquial use of words with their use in physics. But this might also be dangerous, because although finding a way to make physics obey your idea about what a word means works decently in this case, in other instances it’s your expectations for the word that ought to change. (Relativity, with words like “contraction”, “slowing down”, etc. is a good example.)

Mythologist Joseph Campbell believes that we understand the world primarily through story. Perhaps we understand derivations, experimental evidence, and the logic behind physical conclusions as a sort of story, and it’s in building this story that our cognitive chunks are formed.


Mind The Neural Gap Junctions

You are the pattern of neural activity in your brain. When a part of you changes, building a new memory, installing a new habit, or constructing a tool to approach a class of problems, that change must be reflected somewhere in your brain.

Lesswrong user kalla724 describes this process in “Attention control is critical for changing/increasing/altering motivation

First thing to keep in mind is the plasticity of cortical maps. In essence, particular functional areas of our brain can expand or shrink based on how often (and how intensely) they are used. A small amount of this growth is physical, as new axons grow, expanding the white matter; most of it happens by repurposing any less-used circuitry in the vicinity of the active area. For example, our sense of sight is processed by our visual cortex, which turns signals from our eyes into lines, shapes, colors and movement. In blind people, however, this part of the brain becomes invaded by other senses, and begins to process sensations like touch and hearing, such that they become significantly more sensitive than in sighted people. Similarly, in deaf people, auditory cortex (part of the brain that processes sounds) becomes adapted to process visual information and gather language clues by sight.

But, ze cautions, these neural changes occur primarily to those parts of our minds to which we pay conscious attention:

A man is sitting in his living room, in front of a chessboard. Classical music plays in the background. The man is focused, thinking about the next move, about his chess strategy, and about the future possibilities of the game. His neural networks are optimizing, making him a better chess player.

A man is sitting in his living room, in front of a chessboard. Classical music plays in the background. The man is focused, thinking about the music he hears, listening to the chords and anticipating the sounds still to come. His neural networks are optimizing, making him better at understanding music and hearing subtleties within a melody.

A man is sitting in his living room, in front of a chessboard. Classical music plays in the background. The man is focused, gritting his teeth as another flash of pain comes from his bad back. His neural networks are optimizing, making the pain more intense, easier to feel, harder to ignore.

You need to pay attention not just to doing physics, but to the right parts of doing physics – the parts most related to intuition.

James Nearing gave his advice on how to do this in Mathematical Tools for Physicists

“How do you learn intuition?

When you’ve finished a problem and your answer agrees with the back of the book or with your friends or even a teacher, you’re not done. The way do get an intuitive understanding of the mathematics and of the physics is to analyze your solution thoroughly. Does it make sense? There are almost always several parameters that enter the problem, so what happens to your solution when you push these parameters to their limits? In a mechanics problem, what if one mass is much larger than another? Does your solution do the right thing? In electromagnetism, if you make a couple of parameters equal to each other does it reduce everything to a simple, special case? When you’re doing a surface integral should the answer be positive or negative and does your answer agree?

When you address these questions to every problem you ever solve, you do several things. First, you’ll find your own mistakes before someone else does. Second, you acquire an intuition about how the equations ought to behave and how the world that they describe ought to behave. Third, It makes all your later efforts easier because you will then have some clue about why the equations work the way they do. It reifies the algebra.

Does it take extra time? Of course. It will however be some of the most valuable extra time you can spend.

Is it only the students in my classes, or is it a widespread phenomenon that no one is willing to sketch a graph? (\Pulling teeth” is the cliche that comes to mind.) Maybe you’ve never been taught that there are a few basic methods that work, so look at section 1.8. And keep referring to it. This is one of those basic tools that is far more important than you’ve ever been told. It is astounding how many problems become simpler after you’ve sketched a graph. Also, until you’ve sketched some graphs of functions you really don’t know how they behave.”

(To see the advice on graphs, along with a detailed step-by-step example, see his book, free online)

Brown Big Spiders

One of the difficulties with chunks is that they’re mostly subconscious. We may ultimately know of their existence, as did the chess master who told me he knew how each square of the chess board felt, but their precise nature and the process of their creation are almost immune to introspection. The study methods I’ve talked about above are empirically useful in creating chunks, so we have guidelines for how to make new chunks in general, but we usually don’t know which ones we are creating.

Lesswrong user Yvain comments on the essay Being a teacher:

“I used to teach English as a second language. It was a mind trip.

I remember one of my students saying something like “I saw a brown big spider”. I responded “No, it should be ‘big brown spider'”. He asked why. Not only did I not know the rule involved, I had never even imagined that anyone would ever say it the other way until that moment.

Such experiences were pretty much daily occurrences.”

In other words, the chunkiest cognitive process we have – language – develops largely without our awareness. (In retelling this story, I’ve met a surprising number of people who actually did know about adjective order in English, but most of them either learned English as a second language or had studied it in psychology or linguistics course.)

This makes it incredibly difficult for physics teachers or textbook writers to communicate with beginners. It’s inevitable that beginners will say that a certain lecturer or book just doesn’t explain it clearly enough, or needs to give more examples. Meanwhile, the lecturer has no idea why what they said wasn’t already perfectly clear and thinks the example was completely explicit. Neither party can articulate the problem, the student because they can’t see the incorrect assumption they’re making, the professor because they don’t realize they’ve already made such an assumption.

For example, once I was proctoring a test in a physics class for biology majors. A question on the test described a certain situation with light going through a prism and asked, “What is the sign of the phase shift?” A student came up to ask for clarification, and it wasn’t until they’d asked their question three times that I finally got it. They thought they were supposed to find the “sign” as in a signpost, or marker. There would be some sort of observable behavior that would indicate that a phase shift had occurred, and that was the “sign of the phase shift.” Until then, I was only able to think of “sign” as meaning positive or negative – did the wave get advanced or retarded?

If you want to learn a language with all those rules you don’t even know about, you need to immerse yourself. Endless drills and exercises from a book won’t be enough, as millions of Americans a decade out of high school straining to remember, “Dondé esta el baño?” can attest. You need to read, speak, see, and hear that language all around you before it takes.

To learn physics, then, read, speak, and hear it all around you. Attend colloquia. Read papers. Solve problems. Read books. Talk to professors and TA’s, and expose yourself to all the patterns of thought that are the native language of the field.

As you learn, you will build the right chunks to think about physics without realizing what they are. But there’s a flip side to this problem, which is that when you’re not doing physics, you can build the wrong chunks. They can get in the way, and again you don’t realize it.

In Drawing on the Right Side of the Brain, Betty Edwards discusses an exercise she gave her art students:

‘One day, on impulse, I asked the students to copy a Picasso drawing upside down. That small experiment, more than anything else I had tried, showed that something very different is going on during the act of drawing. To my surprise, and to the students’ surprise, the finished drawings were so extremely well done that I asked the class, “How come you can draw upside down when you can’t draw right-side up?” The students responded, “Upside down, we didn’t know what we were drawing.”‘

When we see a recognizable image, unconscious chunking immediately gets to work, interpreting, imparting meaning, and inevitably distorting. Learning to draw, according to Edwards, involves circumventing harmful chunks as much as building helpful ones.

So it is with physics. The ideas about force, animation, and intent discussed in the laptop-and-desk example seem to illustrate just this problem. Five Easy Lessonslists many of the known misconceptions that students have somehow taught themselves in each topic of introductory physics – for example that electric current gets used up as it goes around a circuit. But I think it’s likely that there are many more such obstructive thought patterns that we don’t yet know exist. These might be more general notions about such things as cause and effect, what nature “wants” to accomplish, etc.

I Feel Dumb

Educators are perpetually frustrated by what seems like an outrageous pattern. They explain something clearly. The students all claim to understand perfectly, and can even solve quantitative problems. Still, when you ask the students to answer basic conceptual questions, they get it all wrong. How is this possible?

In this YouTube video, Veritasium explores what happens when you explain something clearly:

Amazingly, the clearer the explanation, the less students learn. Humans have a huge array of cognitive biases. In general, these various biases work so that we’ll keep believing whatever it was we believed to begin with, unless there’s a really good reason not to. Someone giving a clear, authoritative physics lecture does not register in your mind as a good reason to check your beliefs, so you listen happily and rave about what a great lecture it was, all while maintaining your wrong ideas.

However, with the right stimulus you can get your brain to throw out the old, wrong ideas. Entering such a state is a prerequisite to true learning, and fortunately we can detect it in ourselves. We call it confusion.

Confusion is a message from your emotional mind (the part that tells your analytical mind what decisions to start justifying). It’s saying, “Hey, something about our beliefs is very wrong, and this is actually important. Pay attention and figure it out.”

A great lecturer, instead of being clear, will confuse students by asking them to predict ahead of time what a demonstration will show, then do it, and the opposite actually occurs. Or they will ask students to solve questions that sound straightforward, but in fact the students can’t figure out. Only after confusion sets in will the teacher reveal the trick.

You want to defeat your biases, toss out your wrong beliefs, and learn physics to the Feynman level – the level where you create the knowledge as you go along. Even many specialists never fully get there, instead rising to increasingly-sophisticated levels of rehashing the same memorized arguments in a way that can carry them quite far and trick most people. The only way to avoid this is to spend many, many hours thoroughly confused.

Have you ever lost an argument, only to think of the perfect retort two days later when stopped at a traffic light? This shows how your mind will continue working on hard problems in the background. It eventually comes up with a great answer, but only if you first prime it with what to chew on. This works for physics problems just as well as for clever comebacks, once you find good problems to grapple with. I conjecture that engaging this subconscious system requires a strong emotional connection to the problem, such as the frustration or embarrassment of being dumbstruck in an argument or the confusion of being stumped by a hard problem.

Confusion is essential, but often also unpleasant. When you repeatedly feel frustrated or upset by your confusion, your mind unconsciously learns to shy away from hard thinking. You develop an ugh field.

This could happen for different reasons. A common one arises in people who judge themselves by their intellect. Confusion for such people is a harsh reminder of just how limited they are; it’s a challenge to their very identity. Whether for this reason or some other, it’s common for students and academics to fall into patterns of procrastination and impostor syndrome when navigating the maze of confusion that come with their chosen path.

I don’t have the answer for this. I have heard many people tell their stories, but I have yet to figure out my own. Sometimes confusion feels awful, and my story in physics is a jerky, convoluted one because of how I’ve dealt with that. But once in a while a problem is so good that none of that matters. When I find one of these problems, it hijacks my mind like Cordyceps in a bullet ant, jerking me back to a fresh piece of scratch paper again and again, sometimes for days. If you reach this state over and over, you’ll know Feynman meant by, “What I cannot create I do not understand”

Get confused. Solve problems. Repeat. The universe is waiting for you.


In order of appearance in this answer

Feynman’s Tips on Physics: Richard P. Feynman, Michael A. Gottlieb, Ralph Leighton: 9780465027972: Books
soft question – Memorizing theorems – MathOverflow
The Magical Number Seven, Plus or Minus Two (wikipedia)
The Magical Number Seven (original paper)
Google Translate (Chinese phrase)
Knight, Randall. Five Easy Lessons pp 37
Reif and Heller, 1982
Viète’s formula
How To Solve It: A New Aspect of Mathematical Method (Amazon)
How To Solve It (summary)
How to Solve It (Wikipedia)
Learn Faster with the Feynman Technique (Scott Young. His page is start to get spammy.)
Study Hacks ” About (Cal Newport)
Anki – powerful, intelligent flashcards
Spaced repetition (review by Gwern)
K. Anders Ericsson (Wikipedia)
The Role of Deliberate Practice in the Acquisition of Expert Performance
Dual N-Back FAQ (gwern)
Food Rules An Eater`s Manual (Amazon, how to eat)
Core Performance Essentials (Amazon, exercise) Exercise is an interesting case because not everyone responds very well. For the majority of people it’s worth the time.
Howard Gardner (wikipedia)
The Unschooled Mind: How Children Think And How Schools Should Teach: Howard E. Gardner (Amazon)
The Perils and Promises of Praise (article by Dweck)
Mindset, Dweck’s book.
Flow (psychology) (Wikipedia)
Flow: The Psychology of Optimal Experience: Mihaly Csikszentmihalyi: 9780061339202: Books
David Allen, Getting Things Done® and GTD®
Online to-do list and task management (One possible GTD software)
How to Setup Remember The Milk for GTD
George Lakoff (professional site)
George Lakoff (Wikipedia)
Where Mathematics Come From: How The Embodied Mind Brings Mathematics Into Being: George Lakoff, Rafael Nuñez: 9780465037711: Books
Loaded sentences (Hofstadter reviews Pinker)
The Stuff of Thought: Language as a Window into Human Nature: Steven Pinker: 9780143114246: Books
The Power of Myth: Joseph Campbell, Bill Moyers: 9780385418867: Books
Attention control is critical for changing/increasing/altering motivation
Mathematical Tools for Physics (Nearing)
Being a teacher – Less Wrong
Drawing on the Right Side of the Brain: The Definitive, 4th Edition: Betty Edwards: 9781585429202: Books
Veritasium (channel)
List of cognitive biases (wikipedia)
Dunning–Kruger effect (wikipedia)
Ugh fields – Less Wrong

Useful Quora Answers

Anonymous’ answer to Mathematics: What is it like to understand advanced mathematics?

Satvik Beri’s answer to Mathematics: How do math geniuses understand extremely hard math concepts so quickly?

Qiaochu Yuan’s answer to Wikipedia: Why is it almost impossible to learn a mathematical concept on Wikipedia?

Christopher VanLang’s answer to Graduate School: What should I do if my PhD advisor and lab colleagues think I’m stupid?

What did Richard Feynman mean when he said, “What I cannot create, I do not understand”?

Quora User’s answer to Physics: What are some words, phrases, or expressions that physicists frequently use in ordinary conversation?

Paul King’s answer to Neuroscience: How does arbitrary become meaningful?

English (language): What are some English language rules that native speakers don’t know, but still follow?

Raman Shah’s answer to Procrastination: How do I get over my bad habit of procrastinating?

Further Reading

I feel a little sleazy writing this answer because when I mention, for example, Carol Dweck doing research on the psychology of mindsets or K. Anders Ericsson studying deliberate practice, in fact there are thousands of people working in those fields. The ones I’ve mentioned are simply the most public figures or those I’ve come across by chance. I haven’t even read the original research in most of these cases, relying on summaries instead.

The answer is also preliminary and incomplete. There’s lots of research left to be done, and I’m not an expert in what’s out there. Still, here is a guide to some further resources that have informed this answer.

For an overview of the psychology of learning, I like Monisha Pasupathi’s audio course How We Learn from The Teaching Company. It covers many clever experiments designed to help you build a model of what happens in your mind as you learn.

Bret Victor explores software solutions to visualizing the connection between physical world, mathematical representation, and mental models. Check out:

The Ladder of Abstraction
Explorable Explanations

I think it’s helpful to build an innate impression of your mind as not perceiving the world directly, but as concocting its own, tailored interpretation from sense data. All your consciousness ever gets to experience is the highly-censored version. The books of Oliver Sacks are great for making this clear by illustrating what happens with people for whom some of the processing machinery breaks down.

The LessWrong Sequences were, for me, a powerful introduction to the quirks of human thought, preliminary steps towards how to work best with the firmware we’ve got, and what it means to seek truth.

Selected Bibliography

These are some physics books to which have helped me so far. I’m not choosing them for clear exposition or specialty knowledge in a certain subject, but for how I think they helped me understand the way to think about physics generally.

Blandford and Thorne, Applications of Classical Physics
Epstein, Thinking Physics
Feynman, Lectures on Physics
————   The Character of Physical Law
————   QED: The Strange Theory of Light and Matter
————   Tips on Physics
Geroch, General Relativity from A to B
Levi, The Mathematical Mechanic
Lewin, Walter “Classical Mechanics”, “Electricity and Magnetism” (video lectures with demonstrations on MIT OpenCourseWare)
Mahajan, Street-Fighting Mathematics
Morin, Introduction to Classical Mechanics
Nearing, Mathematical Tools for Physics
Purcell, Electricity and Magnetism
———-, Back of the Envelope Problems
Schey, Div, Grad, Curl, and All That
Thomas and Raine, Physics to a Degree
Thompson, Thinking Like a Physicist
Weisskopf, “The Search for Simplicity” (articles in Am. J. Physics)


Feynman’s Tips on Physics, Feynman, Gottlieb, Leighton
Architectural detail- cut stone wall


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s