Parallelism

SIMT

• 8 cores finishes 4 instruction before new instruction if fetched -> 8.4 = 32 (warp size)

Reconvergence

• can create deadlock (unless Volta / Ampere)

if (...)
    1.
else
    2.
// reconvergence point

• Slide 11
  • reconvergence point after while loop causes deadlock - waits for all threads to reach after while, not incrementing s

Thread properties

• GPU threads are hardware scheduled

Memory hierarchy

• important = what is happening in a warp at the moment