SO-101 Robotics Curriculum

Table of Contents

• Overview
• Module 0: Orientation and Intro Robotics
• Module 1: Foundational Mathematics of Robotics
• Module 2: Feedback Control Systems
• Module 3: Kinematics and Dynamics
• Module 4: Computer Vision
• Module 5: Systems Integration and Capstone
• Deferred
• Beyond This Curriculum
• References

Overview

🤖 This curriculum uses the physical SO-101 arm as the hands-on companion for a self-study robotics track, modeled on Carnegie Mellon’s Bachelor of Science in Robotics curriculum. General undergraduate math (calculus, linear algebra, discrete math, probability) and general CS (imperative programming, systems, algorithms) are assumed already known and are not re-taught here — see SO-101 Build & Vendor Comparison for the hardware acquisition decision (currently Seeed Studio Pro, after the Hiwonder DIY kit was found to be missing printed parts) that this curriculum assumes is complete before Module 0 begins.

This is Stage 1 of a longer staged progression toward autonomous quadcopter flight — see Beyond This Curriculum below for Stages 2–6.

Each module maps to one or more CMU BSR courses, substituting the best freely-available public course materials found for each (CMU’s own course sites are largely login-walled). Pacing below assumes roughly 5–8 hrs/week; treat it as a default scaffold, not a fixed calendar — compress or stretch modules freely.

flowchart TD
    M0["Module 0
Orientation and Intro Robotics"]
    M1["Module 1
Foundational Mathematics of Robotics"]
    M2["Module 2
Feedback Control Systems"]
    M3["Module 3
Kinematics and Dynamics"]
    M4["Module 4
Computer Vision"]
    M5["Module 5
Systems Integration and Capstone"]
    M0 --> M1
    M1 --> M2
    M1 --> M3
    M2 --> M5
    M3 --> M5
    M4 --> M5
    M0 --> M4

Module 0: Orientation and Intro Robotics

(≈ 2 weeks — maps to CMU 16-180/16-280/16-281 General Robotics)

Concept goals: systems-thinking overview of what a robot is — sensing, planning, actuation, feedback loops — and orientation to the toolchain before deep-diving into any one subfield.

Materials: - [ ] Modern Robotics (Lynch & Park) — free full PDF, read Ch. 1–2 (configuration space, degrees of freedom) as orientation - [ ] CMU 16-281 labs index — since the SO-101 is a stationary manipulator, not a mobile robot, only a subset of the 10 labs is directly relevant. Do: - [ ] Vision lab - [ ] Controls lab - [ ] Inverse Kinematics lab - Skip as not applicable to a fixed-base arm: Rube Goldberg Machine, Line Following & Odometry, Path Planning, Localization, Remote Control (USAR), Wheel-Free locomotion — these are mobile-robotics-specific

Milestone: ✅ SO-101 fully assembled, wired, motor IDs configured, and calibrated (lerobot-calibrate) per the build doc — this is the hands-on deliverable for this module, since it exercises exactly the sense/act/feedback loop the module is about.

Module 1: Foundational Mathematics of Robotics

(≈ 2–3 weeks — maps to CMU 16-211)

Concept goals: rigid-body motion representations — rotation matrices, homogeneous transforms, exponential coordinates (screw theory) — the coordinate-frame machinery every later module depends on.

Materials: - [ ] Modern Robotics Ch. 3 (Rigid-Body Motions) — rotation matrices, angular velocities, exponential coordinates for rotation - [ ] Modern Robotics Ch. 4 (Forward Kinematics) — product-of-exponentials formula, Denavit-Hartenberg convention - [ ] Selected 16-811 lecture notes (Matt Mason, CMU) for supplementary rigor: - [ ] Solving Linear Equations - [ ] Differential Geometry (Frenet Frames and Surface Curvature) - [ ] Two-Dimensional Configuration Space

Milestone: derive the SO-101’s own forward-kinematics chain by hand, frame-by-frame, using the joint/gear-ratio table already recorded in the build doc — six frames from base to gripper.

Module 2: Feedback Control Systems

(≈ 3–4 weeks — maps to CMU 16-299)

Concept goals: PID control, state-space representations, stability analysis, LQR — the theory underlying what the STS3215 servo firmware is already doing for you, made explicit.

Materials: full public course site at cs.cmu.edu/~cga/controls-intro - [ ] Textbook: Feedback Systems (Åström & Murray) — free full PDF - [ ] Lecture: Introduction and Intuition (the Math) - [ ] Lecture: State-space methods (CGA Lecture 4) - [ ] Lecture: State Estimation (CGA Lecture 6) - [ ] Lecture: Frequency Domain (CGA Lecture 9) - [ ] Lecture: Deriving Dynamics Equations - [ ] Lecture: LQR & DDP - [ ] Assignment 2 (ass2.html) and Assignment 3 (fitting a linear model to real robot data) — the course’s own hardware is an Elegoo-based mobile robot, not the SO-101; treat these as worked exercises and re-derive the equivalent single-joint model for one SO-101 servo instead of replicating on the course’s hardware

Milestone: write a simple position controller for one SO-101 joint from scratch (bypassing the servo’s built-in position-control firmware, if the Feetech SDK exposes raw PWM/current control), and compare its step response against the servo’s native controller.

Module 3: Kinematics and Dynamics

(≈ 3–4 weeks — maps to CMU 16-384)

Concept goals: velocity kinematics (Jacobians), inverse kinematics, equations of motion (Lagrangian dynamics) — completing what Module 1 started.

Materials: continue in Modern Robotics (free PDF) - [ ] Ch. 5 (Velocity Kinematics and Statics) — space/body Jacobians, singularities - [ ] Ch. 6 (Inverse Kinematics) — Newton-Raphson iterative IK - [ ] Ch. 8 (Dynamics of Open Chains) — Lagrangian formulation, mass matrix, Coriolis terms - [ ] Companion exercises/videos at modernrobotics.org

Milestone: implement forward and inverse kinematics for the SO-101 in code (Python), validate against the physical arm by commanding a target end-effector pose and checking the achieved pose; derive the Jacobian and identify the arm’s singular configurations.

Module 4: Computer Vision

(≈ 6–8 weeks — maps to CMU 16-385, the heaviest module)

Concept goals: classical geometric vision (filtering, features, camera models, homographies) through learned vision (CNNs) — the perception stack that imitation-learning policies consume.

Materials: full public course site at 16385.courses.cs.cmu.edu/spring2026 (27 lectures, full topic list with readings) - [ ] Textbook: Computer Vision: Algorithms and Applications (Szeliski) — free PDF - [ ] Core lectures: Image Filtering, Image Pyramids/Frequency Domain, Hough Transform, Feature Detectors/Descriptors, 2D Transformations, Image Homographies, Geometric Camera Models, Neural Networks, Convolutional Neural Networks - [ ] Deep-dive (valuable but skippable under time pressure): Two-View Geometry, Stereo, Radiometry/Reflectance, Photometric Stereo, Digital Photography pipeline - [ ] Assignments (all publicly downloadable): - [ ] Assignment 0: Intro to Python - [ ] Assignment 1: Image Filtering and Hough Transform - [ ] Assignment 2: AR with Planar Homographies - [ ] Assignment 5: Neural Networks for Recognition — most directly relevant to the learned-vision backbones LeRobot policies use - [ ] Optional: Assignment 3 (3D Reconstruction), Assignment 4 (Bag-of-Words Scene Recognition), Assignment 6 (Video Tracking)

Milestone: calibrate the SO-101’s workspace camera(s), and implement one classical-vision task (e.g. homography-based workspace alignment) plus one learned-vision task (e.g. a small object-detection or classification model) against real footage from the arm’s own camera.

Module 5: Systems Integration and Capstone

(≈ 4–6 weeks — maps to CMU 16-450/16-474; no public CMU materials exist for these, since they’re process/project courses)

Concept goals: everything from Modules 0–4 integrated into one working pipeline: hardware, control, kinematics, and perception feeding a learned policy.

Materials: primary anchor is Capuano, Pascal, Zouitine, Wolf & Aractingi — “Robot Learning: A Tutorial” (Oxford/Hugging Face, Oct 2025) — this replaces a vaguer earlier pointer to “LeRobot docs + source code.” It’s written by the LeRobot core team specifically as a guide from classical robotics through RL/behavioral cloning to generalist VLA policies, with runnable code examples in lerobot throughout, and several code snippets load actual public SO-101 datasets (e.g. lerobot/svla_so101_pickplace) directly — this is about as close to purpose-built for this curriculum as an external source gets. Work through it in this order: - [ ] §1 — LeRobotDataset format and the streaming/batching API (code examples included) — do this first regardless of RL vs. BC focus, since every later section depends on it - [ ] §2 — Classical Robotics (explicit/implicit models, planar manipulation, feedback loops, limitations of dynamics-based robotics) — a compact bridge chapter connecting back to Modules 1 and 3; worth reading even though it covers ground already built there - [ ] §3 — Robot Reinforcement Learning (RL primer, real-world RL for robotics with code, limitations: simulators and reward design) - [ ] §4 — Robot Imitation Learning: generative models primer (VAEs, diffusion models, flow matching) → Action Chunking Transformers (ACT) with training code → Diffusion Policy with training code → optimized/async inference - [ ] §5 — Generalist Robot Policies: VLAs, VLMs-for-VLAs, π0 with code, SmolVLA with code - [ ] Getting started with real-world robots tutorial (LeRobot docs) — teleoperation and dataset-recording mechanics not covered by the tutorial’s conceptual focus

Milestones (the actual capstone): - [ ] Collect a demonstration dataset via leader-follower teleoperation for one manipulation task, using LeRobotDataset - [ ] Train a policy (ACT or Diffusion Policy, §4) on the collected dataset using LeRobot - [ ] Evaluate the trained policy on the physical follower arm and characterize failure modes - [ ] (Stretch) Fine-tune or evaluate SmolVLA (§5) on the same task/dataset for comparison — or, before collecting your own data at all, load the public lerobot/svla_so101_pickplace dataset referenced in the tutorial’s own code examples as a first no-hardware-risk trial run of the training pipeline

Deferred

Beyond This Curriculum

🗺️ This SO-101 track is Stage 1 of a longer staged progression toward autonomous quadcopter flight. Each later stage is the cheapest, lowest-risk platform that forces the one new subsystem the previous stages don’t cover, reusing everything learned so far.

flowchart TD
    S1["Stage 1
SO-101 Manipulator
curricula/robotics/curriculum.md"]
    S2["Stage 2
Mobile Robotics
curricula/robotics/mobile-robotics.md"]
    S3["Stage 3
Balancing Robots
curricula/robotics/balancing-robots.md"]
    S4["Stage 4
Quadcopter Simulation
curricula/robotics/quadcopter-autonomy.md"]
    S5["Stage 5
Manual-Flight Quad Build
curricula/robotics/quadcopter-autonomy.md"]
    S6["Stage 6
Autonomous Flight
curricula/robotics/quadcopter-autonomy.md"]
    S1 --> S2
    S2 --> S3
    S3 --> S4
    S4 --> S5
    S5 --> S6

References