Cargo Ops · design notesOpen the consolebisesi.dev

An operator console for robots on a space station

Cargo Ops is a browser simulation of the ground side of station robotics: one person on the ground supervising a fleet of free-flying robots over a slow, unreliable link, stepping in by hand when autonomy fails, and turning each fix into training data. These notes explain how it works and why.

Open the consoleLink modelMessagesData loopWatch the demo

A scripted 94-second run: the first arm drills, the map growing, a robot asking for help and the teleop fix, then the build outside. Sped up about 2×.

What this is, and isn't. A fan-made simulation, not Icarus software. The robots are named for Icarus's JOY, a two-armed free-flyer that is teleoperated from Earth first. Everything they do here is invented, and the console design is mine. Built to show how I think about the operator side of remotely controlled hardware.

The operator's job

Icarus's public plan is teleoperation first, then partial autonomy where one person supervises several robots by choosing higher-level commands ("open the bag", "unstow the items"), then full autonomy, with the data from each stage training the next. Cargo Ops plays the middle stage. The robots fly, route, grab and charge on their own. What they can't do yet is know what the crew needs first, or get unstuck. So the person on the ground:

Three clocks and the link

Everything on the ground is old. A command leaves now, the robot acts on it one link delay later, and the operator sees the result one more delay after that. The console shows both clocks on every event-log line.

GROUND STATION command sent uplink L robot acts telemetry L operator sees it LOSuplink lost,downlink buffered
DelayOne-way latency L (0.3 to 3 s, default 1.2) with ±10% jitter on every packet. Telemetry frames go down at 20 Hz.
Loss of signalScheduled windows when no relay satellite is in view (the zone of exclusion, shown on the orbit view). Uplink packets sent into a window are lost. Downlink is buffered onboard and arrives in a burst at the next acquisition of signal, so the map freezes, then jumps.
Command lifeSENT → ACKED → EXECUTING → DONE, or EXPIRED (no acknowledgement in time) or FAILED (refused onboard). Each command carries an expiry, and the station never acts on a stale one. The ground timeout allows for known LOS windows, so an ACK sitting in the station's buffer is not counted as lost. A late ACK brings an expired command back.
SkyThe Earth below is kept plain so it never competes with the station: its tone and the light shift through day, dusk and night once per compressed orbit, following the orbit view. The integrated truss carries the eight solar array wings and the radiators, drawn see-through at a reduced scale.
RelayTDRS by default (what the ISS uses today: near-continuous coverage with gaps). A what-if option models a low-orbit relay mesh: only brief handover gaps and much lower latency.

Messages

Uplink (ground → station)

PLANA whole checklist in one command, with its LOS policy
MOVEA single transfer, sent now
RETRY / SKIPAnswers to an assist
TELEOPTeleop finished, the robot resumes its item
TAKE / DRIVE / GRAB / RELEASEFirst-person direct control
ESTOP / RESUMEFleet-wide halt and release
HANDOFF / TAKEBACKLend a robot to another console, take it back

Downlink (station → ground)

ACK / NACKReceived; or refused, with a reason
EXEC / STEP / DONEItem assigned to a robot, progress, finished
ASSIST / RESOLVEDA robot is stuck and waiting; the answer took effect
HOLD / RESUMEWhat the fleet did when the signal dropped and came back
ESTOPPED / RESUMEDConfirmation onboard
HANDBACKA lent robot is back under this console
telemetryRobot pose, state, battery, task, route; cargo; segments

Learning the controls first

A first visit opens with a short guided tour of the console while four robots run a real checklist; it ends by starting level 1.0. From there every player starts the same way: one robot, JOY-1, in one corner of the station (Harmony, the PMA-2 hatch and a docked Dragon). It gets no work until you can work its arms. Two drills come first:

  1. Arms, short delay. Free a snagged strap on the teleop bench with the link cut to 0.25 s, to learn the grippers.
  2. Arms, real delay. Open a zip over the full 1 s link. Move, then wait for the arm.

Then comes the first task, and with each level the map grows: the camera eases out as Unity, Destiny, the Russian segment and Kibo fade in. When nothing has changed for a while and something is waiting on you (a draft not uplinked, a robot asking for help, a refused line, the E-stop left on, a drill step that drags on), the guide points at it. In the first levels the button to press next carries a NEXT tag, and every teleop fix ends on a card that says what was logged.

Seeing what the robot sees

Hover any robot to get a small inset from its forward camera. Double-click it to take direct control: the whole map becomes that camera, with racks, lockers, handrails and hatch frames inside each module, and the image is one link delay old. With hand tracking on, your hands work the map: pinch and move to orbit, pinch with both hands to zoom and turn.

Onboard autonomy

Assists, teleop and the data loop

When a robot gets stuck (a snagged strap, a stiff zip, a fastener that won't seat) it stops and waits for a person. The operator can retry, skip, or open the teleop bench: a close-up of the robot's two arms, driven over the same link. With the camera on, each of your hands drives one gripper (MediaPipe hand tracking, pinch to close), and your mirrored camera image sits faintly behind the arms so your fingers line up with the grippers. Mouse and touch work too, one arm at a time.

The learning rule is deliberately simple. The point is the loop, not the model: operator work becomes structured data, and that data visibly reduces operator work.

Sharing the fleet

Mid-shift, a second console (payload ops) asks to borrow a robot. Only one console commands a robot at a time: approving hands it over through the link, the allocator stops giving it work, and it comes back when the experiment ends. You can take it back early, explicitly, with a confirmation. This comes straight from a system I built and ran in classrooms, where teachers keep control of their headsets until another teacher explicitly takes over.

Modelled, approximate, invented

ItemStatus
Which module sits on which portrealISS configuration; zenith and nadir modules drawn beside their port, lengths stretched
Module shapesapproximateZvezda's stepped hull and sphere, Prichal's ball, the Cupola's six-sided dome, PMA cones, vehicle silhouettes
TDRS relay, zones with no coveragereal conceptTiming compressed so a shift sees four
Cargo transfer bags, rack stowagereal conceptCounts and categories invented
JOY's two arms and fansapproximateFrom public descriptions; not its real design
Speeds, batteries, failure reasons and ratesinventedTuned for play
Outpost-1 and the spacecraftinventedLevels 3.1 and 3.2

What I'd build next with a real robot

Built with

Plain JavaScript and canvas, three.js r128 for the 3D station and the teleop bench, MediaPipe Hands for hand tracking, and NASA's ISS model behind the briefing. No build step, about 3,200 lines. Six levels in three stages, a guided tour, a quick reference, replay and JSONL export.

Sources for Icarus's public plan: TechCrunch, Silicon Republic, Payload. Station model: NASA. Nick Bisesi, New York · nick@bisesi.dev