Autograd ops

Unreleased native NuTorch shell reference. Generated by scripts/gen-ops-reference.ts; every op also documents itself with torch <op> --help. Tensor operands are native values, not strings.

backward

backpropagate from a scalar loss (gradients accumulate)

Signature (placeholders describe arguments; this is not a runnable example):

use torch
torch backward <t1>

grad

snapshot of a tensor’s accumulated gradient

Signature (placeholders describe arguments; this is not a runnable example):

use torch
torch grad <t1>

detach

a graph-free reference to the same data

Signature (placeholders describe arguments; this is not a runnable example):

use torch
torch detach <t1>

zero_grad

zero a tensor’s accumulated gradient in place

Signature (placeholders describe arguments; this is not a runnable example):

use torch
torch zero_grad <t1>

Tips

Intro

requires-grad marks a tensor for differentiation.

use torch
let x = torch tensor [2.0] --requires-grad
$x | torch sum

Basic

Sum a tracked vector and read the scalar.

use torch
let x = torch tensor [2.0] --requires-grad
$x | torch sum | torch value

Intermediate

Square, sum, and backpropagate.

use torch
let x = torch tensor [2.0] --requires-grad
$x | torch mul $x | torch sum | torch backward

Advanced

Backpropagate, then read the gradient.

use torch
let x = torch tensor [2.0] --requires-grad
$x | torch mul $x | torch sum | torch backward
$x | torch grad | torch value