I'll try to keep this rather brief. There are probably a ton of possible things …here that I haven't explained very well so please ask questions! And there are a ton of possible extensions or small changes we could make, so please give feedback on that too :)
## Proposal
Programmable Transactions are a new type of transaction that will replace both batch transactions and normal transactions (with the exception of special system transactions). These transactions will allow for a series of `Commands` (mini transactions of sorts) to be executed, where the results of commands can be used in following commands. The effects of these commands will be "all-or-nothing", meaning if one command fails, they all fail; just like is done today inside of Move calls.
The goal here is that this would serve as an easy target for SDK tools to implement the `Pay*` transactions, to apply limits to `Coin` inputs, to construct non-primitive Move inputs, to use Shared objects more than once, etc. The goal though is still to also be simple. We are not going to target repetition or looping. We are also not going to allow returning of references from Move. In both cases, it greatly complicates the runtime behavior and rules. We have a great language already for those cases, Move. To address problems solved by loops or by complicated reference usage, we will be looking to at improvements to the module publishing flow. Specifically, we might add the ability to run code from a package without publishing it; a "one-off" package of sorts.
Here is a sketch of the programmable transaction kind, which will be contained in a `TransactionData` struct (so that's where all the gas and sender information will be). Keep in mind:
***No end user or developer should be expected to program with this directly!***
(unless you really want to of course)
In the short term, the SDK and other tools will generate these transactions for simple things like `Pay*` or Move calls. In the long term, we will build tools and abstractions that use this system as a target. We might start with exposing this directly in a JSON-like transaction builder perhaps, but we will strive to build a tool or very small DSL that compiles to this format instead.
```rust
struct ProgrammableTransactionKind {
inputs: Vec<CallArg>, // these are the input objects or primitive values
commands: Vec<Command>,
}
// we will pick reasonable integer types here depending on the limits for number
enum Argument {
// the gas coin. The gas coin can only be used by-ref, except for with `TransferObjects`, which
// can use it by-value.
GasCoin,
// one of the input objects or primitive values (from inputs above)
Input(u16),
// the result of another command.
Result(u16),
NestedResult(u16),
}
// each variant might want its own type
enum Command {
// the MoveCall here can be any entry function with the existing rules
// or *any public function*. The public functions can have return values,
// but cannot return any references
MoveCall {
package: ObjectRef,
module: Identifier,
function: Identifier,
type_arguments: Vec<TypeTag>,
arguments: Vec<Argument>,
},
// (Vec<forall T:key+store. T>, address)
// sends n-objects to the specified address
// objects must have store (public transfer) and the previous owner
// must be an address
TransferObjects(Vec<Argument>, Argument),
// coin operations
// (&mut Coin<T>, u64) -> Coin<T>
// splits of some amount into a new coin
Split(Argument, Argument),
// (&mut Coin<T>, Vec<Coin<T>>)
// merges n-coins into the first coin
Merge(Argument, Vec<Argument>),
}
struct CommandResultValue {
// A Move type
type_: Type,
// derivable from the type_, but explicitly called out here as they will
// be used in the semantics
abilities: AbilitySet,
// BCS bytes
value: Vec<u8>,
}
// The running result of all commands
type CommandResults = Vec<Option<CommandResultValue>>;
```
## Semantics
Each `Command` takes `Argument`s, and produces a series of outputs, `CommandResult`, which are added to the running series of results, `CommandResults`.
The `inputs: Vec<CallArg>` are objects or pure values that can be used in commands (by-value or by-reference). The gas coin will be a special sort of input that with restricted usage. The results The outputs of commands are also accessible as inputs to future commands, with `Argument::Result` or `Argument::NestedResult`.
The rules for an individual `CommandResultValue` depends on the `abilities` of the value. When used by-value, values with `copy` are copied; otherwise, the value is “consumed” or “transferred” and the entry in the `CommandResult` associated with that value is set to `None`. At the end of the transaction, any value without `drop` must have been transferred in this way. In other words, at the end of executing all commands, all `CommandResultValue`s remaining in the `CommandResults` have the `drop` ability. For unused `inputs`, the same rules apply as apply for `MoveCall` , i.e. the contents might change but the owner does not.
With the whole picture in mind, we can look a bit more in depth at each `Argument` variant
- `GasCoin` like the input objects, does not need to be used. At the beginning of executing the series of `commands`, we will withdraw the maximum amount of gas (as specified from the transaction parameters) from the gas coin. Any unused gas will be returned to the coin at the end of execution.
- To simplify the unused gas logic, the `GasCoin` cannot be used by-value with `MoveCall` or `Merge`. However, it can be used by-value with `TransferObjects`, and doing so is critical for implementing `TransferSui` or `Pay*`
- `Input` can be either an object, a vector of objects, or a series of BCS bytes. Since these rules are largely unchanged from the current Move Call transaction, we will not go more in depth.
- `Result` and `NestedResult` are used for accessing the values of previous transactions. Each value will have a Move type associated with it, which will be strongly typed. This means that the result cannot be used or transmuted to some other value, even if BCS compatible.
For execution, the commands are processed sequentially, where results are accessed by the index system described above. Any errors stop execution and prevent all state changes (though gas will be charged). Similar to the handling of linear/affine values in Move or Rust, values (inputs or results) can be used either by-reference or by-value. This usage is automatic and depends on the signature of the operation. Mutable reference `&mut` inputs must be uniquely used in that command. Immutable reference inputs `&` can be used more than once, but cannot be used owned. Using by-value invalidates the memory location. Meaning if it is an `input` it cannot be used again, and if it is a `CommandResultValue` we will set it’s location to `None`. A bit more in detail for each command:
- `MoveCall` will use its inputs by-value or by-reference according to the Move type of the parameter.
- `TransferObjects` use its objects and address by-value.
- `Split` uses its coin by mutable reference `&mut` and a `u64` amount by-value.
- `Merge` takes its first argument by reference `&mut`, but the rest are by-value.
- Any `&mut` usage updates its cell after execution.
- By-value usage invalidates the cell.
- Reference usage `&` leaves the cell untouched.
At the end of execution of the commands:
- Any remaining `input` (or the `GasCoin`) has it’s ownership unchanged.
- Unused gas is returned to the `GasCoin` regardless of (possibly new) owner.
- Recall that the `GasCoin` cannot be destroyed
- For any `CommandResultValue` it must have `drop` or it is an error.
- Astute readers might notice that we copy any value with `copy`, so what about values with `copy` and not `drop`? We can claim that the last usage of any value with `copy` also invalidates the memory location (transfers the ownership). Such a rule is not observably different from also ignoring values with just `copy` at the end of execution.
## Examples
### Pay* Transactions
```rust
// PaySui
gas: vec![/* gas coins */, ...],
inputs: vec![addr0],
commands: vec![
TransferObjects(vec![GasCoin], Input(0)),
]
// PayAllSui
gas: vec![/* gas coins */, ...],
inputs: vec![amount0, addr0, amount1, addr1, amount2, addr2, ...],
commands: vec![
Split(GasCoin, Input(0)),
TransferObjects(vec![Result(0)], Input(1)),
Split(GasCoin, Input(2)),
TransferObjects(vec![Result(2)], Input(3)),
Split(GasCoin, Input(4)),
TransferObjects(vec![Result(4)], Input(5)),
...
]
// PayAllSui
gas: vec![/* gas coins */, ...],
inputs: vec![coin0, amount0, addr0, coin1, amount1, addr1, coin2, amount2, addr2, ...],
commands: vec![
Split(Input(0), Input(1)),
TransferObjects(vec![Result(0)], Input(2)),
Split(Input(3), Input(4)),
TransferObjects(vec![Result(2)], Input(5)),
Split(Input(6), Input(7)),
TransferObjects(vec![Result(4)], Input(8)),
...
]
```
### Coin Limit
Lets say we want to call `a::m::foo(coin: &mut Coin<_>, ctx: &mut TxContext)` with a coin `C` but limit the amount possibly withdrawn to 100
```rust
gas: vec![...],
inputs: vec![C, 100]
commands: vec![
Split(Input(0), Input(1)),
// pardon the psuedo syntax
MoveCall("a::m::foo", Result(0))
Merge(Input(0), Result(0))
]
```
### Shared Object Interplay
We could imagine having two shared dex objects `A` and `B`. We want to exchange some `SUI` in `A`, take that coin and exchange it in `B`, and take it back to `A` for some `SUI` again.
Assuming a function `exchange<C1, C2>(&mut Dex, coin: Coin<C1>): C2`
```rust
gas: vec![...],
inputs: vec![A, B, 100],
commands: vec![
// take 100 SUI
Split(GasCoin, /* 100 */Input(2)),
// exchange it for X
MoveCall("exchange<SUI, X>", vec![/* A */Input(0), Result(0)]),
// exchange the X for Y
MoveCall("exchange<X, Y>", vec![/* B */Input(1), Result(1)]),
// exchange the Y for SUI
MoveCall("exchange<Y, SUI>", vec![/* A */Input(0), Result(2)),
// merge it
Merge(GasCoin, vec![Result(3)])
]
```
## Deprecation of `SingleTransactionKind` and `TransactionKind::Batch`
Programmable Transactions can cover all variants of `SingleTransactionKind` except for the system related ones, such as `ChangeEpoch` and `Genesis`. Those system related transactions might be fine in their own `SystemTransactionKind` enum, or can remain in `SingleTransactionKind` if we do not deprecate the enum.
While all (non-system) single transaction types can be covered by programmable transactions, it might be cumbersome to do so. Especially depending on the the final layout of the `ProgrammableTransaction` struct itself. All transactions non `Pay*` transactions become just a single `Command`. The `Pay*` transactions are a sequence of `Merge`, `Split`, and `TransferObjects` (depending on the specific `Pay*` variant).
Similarly, this system can replace `Batch` as currently `Batch` is just a special case where no results are used in future commands.