Why does a Bitcoin transaction have two identifiers?

Evidence

Each numbered marker in the text points here. Quotes are verbatim from the BIP files at commit 3a10b5b, including their wiki markup; links open the exact lines; the quoted text sits under each entry. Labels say what kind of statement each is: a rule, the author’s rationale, history, a test vector, or our own inference.

1. Author’s rationale A transaction's effects are determined by the outputs it spends and creates; signatures are only needed to validate the chain state, not to determine it.

   BIP 141 L22

   Quoted source text (1)

   The entirety of the transaction's effects are determined by output consumption (spends) and new output creation. Other transaction data, and signatures in particular, are only required to validate the blockchain state, not to determine it.

2. Author’s rationale Signature data in the witness no longer affects the txid; BIP141 states this prevents involuntary malleability when all inputs are signed with at least one CHECKSIG or CHECKMULTISIG, while an m-of-n multisig spend remains malleable with agreement of m key holders.

   BIP 141 L26 BIP 141 L27 BIP 141 L28

   Quoted source text (3)

   Since signature data is no longer part of the transaction hash, changes to how the transaction was signed are no longer relevant to transaction identification.

   as long as all inputs are signed (with at least one CHECKSIG or CHECKMULTISIG operation)

   In the case of an m-of-n CHECKMULTISIG script, a transaction is malleable only with agreement of m private key holders

3. History BIP141 (assigned 2015) and BIP143 (assigned 2016) are consensus soft-fork proposals whose preambles record the status Deployed.

   BIP 141 L10 BIP 141 L3 BIP 141 L8 BIP 143 L9 BIP 143 L7

   Quoted source text (5)

   Assigned: 2015-12-21

   Layer: Consensus (soft fork)

   Status: Deployed

   Assigned: 2016-01-03

   Status: Deployed

4. Rule BIP141 defines a witness structure, committed to blocks separately from the transaction merkle tree, holding data needed to check validity but not to determine effects; scripts and signatures move into it.

   BIP 141 L16

   Quoted source text (1)

   This BIP defines a new structure called a "witness" that is committed to blocks separately from the transaction merkle tree. This structure contains data required to check transaction validity but not required to determine transaction effects. In particular, scripts and signatures are moved into this new structure.

5. Rule Each transaction has two IDs: the txid is the double SHA256 of [nVersion][txins][txouts][nLockTime]; the wtxid is the double SHA256 of [nVersion][marker][flag][txins][txouts][witness][nLockTime].

   BIP 141 L41 BIP 141 L43 BIP 141 L45 BIP 141 L47 BIP 141 L49

   Quoted source text (5)

   Each transaction will have 2 IDs.

   Definition of <code>txid</code> remains unchanged: the double SHA256 of the traditional serialization format:

   [nVersion][txins][txouts][nLockTime]

   A new <code>wtxid</code> is defined: the double SHA256 of the new serialization with witness data:

   [nVersion][marker][flag][txins][txouts][witness][nLockTime]

6. Rule Block weight is base size × 3 + total size, and must be at most 4,000,000; base size excludes witness-related data as a non-upgraded node sees it.

   BIP 141 L117 BIP 141 L119 BIP 141 L123

   Quoted source text (3)

   ''Block weight'' is defined as ''Base size'' * 3 + ''Total size''.

   ''Base size'' is the block size in bytes with the original transaction serialization without any witness-related data, as seen by a non-upgraded node.

   The new rule is ''block weight'' ≤ 4,000,000.

7. Test vector BIP143 publishes complete example transactions with their preimages and sighashes, and asks developers to test against them.

   BIP 143 L137 BIP 143 L190 BIP 143 L250

   Quoted source text (3)

   developers should test their implementations against all the tests below.

   The serialized signed transaction is:

   The serialized signed transaction is:

8. Rule The witness is committed in a tree nested into the block's merkle root via the coinbase; the commitment is required when a block has witness data and optional otherwise.

   BIP 141 L18 BIP 141 L63 BIP 141 L80

   Quoted source text (3)

   The witness is committed in a tree that is nested into the block's existing merkle root via the coinbase transaction

   A new block rule is added which requires a commitment to the <code>wtxid</code>.

   If all transactions in a block do not have witness data, the commitment is optional.

9. Author’s rationale Transmitting witness data is needed only for a peer that validates a transaction, not one that only checks its existence.

   BIP 141 L31

   Quoted source text (1)

   It is needed only if a peer is trying to validate a transaction instead of just checking its existence.

10. Rule The marker must be 0x00 and the flag a non-zero byte, currently 0x01.

    BIP 141 L53 BIP 141 L55

    Quoted source text (2)

    The <code>marker</code> MUST be a 1-byte zero value: <code>0x00</code>.

    The <code>flag</code> MUST be a 1-byte non-zero value. Currently, <code>0x01</code> MUST be used.

11. Rule Each input has a witness field: a var_int count of stack items, then each item with a var_int length. Witness data is not script.

    BIP 141 L57

    Quoted source text (1)

    Each txin is associated with a witness field. A witness field starts with a <code>var_int</code> to indicate the number of stack items for the txin. It is followed by stack items, with each item starts with a <code>var_int</code> to indicate the length. Witness data is NOT script.

12. Rule A non-witness-program input must have an empty witness field, written 0x00; if no input is a witness program, wtxid equals txid.

    BIP 141 L59

    Quoted source text (1)

    A non-witness program (defined hereinafter) txin MUST be associated with an empty witness field, represented by a <code>0x00</code>. If all txins are not witness program, a transaction's <code>wtxid</code> is equal to its <code>txid</code>.

13. Rule Blocks containing witness data must commit to the wtxids; the witness root is built like the merkle root, recorded in a coinbase output, and the coinbase's own wtxid is taken as all zeros.

    BIP 141 L63 BIP 141 L65 BIP 141 L67 BIP 141 L80

    Quoted source text (4)

    A new block rule is added which requires a commitment to the <code>wtxid</code>. The <code>wtxid</code> of coinbase transaction is assumed to be <code>0x0000....0000</code>.

    A <code>witness root hash</code> is calculated with all those <code>wtxid</code> as leaves, in a way similar to the <code>hashMerkleRoot</code> in the block header.

    The commitment is recorded in a <code>scriptPubKey</code> of the coinbase transaction.

    If all transactions in a block do not have witness data, the commitment is optional.

14. Test vector The native P2WPKH example spends one ordinary P2PK output and one P2WPKH output; the P2SH-P2WPKH example spends one P2SH-wrapped witness program.

    BIP 143 L151 BIP 143 L155 BIP 143 L214

    Quoted source text (3)

    The first input comes from an ordinary P2PK:

    The second input comes from a P2WPKH witness program:

    The input comes from a P2SH-P2WPKH witness program:

15. Rule A version 0, 20-byte program is P2WPKH: the witness is exactly two items, a signature and a public key whose HASH160 matches the program.

    BIP 141 L95 BIP 141 L96 BIP 141 L97

    Quoted source text (3)

    It is interpreted as a pay-to-witness-public-key-hash (P2WPKH) program.

    The witness must consist of exactly 2 items (≤ 520 bytes each). The first one a signature, and the second one a public key.

    The HASH160 of the public key must match the 20-byte witness program.

16. Author’s rationale BIP141 says this allows unconfirmed transaction dependency chains without counterparty risk, important for off-chain protocols such as the Lightning Network.

    BIP 141 L30

    Quoted source text (1)

    It allows creation of unconfirmed transaction dependency chains without counterparty risk, an important feature for offchain protocols such as the Lightning Network

17. Rule A version 0, 32-byte program is P2WSH: the witness is an input stack for the script followed by the serialized script itself.

    BIP 141 L101 BIP 141 L102

    Quoted source text (2)

    It is interpreted as a pay-to-witness-script-hash (P2WSH) program.

    The witness must consist of an input stack to feed to the script, followed by a serialized script (<code>witnessScript</code>).

18. Rule Witness validation is also triggered when a P2SH redeemScript pushed in the scriptSig is exactly a version byte plus a witness program.

    BIP 141 L92

    Quoted source text (1)

    Triggered when a <code>scriptPubKey</code> is a P2SH script, and the BIP16 <code>redeemScript</code> pushed in the <code>scriptSig</code> is exactly a push of a version byte plus a push of a witness program.

19. Author’s rationale BIP141's motivation lists that moving data into a structure unknown to old nodes lets a soft fork discount witness size when calculating block size.

    BIP 141 L32–33

    Quoted source text (1)

    '''Some constraints could be bypassed with a soft fork''' by moving part of the transaction data to a structure unknown to current protocol, for example: #* Size of witness could be ignored / discounted when calculating the block size, effectively increasing the block size to some extent

20. History When BIP141 was written, blocks were limited to 1,000,000 bytes total size.

    BIP 141 L115

    Quoted source text (1)

    Blocks are currently limited to 1,000,000 bytes (1MB) total size.

21. Editorial inference Since weight = 3 × base + total = 4 × base + (total − base), each base byte adds 4 weight units and each witness-related byte (marker, flag, witness) adds 1.

    BIP 141 L117 BIP 141 L121

    Quoted source text (2)

    ''Block weight'' is defined as ''Base size'' * 3 + ''Total size''.

    including base data and witness data.

22. Author’s rationale BIP141 chose one composite limit because two separate limits would make mining and fee estimation nearly impossible.

    BIP 141 L117

    Quoted source text (1)

    Using two separate limits would make mining and fee estimation nearly impossible.

23. Editorial inference Because weight = 4 × base + witness ≤ 4,000,000, a block's base data cannot exceed 1,000,000 bytes; only witness data lets its total size exceed that.

    BIP 141 L117 BIP 141 L123

    Quoted source text (2)

    ''Block weight'' is defined as ''Base size'' * 3 + ''Total size''.

    The new rule is ''block weight'' ≤ 4,000,000.

24. Rule Transaction weight is defined the same way, and virtual size is weight / 4 rounded up; these are suggested terms, not consensus limits.

    BIP 141 L136 BIP 141 L140 BIP 141 L142

    Quoted source text (3)

    The following definitions are not used for consensus limits, but are suggested to provide language consistent with the terminology introduced above.

    ''Transaction weight'' is defined as ''Base transaction size'' * 3 + ''Total transaction size''

    ''Virtual transaction size'' is defined as ''Transaction weight'' / 4 (rounded up to the next integer).

25. Test vector The tested model reproduces both published examples' serializations, preimages and sighashes byte for byte, and computes their sizes and weights.

    BIP 143 L174 BIP 143 L187

    Quoted source text (2)

    hash preimage:

    sigHash:

26. Test vector Parsed, the native P2WPKH example has 233 base bytes and 343 bytes in total: weight 1,042, virtual size 261.

    BIP 143 L190 BIP 141 L140

    Quoted source text (2)

    The serialized signed transaction is:

    ''Transaction weight'' is defined as ''Base transaction size'' * 3 + ''Total transaction size''

27. Rule BIP143 defines a new digest for signatures in version 0 witness programs, to minimize redundant hashing and to cover the input value.

    BIP 143 L14

    Quoted source text (1)

    This proposal defines a new transaction digest algorithm for signature verification in version 0 witness program, in order to minimize redundant data hashing in verification, and to cover the input value by the signature.

28. Author’s rationale With the original digest, hashing per signature grows with transaction size, so total hashing grows as O(n^2) in the number of sigops.

    BIP 143 L21

    Quoted source text (1)

    For the verification of each signature, the amount of data hashing is proportional to the size of the transaction. Therefore, data hashing grows in O(n²) as the number of sigops in a transaction increases.

29. Rule The BIP143 preimage has ten items: nVersion, hashPrevouts, hashSequence, outpoint, scriptCode, amount, nSequence, hashOutputs, nLocktime and sighash type.

    BIP 143 L29 BIP 143 L34 BIP 143 L38

    Quoted source text (3)

    1. nVersion of the transaction (4-byte little endian)

    6. value of the output spent by this input (8-byte little endian)

    10. sighash type of the signature (4-byte little endian)

30. Rule hashPrevouts, hashSequence and hashOutputs can be reused across inputs, reducing total hashing from O(n^2) to O(n).

    BIP 143 L70

    Quoted source text (1)

    may be reused in other inputs of the same transaction, so that the time complexity of the whole hashing process reduces from O(n²) to O(n).

31. Rule Without ANYONECANPAY, hashPrevouts is the double SHA256 of all input outpoints; for SIGHASH_ALL, hashSequence covers all nSequence values and hashOutputs all outputs.

    BIP 143 L58 BIP 143 L62 BIP 143 L66

    Quoted source text (3)

    If the <code>ANYONECANPAY</code> flag is not set, <code>hashPrevouts</code> is the double SHA256 of the serialization of all input outpoints;

    If none of the <code>ANYONECANPAY</code>, <code>SINGLE</code>, <code>NONE</code> sighash type is set, <code>hashSequence</code> is the double SHA256 of the serialization of <code>nSequence</code> of all inputs;

    If the sighash type is neither <code>SINGLE</code> nor <code>NONE</code>, <code>hashOutputs</code> is the double SHA256 of the serialization of all output amount (8-byte little endian) with <code>scriptPubKey</code> (serialized as scripts inside CTxOuts);

32. Rule Every BIP143 sighash type commits to the amount spent by the signed input; item 6 is that 8-byte amount.

    BIP 143 L42 BIP 143 L55

    Quoted source text (2)

    All sighash types commit to the amount being spent by the signed input;

    The item 6 is a 8-byte value of the amount of bitcoin spent in this input.

33. Author’s rationale Without the amount, an offline signer cannot compute the amount spent or the fee; with the input's amount in its digest, a wrong value makes that signature invalid and no funding might be lost.

    BIP 143 L22 BIP 143 L22

    Quoted source text (2)

    For an offline transaction signing device ("cold wallet"), however, the unknowing of input amount makes it impossible to calculate the exact amount being spent and the transaction fee.

    In the case that a wrong value is provided and signed, the signature would be invalid and no funding might be lost.

34. Rule For P2WPKH the scriptCode is 0x1976a914{20-byte-pubkey-hash}88ac.

    BIP 143 L50

    Quoted source text (1)

    For <code>P2WPKH</code> witness program, the <code>scriptCode</code> is <code>0x1976a914{20-byte-pubkey-hash}88ac</code>.

35. Rule The BIP143 digest applies only to sigops in version 0 witness programs.

    BIP 143 L27 BIP 141 L152

    Quoted source text (2)

    A new transaction digest algorithm is defined, but only applicable to sigops in version 0 witness program:

    a new transaction digest algorithm for signature verification in version 0 witness program.

36. Author’s rationale The original signature digest does not involve the amount being spent by the input.

    BIP 143 L22

    Quoted source text (1)

    The algorithm does not involve the amount of Bitcoin being spent by the input.