>> (p.1)
    Author Topic: Notes on the use of midstate compression for coinbase transaction commitments  (Read 1079 times)
    gmaxwell (OP)
    Moderator
    Legendary
    *
    expert
    Offline Offline

    Activity: 4494
    Merit: 9673



    View Profile WWW
    November 19, 2015, 11:31:55 PM
    Merited by ABCbits (3)
     #1

    I thought I'd leave a note for posterity related to the reoccurring scheme of using mid-state compression to avoid sending coinbase transactions; as it seems to get reinvented from time to time.  This has been discussed several times scattered about in various places, and I thought it useful to put the folklore understanding in one place.

    In Bitcoin today various schemes like P2Pool or UTXO set commitments are suggested that require adding some mandatory data to a coinbase transaction.  To prove the mandatory data is present you must then transmit the whole coinbase transaction.  Coinbase transactions can be large, e.g. in p2pool and Eligius like pools that pay directly or for other reasons. In theory, a coinbase transaction can be upto the size of the whole block.

    The reoccurring optimization is to put your mandatory data at the end of the coinbase transaction, and then send a sha256 midstate. The receiver can then complete the hashing with the mandatory data and verify agreement.

    I personally think this design is inadvisable because it peels back the black box of the already fragile merkle-damgård construction and may open some novel prefix-extension attacks that no one has thought to check sha2 for... but if it is done, implementations _must_ be very careful to avoid complications from serialization capture. E.g. if you require your last commitment to be a free standing output "OP_RETURN PUSH(data)" someone can instead make their final output "OP_RETURN [other stuff] PUSH__8+4+data+size", and then that output will consume the appended commitment, and then it's possible to send a misleading midstate proof, that would be parsed differently by a midstate only client and implementations that work with the whole coinbase transaction; opening up the avenue to attacks.

    Another way of saying it is that due to the use of variable length fields the transaction serialization cannot be read backwards, and so no application using midstate compression-- which inherently hides the front of transaction-- can have any dependency on the serialized structure beyond "ends with these bytes"..

    For commitment applications, these problems can be avoided like p2pool does by making your rule be specific to the last N bytes of the transaction (your commitment + nlocktime), regardless of what comes before them (e.g. embedded in another output is fine)... but I  still think it's best to avoid this construction entirely.
Page 1
Viewing Page: 1