Encoding VarInts

0 votes
asked Sep 17, 2016 by dmaclach (160 points)

I'm trying to determine how to convert a varint into a number using Synalysis. My varint is integers in 7-bit chunks, little-endian order. The high-bit in each byte signifies if it is the last byte. So for values from 0 to 127 it would be 1b6b5b4b3b2b1band for values 128-16383 it would be 0b6b5b4b3b2b1b1b13b12b11b10b9b8b7.

Thoughts?

1 Answer

0 votes
answered Sep 17, 2016 by andreas (2,180 points)

The easiest way is probably to write a custom (scripted) data type like the ones on https://www.synalysis.net/scripts.html and implement it in Python using existing code: https://github.com/google/protobuf/blob/master/python/google/protobuf/internal/decoder.py or https://github.com/dowski/misc/blob/master/varints.py.

I don't understand why you write it's little endian - has the byte order actually be reversed before decoding the varint?

Does this help?

Andreas

commented Sep 17, 2016 by dmaclach (160 points)
Yep.. I actually hadn't found the scripted data types.

def parseByteRange(element, byteView, bitPos, bitLength, results):
    processedBytes = 0
    num = 0
    done = 0
    while done == 0:
        done = byteView.readUnsignedIntBits(bitPos, 1, ENDIAN_LITTLE)
        bitPos += 1
        num = num + (byteView.readUnsignedIntBits(bitPos, 7, ENDIAN_LITTLE) >> 1)
        bitPos +=  7
        processedBytes += 1
   
    value = Value()
    value.setUnsigned(num)

    results.addElement(element, processedBytes, 0, value)
    return processedBytes

Does a pretty good job.

BTW is there anymore documentation anywhere on writing Lua scripts? I only found one example after a cursory search, and it wasn't great.
commented Sep 17, 2016 by dmaclach (160 points)
Found some better lua scripts by digging through some of the other grammars.
...