Better Truncated Base64 Decoding

[unkemptneckbeard]

Hello,

I see in in /bin/lib/decryptlib.py there is padding check for b64. Would be it possible to add a similar check for b32?

@numargs(0)
def fn_atob(data, args):
data = data.encode() if type(data) == str else data
return base64.b64decode(data + "====".encode()) # b64decode ignores extra padding

@numargs(0)
def fn_b32(data, args):
padding = "=" * (8 - (len(data) % 8)) if len(data) % 8 != 0 else ""
data = data.encode() if type(data) == str else data
return base64.b32decode(data + padding.encode())

[gjanders]

Reading the code the base64 function looks like it always adds 4 equal signs, "====", the base32 function has a check to see if padding is required...I'm not 100% sure what the question is here.

[malvidin]

There is an issue when the base64 encoded content length is one more than a multiple of four (e.g. 9, 13, etc.).

For example, AAAAA fails even if ==== is appended. The resulting string of AAAAA==== doesn't decode. This error is not exposed by the decrypt command to the user.

from base64 import b64decode

print(b64decode(b"AAAAA===="))

binascii.Error: Invalid base64-encoded string: number of data characters (5) cannot be 1 more than a multiple of 4

One option would be to truncate the last character to the last full block, or to append a character (like A) to inputs that are a 4*n + 1 characters long. Because b64decode ignores all non-alphabet characters, calculating the length is not simple. Appending A==== if it raises an error will add two bits (0b00) to the end of the last decoded byte. This would not fix all potential subsequent decoding errors and could be misleading if someone did not understand this implementation.

@numargs(0)
def fn_atob(data, args):
    data = data.encode() if type(data) == str else data
    try:
        return base64.b64decode(data + "====".encode()) # b64decode ignores extra padding
    except binascii.Error:
        return base64.b64decode(data + "A====".encode())
        # return base64.b64decode(data + "A====".encode())[:-1]  # only the last two bits are unknown, is truncating the entire byte desired?

[gjanders]

Agreed, I'm wondering if working for "most cases" is better but it would definitely require someone to either check the README or the code to understand what happened.

Unless a message is returned to Splunk to advise this has happened (which should be possible)

[malvidin]

It looks like there is a bug how Splunk's web UI lists fields when data includes null values for the Events view. The issue is avoided if you download the data (e.g. CSV output) or push it to a statistics table.

I will share the actual search/makeresults command later, but it looked something like this:

| search * | head 1 ``` get any event ```
| makeresults format=csv data="index,encoded,field1
test,AAAA,data1
test,AAAAAAAA,data1
"
| eval _time = now(), _raw = encoded | search index="test"
| decrypt field=encoded atob decode('utf8') emit('decoded')

On the field summary portion of the UI on the left side, the field names were incomplete, some of the field names were invalid, and the field values were incorrect.