🔍 Code Extractor

function unpack

Maturity: 24

Reads binary data from a stream and unpacks it according to a specified struct format string.

File:
/tf/active/vicechatdev/rmcl/zipdir.py
Lines:
10 - 12
Complexity:
simple

Purpose

This function provides a convenient wrapper around Python's struct module for reading and unpacking binary data from file-like objects or streams. It calculates the required buffer size based on the format string, reads that exact amount of data from the stream, and unpacks it into a tuple of Python values. This is commonly used for parsing binary file formats, network protocols, or any structured binary data.

Source Code

def unpack(fmt, stream):
    buffer = stream.read(struct.calcsize(fmt))
    return struct.unpack(fmt, buffer)

Parameters

Name Type Default Kind
fmt - - positional_or_keyword
stream - - positional_or_keyword

Parameter Details

fmt: A format string following Python's struct module conventions (e.g., '<I' for little-endian unsigned int, '>2H' for two big-endian unsigned shorts). Defines the structure and byte order of the binary data to be unpacked. Must be a valid struct format string.

stream: A file-like object or stream that supports the read() method (e.g., open file handle, io.BytesIO, socket). The stream must be opened in binary mode and positioned at the location where data should be read from.

Return Value

Returns a tuple containing the unpacked values according to the format string. The number and types of elements in the tuple depend on the format string. For example, format '<IH' returns a tuple of (int, int), while '<I' returns a single-element tuple (int,). If the format string specifies multiple values, they are returned in order.

Dependencies

  • struct

Required Imports

import struct

Usage Example

import struct
import io

def unpack(fmt, stream):
    buffer = stream.read(struct.calcsize(fmt))
    return struct.unpack(fmt, buffer)

# Example 1: Unpack an unsigned integer and a short from binary data
binary_data = b'\x01\x00\x00\x00\x02\x00'
stream = io.BytesIO(binary_data)
result = unpack('<IH', stream)
print(result)  # Output: (1, 2)

# Example 2: Reading from a binary file
with open('data.bin', 'rb') as f:
    # Read a 4-byte integer followed by a 2-byte short
    values = unpack('<IH', f)
    integer_val, short_val = values
    print(f'Integer: {integer_val}, Short: {short_val}')

# Example 3: Unpack multiple values
stream = io.BytesIO(b'\x00\x01\x02\x03\x04\x05\x06\x07')
result = unpack('>2I', stream)  # Two big-endian unsigned ints
print(result)  # Output: (66051, 67438087)

Best Practices

  • Always ensure the stream is opened in binary mode ('rb') before passing it to this function
  • Verify that the stream has enough data available to satisfy the format string requirements, otherwise struct.unpack will raise struct.error
  • Be aware of byte order (endianness) when specifying format strings - use '<' for little-endian, '>' for big-endian, or '=' for native byte order
  • Handle potential exceptions: struct.error if the buffer size doesn't match the format, or if the stream doesn't have enough data to read
  • Remember that the return value is always a tuple, even for single values (e.g., unpack('<I', stream) returns (value,) not value)
  • Consider wrapping this function in try-except blocks to handle EOF or malformed binary data gracefully
  • The stream position advances by struct.calcsize(fmt) bytes after each call

Tags

binary-data struct unpacking serialization deserialization file-parsing binary-format stream-processing data-conversion

Similar Components

AI-powered semantic similarity - components with related functionality:

  • function unpack_group 42.5% similar

    Unpacks a pandas DataFrame group by iterating over rows and yielding tuples of keys and objects, with special handling for objects with 'kdims' attribute.

    From: /tf/active/vicechatdev/patches/util.py

  • function format_file_size_v1 39.9% similar

    Converts a file size in bytes to a human-readable string format with appropriate units (B, KB, MB, GB, TB).

    From: /tf/active/vicechatdev/SPFCsync/dry_run_test.py

  • function _int_to_bytes 39.8% similar

    Converts a signed integer to its little-endian byte representation, automatically determining the minimum number of bytes needed based on the integer's bit length.

    From: /tf/active/vicechatdev/patches/util.py

  • function wrap_tuple_streams 38.5% similar

    Fills in None values in a tuple with corresponding dimensioned stream values based on matching key dimension names.

    From: /tf/active/vicechatdev/patches/util.py

  • function format_file_size_v1 37.6% similar

    Converts a file size in bytes to a human-readable string format with appropriate units (B, KB, MB, GB, TB).

    From: /tf/active/vicechatdev/SPFCsync/test_upload_modalities.py
← Back to Browse