Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: cannot convert 'int' object to bytes #77

Open
hoday opened this issue May 10, 2020 · 2 comments
Open

TypeError: cannot convert 'int' object to bytes #77

hoday opened this issue May 10, 2020 · 2 comments

Comments

@hoday
Copy link

hoday commented May 10, 2020

I had an error when trying to open a parquet file:

Traceback (most recent call last):
File "/local/workplace/lib/python3.6/site-packages/lambda_handlers/parquet_test.py", line 57, in lambda_handler
for row in parquet.reader(fin):
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 470, in reader
dict_items)
File "/local/workplace/lib/python3.6/site-packages/parquet/init.py", line 340, in read_data_page
if schema_element.converted_type is not None else read_values
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in convert_column
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 68, in
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "/local/workplace/lib/python3.6/site-packages/parquet/converted_types.py", line 42, in intbig
return int.from_bytes(data, 'big', signed=True)
TypeError: cannot convert 'int' object to bytes

@mauryfs
Copy link

mauryfs commented Dec 3, 2020

I got a similar error and determined it was choking on a column of type FIXED_LEN_BYTE_ARRAY. I was able to work around this by excluding the column.

Traceback (most recent call last):
File ".\parquetexport3.py", line 102, in
for row in parquet.DictReader(fileIn, columns=targetColumns):
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 415, in DictReader
for row in reader(file_obj, columns):
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 464, in reader
values = read_data_page(file_obj, schema_helper, page_header, cmd,
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet_init_.py", line 334, in read_data_page
read_values = convert_column(read_values, schema_element)
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 66, in convert_column
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 66, in
return [Decimal(intbig(unscaled)) * scale_factor for unscaled in data]
File "C:\Users\m\AppData\Local\Programs\Python\Python38-32\lib\site-packages\parquet\converted_types.py", line 40, in intbig
return int.from_bytes(data, 'big', signed=True)
TypeError: cannot convert 'int' object to bytes

@hoday
Copy link
Author

hoday commented Dec 4, 2020

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants