Write to DeltaTable using Python WITHOUT pandas (using pyarrow)

 I am trying to write data in DeltaTable format from an AWS Lambda Function, but AWS Lambda Function limits to 250MB. The deltalake library takes 247MB, which exceeds the limit along with pandas. Since the deltalake library included pyarrow, I need to find a way to write a data frame without including pandas.

Here is how:

	import pyarrow as pa
	from deltalake.writer import write_deltalake
	n_legs = pa.array([2, 4, 5, 100])
	animals = pa.array(["Flamingo", "Horse", "Brittle stars", "Centipede"])
	names = ["n_legs", "animals"]
	table = pa.Table.from_arrays([n_legs, animals], names=names)
	write_deltalake('/tmp/delta_table', table, mode = 'append')

view raw python-write-delta-table.py hosted with ❤ by GitHub