help wanted
Repository metrics
- Stars
- (1,953 stars)
- PR merge metrics
- (Avg merge 1d 7h) (49 merged PRs in 30d)
Description
It would be nice to have an interface that converts a SedonaSpark DataFrame to a SedonaDB DataFrame easily. Here is a current solution that works:
import sedona.db
sd = sedona.db.connect()
df = sd.create_data_frame(dataframe_to_arrow(spark_df))
This could be nice:
spark_df.to_sedonadb()
But maybe we'd have to do this:
spark_df.to_sedonadb(sd)
This would allow for cool spatial workflows, like this:
- Read an Iceberg table with SedonaSpark and perform big data operations with a filtering operation at the end to make the data small enough to fit on a single machine
- Convert the SedonaSpark DataFrame to SedonaDB
- Use a library that's compatible with SedonaDB, like lonboard, to create a graph
Let me know what you think!