[feat]: CGO packed writer api #160
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: shaoting-huang The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
shaoting-huang
force-pushed
the
cgo_writer
branch
from
756ed27 to
0f7a773
Compare
cpp/src/packed/writer_c.cpp
Outdated
| } | ||
| } | ||
|
|
||
| void DeletePackedWriter(CPackedWriter c_packed_writer) { |
There was a problem hiding this comment.
Any particular reason to separate the Close and Delete... functions?
There was a problem hiding this comment.
close still writes buffers to files. delete just free the memory
cpp/src/packed/writer_c.cpp
Outdated
| auto writer = | ||
| std::make_unique<milvus_storage::PackedRecordBatchWriter>(buffer_size, trueSchema, trueFs, truePath, conf); | ||
|
|
||
| *packed_writer = writer.release(); |
There was a problem hiding this comment.
to initialize packed_writer pointer
|
|
||
| int WriteRecordBatch(CPackedWriter c_packed_writer, struct ArrowArray* array, struct ArrowSchema* schema); | ||
|
|
||
| int Close(CPackedWriter c_packed_writer); |
There was a problem hiding this comment.
Additional information required on Close, include but not limited to: i) files written, ii) schema mapping on those files written, and iii) statistics and tracing data.
shaoting-huang
force-pushed
the
cgo_writer
branch
from
3bf1728 to
7d83835
Compare
Signed-off-by: shaoting-huang <[email protected]>
shaoting-huang
force-pushed
the
cgo_writer
branch
from
41567e3 to
e7ab8b4
Compare
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
Signed-off-by: shaoting-huang <[email protected]>
| arrow::fs::FileSystem& fs, | ||
| const std::string& file_path, | ||
| const StorageConfig& storage_config, | ||
| const parquet::WriterProperties& props); |
There was a problem hiding this comment.
Why delete the parquet properties support?
| const int pk_index, | ||
| const int ts_index, |
There was a problem hiding this comment.
How do we use these two arguments?
Signed-off-by: shaoting-huang <[email protected]>
|
/lgtm |
related: #158