Skip to content

MB-65860: Added new file writers and readers#337

Open
Likith101 wants to merge 7 commits intomasterfrom
fileCallbacks
Open

MB-65860: Added new file writers and readers#337
Likith101 wants to merge 7 commits intomasterfrom
fileCallbacks

Conversation

@Likith101
Copy link
Member

No description provided.

@Likith101 Likith101 changed the title MB-65860: Added new file writers and readers MB-65860: Added new file writers and readers (WIP) Jul 9, 2025
@Likith101 Likith101 changed the title MB-65860: Added new file writers and readers (WIP) MB-65860: Added new file writers and readers Aug 13, 2025
@CascadingRadium
Copy link
Member

I think i get the main idea here, but want some clarifications regarding the following points:

  • When the system's creating a segment -> a []byte basically, we use the CountHashWriter to write any data we want into the []byte, before flushing to disk. Is there any reason why we need to manually process each bit of data selectively before writing to []byte? Can we not just interleave the code to encrypt data within the existing code? (within the Write method itself?)
    I have added some code that demonstrates optimal callback use (which is more readable as well). Check the code added and let me know if there's a misunderstanding here.

  • You have mentioned that the reader should be able to discern the WriteToken (counter) itself. I believe this is something that the user should handle. How do you plan on handling overflows with this? If the original Counter overflowed (say) and created a new counter (im not sure if thats the right way to handle this). How will the reader be aware of this fact? (I am assuming that the current system uses exactly one counter from both the writer's and reader's POV makit it easier to synchronize the two.)

Will give another pass when you provide some clarity regarding the aforementioned points.

@abhinavdangeti
Copy link
Member

abhinavdangeti commented Nov 3, 2025

Per plan, let's target this to the unstable-v17 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants