Skip to content

[fix] Better uploading#1

Merged
dacort merged 11 commits intomainfrom
fix/big-files
Sep 17, 2025
Merged

[fix] Better uploading#1
dacort merged 11 commits intomainfrom
fix/big-files

Conversation

@dacort
Copy link
Copy Markdown
Owner

@dacort dacort commented Sep 17, 2025

Several fixes including:

  • The ability to actually upload large files
  • Removing some dupe code
  • Cleanup of dangling sessions
  • Condensing of validation code so it doesn't create a new session for each command

@dacort dacort requested a review from Copilot September 17, 2025 19:03
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR improves file upload functionality by fixing large file uploads, consolidating validation code, and implementing proper session cleanup. The changes address upload reliability issues and reduce duplicate code paths.

Key changes:

  • Reduced chunk size from 64KB to 32KB to accommodate base64 encoding overhead within AWS SSM frame limits
  • Consolidated session creation and validation logic into the file transfer client
  • Added comprehensive session cleanup in finally blocks to prevent dangling sessions

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/test_file_transfer.py Updated test to reflect new 32KB chunk size default (was 64KB)
src/pyssm_client/utils/command.py Added proper session cleanup in finally block with both client and AWS-side termination
src/pyssm_client/file_transfer/types.py Reduced default chunk size to 32KB with explanatory comment about base64 overhead
src/pyssm_client/file_transfer/client.py Major refactoring: consolidated session management, improved upload logic with size verification, added comprehensive cleanup
src/pyssm_client/communicator/websocket_channel.py Fixed websocket configuration to use frame size limit instead of message size
src/pyssm_client/communicator/types.py Added max_frame_size configuration for websocket frames
src/pyssm_client/cli/types.py Updated CLI default chunk size to 32KB with documentation
src/pyssm_client/cli/main.py Updated CLI help text and default for chunk size option
src/pyssm_client/cli/coordinator.py Removed duplicate file transfer methods, consolidating functionality into FileTransferClient

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

# Valid options
options = FileTransferOptions()
assert options.chunk_size == 65536
assert options.chunk_size == 49152
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test assertion expects chunk_size to be 49152, but the actual default in FileTransferOptions is now 32768 (32KB). This will cause the test to fail.

Suggested change
assert options.chunk_size == 49152
assert options.chunk_size == 32768

Copilot uses AI. Check for mistakes.
Comment on lines 59 to 60
local_mode = local_file.stat().st_mode & 0o777

Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The variable local_mode is computed here but only used much later in the upload process (line 543). Consider moving this calculation closer to where it's used to improve code clarity.

Suggested change
local_mode = local_file.stat().st_mode & 0o777

Copilot uses AI. Check for mistakes.
Comment on lines +495 to +496
# Send chunk using simple append (no heredoc complexity)
append_cmd = f"echo -n '{encoded_chunk}' | base64 -d >> '{temp_remote}'\n"
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The base64 encoded chunk is inserted directly into a shell command without proper escaping. If the encoded data contains single quotes, this could break the command or potentially cause command injection. Consider using a safer method for transferring the data.

Suggested change
# Send chunk using simple append (no heredoc complexity)
append_cmd = f"echo -n '{encoded_chunk}' | base64 -d >> '{temp_remote}'\n"
# Send chunk using heredoc to safely handle any characters
append_cmd = (
f"base64 -d >> '{temp_remote}' <<'EOF'\n"
f"{encoded_chunk}\n"
f"EOF\n"
)

Copilot uses AI. Check for mistakes.
@dacort dacort merged commit ef6ee33 into main Sep 17, 2025
3 checks passed
@dacort dacort deleted the fix/big-files branch September 17, 2025 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants