Batching executor

[normanrz]

Description:

• Add BatchingExecutor

Todos:

Make sure to delete unnecessary points or to check all before merging:

• Updated Changelog
• Added / Updated Tests

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a BatchingExecutor to optimize task submission for existing executors. By grouping individual tasks into larger batches, it aims to reduce overhead associated with frequent task submissions, particularly beneficial for executors that incur significant per-task costs. This enhancement provides a flexible way to manage workload distribution and improve overall processing efficiency without altering the core logic of the underlying executors.

Highlights

• New BatchingExecutor: A new BatchingExecutor class has been introduced, designed to wrap existing executors and process tasks in configurable batches.
• Batching Logic for Map Operations: The BatchingExecutor implements batching logic for both map and map_to_futures methods, optimizing task submission by grouping individual items into larger batches before delegating to the underlying executor.
• Comprehensive Testing: Extensive unit tests have been added for the BatchingExecutor, covering various scenarios for map and map_to_futures, including edge cases like empty inputs, single items, and exact batch boundaries, and verifying correct batching behavior.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new BatchingExecutor that wraps an existing executor to process items in configurable batches, enhancing efficiency for certain workloads. The implementation includes map and map_to_futures methods, along with corresponding unit tests. Review feedback indicates that the map_to_futures method is currently blocking and should be refactored to be non-blocking, possibly using a background thread. Additionally, the map method's handling of iterables could lead to high memory consumption for large inputs, suggesting a more memory-efficient, lazy batching approach. The interaction of the chunksize parameter in BatchingExecutor.map is ambiguous and should be clarified or disallowed. Finally, one of the new tests (test_map_calls_inner_with_batches) needs to be improved to effectively verify the batching behavior of the inner executor, potentially by using a mock object.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
11315	9530	84%	80%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: e343a78 by action🐍

[normanrz]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new BatchingExecutor that wraps an existing executor to process tasks in batches, improving efficiency for certain workloads. The get_executor factory function has been updated to support this new executor type, and comprehensive unit tests have been added. The review identified a critical issue in get_executor where the kwargs parsing for the 'batching' environment could lead to KeyError or TypeError due to improper handling of the inner executor's configuration. Additionally, a high-severity memory concern was raised in BatchingExecutor.map_to_futures, where the input iterable is fully materialized into memory, potentially causing OutOfMemoryError for large datasets.