[Examples] Add autonomous code optimization example

[alex000kim]

Add a new example for autonomous code optimization using SkyPilot, where a coding agent (Claude Code, Codex, etc.) uses SkyPilot VMs to parallelize performance experiments on any open-source project

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a powerful new example that enables autonomous code optimization for open-source projects. By integrating coding agents with SkyPilot, the system can intelligently identify performance bottlenecks, search academic literature for optimization strategies, and execute parallel experiments across cloud virtual machines to iteratively improve code efficiency. This significantly streamlines the process of performance tuning and research for developers.

Highlights

• New Example Added: A new example demonstrating autonomous code optimization has been added, showcasing how coding agents can leverage SkyPilot for performance experiments.
• SkyPilot Integration: The example utilizes SkyPilot to provision and manage cloud VMs, enabling parallel execution of optimization experiments on any open-source project.
• Agent-Driven Optimization: It outlines a methodology where coding agents (e.g., Claude Code, Codex) perform literature-guided autoresearch, profile bottlenecks, edit source code, and run experiments to improve performance.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new "Autonomous Code Optimization" example for SkyPilot, encompassing documentation, a detailed README, a SkyPilot experiment configuration, agent instructions, and a setup script. Feedback from the review suggests several improvements: renaming pi-autoresearch to autonomous-code-optimization for clarity in the documentation, correcting MyST directive syntax in the new documentation page, mitigating a potential command injection vulnerability in the experiment.yaml due to eval usage, and improving the efficiency of work directory preparation in the agent instructions to avoid copying unnecessary files.

[gemini-code-assist]

The MyST directive syntax appears to be incorrect. Directives should be opened and closed with three colons (:::), but four colons (::::) are used here and in other places in the file. This will likely break documentation rendering.

This block should be:

:::{admonition} experiment.yaml
:class: dropdown

:::{literalinclude} ../../../../examples/autonomous-code-optimization/experiment.yaml
:language: yaml
:::
:::

Please correct all similar occurrences in this file.

[gemini-code-assist]

Using eval with a user-configurable environment variable like BUILD_CMD can introduce a command injection vulnerability. While users of SkyPilot generally control their own execution environments, it's a security best practice to avoid eval when possible. A similar issue exists on line 57. Consider alternatives that don't involve evaluating a raw command string, such as using a dedicated build script and passing safe parameters to it.

[gemini-code-assist]

The instruction to use cp -r . /tmp/autoresearch/exp-03/ to create an isolated workdir can be inefficient and risky. It will copy all files from the current directory, including build artifacts, version control directories (.git), and potentially very large files not needed for the experiment. This can slow down experiment setup and lead to unexpected behavior.

Consider suggesting a more robust method, such as:

• Using rsync with an exclude list to avoid copying unnecessary files.
• Using git archive to create a clean export of the repository files.

For example:

# Using rsync
rsync -av --exclude='.git' --exclude='build/' . /tmp/autoresearch/exp-03/

This would provide a cleaner and more efficient way to prepare the experiment directory.