Conversation
- Update activation instructions to use AWS Marketplace usage instructions link - Add supported Glue versions: Glue 4 (Spark 3.3) and Glue 5 (Spark 3.5) - Simplify manual installation to reference only the Spark connector JAR - Fix code examples to use proper Glue patterns (custom.spark with DynamicFrames) - Add Visual Editor example showing drag-and-drop connector usage - Add screenshots for marketplace usage instructions and visual editor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
4 Skipped Deployments
|
BentsiLeviav
left a comment
BentsiLeviav
left a comment
There was a problem hiding this comment.
@ShimonSte in addition to these comments/questions - can seperate this line
For more details, please visit our Spark documentation.
to a seperate section, and add more links linking to Glue docs?
|
|
||
| 3. <h3 id="activate-the-connector">Activate the Connector & Create a Connection</h3> | ||
| You can activate the connector and create a connection directly by clicking [this link](https://console.aws.amazon.com/gluestudio/home#/connector/add-connection?connectorName="ClickHouse%20AWS%20Glue%20Connector"&connectorType="Spark"&connectorUrl=https://709825985650.dkr.ecr.us-east-1.amazonaws.com/clickhouse/clickhouse-glue:1.0.0&connectorClassName="com.clickhouse.spark.ClickHouseCatalog"), which opens the Glue connection creation page with key fields pre-filled. Give the connection a name, and press create (no need to provide the ClickHouse connection details at this stage). | ||
| After subscribing, select the Glue version that matches your job requirements. In the **Additional details** section, under **Usage instructions**, click the link to **Open Glue Studio - Add ClickHouse connector**. This opens the Glue connection creation page with key fields pre-filled. Give the connection a name and press create (no need to provide the ClickHouse connection details at this stage). |
There was a problem hiding this comment.
The link we had here was meant to help users to create a connection easily. If you remove that, please elaborate on how to create this connection.
I took the "new user path", and after subscribing, I don't see any option for selecting a Glue version. Have you validated these steps?
- I do see an option to select a glue version and a connector version - how should a user proceed from that point?
- I don't see ClickHouse as an option when I try to create a connection:
There was a problem hiding this comment.
The reason i removed the link was that now we have multiple Glue options and it is not one link.
To choose the Glue version is as you mentioned ( In the drop down)
After subscribing and selecting the glue version you click the link that is in the In the Additional details section, under Usage instructions.
I don't see ClickHouse as an option when I try to create a connection
You are supposed to press the link in the Glue page, that will open a create connection page
There was a problem hiding this comment.
We should update these instructions, I don't see an option to Continue to Configure after subscribing.
| "http_port" -> "<your-clickhouse-port>", | ||
| "protocol" -> "https", | ||
| "user" -> "default", | ||
| "password" -> "<your-password>", |
There was a problem hiding this comment.
Can we somehow add the credentials in the connection creation pahse? I see other connectors give that ability, and wonder if that option is only available to native connector vs marketplace connectors.
There was a problem hiding this comment.
There is, I'll add it to the documentation
Can you elaborate a bit, i am not sure i fully understand you |
Add a section explaining how to store ClickHouse credentials in AWS Secrets Manager and reference them via the Glue connection or the secretId connection option, so user and password don't need to be hardcoded in jobs.
Product name follows official capitalization per the style guide, matching how AWS Marketplace and AWS Lambda are already handled.
Replace with the current AWS Marketplace "Additional details" view showing the Open Glue Studio link for the ClickHouse connector.
Update the secrets section's Option 2 and the Examples section to use connection_type="marketplace.spark" and reference the connector by connectionName, matching the script Glue Studio auto-generates. Note that Manual Installation users should keep custom.spark with inline className/host/credentials, and that secretId can be omitted when the secret is already attached to the Glue connection.
3 participants
Updated glue documentation to support our new connector