[SS-44] Add COPY FROM s3 Docs#35301
Open
patrickwwbutler wants to merge 3 commits intoMaterializeInc:mainfrom
Open
[SS-44] Add COPY FROM s3 Docs#35301patrickwwbutler wants to merge 3 commits intoMaterializeInc:mainfrom
patrickwwbutler wants to merge 3 commits intoMaterializeInc:mainfrom
Conversation
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
patrickwwbutler
changed the title
martykulma
reviewed
Mar 3, 2026
Contributor
martykulma
left a comment
martykulma
left a comment
There was a problem hiding this comment.
Nice! The doc should also enumerate the S3 bucket/object ACLs needed for MZ to read the data.
| `bool` | `BOOLEAN` | | [`boolean`](/sql/types/boolean/) | ||
| `date32` | `INT32` | `DATE` | [`date`](/sql/types/date/) | ||
| `decimal128[38, 10 or max-scale]` | `FIXED_LEN_BYTE_ARRAY` | `DECIMAL` | [`numeric`](/sql/types/numeric/) | ||
| `fixed_size_binary(16)` | `FIXED_LEN_BYTE_ARRAY` | | [`bytea`](/sql/types/bytea/) <--- what about using the UUID logical type? |
Contributor
There was a problem hiding this comment.
Extra comment in the Materialize type column
Comment on lines
+25
to
+29
| `DELIMITER` | Single-quoted one-byte character | Format-dependent | Overrides the format's default column delimiter. | ||
| `NULL` | Single-quoted strings | Format-dependent | Specifies the string that represents a _NULL_ value. | ||
| `QUOTE` | Single-quoted one-byte character | `"` | Specifies the character to signal a quoted string, which may contain the `DELIMITER` value (without beginning new columns). To include the `QUOTE` character itself in column, wrap the column's value in the `QUOTE` character and prefix all instance of the value you want to literally interpret with the `ESCAPE` value. _`FORMAT CSV` only_ | ||
| `ESCAPE` | Single-quoted strings | `QUOTE`'s value | Specifies the character to allow instances of the `QUOTE` character to be parsed literally as part of a column's value. _`FORMAT CSV` only_ | ||
| `HEADER` | `boolean` | `false` | Specifies that the file contains a header line with the names of each column in the file. The first line is ignored on input. _`FORMAT CSV` only._ |
Contributor
There was a problem hiding this comment.
Do these apply only to CSV, or also to TEXT? Do we need to add something like _FORMAT X,Y only_ for each of these?
| `QUOTE` | Single-quoted one-byte character | `"` | Specifies the character to signal a quoted string, which may contain the `DELIMITER` value (without beginning new columns). To include the `QUOTE` character itself in column, wrap the column's value in the `QUOTE` character and prefix all instance of the value you want to literally interpret with the `ESCAPE` value. _`FORMAT CSV` only_ | ||
| `ESCAPE` | Single-quoted strings | `QUOTE`'s value | Specifies the character to allow instances of the `QUOTE` character to be parsed literally as part of a column's value. _`FORMAT CSV` only_ | ||
| `HEADER` | `boolean` | `false` | Specifies that the file contains a header line with the names of each column in the file. The first line is ignored on input. _`FORMAT CSV` only._ | ||
| `AWS CONNECTION` | _connection_name_ | | The name of the AWS connection to use in the `COPY FROM` command. If using an s3 URI, must be specified. For details on creating connections, check the [`CREATE CONNECTION`](/sql/create-connection/#aws) documentation page. _Only valid with an S3._ |
Contributor
There was a problem hiding this comment.
with an S3 should probably just be with S3
| `large_binary` | `BYTE_ARRAY` | | [`bytea`](/sql/types/bytea/) | ||
| `large_utf8` | `BYTE_ARRAY` | | [`jsonb`](/sql/types/jsonb/) | ||
| `list` | Nested | | [`list`](/sql/types/list/) | ||
| CUSTOM TYPE `struct` | Nested | | [Arrays](/sql/types/array/) (`[]`) |
| `uint32` | `INT32` | `INT(32, false)` | [`uint4`](/sql/types/uint/#uint4-info) | ||
| `uint64` | `INT64` | `INT(64, false)` | [`uint8`](/sql/types/uint/#uint8-info) | ||
| `utf8` or `large_utf8` | `BYTE_ARRAY` | `STRING` | [`text`](/sql/types/text/) | ||
| (COMING SOON) `map` (`struct` with fields `keys` and `values`) | Nested | `MAP` | [`map`](/sql/types/map/) |
Contributor
There was a problem hiding this comment.
Should this get marked as unsupported along with interval?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates the documentation for
COPY FROMmzsql command to include information and syntax on the newCOPY FROM s3feature.Motivation
https://linear.app/materializeinc/issue/SS-44/write-user-facing-docs-for-copy-from-s3-statement-csv
Description
Adds a new syntax file for copy from s3/url, adds tab to SQL command reference page, and information on how to use it.