changed test structure

Author: sfc-gh-mkowalski

pkg/manual_tests/migration_script/README.md

@@ -2,15 +2,6 @@
 
 This directory contains end-to-end tests for the migration script. Each object type has its own folder with test configuration.
 
-## Test Approach
-
-The test validates the migration script by:
-1. Creating test objects on Snowflake
-2. Fetching objects via data source and generating a CSV
-3. Running the migration script to generate Terraform code with import blocks
-4. Applying the generated code (which imports existing objects)
-5. **Verifying the plan is empty** (no changes needed = successful import)
-
 ## Quick Start
 
 ### Step 1: Navigate to object type folder
@@ -22,183 +13,73 @@ cd grants  # or schemas, warehouses, users, etc.
 ### Step 2: Clean up any previous state
 
 ```bash
-rm -rf .terraform terraform.tfstate terraform.tfstate.backup generated_output.tf
+rm -rf .terraform terraform.tfstate terraform.tfstate.backup import/.terraform import/terraform.tfstate import/terraform.tfstate.backup import/main.tf
 ```
 
-### Step 3: Initialize Terraform
+### Step 3: Initialize and create test objects
 
 ```bash
 terraform init
-```
-
-### Step 4: Create test objects AND generate CSV
-
-```bash
 terraform apply -auto-approve
 ```
 
 This creates objects from `objects_def.tf` and generates `objects.csv` via `datasource.tf`.
 
-### Step 5: Run migration script
+### Step 4: Run migration script (output to import directory)
 
 ```bash
-go run github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/scripts/migration_script@main -import=block grants < objects.csv > generated_output.tf
+go run github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/scripts/migration_script@dev \
+  -import=block grants < objects.csv > import/main.tf
 ```
 
-### Step 6: Apply generated output (imports existing objects)
+### Step 5: Import in the separate directory
 
 ```bash
-terraform apply -auto-approve
+cd import
+terraform init
+terraform apply
 ```
 
-This imports the existing Snowflake objects into Terraform state.
+This imports the existing Snowflake objects into a fresh state.
 
-### Step 7: Verify plan is empty
+### Step 6: Verify plan is empty
 
 ```bash
 terraform plan
 ```
 
-**Success criteria:** The plan should show "No changes. Your infrastructure matches the configuration."
-
-If there are changes, the migration script generated incorrect values.
-
-### Step 8: Cleanup (when done)
+### Step 7: Cleanup
 
 ```bash
-terraform destroy -auto-approve
-```
-
-## Directory Structure
-
-```
-manual_tests/
-├── README.md                # This file
-├── users/                   # Users test
-│   ├── objects_def.tf       # Creates test users on Snowflake
-│   ├── datasource.tf        # Fetches users, generates CSV
-│   ├── objects.csv          # Generated CSV (after terraform apply)
-│   └── generated_output.tf  # Generated by migration script
-├── schemas/
-├── warehouses/
-├── grants/
-└── <new_object_type>/       # Add new object types here
+cd ..
+terraform destroy
 ```
 
-## Adding a New Object Type
+## Test Assertions
 
-To add tests for a new object type (e.g., `warehouses`):
+The `datasource.tf` includes a **precondition assertion** that fails if no grants are found. This prevents generating an empty CSV that would cause silent failures.
 
-### Step 1: Create the folder
-
-```bash
-mkdir -p warehouses
-```
-
-### Step 2: Create `objects_def.tf`
-
-Create test objects with various configurations:
+### How it works
 
 ```hcl
-terraform {
-  required_providers {
-    snowflake = {
-      source = "snowflakedb/snowflake"
+resource "local_file" "grants_csv" {
+  # ...
+  lifecycle {
+    precondition {
+      condition     = length(local.grants_csv_rows_unique) > 0
+      error_message = "TEST ASSERTION FAILED: No grants found. Make sure objects_def.tf resources were created first."
     }
   }
 }
-
-provider "snowflake" {}
-
-# Basic warehouse
-resource "snowflake_warehouse" "basic" {
-  name = "MIGRATION_TEST_WH_BASIC"
-}
-
-# Warehouse with all parameters
-resource "snowflake_warehouse" "complete" {
-  name           = "MIGRATION_TEST_WH_COMPLETE"
-  comment        = "Test warehouse for migration"
-  warehouse_size = "XSMALL"
-  # ... more parameters
-}
 ```
 
-**Important naming convention:** Use `MIGRATION_TEST_` prefix for all test objects.
+### Testing the assertion
 
-### Step 3: Create `datasource.tf`
-
-Fetch the objects and generate CSV:
-
-```hcl
-# Fetch test warehouses
-data "snowflake_warehouses" "test_warehouses" {
-  like = "MIGRATION_TEST_WH_%"
-}
-
-locals {
-  # Flatten the data source output
-  warehouses_flattened = [
-    for wh in data.snowflake_warehouses.test_warehouses.warehouses :
-    wh.show_output[0]
-  ]
-
-  # Create CSV header
-  csv_header = length(local.warehouses_flattened) > 0 ? join(",", [
-    for key in keys(local.warehouses_flattened[0]) : "\"${key}\""
-  ]) : ""
-
-  # CSV escape function
-  csv_escape = length(local.warehouses_flattened) > 0 ? {
-    for wh in local.warehouses_flattened :
-    wh.name => {
-      for key in keys(local.warehouses_flattened[0]) :
-      key => replace(
-        replace(
-          replace(tostring(lookup(wh, key, "")), "\\", "\\\\"),
-          "\n", "\\n"
-        ),
-        "\"", "\"\""
-      )
-    }
-  } : {}
-
-  # Create CSV rows
-  csv_rows = length(local.warehouses_flattened) > 0 ? [
-    for wh in local.warehouses_flattened :
-      join(",", [
-        for key in keys(local.warehouses_flattened[0]) :
-        "\"${local.csv_escape[wh.name][key]}\""
-      ])
-  ] : []
-
-  csv_content = join("\n", concat([local.csv_header], local.csv_rows))
-}
-
-# Write CSV file
-resource "local_file" "csv" {
-  content  = local.csv_content
-  filename = "${path.module}/objects.csv"
-}
-
-# Debug outputs
-output "objects_found" {
-  value = length(local.warehouses_flattened)
-}
-```
-
-### Step 4: Test it
+To verify the assertion works correctly, use the `test_assertion/` subdirectory:
 
 ```bash
-cd warehouses
-rm -rf .terraform terraform.tfstate* generated_output.tf
+cd test_assertion
 terraform init
-terraform apply -auto-approve
-
-cd ..
-go run . -import=block warehouses < manual_tests/warehouses/objects.csv > manual_tests/warehouses/generated_output.tf
-
-cd manual_tests/warehouses
-terraform apply -auto-approve
-terraform plan  # Should show no changes!
+terraform apply
+# Expected error: "TEST ASSERTION FAILED: No grants found..."
 ```

pkg/manual_tests/migration_script/grants/datasource.tf

@@ -59,7 +59,7 @@ data "snowflake_grants" "of_child_role" {
 # Fetch all grants TO the privilege database role
 data "snowflake_grants" "to_priv_db_role" {
   grants_to {
-    database_role = "\"${snowflake_database.test_db.name}\".\"${snowflake_database_role.priv_db_role.name}\""
+    database_role = snowflake_database_role.priv_db_role.fully_qualified_name
   }
 
   depends_on = [
@@ -73,7 +73,7 @@ data "snowflake_grants" "to_priv_db_role" {
 # Fetch all grants TO the parent database role
 data "snowflake_grants" "to_parent_db_role" {
   grants_to {
-    database_role = "\"${snowflake_database.test_db.name}\".\"${snowflake_database_role.parent_db_role.name}\""
+    database_role = snowflake_database_role.parent_db_role.fully_qualified_name
   }
 
   depends_on = [
@@ -88,7 +88,7 @@ data "snowflake_grants" "to_parent_db_role" {
 # Fetch grants OF the child database role
 data "snowflake_grants" "of_child_db_role" {
   grants_of {
-    database_role = "\"${snowflake_database.test_db.name}\".\"${snowflake_database_role.child_db_role.name}\""
+    database_role = snowflake_database_role.child_db_role.fully_qualified_name
   }
 
   depends_on = [
@@ -99,7 +99,7 @@ data "snowflake_grants" "of_child_db_role" {
 # Fetch grants OF the priv database role (granted to account role)
 data "snowflake_grants" "of_priv_db_role" {
   grants_of {
-    database_role = "\"${snowflake_database.test_db.name}\".\"${snowflake_database_role.priv_db_role.name}\""
+    database_role = snowflake_database_role.priv_db_role.fully_qualified_name
   }
 
   depends_on = [
@@ -119,42 +119,38 @@ locals {
     [for g in data.snowflake_grants.of_priv_db_role.grants : g]
   )
 
-  # Filter to only privilege grants:
-  # - Must contain MIGRATION_TEST in grantee_name or name
-  # - Must have a non-empty granted_by (implicit grants from Snowflake have empty granted_by
-  #   and cannot be managed by Terraform)
-  # - Must have a non-empty privilege (grants_of returns role membership info with empty privilege,
-  #   these are not privilege grants and the migration script can't handle them)
+  # Filter grants:
+  # - Must contain MIGRATION_TEST in grantee_name or name (test objects only)
+  # - Must have a non-empty privilege (grants_of returns role membership rows with
+  #   empty privilege - the migration script can't handle these)
   test_grants = [
     for g in local.all_grants : g
     if (can(regex("MIGRATION_TEST", g.grantee_name)) || can(regex("MIGRATION_TEST", g.name))) &&
-       g.granted_by != "" &&
        g.privilege != ""
   ]
 
+  # CSV column definitions - order matters for output
+  csv_columns = ["privilege", "granted_on", "grant_on", "name", "granted_to", "grant_to", "grantee_name", "grant_option", "granted_by"]
+
   # CSV header - matches the GrantCsvRow struct
-  grants_csv_header = "\"privilege\",\"granted_on\",\"grant_on\",\"name\",\"granted_to\",\"grant_to\",\"grantee_name\",\"grant_option\",\"granted_by\""
+  grants_csv_header = join(",", [for col in local.csv_columns : "\"${col}\""])
 
-  # CSV escape function
-  csv_escape_grant = {
+  # CSV escape helper - escapes special chars for CSV format
+  csv_escape = {
     for idx, grant in local.test_grants :
     idx => {
-      privilege   = replace(replace(replace(tostring(grant.privilege), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
-      granted_on  = replace(replace(replace(tostring(grant.granted_on), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
-      grant_on    = ""
-      name        = replace(replace(replace(tostring(grant.name), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
-      granted_to  = replace(replace(replace(tostring(grant.granted_to), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
-      grant_to    = ""
-      grantee_name = replace(replace(replace(tostring(grant.grantee_name), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
-      grant_option = tostring(grant.grant_option)
-      granted_by  = replace(replace(replace(tostring(grant.granted_by), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
+      for col in local.csv_columns :
+      col => col == "grant_on" || col == "grant_to" ? "" : (
+        col == "grant_option" ? tostring(grant[col]) :
+        replace(replace(replace(tostring(grant[col]), "\\", "\\\\"), "\n", "\\n"), "\"", "\"\"")
+      )
     }
   }
 
   # Convert each grant to CSV row
   grants_csv_rows = [
     for idx, grant in local.test_grants :
-    "\"${local.csv_escape_grant[idx].privilege}\",\"${local.csv_escape_grant[idx].granted_on}\",\"${local.csv_escape_grant[idx].grant_on}\",\"${local.csv_escape_grant[idx].name}\",\"${local.csv_escape_grant[idx].granted_to}\",\"${local.csv_escape_grant[idx].grant_to}\",\"${local.csv_escape_grant[idx].grantee_name}\",\"${local.csv_escape_grant[idx].grant_option}\",\"${local.csv_escape_grant[idx].granted_by}\""
+    join(",", [for col in local.csv_columns : "\"${local.csv_escape[idx][col]}\""])
   ]
 
   # Remove duplicates by converting to set and back

pkg/manual_tests/migration_script/grants/import/provider.tf

@@ -0,0 +1,15 @@
+# Provider configuration for import directory
+# This file is kept separate from generated main.tf
+
+terraform {
+  required_providers {
+    snowflake = {
+      source = "snowflakedb/snowflake"
+    }
+  }
+}
+
+provider "snowflake" {
+  # Uses default configuration from ~/.snowflake/config or environment variables
+}
+