feat(bloblang): Improve Bloblang playground with Go/WASM migration by iamramtin · Pull Request #570 · warpstreamlabs/bento

iamramtin · 2025-11-24T11:40:33Z

Summary

Improves the Bloblang playground by migrating core logic from JavaScript to Go/WASM.

The playground previously duplicated some of Bloblang logic in JavaScript, creating maintenance overhead and potential inconsistencies. By moving to Go/WASM, we:

Better align it with the rest of the codebase
Reuse existing Bloblang environment and validation logic directly
Eliminate duplication between Go and JS implementations
Improve type safety and error handling
Maintain a closer source of truth for Bloblang behavior

Changes

Go/WASM Migration:

Migrated execution, validation, formatting, autocompletion, and other utilities from JS to Go/WASM
Split blobl package into focused modules (core, editor, server, wasm, types)
Simplified WASM initialization and auto-register server/WASM handlers
Added elementary test suite for core playground functions (verifies playground-specific behavior without duplicating existing Bloblang tests)

JavaScript Simplification:

Extracted BloblangAPI abstraction layer for WASM/server routing
Removed unused code, dead methods, redundant functionality
Improved error handling and WASM initialization flow

Other Improvements:

Improved autocomplete with pre-rendered HTML docs and smarter context detection
Replaced hardcoded values with CSS variables and duplicate rules
Improved mobile responsive layout and overall UX

Testing

1. Run Unit Tests

make playground-test

2. Test Server Mode (Go backend via HTTP)

# Build and run playground server
go build -o bento ./cmd/bento/main.go
make playground
./bento blobl playground -p 3001 --no-open

Visit playground and verify:

Playground loads without errors
Execute mappings (try: root.name = this.name.uppercase())
Format mappings (Cmd/Ctrl+Shift+F or Format button)
Autocomplete works (type root = and trigger autocomplete)
Error messages display correctly for invalid input/mappings
Theme toggle works (light/dark mode)

3. Test WASM Mode (Client-side execution)

# Clean and rebuild
rm -rf website/static/playground
make docs
make playground
cd website && yarn start

Visit the playground in the docs and verify:

Playground loads in iframe without errors
All functionality from server mode works (execution, formatting, autocomplete)
Theme syncs with Docusaurus theme
Responsive layout works on mobile viewport (resize browser)
Full-screen mode works (click "Open in Full Screen" button)

Breaking Changes

None

- Replaced JavaScript integration with Go and WASM instead - Improved WASM API exposure and updated JS integration - Added full syntax support to formatter (operators, keywords, pipes) Signed-off-by: Ramtin Mesgari <[email protected]>

Signed-off-by: Ramtin Mesgari <[email protected]>

…lidation API - Moved documentation HTML generation from JS to Go - Pre-generate docHTML for functions/methods - Added validation function - Added JSON utilities (format/minify/validate) with WASM and HTTP endpoints - Updated JS to prioritise WASM with native fallbacks - Centralized WASM function registration Signed-off-by: Ramtin Mesgari <[email protected]>

- Tests core functions: execute, validate, syntax, format, autocomplete - Tests JSON utilities: formatJSON, minifyJSON, validateJSON Signed-off-by: Ramtin Mesgari <[email protected]>

- Replace hardcoded values with CSS variables - Remove duplicate CSS rules - Fix mobile responsive layout - Add Bento-themed scrollbars to all editors Signed-off-by: Ramtin Mesgari <[email protected]>

Also simplified WASM initialization and auto-register server/WASM handlers. Signed-off-by: Ramtin Mesgari <[email protected]>

- Remove redundant JSON utilities from Go/WASM (use native JS instead) - Extract BloblangAPI abstraction layer for WASM/server routing - Remove unused code and dead methods - Improve error handling and WASM initialization flow - Improve UX Signed-off-by: Ramtin Mesgari <[email protected]>

- Fix type mismatch in getCompletions() for functionSpecWithHTML/methodSpecWithHTML - Improve autocompletion suggestions for Bloblang keywords - Add tests to verify completion types and structure Signed-off-by: Ramtin Mesgari <[email protected]>

Signed-off-by: Ramtin Mesgari <[email protected]>

iamramtin · 2026-01-22T16:15:18Z

Hey @gregfurman @jem-davies just a friendly ping to see if we can get this merged in?

gregfurman · 2026-01-22T16:17:54Z

@iamramtin hey! Thanks for the ping and sorry for delay, will give this a priority review 👍

gregfurman

I'll do another pass but have left some comments on structure and the pattern(s) used. Lmk thoughts!

internal/cli/blobl/core.go

gregfurman · 2025-11-25T10:36:02Z

internal/cli/blobl/core.go

+		return AutocompletionResponse{
+			Completions: []CompletionItem{},
+			Success:     false,
+			Error:       fmt.Sprintf("Failed to get syntax data: %v", err),


Suggested change

Error: fmt.Sprintf("Failed to get syntax data: %v", err),

Error: fmt.Sprintf("Failed to get syntax data: %s", err),

Why should we use %s here for error types?

Ahh sorry. Meant to say err.Error(), since that would extract the string form of the error:

Suggested change

Error: fmt.Sprintf("Failed to get syntax data: %v", err),

Error: fmt.Sprintf("Failed to get syntax data: %s", err.Error()),

Are these not effectively the same thing when the error is not nil?

gregfurman · 2025-12-22T09:59:41Z

Makefile

+
+playground-test:
+	@go test ./internal/cli/blobl -timeout 3m 


Wont this just run as a part of the normal test suite? Why do we need to add an extra make command?

gregfurman · 2026-01-22T17:38:32Z

internal/cli/blobl/wasm.go

+//
+// Arguments:
+//   - args[0]: JSON string containing AutocompletionRequest with line, column, prefix, beforeCursor
+//
+// Returns a JS object with:
+//   - "completions": array of completion items with caption, value, meta, type, score, docHTML
+//   - "success":     true if autocompletion succeeded, false otherwise
+//   - "error":       error message if autocompletion failed, else nil


I don't think this level of verbosity is necessary

Suggested change

//

// Arguments:

// - args[0]: JSON string containing AutocompletionRequest with line, column, prefix, beforeCursor

//

// Returns a JS object with:

// - "completions": array of completion items with caption, value, meta, type, score, docHTML

// - "success": true if autocompletion succeeded, false otherwise

// - "error": error message if autocompletion failed, else nil

gregfurman · 2026-01-22T18:58:28Z

internal/cli/blobl/wasm.go

+// Excludes functions that depend on system resources (env vars, filesystem).
+var getWasmEnvironment = sync.OnceValue(func() *bloblang.Environment {
+	return bloblang.GlobalEnvironment().WithoutFunctions("env", "file")
+})


Already mentioned, but why not we replace this with an init and run the InitializeWASM logic as well as all JS function registration when the package gets initialised i.e

func init() { env := bloblang.GlobalEnvironment().WithoutFunctions("env", "file") InitializeWASM() // .. pass 'env' around }

Then we can import

gregfurman · 2026-01-22T19:08:04Z

internal/cli/blobl/types.go

+type FormatMappingResponse struct {
+	Success   bool   `json:"success"`
+	Formatted string `json:"formatted"`
+	Error     string `json:"error,omitempty"`


I think we can use the globally available Error function from the JS environment i.e

var errorResp FormatMappingResponse err := js.Global().Get("Error").New(errorResp.Error) panic(err) // or maybe ignore if error

Then we can treat the error case as a JS-level error and handle with a try-catch instead of this error payload approach.

Also, that way we don't need to pass around an error attribute and can focus on payload responses.

wdyt?

See 24c6fa92f2a9, lmk wyt!

gregfurman · 2026-01-22T19:10:05Z

internal/cli/blobl/types.go

+
+// FunctionSpecWrapper wraps query.FunctionSpec to implement Spec interface
+type FunctionSpecWrapper struct {
+	query.FunctionSpec


I don't think the entire struct needs to be embedded here since you're just using some attributes. Consider

Suggested change

query.FunctionSpec

spec query.FunctionSpec

and then

func (f FunctionSpecWrapper) GetDescription() string { return f.spec.Description }

Good call, I think this explicitness is much better 👍

gregfurman · 2026-01-22T19:12:25Z

internal/cli/blobl/types.go

+}
+
+// Spec represents a unified interface for function and method specs
+type Spec interface {


Why not use a dedicated Spec struct here instead with the attributes we need? Feels like 2 more structs + an interface is a bit of overkill.

How would you feel about me adding a BaseSpec to internal/bloblang/query/docs.go and having FunctionSpec and MethodSpec embed those shared attributes instead of duplicating all this logic in this file?

See commit 407ffa62b and LMK what you think, this is non-breaking and fully compatible and should let me not have to redeclare these shared attributes

And follow up 1da4e310a to see how it affects the current code

gregfurman · 2026-01-22T19:14:52Z

internal/cli/blobl/wasm/wasm.go

Wdyt of placing all of these WASM playground related functions in their own package i.e cli/blobl/wasm or /playground?

See 2ee4747ac

…nd MethodSpec Signed-off-by: Ramtin Mesgari <[email protected]>

Signed-off-by: Ramtin Mesgari <[email protected]>

iamramtin · 2026-01-25T05:56:28Z

@gregfurman addressed comments, LMK WYT 🏁

gregfurman · 2026-01-29T13:24:59Z

internal/cli/blobl/core_test.go

+	}
+}
+
+// Test validate (Bloblang validation)


Suggested change

// Test validate (Bloblang validation)

gregfurman · 2026-01-29T13:27:01Z

internal/cli/blobl/core_test.go

+		{
+			name:    "invalid syntax",
+			mapping: `root.bad =`,
+			valid:   false,


valid is implied to be false if empty, so it can just be omitted

gregfurman · 2026-01-29T13:30:01Z

internal/cli/blobl/core_test.go

+			name:    "unknown function",
+			mapping: `root = fake_function()`,
+			valid:   false,
+		},


Perhaps add some more negative test cases i.e

Suggested change

},

},

{

name: "invalid formatting",

mapping: `root=fake_function()`,

valid: false, // should this be false?

},

{

name: "invalid root name",

mapping: `ROOT = this`,

valid: false,

},

ROOT = this would be valid because it would be assigning this to a variable called ROOT rather than Bloblang special root key, i.e.

{ "ROOT": { "name": "bento", "stars": 1500 } }

instead of

{ "name": "bento", "stars": 1500 }

gregfurman · 2026-01-29T13:35:48Z

internal/cli/blobl/core_test.go

+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			valid, err := ValidateBloblangMapping(env, tt.mapping)


Since valid will always be false on error and true when no error is present, I can't really see a need for checking (and returning) both values.

gregfurman · 2026-01-29T13:39:31Z

internal/cli/blobl/core.go

+// ValidateBloblangMapping validates a Bloblang mapping without executing it.
+// Note: Not currently used by the playground UI ('execute' already validates).
+func ValidateBloblangMapping(env *bloblang.Environment, mapping string) (bool, error) {
+	if mapping == "" {
+		return false, errors.New("mapping cannot be empty")
+	}
+
+	_, err := env.NewMapping(mapping)
+	if err != nil {
+		return false, fmt.Errorf("could not parse mapping: %v", err)
+	}
+
+	return true, nil
+}


Is this extra utility function necessary? Can we not just use the result of the underlying NewMapping?

_, err := env.NewMapping(mapping) if err != nil { // ... }

gregfurman · 2026-01-29T14:41:33Z

internal/cli/blobl/editor.go

+func getParamTypeString(param query.ParamDefinition) string {
+	switch param.ValueType {
+	case value.TString:
+		return "string"
+	case value.TInt:
+		return "integer"
+	case value.TFloat:
+		return "number"
+	case value.TBool:
+		return "boolean"
+	case value.TArray:
+		return "array"
+	case value.TObject:
+		return "object"
+	case value.TTimestamp:
+		return "timestamp"
+	default:
+		return "any"
+	}
+}


Is this necessary?

The underlying value.Type is just an alias to a string so running string(param.ValueType) would convert the type into it's string equivalent

Suggested change

func getParamTypeString(param query.ParamDefinition) string {

switch param.ValueType {

case value.TString:

return "string"

case value.TInt:

return "integer"

case value.TFloat:

return "number"

case value.TBool:

return "boolean"

case value.TArray:

return "array"

case value.TObject:

return "object"

case value.TTimestamp:

return "timestamp"

default:

return "any"

}

}

func getParamTypeString(param query.ParamDefinition) string {

return string(param.ValueType)

}

gregfurman · 2026-01-29T15:00:15Z

internal/cli/blobl/editor.go

+}
+
+// formatBloblang formats Bloblang code with indentation and consistent spacing
+func formatBloblang(originalMapping string) (string, error) {


ok wow I can't pretend to say I understand much of what is happening here re: all of this parsing. How did we do it previously in the playground?

gregfurman · 2026-01-29T15:12:30Z

internal/cli/blobl/server.go

-	if !c.Bool("no-open") {
-		u, err := url.Parse("http://localhost:" + port)
-		if err != nil {
-			return fmt.Errorf("failed to parse URL: %w", err)
-		}
-		openBrowserAt(u.String())


Why the refactor here?

gregfurman · 2026-01-29T15:13:50Z

internal/cli/blobl/server.go

Would it be possible to revert this refactor? I'm finding it a bit tough to see what was added versus what was refactored

I understand the difficulty in reviewing this as a mixed change, but I'd prefer not to revert as most of the diff is structural rather than behavioural

The refactor separates server lifecycle, routing, and handlers into explicit types and methods, removes large inline closures, and centralises the state into a server type. This was necessary to make this file more idiomatic when adding the new endpoints cleanly without further growing what was previously essentially just a single function

Functionally, behaviuor is the same, main change is turning implicit structure into explicit structure so future additions don't compound the layout

I’m happy to walk through the structure or split follow-ups if that helps

gregfurman · 2026-01-29T15:17:09Z

internal/bloblang/query/docs.go

+// ------------------------------------------------------------------------------
+
+// BaseSpec contains fields shared by function and method specifications.
+type BaseSpec struct {


Thoughts on just having a custom struct inside the blobl package that maps FunctionSpec or MethodSpec to a common single type? That way, the internal/bloblang functionality does not need to be touched at all and we can keep this unified approach logic where its needed

iamramtin added 7 commits October 2, 2025 18:43

feat(bloblang): Improve Bloblang autocomplete system

effe2d2

Signed-off-by: Ramtin Mesgari <[email protected]>

test(bloblang): Add tests for core playground functions

7c404c6

- Tests core functions: execute, validate, syntax, format, autocomplete - Tests JSON utilities: formatJSON, minifyJSON, validateJSON Signed-off-by: Ramtin Mesgari <[email protected]>

refactor(bloblang): Consolidate playground CSS

fd279e9

- Replace hardcoded values with CSS variables - Remove duplicate CSS rules - Fix mobile responsive layout - Add Bento-themed scrollbars to all editors Signed-off-by: Ramtin Mesgari <[email protected]>

refactor(bloblang): Split blobl package into focused modules

63d17c1

Also simplified WASM initialization and auto-register server/WASM handlers. Signed-off-by: Ramtin Mesgari <[email protected]>

iamramtin force-pushed the playground-v2 branch from 0b9da37 to 0d0b520 Compare November 25, 2025 03:36

iamramtin marked this pull request as ready for review November 25, 2025 03:54

iamramtin requested review from gregfurman and jem-davies as code owners November 25, 2025 03:54

iamramtin force-pushed the playground-v2 branch from 0d0b520 to fe4aa15 Compare November 25, 2025 04:02

gregfurman self-assigned this Nov 25, 2025

iamramtin mentioned this pull request Dec 4, 2025

Playground V3: Feature enhancements #600

Open

iamramtin force-pushed the playground-v2 branch from a6dd66b to 4e96b2c Compare December 6, 2025 06:22

refactor(bloblang): Improve server and WASM implementation

b98ab91

Signed-off-by: Ramtin Mesgari <[email protected]>

iamramtin force-pushed the playground-v2 branch from 4e96b2c to b98ab91 Compare December 6, 2025 06:47

gregfurman reviewed Jan 22, 2026

View reviewed changes

iamramtin added 5 commits January 25, 2026 10:57

Refactor(bloblang): Introduce BaseSpec and embed it in FunctionSpec a…

407ffa6

…nd MethodSpec Signed-off-by: Ramtin Mesgari <[email protected]>

Refactor(bloblang): Remove function and method spec wrappers

1da4e31

Signed-off-by: Ramtin Mesgari <[email protected]>

Refactor(bloblang): Make environment injection explicit

8efe5ab

Signed-off-by: Ramtin Mesgari <[email protected]>

Refactor(bloblang): Shift error handling to JS/Go boundary

24c6fa9

Signed-off-by: Ramtin Mesgari <[email protected]>

Refactor(bloblang): Create separate WASM subpackage

2ee4747

Signed-off-by: Ramtin Mesgari <[email protected]>

iamramtin requested a review from gregfurman January 25, 2026 05:55

gregfurman requested changes Jan 29, 2026

View reviewed changes

	Error: fmt.Sprintf("Failed to get syntax data: %v", err),
	Error: fmt.Sprintf("Failed to get syntax data: %s", err),

-		},
+		},
+   	{
+			name:    "invalid formatting",
+			mapping: `root=fake_function()`,
+			valid:   false, // should this be false?
+		},
+   	{
+			name:    "invalid root name",
+			mapping: `ROOT = this`,
+			valid:   false,
+		},

Conversation

iamramtin commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing

1. Run Unit Tests

2. Test Server Mode (Go backend via HTTP)

3. Test WASM Mode (Client-side execution)

Breaking Changes

Uh oh!

iamramtin commented Jan 22, 2026

Uh oh!

gregfurman commented Jan 22, 2026

Uh oh!

gregfurman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamramtin commented Jan 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamramtin Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamramtin commented Nov 24, 2025 •

edited

Loading

iamramtin Feb 1, 2026 •

edited

Loading