perf: Adopt `Span<T>` and `Memory<T>`

[currantw]

Summary

The codebase creates many byte arrays throughout the project, causing significant performance overhead.

Description of the Status Quo

Currently, byte array operations throughout the codebase allocate new arrays for sending and receiving data from the Rust core. Allocating and deallocating all this memory reduces throughput and increases memory usage.

Steps to Reproduce

⚪

Proposed Solution

Adopt Span<T> and Memory<T> patterns to eliminate unnecessary allocations, based on the approach in Real world example of reducing allocations using Span and Memory:

1. Binary serialization/deserialization - Use ReadOnlySpan<byte> for parsing protocol data instead of allocating intermediate byte arrays.
2. Buffer management - Use MemoryPool<byte>.Shared for reusable buffers instead of allocating new arrays.
3. String-to-UTF8 conversion - Write directly to existing buffers using Encoding.UTF8.GetBytes(string, offset, length, array, arrayOffset).
4. Command serialization - Pre-calculate buffer sizes and serialize directly to pooled memory.

Effort & Complexity Estimates

	Estimate	Explanation
Effort	Medium	Significant refactoring. Requires performance benchmarking.
Complexity	High	Requires careful memory management, understanding of Span/Memory semantics, and thorough performance testing

Related Issues

• Real world example of reducing allocations using Span and Memory
• Dynamic Pub/Sub StackExchange.Redis Compatibility #202 (see this comment).