Enhancements and Test fixes (#27)

Author: msmygit

.github/workflows/release-on-main.yml

@@ -34,7 +34,7 @@ jobs:
       - name: Run tests
         run: npm test
         # Temporary: skip tests for now
-        continue-on-error: true
+        #continue-on-error: true
 
       # Optional: build if needed (for TypeScript, bundlers, etc.)
       - name: Build package

README.md

@@ -86,22 +86,136 @@ Once added, your editor will be fully connected to your Astra DB database.
 
 The server provides the following tools for interacting with Astra DB:
 
+### Collection Management
 - `GetCollections`: Get all collections in the database
-- `CreateCollection`: Create a new collection in the database
+- `CreateCollection`: Create a new collection in the database (with vector support)
 - `UpdateCollection`: Update an existing collection in the database
 - `DeleteCollection`: Delete a collection from the database
+- `EstimateDocumentCount`: Get estimate of the number of documents in a collection
+
+### Record Operations
 - `ListRecords`: List records from a collection in the database
 - `GetRecord`: Get a specific record from a collection by ID
 - `CreateRecord`: Create a new record in a collection
 - `UpdateRecord`: Update an existing record in a collection
 - `DeleteRecord`: Delete a record from a collection
 - `FindRecord`: Find records in a collection by field value
+- `FindDistinctValues`: Find distinct values for a specific field in a collection
+
+### Bulk Operations
 - `BulkCreateRecords`: Create multiple records in a collection at once
 - `BulkUpdateRecords`: Update multiple records in a collection at once
 - `BulkDeleteRecords`: Delete multiple records from a collection at once
+
+### Vector Search
+- `VectorSearch`: Perform vector similarity search on vector embeddings
+- `HybridSearch`: Combine vector similarity search with text search
+
+### Utility
 - `OpenBrowser`: Open a web browser for authentication and setup
 - `HelpAddToClient`: Get assistance with adding Astra DB client to your MCP client
-- `EstimateDocumentCount`: Get estimate of the number of documents in a collection
+## New Features and Capabilities
+
+### Vector Search Capabilities
+
+The Astra DB MCP server now includes powerful vector search capabilities for AI applications:
+
+#### VectorSearch
+
+Perform similarity search on vector embeddings:
+
+```javascript
+// Example usage
+const results = await VectorSearch({
+  collectionName: "my_vector_collection",
+  queryVector: [0.1, 0.2, 0.3, ...], // Your embedding vector
+  limit: 5,                          // Optional: Number of results to return (default: 10)
+  minScore: 0.7,                     // Optional: Minimum similarity score threshold
+  filter: { category: "article" }    // Optional: Additional filter criteria
+});
+```
+
+#### HybridSearch
+
+Combine vector similarity search with text search for more accurate results:
+
+```javascript
+// Example usage
+const results = await HybridSearch({
+  collectionName: "my_vector_collection",
+  queryVector: [0.1, 0.2, 0.3, ...], // Your embedding vector
+  textQuery: "climate change",        // Text query to search for
+  weights: {                          // Optional: Weights for hybrid search
+    vector: 0.7,                      // Weight for vector similarity (0.0-1.0)
+    text: 0.3                         // Weight for text relevance (0.0-1.0)
+  },
+  limit: 5,                           // Optional: Number of results to return
+  fields: ["title", "content"]        // Optional: Fields to search in for text query
+});
+```
+
+### Enhanced Collection Creation
+
+The `CreateCollection` tool now supports more vector configuration options:
+
+```javascript
+// Example usage
+const result = await CreateCollection({
+  collectionName: "my_vector_collection",
+  vector: true,                       // Enable vector search
+  dimension: 1536,                    // Vector dimension (e.g., 1536 for OpenAI embeddings)
+  metric: "cosine"                    // Similarity metric: "cosine", "euclidean", or "dot_product"
+});
+```
+
+### Finding Distinct Values
+
+The new `FindDistinctValues` tool allows you to find unique values for a field:
+
+```javascript
+// Example usage
+const distinctValues = await FindDistinctValues({
+  collectionName: "my_collection",
+  field: "category",                  // Field to find distinct values for
+  filter: { active: true }            // Optional: Filter to apply
+});
+```
+
+### Optimized Bulk Operations
+
+Bulk operations now use native batch processing for better performance:
+
+```javascript
+// Example: Bulk create records
+const result = await BulkCreateRecords({
+  collectionName: "my_collection",
+  records: [
+    { title: "Record 1", content: "Content 1" },
+    { title: "Record 2", content: "Content 2" },
+    // ... more records
+  ]
+});
+
+// Example: Bulk update records
+const updateResult = await BulkUpdateRecords({
+  collectionName: "my_collection",
+  records: [
+    { id: "record1", record: { title: "Updated Title 1" } },
+    { id: "record2", record: { title: "Updated Title 2" } },
+    // ... more records
+  ]
+});
+
+// Example: Bulk delete records
+const deleteResult = await BulkDeleteRecords({
+  collectionName: "my_collection",
+  recordIds: ["record1", "record2", "record3"]
+});
+```
+
+### Improved Error Handling
+
+The server now provides more detailed error messages with error codes to help diagnose issues more easily.
 
 ## Changelog
 All notable changes to this project will be documented in [this file](./CHANGELOG.md).

docs/improvement_sep2025.md

@@ -0,0 +1,198 @@
+# Astra DB MCP Implementation Analysis and Recommendations
+
+## Executive Summary
+
+The Astra DB MCP Server provides a valuable integration between Large Language Models (LLMs) and Astra DB databases through the Model Context Protocol. After a thorough analysis of the implementation and comparison with the Astra DB API documentation, I've identified several opportunities for optimization and enhancement. The current implementation provides a solid foundation but could benefit from improvements in performance, functionality, security, and documentation.
+
+## Key Findings
+
+1. **Architecture and Implementation**: The server is well-structured with clear separation of concerns between server setup, database connection, tool implementation, and security features. It provides a comprehensive set of tools for basic database operations.
+
+2. **API Coverage**: While the implementation covers most essential operations, there are gaps compared to the full Astra DB API, particularly in advanced vector capabilities, specialized query operations, and configuration options.
+
+3. **Performance Considerations**: The current implementation of bulk operations uses multiple individual operations rather than native batch processing, which impacts performance. There are also opportunities for connection optimization and caching.
+
+4. **Security and Error Handling**: The implementation includes good sanitization for prompt injection prevention but could benefit from enhanced error handling, structured error responses, and additional security features like rate limiting.
+
+5. **Vector Database Support**: Basic vector collection creation is supported, but advanced vector search capabilities, hybrid search, and specialized vector configurations are not fully implemented.
+
+6. **Documentation**: The current documentation provides basic setup instructions but lacks comprehensive tool reference, examples, tutorials, and architectural guidance.
+
+## Detailed Recommendations
+
+### 1. Performance Optimizations
+
+#### Bulk Operations Enhancement
+```typescript
+// Current implementation
+const insertPromises = records.map((record) => collection.insertOne(record));
+const results = await Promise.all(insertPromises);
+
+// Recommended enhancement
+const result = await collection.insertMany(records);
+```
+
+**Benefits**: Significant performance improvement by reducing network requests and leveraging native batch processing.
+
+#### Connection Optimization
+Implement connection pooling or reuse to avoid creating new database connections for each operation.
+
+#### Caching Mechanism
+Add caching for frequently accessed data with TTL-based invalidation to reduce database load.
+
+### 2. Vector Database Capabilities
+
+#### Vector Search Implementation
+```typescript
+export async function VectorSearch({
+  collectionName,
+  queryVector,
+  limit = 10,
+  minScore = 0.0,
+  filter = {},
+}) {
+  const collection = db.collection(collectionName);
+  
+  const results = await collection.find({
+    $vector: {
+      vector: queryVector,
+      limit: limit,
+      minScore: minScore,
+    },
+    ...filter,
+  }).toArray();
+  
+  return sanitizeRecordData(results);
+}
+```
+
+#### Hybrid Search Support
+Implement hybrid search combining vector similarity with traditional text search.
+
+#### Enhanced Vector Configuration
+Add support for different similarity metrics, indexing algorithms, and configuration parameters.
+
+### 3. Error Handling and Security
+
+#### Structured Error Responses
+Implement consistent error response structure with error codes, messages, and troubleshooting information.
+
+#### Rate Limiting
+Add rate limiting to prevent abuse and ensure system stability.
+
+#### Audit Logging
+Implement audit logging for security-sensitive operations.
+
+### 4. API Coverage Expansion
+
+#### Find Distinct Values
+Implement the missing "Find distinct values" operation from the API.
+
+#### Advanced Query Filters
+Enhance query capabilities with support for complex filters, nested fields, and range queries.
+
+#### Cursor-based Pagination
+Implement cursor-based pagination for efficient handling of large result sets.
+
+### 5. Documentation Improvements
+
+#### Comprehensive Tool Reference
+Create detailed documentation for each tool with parameters, response formats, examples, and error scenarios.
+
+#### Interactive Examples
+Develop workflow examples, code snippets, and integration examples with popular LLM platforms.
+
+#### Architecture Documentation
+Create diagrams and explanations of the system architecture, data flow, and integration points.
+
+## Prioritized List of Recommendations
+
+### High Priority (Critical Improvements)
+
+1. **Optimize Bulk Operations**
+   - Replace individual operations with native batch processing
+   - Implement chunking for large datasets
+   - Add transaction support for atomic operations
+   - Estimated impact: High (significant performance improvement)
+
+2. **Enhance Vector Database Capabilities**
+   - Implement vector search functionality
+   - Add support for hybrid search
+   - Support additional vector configuration options
+   - Estimated impact: High (enables key AI use cases)
+
+3. **Improve Error Handling**
+   - Implement structured error responses
+   - Add specific error types and codes
+   - Enhance error recovery mechanisms
+   - Estimated impact: High (improves reliability and user experience)
+
+4. **Enhance Security Features**
+   - Implement rate limiting
+   - Add audit logging
+   - Improve credential validation
+   - Estimated impact: High (addresses security concerns)
+
+### Medium Priority (Significant Enhancements)
+
+5. **Expand API Coverage**
+   - Implement missing API endpoints (find distinct values, replace document)
+   - Add support for advanced query filters
+   - Implement cursor-based pagination
+   - Estimated impact: Medium (increases functionality)
+
+6. **Improve Documentation**
+   - Create comprehensive tool reference
+   - Add examples and tutorials
+   - Enhance code documentation
+   - Estimated impact: Medium (improves developer experience)
+
+7. **Implement Performance Optimizations**
+   - Add caching mechanisms
+   - Implement connection pooling
+   - Add query optimization options
+   - Estimated impact: Medium (improves performance)
+
+8. **Enhance Collection Management**
+   - Add support for indexing strategies
+   - Implement field-specific indexing
+   - Add collection metadata support
+   - Estimated impact: Medium (improves flexibility)
+
+### Lower Priority (Nice-to-Have Improvements)
+
+9. **Improve User Experience**
+   - Enhance response formats
+   - Add contextual help
+   - Implement better parameter validation
+   - Estimated impact: Low to Medium (improves usability)
+
+10. **Add Advanced Features**
+    - Implement embedding generation integration
+    - Add support for data masking
+    - Implement asynchronous processing
+    - Estimated impact: Low to Medium (adds specialized capabilities)
+
+## Implementation Roadmap
+
+### Phase 1: Foundation Improvements (1-3 months)
+- Optimize bulk operations
+- Enhance error handling
+- Improve basic security features
+- Update core documentation
+
+### Phase 2: Feature Expansion (3-6 months)
+- Enhance vector database capabilities
+- Expand API coverage
+- Implement performance optimizations
+- Improve collection management
+
+### Phase 3: Advanced Capabilities (6+ months)
+- Add advanced security features
+- Implement advanced vector search capabilities
+- Enhance user experience
+- Add specialized features for AI applications
+
+## Conclusion
+
+The Astra DB MCP Server provides a valuable integration between LLMs and Astra DB, but there are significant opportunities for enhancement. By implementing the recommended improvements, the server can offer better performance, more comprehensive functionality, enhanced security, and improved developer experience. The prioritized approach allows for incremental improvements while focusing first on the most critical aspects.