Merge branch 'main' into patch-11

Author: mariadb-stefan-hinz

analytics/.gitbook/includes/the-instructions-were-teste....md

@@ -0,0 +1,7 @@
+---
+title: The instructions were teste...
+---
+
+{% hint style="info" %}
+The instructions were tested against ColumnStore 23.10.
+{% endhint %}

analytics/SUMMARY.md

@@ -4,24 +4,24 @@
 * [MariaDB ColumnStore](mariadb-columnstore/README.md)
   * [Quickstart Guides](mariadb-columnstore/columnstore-quickstart-guides/README.md)
     * [MariaDB ColumnStore Guide](mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-guide.md)
-    * [MariaDB ColumnStore System Usage](mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-usage-guide.md)
     * [MariaDB ColumnStore Hardware Guide](mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md)
+    * [MariaDB ColumnStore System Usage](mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-usage-guide.md)
   * [ColumnStore Architecture](mariadb-columnstore/architecture/README.md)
     * [Topologies Overview](mariadb-columnstore/architecture/topologies-overview.md)
     * [ColumnStore Architectural Overview](mariadb-columnstore/architecture/columnstore-architectural-overview.md)
-    * [ColumnStore Performance Module](mariadb-columnstore/architecture/columnstore-performance-module.md)
-    * [ColumnStore Query Processing](mariadb-columnstore/architecture/columnstore-query-processing.md)
     * [ColumnStore Storage Architecture](mariadb-columnstore/architecture/columnstore-storage-architecture.md)
-    * [ColumnStore System Databases](mariadb-columnstore/architecture/columnstore-system-databases.md)
-    * [ColumnStore User Module](mariadb-columnstore/architecture/columnstore-user-module.md)
-    * [MariaDB Enterprise Columnstore Locking](mariadb-columnstore/architecture/mariadb-enterprise-columnstore-locking.md)
-    * [Mariadb Enterprise Columnstore Query Evaluation](mariadb-columnstore/architecture/mariadb-enterprise-columnstore-query-evaluation.md)
     * [ColumnStore Storage Engine](mariadb-columnstore/architecture/columnstore-storage-engine-overview.md)
     * [ColumnStore Read Replicas](mariadb-columnstore/architecture/columnstore-read-replicas.md)
+    * [ColumnStore System Databases](mariadb-columnstore/architecture/columnstore-system-databases.md)
+    * [ColumnStore Query Processing](mariadb-columnstore/architecture/columnstore-query-processing.md)
+    * [Mariadb Enterprise Columnstore Query Evaluation](mariadb-columnstore/architecture/mariadb-enterprise-columnstore-query-evaluation.md)
+    * [MariaDB Enterprise Columnstore Locking](mariadb-columnstore/architecture/mariadb-enterprise-columnstore-locking.md)
+    * [ColumnStore Performance Module](mariadb-columnstore/architecture/columnstore-performance-module.md)
+    * [ColumnStore User Module](mariadb-columnstore/architecture/columnstore-user-module.md)
   * [Managing ColumnStore](mariadb-columnstore/management/README.md)
     * [Deployment](mariadb-columnstore/management/deployment/README.md)
-      * [ColumnStore Minimum Hardware Specification](mariadb-columnstore/management/deployment/columnstore-minimum-hardware-specification.md)
       * [Installing ColumnStore](mariadb-columnstore/management/deployment/install-guide/README.md)
+        * [MariaDB ColumnStore Hardware Guide](mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md)
         * [Single-Node Localstorage](mariadb-columnstore/management/deployment/install-guide/singlenode-localstorage/README.md)
           * [Step 1: Prepare Systems for Enterprise ColumnStore Nodes](mariadb-columnstore/management/deployment/install-guide/singlenode-localstorage/step-1-prepare-systems-for-enterprise-columnstore-nodes.md)
           * [Step 2: Install Enterprise ColumnStore](mariadb-columnstore/management/deployment/install-guide/singlenode-localstorage/step-2-install-enterprise-columnstore.md)

analytics/mariadb-columnstore/architecture/README.md

@@ -8,3 +8,39 @@ icon: house-blank
 
 # ColumnStore Architecture
 
+{% content-ref url="topologies-overview.md" %}
+[topologies-overview.md](topologies-overview.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-architectural-overview.md" %}
+[columnstore-architectural-overview.md](columnstore-architectural-overview.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-storage-architecture.md" %}
+[columnstore-storage-architecture.md](columnstore-storage-architecture.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-storage-engine-overview.md" %}
+[columnstore-storage-engine-overview.md](columnstore-storage-engine-overview.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-read-replicas.md" %}
+[columnstore-read-replicas.md](columnstore-read-replicas.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-system-databases.md" %}
+[columnstore-system-databases.md](columnstore-system-databases.md)
+{% endcontent-ref %}
+
+{% content-ref url="columnstore-query-processing.md" %}
+[columnstore-query-processing.md](columnstore-query-processing.md)
+{% endcontent-ref %}
+
+{% content-ref url="mariadb-enterprise-columnstore-query-evaluation.md" %}
+[mariadb-enterprise-columnstore-query-evaluation.md](mariadb-enterprise-columnstore-query-evaluation.md)
+{% endcontent-ref %}
+
+{% content-ref url="mariadb-enterprise-columnstore-locking.md" %}
+[mariadb-enterprise-columnstore-locking.md](mariadb-enterprise-columnstore-locking.md)
+{% endcontent-ref %}
+

analytics/mariadb-columnstore/columnstore-quickstart-guides/README.md

@@ -7,3 +7,11 @@ icon: rabbit-running
 
 # Quickstart Guides
 
+{% content-ref url="mariadb-columnstore-guide.md" %}
+[mariadb-columnstore-guide.md](mariadb-columnstore-guide.md)
+{% endcontent-ref %}
+
+{% content-ref url="mariadb-columnstore-hardware-guide.md" %}
+[mariadb-columnstore-hardware-guide.md](mariadb-columnstore-hardware-guide.md)
+{% endcontent-ref %}
+

analytics/mariadb-columnstore/columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md

@@ -1,5 +1,7 @@
 ---
-description: Quickstart guide for MariaDB ColumnStore hardware requirements
+description: >-
+  This page details MariaDB ColumnStore hardware requirements (CPU, RAM,
+  storage, and network).
 ---
 
 # MariaDB ColumnStore Hardware Guide
@@ -8,40 +10,55 @@ description: Quickstart guide for MariaDB ColumnStore hardware requirements
 
 MariaDB ColumnStore is designed for analytical workloads and scales linearly with hardware resources. While the performance generally improves with more CPU cores, memory, and servers, understanding the minimum hardware specifications is crucial for successful deployment, especially in development and production environments.
 
-MariaDB ColumnStore's performance directly benefits from additional hardware resources. More CPU cores enable greater parallel processing, increased memory allows for more data caching (reducing I/O), and more servers enable a larger distributed architecture.
+MariaDB ColumnStore's performance directly benefits from additional hardware resources:
+
+* **More CPU cores** enable greater parallel processing, improving query processing time.
+* **More memory** allows for more data caching (reducing I/O), and more servers enable a larger distributed architecture.
+* **HDDs vs. SSDs:** SSD[^1]s don't deliver as much benefit as you might assume because ColumnStore is optimized towards block streaming, which usually performs well enough on HDD[^2]s.
+* **Bare metal vs. virtual servers:** Bare metal servers are recommended — they provide additional performance because ColumnStore can fully consume CPU cores and memory.
 
 ## Minimum Hardware Recommendations
 
 The specifications differentiate between a basic development environment and a production-ready setup:
 
-**1. For Development Environments:**
+### **For Development Environments**
 
 * **CPU:** A minimum of **8 CPU cores**.
 * **Memory (RAM):** A minimum of **32 GB**.
 * **Storage:** Local disk storage is acceptable for development purposes.
 
-**2. For Production Environments:**
+### **For Production Environments**
 
 * **CPU:** A minimum of **64 CPU cores**.
-  * _Note:_ This recommendation underscores the highly parallel nature of ColumnStore, which can effectively utilize a large number of cores for analytical processing.
+  * This recommendation underscores the highly parallel nature of ColumnStore, which can effectively utilize a large number of cores for analytical processing.
 * **Memory (RAM):** A minimum of **128 GB**.
-  * _Note:_ Adequate memory is critical for caching data and intermediate results, directly impacting query performance.
+  * Adequate memory is critical for caching data and intermediate results, directly impacting query performance.
 * **Storage:** **StorageManager (S3)** is recommended.
-  * _Note:_ This implies leveraging cloud-object storage (like AWS S3 or compatible services) for scalable and durable data persistence in production.
+  * This implies leveraging cloud-object storage (like AWS S3 or compatible services) for scalable and durable data persistence in production.
+
+## Network Interconnectivity
 
-## Network Interconnectivity (for Multi-Server Deployments)
+Network interconnectivity plays a role for multi-server deployments.
 
-* **Minimum Network:** For multi-server ColumnStore deployments, a minimum of a **1 Gigabit (1G) network** is recommended.
-  * _Note:_ This facilitates efficient data transfer between nodes via TCP/IP for replication and query processing across the distributed architecture. For optimal performance in heavy-load scenarios, higher bandwidth (e.g., 10G or more) is highly beneficial.
+* **Minimum Network:** A minimum of a **1 Gigabit (1G) network** is recommended.
+  * This facilitates efficient data transfer between nodes via TCP/IP for replication and query processing across the distributed architecture. For optimal performance in heavy-load scenarios, higher bandwidth (for instance, 10G or more) is highly beneficial.
 
 Adhering to these minimum specifications will provide a baseline for ColumnStore functionality. For specific workload requirements, it's always advisable to conduct performance testing and scale hardware accordingly.
 
+## AWS Instance Sizes <a href="#aws-instance-sizes" id="aws-instance-sizes"></a>
+
+For AWS, ColumnStore internal testing generally uses `m4.4xlarge` instance types as a cost-effective middle ground. The `R4.8xlarge` has also been tested, and performs about twice as fast for about twice the price.
+
 ## See Also
 
-* [MariaDB ColumnStore Minimum Hardware Specification Documentation](../management/deployment/columnstore-minimum-hardware-specification.md)
+* [MariaDB ColumnStore Minimum Hardware Specification Documentation](/broken/pages/ksFdboCNE70th9VaY7pM)
 * [MariaDB ColumnStore Overview](https://mariadb.com/products/columnstore/)
 * [MariaDB documentation: MariaDB ColumnStore](../)
 
 {% include "https://app.gitbook.com/s/SsmexDFPv2xG2OTyO5yV/~/reusable/pNHZQXPP5OEz2TgvhFva/" %}
 
 {% @marketo/form formId="4316" %}
+
+[^1]: Solid state drive
+
+[^2]: Hard disk drive

analytics/mariadb-columnstore/management/deployment/README.md

@@ -17,3 +17,15 @@ layout:
 
 # Deployment
 
+{% content-ref url="../../columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md" %}
+[mariadb-columnstore-hardware-guide.md](../../columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md)
+{% endcontent-ref %}
+
+{% content-ref url="install-guide/" %}
+[install-guide](install-guide/)
+{% endcontent-ref %}
+
+{% content-ref url="upgrades/" %}
+[upgrades](upgrades/)
+{% endcontent-ref %}
+

analytics/mariadb-columnstore/management/deployment/columnstore-minimum-hardware-specification.md

@@ -19,5 +19,25 @@ layout:
     visible: true
 ---
 
-# Installation Guides
+# Installing ColumnStore
+
+{% content-ref url="../../../columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md" %}
+[mariadb-columnstore-hardware-guide.md](../../../columnstore-quickstart-guides/mariadb-columnstore-hardware-guide.md)
+{% endcontent-ref %}
+
+{% content-ref url="singlenode-localstorage/" %}
+[singlenode-localstorage](singlenode-localstorage/)
+{% endcontent-ref %}
+
+{% content-ref url="singlenode-localstorage/" %}
+[singlenode-localstorage](singlenode-localstorage/)
+{% endcontent-ref %}
+
+{% content-ref url="single-node-s3/" %}
+[single-node-s3](single-node-s3/)
+{% endcontent-ref %}
+
+{% content-ref url="multinode-s3/" %}
+[multinode-s3](multinode-s3/)
+{% endcontent-ref %}
 

analytics/mariadb-columnstore/management/deployment/install-guide/README.md

@@ -25,10 +25,12 @@ layout:
 
 This procedure describes the deployment of the Single-Node Enterprise ColumnStore topology with Object storage.
 
-MariaDB Enterprise ColumnStore 23.10 is a columnar storage engine for MariaDB Enterprise Server and Enterprise ColumnStore is best suited for Online Analytical Processing (OLAP) workloads.
+MariaDB Enterprise ColumnStore is a columnar storage engine for MariaDB Enterprise Server and Enterprise ColumnStore is best suited for Online Analytical Processing (OLAP) workloads.
 
 This procedure has 5 steps, which are executed in sequence.
 
+{% include "../../../../../.gitbook/includes/the-instructions-were-teste....md" %}
+
 This page provides an overview of the topology, requirements, and deployment procedures.
 
 Please read and understand this procedure before executing.

analytics/mariadb-columnstore/management/deployment/install-guide/multinode-s3/README.md

@@ -8,7 +8,9 @@ description: 'Step 1: Prepare Systems for Enterprise ColumnStore Nodes'
 
 This page details step 1 of a 5-step procedure for deploying Single-Node Enterprise ColumnStore with Object storage.
 
-This step prepares the system to host MariaDB Enterprise Server and MariaDB Enterprise ColumnStore 23.10.
+This step prepares the system to host MariaDB Enterprise Server and MariaDB Enterprise ColumnStore.
+
+{% include "../../../../../.gitbook/includes/the-instructions-were-teste....md" %}
 
 Interactive commands are detailed. Alternatively, the described operations can be performed using automation.