Best for
- Use when someone asks 'data doesn't fit on one node', 'too much data', 'need more storage', 'vertical or horizontal scaling', 'tenant scaling', 'time window rotation', or 'data growth exceeds capacity'.
github/awesome-copilot/skills/qdrant-scaling/scaling-data-volume/SKILL.md
Guides Qdrant data volume scaling decisions. Use when someone asks 'data doesn't fit on one node', 'too much data', 'need more storage', 'vertical or horizontal scaling', 'tenant scaling', 'time window rotation', or 'data growth exceeds capacity'.
Decision brief
This document covers data volume scaling scenarios, where the total size of the dataset exceeds the capacity of a single node.
Compatibility matrix
| Platform | Status | Evidence | What to check |
|---|---|---|---|
| Codex | Not declared | No explicit evidence | Portability before use |
| Claude Code | Not declared | No explicit evidence | Portability before use |
| Cursor | Not declared | No explicit evidence | Portability before use |
| Gemini CLI | Not declared | No explicit evidence | Portability before use |
Installation
The source command is displayed only when detected. A safe inspection prompt is always available so your agent can explain every action before execution.
npx skills add https://github.com/github/awesome-copilot --skill "skills/qdrant-scaling/scaling-data-volume"Inspect the Agent Skill "qdrant-scaling-data-volume" from https://github.com/github/awesome-copilot/blob/9933dcad5be5caeb288cebcd370eeeb2fc2f1685/skills/qdrant-scaling/scaling-data-volume/SKILL.md at commit 9933dcad5be5caeb288cebcd370eeeb2fc2f1685. List every install step, command, network request, credential, file read/write, external action, and rollback step. Explain whether it fits my task. Do not install or execute anything until I approve.
Workflow
If the use case is multi-tenant, meaning that each user only has access to a subset of the data, and we never need to query across all the data, then we can use multi-tenancy patterns to scale.
Some use-cases are based on a sliding time window, where only the most recent data is relevant. For example an index for social media posts, where only the last 6 months of data require fast search.
Most general use-cases require global search across all data. In these situations, we might need to fall back to vertical scaling, and then horizontal scaling when we reach the limits of vertical scaling.
When data doesn't fit in a single node, the first approach is to scale the node itself — more RAM, better disk, quantization, mmap. Exhaust vertical options before going horizontal, as horizontal scaling adds permanent operational complexity.
Permission review
No configured static risk pattern was detected
This is not proof of safety. Runtime behavior, indirect dependencies, and hidden external systems are outside the static scan.
Evidence record
| Signal | Value | Evidence type | Meaning |
|---|---|---|---|
| Quality score | 71/100 | Computed | Documentation, specificity, maintenance, and trust rules |
| Repository stars | 37,126 | Source | Repository attention, not individual Skill quality |
| Compatibility | 0 platforms | Source | Declared in the catalog source record |
| Usage guide | automated source guide | Editorial | Generated or reviewed according to the visible evidence level |
Pinned source
This document covers data volume scaling scenarios, where the total size of the dataset exceeds the capacity of a single node.
If the use case is multi-tenant, meaning that each user only has access to a subset of the data, and we never need to query across all the data, then we can use multi-tenancy patterns to scale.
The recommended way is to use multi-tenant workloads with payload partitioning, per-tenant indexes, and tiered multitenancy.
Learn more Tenant Scaling
Some use-cases are based on a sliding time window, where only the most recent data is relevant. For example an index for social media posts, where only the last 6 months of data require fast search.
Learn more Sliding Time Window
Most general use-cases require global search across all data. In these situations, we might need to fall back to vertical scaling, and then horizontal scaling when we reach the limits of vertical scaling.
When data doesn't fit in a single node, the first approach is to scale the node itself — more RAM, better disk, quantization, mmap. Exhaust vertical options before going horizontal, as horizontal scaling adds permanent operational complexity.
Learn more Vertical Scaling
When a single node can't hold the data even with quantization and mmap, distribute data across multiple nodes via sharding.
Learn more Horizontal Scaling