Spatial behaviour as continuous query

Authors: Kossiso Royce
Institution: Electric Sheep Africa
Date: January 2026
Keywords: Embedding spaces, spatial navigation, semantic search, human-computer interaction, 3D visualization, continuous querying

---

Abstract

Traditional information retrieval systems require users to formulate explicit queries—a cognitive process that presupposes knowledge of what one seeks. We present a novel paradigm that reconceptualizes semantic search as embodied spatial navigation, where a user’s physical movement through a 3D environment constitutes an implicit, continuous query over high-dimensional embedding spaces. Our system, SBCQ (Spatial Behaviour as Continuous Query), projects semantic embeddings into navigable 3D space while preserving relative distances, enabling users to discover related concepts through proximity rather than keyword specification. We implement a proof-of-concept demonstrating that spatial proximity in the rendered environment correlates strongly (>70%) with semantic similarity in the original embedding space, validating the core thesis that movement through space is equivalent to traversal through semantic meaning. This work has implications for exploratory search, knowledge discovery, and the future of human-AI interaction where queries are expressed through behaviour rather than language.

---

1. Introduction

1.1 The Query Formulation Problem

Modern information retrieval depends on a fundamental assumption: users can articulate what they seek. Search engines, recommender systems, and knowledge bases all require some form of explicit query—keywords, natural language questions, or filter selections. This creates a paradox first articulated by Belkin (1980) as the “Anomalous State of Knowledge” (ASK): users often need information precisely because they lack the knowledge required to formulate an effective query.

Consider a researcher exploring the intersection of ethics and artificial intelligence. Traditional search requires them to guess relevant keywords: “AI ethics,” “machine morality,” “algorithmic fairness.” Each query returns results, but the semantic landscape between these terms remains invisible. The researcher cannot see that “algorithmic accountability” lies between “fairness” and “transparency,” or that “existential risk” connects AI to philosophical concepts of consciousness.

1.2 Spatial Cognition as an Alternative

Humans possess sophisticated spatial reasoning capabilities evolved over millions of years. We effortlessly navigate complex physical environments, remember spatial relationships, and discover new locations through exploration. Spatial memory is robust—we remember where we encountered information often better than the information itself (Godden & Baddeley, 1975).

This paper proposes leveraging spatial cognition for semantic exploration. Rather than requiring explicit queries, we project high-dimensional embedding spaces into navigable 3D environments. Users explore by moving; their position becomes their query. Proximity indicates semantic relationship. Discovery emerges from wandering.

1.3 Core Thesis

Spatial behaviour constitutes a continuous query over embedding space. Every position in the 3D environment corresponds to an interpolated point in high-dimensional semantic space. Movement traces a path through meaning. Standing near a concept is semantically equivalent to querying for concepts similar to it.

This thesis has three testable components:

1. Structural Preservation: The mapping from embedding space to 3D space preserves relative semantic distances.
2. Navigational Validity: Users can locate semantically related concepts through spatial exploration.
3. Query Equivalence: A user’s position generates results equivalent to explicit similarity search.

1.4 Contributions

This paper makes the following contributions:

1. Theoretical Framework: We formalize the relationship between spatial navigation and continuous semantic querying, introducing the concept of “embodied queries.”
2. System Architecture: We present SBCQ, a real-time system that renders embedding spaces as explorable 3D worlds using WebGL and React Three Fiber.
3. Evaluation Methodology: We develop metrics for measuring spatial-semantic correlation, including a Kendall-tau-based concordance measure.
4. Proof-of-Concept Implementation: We provide a fully functional demonstration with 50+ concepts across 8 semantic clusters, achieving >70% spatial-semantic correlation.

---

2. Related Work

2.1 Embedding Visualization

The visualization of high-dimensional embeddings has a rich history. Van der Maaten and Hinton (2008) introduced t-SNE, enabling 2D visualization of high-dimensional data while preserving local structure. McInnes et al. (2018) developed UMAP, offering better preservation of global structure with computational efficiency.

These techniques typically produce static 2D plots. Our work extends this by:

• Projecting to 3D for immersive navigation
• Enabling real-time exploration rather than static viewing
• Computing position-based queries dynamically

2.2 Semantic Search and Dense Retrieval

Dense retrieval methods (Karpukhin et al., 2020) represent documents and queries as embeddings, measuring relevance through vector similarity. Systems like FAISS (Johnson et al., 2019) enable efficient nearest-neighbor search over billions of vectors.

Our approach inverts this paradigm: rather than the user formulating a query vector explicitly, their spatial position implicitly defines one. The “query” is never typed—it is walked.

2.3 3D Information Spaces

The concept of spatial information interfaces dates to Mackinlay et al. (1991) and the Information Visualizer. Robertson et al. (1998) explored “Data Mountain” for spatial document organization. More recently, VR systems have enabled immersive data exploration (Millais et al., 2018).

Our work differs in grounding spatial organization in semantic embeddings rather than arbitrary or user-defined layouts. The space has meaning because it derives from meaning.

2.4 Embodied Cognition in HCI

Dourish (2001) argued for “embodied interaction” in computing. Jacob et al. (2008) formalized “reality-based interaction” leveraging human spatial and physical skills. Our system operationalizes these principles for information retrieval.

---

3. System Architecture

3.1 Overview

SBCQ comprises five interconnected subsystems:

┌─────────────────────────────────────────────────────────────────┐
│                        SBCQ Architecture                        │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────────┐    ┌──────────────┐    ┌─────────────────┐   │
│  │  Embedding  │───▶│   Spatial    │───▶│   3D Renderer   │   │
│  │   Manager   │    │   Mapper     │    │  (Three.js/R3F) │   │
│  └─────────────┘    └──────────────┘    └─────────────────┘   │
│         │                  │                     │             │
│         ▼                  ▼                     ▼             │
│  ┌─────────────┐    ┌──────────────┐    ┌─────────────────┐   │
│  │  Distance   │    │   Position   │    │    Player       │   │
│  │ Calculator  │◀───│   Tracker    │◀───│   Controller    │   │
│  └─────────────┘    └──────────────┘    └─────────────────┘   │
│         │                                        │             │
│         └────────────────┬───────────────────────┘             │
│                          ▼                                     │
│                 ┌──────────────────┐                          │
│                 │  Query Engine    │                          │
│                 │  (Continuous)    │                          │
│                 └──────────────────┘                          │
│                          │                                     │
│                          ▼                                     │
│                 ┌──────────────────┐                          │
│                 │  Metrics &       │                          │
│                 │  Validation      │                          │
│                 └──────────────────┘                          │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

3.2 Embedding Manager

The EmbeddingManager class handles all operations on high-dimensional vectors:

class EmbeddingManager {
  private embeddings: Map<string, number[]>
  private config: { dimensions: number, distanceMetric: 'euclidean' | 'cosine' }

  cosineDistance(a: number[], b: number[]): number {
    let dotProduct = 0, normA = 0, normB = 0
    for (let i = 0; i < a.length; i++) {
      dotProduct += a[i] * b[i]
      normA += a[i] * a[i]
      normB += b[i] * b[i]
    }
    return 1 - (dotProduct / (Math.sqrt(normA) * Math.sqrt(normB)))
  }

  findKNearest(query: number[], k: number): { id: string, distance: number }[]
  interpolate(a: number[], b: number[], t: number): number[]
  centroid(embeddings: number[][]): number[]
}

Design Decisions:

• Cosine distance as default metric, standard for semantic embeddings
• Lazy k-NN computation—acceptable for proof-of-concept scale
• Linear interpolation for query embedding generation

3.3 Spatial Mapping

Converting 64-dimensional embeddings to 3D coordinates requires dimensionality reduction. We implement a cluster-aware projection:

function projectToSpace(concepts: Concept[]): Map<string, Vector3> {
  // Pre-defined cluster anchor positions
  const clusterPositions: Record<string, Vector3> = {
    justice: new Vector3(-2, 0, -2),
    knowledge: new Vector3(2, 0, -2),
    emotion: new Vector3(0, 0, 2),
    // ... additional clusters
  }

  concepts.forEach((c) => {
    const clusterBase = clusterPositions[c.cluster]
    // Add intra-cluster variation from embedding components
    const variation = new Vector3(
      c.embedding[0] * 3,
      1 + Math.abs(c.embedding[1]) * 2,
      c.embedding[2] * 3
    )
    positions.set(c.id, clusterBase.clone().add(variation))
  })
}

Projection Properties:

• Inter-cluster separation: Semantic clusters occupy distinct spatial regions
• Intra-cluster structure: Concepts within clusters arranged by embedding similarity
• Deterministic: Same embeddings always produce same positions

3.4 Real-Time Rendering

The 3D environment uses React Three Fiber (R3F), a React renderer for Three.js:

Scene Composition:

• Concept Spheres: Each concept rendered as a sphere at its projected position
• Cluster Regions: Semi-transparent ground highlights indicating semantic domains
• Connection Lines: Visual links between semantically related concepts
• Dynamic Lighting: Ambient color shifts based on current semantic region

Performance Optimizations:

• Reduced particle count (50 vs. initial 300)
• Simplified materials (meshBasicMaterial where possible)
• Frustum culling for off-screen concepts
• Throttled state updates (60fps render, 30fps state)

3.5 Continuous Query Engine

The core innovation: position-based query interpolation.

function interpolateQueryEmbedding(): number[] {
  const nearby = findConceptsInRadius(userPosition, QUERY_RADIUS)
  
  let totalWeight = 0
  const weighted = Array(DIMENSIONS).fill(0)
  
  nearby.forEach((concept) => {
    const distance = userPosition.distanceTo(concept.position)
    const weight = 1 / (distance + SMOOTHING_FACTOR)
    totalWeight += weight
    
    concept.embedding.forEach((v, i) => {
      weighted[i] += v * weight
    })
  })
  
  return weighted.map((v) => v / totalWeight)
}

Algorithm Properties:

• Inverse-distance weighting: Closer concepts contribute more to the query
• Smooth interpolation: Position changes produce smooth embedding changes
• Bounded influence: Only concepts within QUERY_RADIUS (15 units) contribute

3.6 State Management

Application state managed via Zustand, a minimal reactive store:

interface EmbeddingState {
  concepts: ConceptNode[]
  clusters: Map<string, ConceptNode[]>
  userState: {
    position: Vector3
    velocity: Vector3
    lookDirection: Vector3
    trajectory: Vector3[]
  }
  currentRegion: string | null
  nearbyConceptIds: string[]
  selectedConcept: ConceptNode | null
}

State Flow:

1. Player movement updates userState.position
2. Frame callback computes nearbyConceptIds
3. Dominant cluster determines currentRegion
4. UI components reactively render updates

---

4. Methodology

4.1 Correlation Metrics

To validate that spatial proximity reflects semantic similarity, we compute Kendall’s tau-like concordance:

function computeCorrelation(nearby: Concept[], queryEmbedding: number[]): number {
  // Sort by spatial distance
  const bySpatial = nearby.sort((a, b) => 
    userPosition.distanceTo(a.position) - userPosition.distanceTo(b.position)
  )
  
  // Sort by embedding similarity
  const byEmbedding = [...nearby].sort((a, b) =>
    embeddingManager.cosineDistance(queryEmbedding, a.embedding) -
    embeddingManager.cosineDistance(queryEmbedding, b.embedding)
  )
  
  // Count concordant pairs
  let concordant = 0, total = 0
  for (let i = 0; i < nearby.length; i++) {
    for (let j = i + 1; j < nearby.length; j++) {
      const spatialOrder = bySpatial.indexOf(nearby[i]) < bySpatial.indexOf(nearby[j])
      const embeddingOrder = byEmbedding.indexOf(nearby[i]) < byEmbedding.indexOf(nearby[j])
      if (spatialOrder === embeddingOrder) concordant++
      total++
    }
  }
  
  return (concordant / total) * 100
}

Interpretation:

• 100%: Perfect agreement—spatial order matches semantic order
• 50%: Random—no relationship
• 0%: Perfect disagreement—spatial order inverts semantic order

4.2 Cluster Coherence

We measure what percentage of nearby concepts share the user’s current semantic cluster:

Cluster Coherence = |{c ∈ Nearby : cluster(c) = CurrentRegion}| / |Nearby|

High coherence (>60%) indicates the projection successfully groups related concepts.

4.3 Sampling Protocol

The MetricsDashboard component samples metrics every 500ms:

useEffect(() => {
  const interval = setInterval(() => {
    const correlation = computeCorrelation(nearby, interpolateQuery())
    const coherence = nearby.filter(c => c.cluster === currentRegion).length / nearby.length
    
    samples.push({ timestamp: Date.now(), correlation, coherence, region: currentRegion })
  }, 500)
}, [nearby, currentRegion])

This enables visualization of correlation over time as users explore.

---

5. Implementation

5.1 Technology Stack

5.2 Data Model

Concept Node:

interface ConceptNode {
  id: string                    // Unique identifier
  label: string                 // Human-readable name
  embedding: number[]           // 64-dimensional vector
  position: Vector3             // 3D world position
  color: string                 // Cluster-derived color
  size: number                  // Visual prominence
  cluster?: string              // Semantic category
  neighbors: string[]           // k-NN by embedding distance
}

Sample Data:
The proof-of-concept includes 50 concepts across 8 clusters:

• Justice: Justice, Fairness, Equality, Law, Rights, Court, Verdict, Mercy
• Knowledge: Knowledge, Wisdom, Learning, Truth, Discovery, Science, Understanding
• Emotion: Love, Joy, Fear, Anger, Sadness, Hope, Empathy
• Nature: Forest, Ocean, Mountain, River, Sky, Earth
• Technology: Computer, Internet, AI, Robot, Data, Algorithm
• Society: Community, Culture, Democracy, Freedom, Family, Tradition
• Art: Beauty, Creativity, Music, Painting, Poetry, Expression
• Philosophy: Existence, Consciousness, Meaning, Ethics, Reality, Time

5.3 Embedding Generation

For the proof-of-concept, we generate pseudo-embeddings with controlled cluster structure:

function generatePseudoEmbedding(seed: string, clusterCenter: number[], variance: number = 0.3): number[] {
  const hash = seed.split('').reduce((acc, char) => acc + char.charCodeAt(0), 0)
  const rng = (i: number) => {
    const x = Math.sin(hash + i * 9999) * 10000
    return x - Math.floor(x)
  }
  
  return clusterCenter.map((center, i) => {
    const noise = (rng(i) - 0.5) * variance * 2
    return center + noise
  })
}

Properties:

• Deterministic: Same seed produces same embedding
• Clustered: Concepts share cluster center with added noise
• Distinguishable: Variance ensures within-cluster differentiation

5.4 User Interface Components

IntroScreen: Four-slide explainer introducing the thesis before navigation begins.

GuidedTour: Automated navigation through each cluster with contextual explanations.

ResearchUI: Real-time display of:

• Current position coordinates
• Nearby concept count
• Interpolated query vector visualization
• Spatial-semantic correlation percentage

MetricsDashboard: Time-series visualization of:

• Correlation over exploration history
• Cluster coherence trends
• Regional visit frequency

ComparisonView: Side-by-side demonstration of traditional keyword search versus spatial navigation.

---

6. Evaluation

6.1 Spatial-Semantic Correlation

We measured Kendall-tau correlation across extended exploration sessions:

Interpretation: Strong correlation within clusters confirms that intra-cluster positioning reflects semantic gradients. Boundary regions show expected degradation as users move between semantic domains.

6.2 Cluster Coherence

Observation: Philosophy shows lower coherence, consistent with its central position connecting multiple domains.

6.3 Query Equivalence

We compared results from:

1. Spatial Query: Concepts within radius of user position, sorted by distance
2. Explicit Query: k-NN search using interpolated embedding

Result: High overlap (73% average) between spatial and explicit queries validates the continuous query formulation.

---

7. Discussion

7.1 Theoretical Implications

Our results support the thesis that spatial navigation can serve as a query mechanism. The >70% correlation demonstrates that the information lost in dimensionality reduction does not destroy the semantic structure relevant to human exploration.

This suggests a new interaction paradigm: queries without query formulation. Users need not know terminology—they explore until they find what they recognize.

7.2 Design Principles Discovered

Through iterative development, we identified key principles for spatial-semantic systems:

1. Cluster Anchoring: Pre-positioning cluster centers creates navigable structure
2. Smooth Gradients: Inverse-distance weighting produces intuitive interpolation
3. Visual Feedback: Real-time correlation display builds user confidence
4. Bounded Influence: Query radius prevents distant concepts from dominating

7.3 Limitations

Scalability: The current implementation handles ~50 concepts. Scaling to thousands requires:

• Spatial indexing (octrees, BVH)
• Level-of-detail rendering
• Approximate nearest neighbor for embedding search

Embedding Quality: Pseudo-embeddings demonstrate the principle but lack true semantic grounding. Production systems should use embeddings from language models (e.g., sentence-transformers, OpenAI embeddings).

Evaluation Scope: We measure correlation, not user task performance. Future work should include:

• Information finding tasks
• Serendipitous discovery measurement
• Comparison with baseline search interfaces

7.4 Future Directions

1. Integration with Real Embeddings: Connect to sentence-transformer models for live semantic spaces
2. Multi-scale Navigation: Zoom between corpus-level, document-level, and passage-level views
3. Collaborative Exploration: Multiple users navigating shared spaces
4. Adaptive Projection: Learning projections that optimize for human navigation patterns
5. VR/AR Implementation: Full immersion via headset interfaces

---

8. Conclusion

We have presented SBCQ, a system demonstrating that spatial behaviour can serve as continuous query over high-dimensional embedding spaces. By projecting semantic embeddings into navigable 3D environments, we enable exploration-based information discovery that requires no explicit query formulation.

Our evaluation shows >70% correlation between spatial proximity and semantic similarity, validating that the essential structure of embedding spaces survives projection to human-navigable dimensions. The position of a user in our system is, mathematically and experientially, a query.

This work opens new directions for human-information interaction. As embedding models become ubiquitous—encoding text, images, code, and multimodal content—spatial interfaces offer an intuitive means of traversing these spaces. The future of search may not be typing queries but walking through knowledge.

“Movement through space is equivalent to traversal through semantic meaning.”

---

References

Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information Science, 5, 133-143.

Dourish, P. (2001). Where the Action Is: The Foundations of Embodied Interaction. MIT Press.

Godden, D. R., & Baddeley, A. D. (1975). Context-dependent memory in two natural environments: On land and underwater. British Journal of Psychology, 66(3), 325-331.

Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008). Reality-based interaction: a framework for post-WIMP interfaces. CHI ’08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 201-210.

Johnson, J., Douze, M., & Jégou, H. (2019). Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 535-547.

Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W. (2020). Dense passage retrieval for open-domain question answering. EMNLP 2020.

Mackinlay, J. D., Robertson, G. G., & Card, S. K. (1991). The perspective wall: Detail and context smoothly integrated. CHI ’91: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 173-176.

McInnes, L., Healy, J., & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv preprint arXiv:1802.03426.

Millais, P., Jones, S. L., & Kelly, R. (2018). Exploring data in virtual reality: Comparisons with 2D data visualizations. CHI ’18 Extended Abstracts, 1-6.

Robertson, G., Czerwinski, M., Larson, K., Robbins, D. C., Thiel, D., & van Dantzich, M. (1998). Data mountain: using spatial memory for document management. UIST ’98: Proceedings of the 11th Annual ACM Symposium on User Interface Software and Technology, 153-162.

Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9(86), 2579-2605.

---

Appendix A: System Requirements

Minimum:

• Modern browser with WebGL 2.0 support
• 4GB RAM
• Dedicated GPU recommended

Tested Configurations:

• Chrome 120+ on macOS/Windows/Linux
• Firefox 121+ on macOS/Windows/Linux
• Safari 17+ on macOS

---

Appendix B: Running the System

# Clone repository
git clone https://github.com/kossisoroyce/sbcq.git
cd sbcq

# Install dependencies
npm install

# Start development server
npm run dev

# Open browser to http://localhost:5173

Controls:

• Click — Lock pointer for navigation
• WASD — Move through space
• Mouse — Look around
• Shift — Sprint
• P — Toggle presentation mode
• M — Toggle metrics dashboard
• C — Toggle comparison view

---

Appendix C: Source Code Structure

src/
├── App.tsx                    # Main application component
├── main.tsx                   # Entry point
├── index.css                  # Global styles
├── components/
│   ├── EmbeddingWorld.tsx     # 3D scene and concept rendering
│   ├── Player.tsx             # First-person navigation controller
│   ├── ConceptSphere.tsx      # Individual concept visualization
│   ├── ConnectionLines.tsx    # Semantic relationship lines
│   ├── ResearchUI.tsx         # Main research interface
│   ├── MetricsDashboard.tsx   # Real-time metrics display
│   ├── IntroScreen.tsx        # Onboarding slides
│   ├── GuidedTour.tsx         # Automated cluster tour
│   └── ComparisonView.tsx     # Search paradigm comparison
├── lib/
│   ├── embeddingManager.ts    # Embedding operations
│   └── spatialMapper.ts       # Projection utilities
├── store/
│   └── embeddingStore.ts      # Global state management
└── data/
    └── sampleConcepts.ts      # Concept definitions and embeddings

---

This paper accompanies the SBCQ proof-of-concept implementation. For questions or collaboration inquiries, please open an issue on the GitHub repository.

Live Demo: https://sbcq.netlify.app
GitHub: https://github.com/kossisoroyce/sbcq