Zum Inhalt

Overview

package wittgenstein_embeddings

Classes

  • EmbeddingClient — Query/passage embedding interface (the E5 asymmetric-encoding contract).

  • GeminiEmbedder — Gemini embeddings via google-genai, using its native query/document task types — RETRIEVAL_QUERY / RETRIEVAL_DOCUMENT — instead of a text prefix convention.

  • LocalEmbedder — Sentence-transformers embedder — auto-downloads from HuggingFace, CPU by default.

  • OllamaEmbeddingClient — Sync HTTP client for Ollama /api/embeddings — symmetric provider.

  • OpenAIEmbeddingClient — OpenAI embeddings — symmetric provider (query == passage).

wittgenstein_embeddings.EmbeddingClient

mkapi_definition_mkapi class EmbeddingClient()

Bases : ABC

Query/passage embedding interface (the E5 asymmetric-encoding contract).

intfloat/multilingual-e5-* models require literal "query: "/"passage: " prefixes depending on which side of a retrieval pair is embedded. Symmetric providers (OpenAI, Gemini, Ollama) implement both methods identically — callers never need to know which kind of model backs the instance.

Attributes

  • dimension : int — Embedding width.

Methods

wittgenstein_embeddings.EmbeddingClient.query

mkapi_definition_mkapi method EmbeddingClient.query(text: str) → list[float]

wittgenstein_embeddings.EmbeddingClient.passage

mkapi_definition_mkapi method EmbeddingClient.passage(text: str) → list[float]

wittgenstein_embeddings.EmbeddingClient.query_batch

mkapi_definition_mkapi method EmbeddingClient.query_batch(texts: list[str]) → list[list[float]]

Default: sequential calls. Providers with real batch APIs override this.

wittgenstein_embeddings.EmbeddingClient.passage_batch

mkapi_definition_mkapi method EmbeddingClient.passage_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.EmbeddingClient.dimension

mkapi_definition_mkapi property EmbeddingClient.dimension: int

Embedding width.

Static where the SDK exposes it (sentence-transformers' get_sentence_embedding_dimension()); lazily probed via a one-off query() call otherwise.

wittgenstein_embeddings.GeminiEmbedder

mkapi_definition_mkapi class GeminiEmbedder(api_key: str, model: str = _DEFAULT_MODEL, output_dimensionality: int | None = None)

Bases : EmbeddingClient

Gemini embeddings via google-genai, using its native query/document task types — RETRIEVAL_QUERY / RETRIEVAL_DOCUMENT — instead of a text prefix convention.

Methods

wittgenstein_embeddings.GeminiEmbedder.query

mkapi_definition_mkapi method GeminiEmbedder.query(text: str) → list[float]

wittgenstein_embeddings.GeminiEmbedder.passage

mkapi_definition_mkapi method GeminiEmbedder.passage(text: str) → list[float]

wittgenstein_embeddings.GeminiEmbedder.query_batch

mkapi_definition_mkapi method GeminiEmbedder.query_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.GeminiEmbedder.passage_batch

mkapi_definition_mkapi method GeminiEmbedder.passage_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.GeminiEmbedder.dimension

mkapi_definition_mkapi property GeminiEmbedder.dimension: int

wittgenstein_embeddings.LocalEmbedder

mkapi_definition_mkapi class LocalEmbedder(model_name: str = 'intfloat/multilingual-e5-base', device: str = 'cpu', query_prefix: str = 'query: ', passage_prefix: str = 'passage: ', dtype: str | None = None)

Bases : EmbeddingClient

Sentence-transformers embedder — auto-downloads from HuggingFace, CPU by default.

No new caching layer: relies on sentence-transformers' own reuse of HF's default cache dir (~/.cache/huggingface), same as this platform's GPU worker (apps/_platform/workers/python_gpu/worker.py) already does.

device defaults to "cpu" and is never auto-detected here — this is a generic shared lib, not a GPU-provisioned worker; a caller who wants CUDA computes "cuda" if torch.cuda.is_available() else "cpu" and passes it.

dtype defaults to None (framework default, float32). Pass "bfloat16" to roughly halve resident memory — NOT "float16": plain fp16 has poor native CPU support in PyTorch and can be slower than fp32 there, while bfloat16 is well-supported on CPU since PyTorch 1.10+. GPU callers may still want float16 explicitly; this default just avoids the CPU footgun.

Methods

wittgenstein_embeddings.LocalEmbedder.query

mkapi_definition_mkapi method LocalEmbedder.query(text: str) → list[float]

wittgenstein_embeddings.LocalEmbedder.passage

mkapi_definition_mkapi method LocalEmbedder.passage(text: str) → list[float]

wittgenstein_embeddings.LocalEmbedder.query_batch

mkapi_definition_mkapi method LocalEmbedder.query_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.LocalEmbedder.passage_batch

mkapi_definition_mkapi method LocalEmbedder.passage_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.LocalEmbedder.dimension

mkapi_definition_mkapi property LocalEmbedder.dimension: int

wittgenstein_embeddings.OllamaEmbeddingClient

mkapi_definition_mkapi class OllamaEmbeddingClient(host: str, model: str, timeout_seconds: float = 60.0)

Bases : EmbeddingClient

Sync HTTP client for Ollama /api/embeddings — symmetric provider.

No batch endpoint on the Ollama side (/api/embeddings takes one prompt), so passage_batch stays a sequential loop, same as before.

Methods

wittgenstein_embeddings.OllamaEmbeddingClient.query

mkapi_definition_mkapi method OllamaEmbeddingClient.query(text: str) → list[float]

wittgenstein_embeddings.OllamaEmbeddingClient.passage

mkapi_definition_mkapi method OllamaEmbeddingClient.passage(text: str) → list[float]

wittgenstein_embeddings.OllamaEmbeddingClient.dimension

mkapi_definition_mkapi property OllamaEmbeddingClient.dimension: int

wittgenstein_embeddings.OpenAIEmbeddingClient

mkapi_definition_mkapi class OpenAIEmbeddingClient(api_key: str, model: str = 'text-embedding-3-small', base_url: str = _DEFAULT_BASE_URL)

Bases : EmbeddingClient

OpenAI embeddings — symmetric provider (query == passage).

base_url defaults to the real OpenAI endpoint explicitly, rather than letting the openai SDK silently inherit OPENAI_BASE_URL — a gateway routing hazard already worked around by this platform's other direct OpenAI embedding call (a knowledge-graph indexer).

Methods

wittgenstein_embeddings.OpenAIEmbeddingClient.query

mkapi_definition_mkapi method OpenAIEmbeddingClient.query(text: str) → list[float]

wittgenstein_embeddings.OpenAIEmbeddingClient.passage

mkapi_definition_mkapi method OpenAIEmbeddingClient.passage(text: str) → list[float]

wittgenstein_embeddings.OpenAIEmbeddingClient.query_batch

mkapi_definition_mkapi method OpenAIEmbeddingClient.query_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.OpenAIEmbeddingClient.passage_batch

mkapi_definition_mkapi method OpenAIEmbeddingClient.passage_batch(texts: list[str]) → list[list[float]]

wittgenstein_embeddings.OpenAIEmbeddingClient.dimension

mkapi_definition_mkapi property OpenAIEmbeddingClient.dimension: int