Ir para o conteúdo

Overview

package wittgenstein_schema_discovery

wittgenstein-schema-discovery — DDL + documentation catalog ingestion and semantic retrieval.

Classes

  • CatalogIngester — Ingests catalog YAML and glossary YAML into the pgvector 'ddl' and 'documentation' collections.

  • SchemaRetriever — Retrieves DDL and documentation from pgvector for a natural-language question.

Functions

  • build_column_doc — Build a one-line column description for documentation indexing.

  • build_ddl — Build a CREATE TABLE DDL string for the given table and columns.

  • build_glossary_entry — Build a one-line glossary entry for documentation indexing.

  • build_table_doc — Build a one-line table description for documentation indexing.

wittgenstein_schema_discovery.build_column_doc

mkapi_definition_mkapi build_column_doc(table_name: str, col_name: str, info: dict[str, Any]) → str

Build a one-line column description for documentation indexing.

wittgenstein_schema_discovery.build_ddl

mkapi_definition_mkapi build_ddl(table_name: str, columns: dict[str, Any]) → str

Build a CREATE TABLE DDL string for the given table and columns.

Parameters

  • table_name : str — PostgreSQL table name.

  • columns : dict[str, Any] — dict of {column_name: {type, ...}} from catalog.yaml.

Returns

  • str — DDL string suitable for indexing into pgvector 'ddl' collection.

wittgenstein_schema_discovery.build_glossary_entry

mkapi_definition_mkapi build_glossary_entry(prefix: str, key: str, body: dict[str, Any]) → str

Build a one-line glossary entry for documentation indexing.

wittgenstein_schema_discovery.build_table_doc

mkapi_definition_mkapi build_table_doc(table_name: str, meta: dict[str, Any]) → str

Build a one-line table description for documentation indexing.

wittgenstein_schema_discovery.CatalogIngester

mkapi_definition_mkapi class CatalogIngester(connection_string: str, pg_dsn: str, embedder: Embeddings)

Ingests catalog YAML and glossary YAML into the pgvector 'ddl' and 'documentation' collections.

Each call to ingest_catalog() or ingest_glossary() appends to the existing collections. Call clear_collections() before re-ingesting to avoid duplicate embeddings.

Parameters

  • connection_string : str — psycopg3 URL for LangChain PGVector (e.g. 'postgresql+psycopg://user:pass@host/db').

  • pg_dsn : str — psycopg2 DSN for the clear_collections direct-SQL operation (e.g. 'postgresql://user:pass@host:port/db').

  • embedder : Embeddings — any LangChain Embeddings instance.

Methods

  • ingest_catalog — Index DDL + table/column/FK docs from catalog.yaml.

  • ingest_glossary — Index all glossary entries (phases, typologies, terms) from domain_glossary.yaml.

  • clear_collections — Delete all embeddings from the 'sql', 'ddl', and 'documentation' collections.

wittgenstein_schema_discovery.CatalogIngester.ingest_catalog

mkapi_definition_mkapi method CatalogIngester.ingest_catalog(yaml_path: Path, db_tables: set[str]) → dict[str, int]

Index DDL + table/column/FK docs from catalog.yaml.

Only tables that appear in db_tables are indexed (skips tables absent from the actual DB to avoid confusing the agent).

Parameters

  • yaml_path : Path — path to catalog.yaml.

  • db_tables : set[str] — set of table names that exist in the target DB (from wittgenstein_sql_runner.discover_db_tables()).

Returns

  • Dict with counts — {'ddl', 'doc_table', 'doc_col', 'fk', 'skipped'}.

wittgenstein_schema_discovery.CatalogIngester.ingest_glossary

mkapi_definition_mkapi method CatalogIngester.ingest_glossary(yaml_path: Path) → int

Index all glossary entries (phases, typologies, terms) from domain_glossary.yaml.

Parameters

  • yaml_path : Path — path to domain_glossary.yaml.

Returns

  • int — Number of entries indexed.

wittgenstein_schema_discovery.CatalogIngester.clear_collections

mkapi_definition_mkapi method CatalogIngester.clear_collections() → int

Delete all embeddings from the 'sql', 'ddl', and 'documentation' collections.

Returns the count of removed rows. Returns 0 if LangChain tables don't exist yet.

wittgenstein_schema_discovery.SchemaRetriever

mkapi_definition_mkapi class SchemaRetriever(connection_string: str, embedder: Embeddings)

Retrieves DDL and documentation from pgvector for a natural-language question.

Parameters

  • connection_string : str — psycopg3 URL for LangChain PGVector (e.g. 'postgresql+psycopg://user:pass@host/db').

  • embedder : Embeddings — any LangChain Embeddings instance.

Methods

mkapi_definition_mkapi method SchemaRetriever.get_related_ddl(question: str, k: int = 10) → list[str]

Return the k most relevant DDL strings for the question.

Parameters

  • question : str — natural-language question or table name.

  • k : int — maximum number of DDL entries to return.

Returns

  • list[str] — List of DDL strings (CREATE TABLE ...).

mkapi_definition_mkapi method SchemaRetriever.get_related_documentation(question: str, k: int = 10) → list[str]

Return the k most relevant documentation entries for the question.

Parameters

  • question : str — natural-language question or column/table name.

  • k : int — maximum number of documentation entries to return.

Returns

  • list[str] — List of documentation strings.