Skip to content

Declare column types at runtime (backend-neutral) and add a parameterized query API before v1.0 #28

Open

Description

@yancya

Background

Relationizer needs column types at runtime, the moment it builds the SQL. Today the only runtime type source is the tuple values themselves (plus optional backend-specific symbols in the schema hash). That is the root of the library's recurring pain points:

  • empty tuples and all-nil columns carry zero type information (TypeNotFoundError / ReasonlessTypeError)
  • mixed Integer/Float columns fail inference
  • the schema vocabulary is backend-specific (:INT64 vs :INT8 vs :BIGINT), so the same schema can't be reused across backends

Separately, values are hand-escaped into SQL strings. Every escaping rule is a place for injection-class bugs to hide; #21 and the BigQuery single-quote no-op (511412d) were both real ones. All three databases can take the same data as bound parameters instead:

  • BigQuery: UNNEST(@rows) with an ARRAY<STRUCT<...>> query parameter
  • PostgreSQL: jsonb_to_recordset($1) (or unnest($1::int8[], ...))
  • MySQL: JSON_TABLE(?, ...), since the backend already builds a JSON document

Static typing (RBS) was considered as the type source and rejected for that role: RBS is gone at runtime, and reading sig files at runtime would mean depending on the rbs gem. RBS still fits as a static check layered on top.

Direction

  1. Explicit runtime type declaration with a backend-neutral vocabulary (e.g. :integer, :float, :decimal, :string, :boolean, :date, :timestamp, [:array, :integer]). Each backend maps it to its own SQL type. Value-based inference stays as the fallback when a type is omitted.
  2. Accept Data classes as row input (Data.define(:id, :name)) alongside Array<Array>. Column order and names come from the Data members.
  3. Parameterized API: a new method (working name create_relation_query) returning [sql, params] for each backend. create_relation_literal stays as the convenience string API.
  4. Ship sig/relationizer.rbs covering the public API (both methods, error classes, the type vocabulary as a type alias).

Prerequisites

  • None to start. Must be decided and merged before Cut v1.0.0 (API freeze) #25 (v1.0.0 API freeze), because adding it afterwards changes the frozen surface.
  • Can land after Cut v0.5.0 #24 (v0.5.0) or be part of it. That is a human call.

Open questions (human decision)

  • Final names of the neutral type symbols, and whether backend-specific symbols (:INT64 etc.) stay accepted as an escape hatch
  • Method name for the parameterized API, and the shape of params per backend (positional array for PG/MySQL vs. a named hash for BigQuery)
  • Whether to add callable module functions (Relationizer::MySQL.create_relation_query(...)) in addition to include
  • Split into sub-issues per step above, or ship as one

Acceptance criteria

  • Neutral type vocabulary implemented in all three backends, with a mapping table in README
  • Empty tuples and all-nil columns work without errors when types are declared neutrally
  • Data instances accepted as rows; mismatched members vs. schema raise a gem-level error
  • create_relation_query implemented for all three backends; the generated SQL contains no user values (only placeholders)
  • sig/relationizer.rbs added and steep check (or rbs validate) passes in CI
  • test/consistency_test.rb extended to cover the neutral types and the parameterized API across backends
  • All new behavior done TDD; bundle exec rake green on the CI matrix

Verification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions