Raw SQL
When no shortcut writes the query you need (a join, a window function, an INSERT … SELECT), write it in the sql template, and keep every value that comes from outside out of its text. You already know the shortcuts from the tutorial; every interpolation is in SQL template.
Write the query, its names interpolated
Write the SQL as you would in psql, but put each table and column name between ${…} instead of in the text. Give sql the tables the query may name, then the type of the rows you expect:
TSYou write Open in playground
ts
type Tables = s.films.SQL | s.screenings.SQL | s.cinemas.SQL;
type Listing = Pick<s.films.Selectable, 'title'> & Pick<s.screenings.Selectable, 'starts_at'>;
function listings(cinema: string) {
return db.sql<Tables, Listing[]>`SELECT ${'films'}.${'title'}, ${'screenings'}.${'starts_at'}
FROM ${'screenings'}
JOIN ${'films'} ON ${'films'}.${'id'} = ${'screenings'}.${'film_id'}
JOIN ${'cinemas'} ON ${'cinemas'}.${'id'} = ${'screenings'}.${'cinema_id'}
WHERE ${'cinemas'}.${'name'} = ${db.param(cinema)}
ORDER BY ${'screenings'}.${'starts_at'}`;
}
const listed = await listings('The Lantern').run(pool);SQLPostgres runs
sql
SELECT "films"."title", "screenings"."starts_at"
FROM "screenings"
JOIN "films" ON "films"."id" = "screenings"."film_id"
JOIN "cinemas" ON "cinemas"."id" = "screenings"."cinema_id"
WHERE "cinemas"."name" = $1
ORDER BY "screenings"."starts_at"
values ["The Lantern"]
::TypeScript knows
ts
const listed: Listing[]
// that is
const listed: {
title: string;
starts_at: Date;
}[]
- An interpolated name is quoted,
"films", and checked against the first type argument: hereTables, theSQLtype of each table the query names. Qualify a column as${'films'}.${'title'}: the type knows each table's columns, not'films.title'. - The second type argument is your word for what comes back: nothing checks it against the SQL. The rows come as
pgparses them, as aSelectabledescribes them (aDatefor atimestamptz), not as the JSON of the shortcuts'JSONSelectable. Acount(*)is abigint, whichpggives as a string:'4'. - Keywords stay in the text. SQL you write yourself is safe there; only what comes from outside needs the steps below.
A name the type doesn't know fails to compile: a query that names screenings needs s.screenings.SQL in its first type argument.
ts
db.sql<s.films.SQL>`SELECT * FROM ${'screenings'}`;A write is written the same way. This INSERT … SELECT copies one film's tags to another, and RETURNING * gives back the rows it added:
TSYou write Open in playground
ts
function copyTags(fromFilm: number, toFilm: number) {
const columns: s.tags.Column[] = ['film_id', 'tag'];
return db.sql<s.tags.SQL, s.tags.Selectable[]>`INSERT INTO ${'tags'} (${db.cols(columns)})
SELECT ${db.param(toFilm)}, ${'tag'} FROM ${'tags'} WHERE ${{ film_id: fromFilm }}
ON CONFLICT DO NOTHING RETURNING *`;
}
const copied = await copyTags(1000, 1003).run(pool);SQLPostgres runs
sql
INSERT INTO "tags" ("film_id", "tag")
SELECT $1, "tag"
FROM "tags"
WHERE ("film_id" = $2)
ON CONFLICT DO NOTHING
RETURNING *
values [1003, 1000]
::TypeScript knows
ts
const copied: s.tags.Selectable[]
// that is
const copied: {
film_id: number;
tag: string;
}[]
db.cols()writes the column list,"film_id", "tag".columns, typeds.tags.Column[], checks the names:db.cols()alone takes any, so'tagz'would compile, and Postgres refuse it (42703).ON CONFLICT DO NOTHINGskips a tag the film has already. Without it, Postgres refuses the wholeINSERTwith 23505, and adds none.
Pass each value as a parameter
A value from a caller (a form, a URL, a row you read elsewhere) goes in as db.param(), or in a Whereable object, which makes a parameter of each value:
TSYou write Open in playground
ts
function studioFilms(studioId: number, genre: string) {
return db.sql<s.films.SQL, s.films.Selectable[]>`SELECT * FROM ${'films'}
WHERE ${{ studio_id: studioId }} AND ${db.param(genre)} = ANY(${'genres'})`;
}
const dramas = await studioFilms(1000, 'drama').run(pool);SQLPostgres runs
sql
SELECT *
FROM "films"
WHERE ("studio_id" = $1) AND $2 = ANY("genres")
values [1000, "drama"]
::TypeScript knows
ts
const dramas: s.films.Selectable[]
// that is
const dramas: {
genres: string[];
id: number;
release_year: number;
runtime_minutes: number;
studio_id: number;
tagline: string | null;
title: string;
}[]
${{ studio_id: studioId }}is("studio_id" = $1): a filter offilms, as thewhereofselect()takes it, so conditions work in it too:gte()in{ release_year: db.conditions.gte(1990) }.${db.param(genre)}is$2, in$2 = ANY("genres"): a parameter fits wherever SQL takes a value.- An array is one parameter:
${'id'} = ANY(${db.param(ids)})sendsidsas a Postgres array by default; withcastArrayParamsToJsonon, it does not (below). - A part of the query that only some calls need is a nested fragment, or
[]for nothing (Nested fragments).
A bare string between ${…} is a name, not a value, so the type refuses it:
ts
function filmTitled(title: string) {
return db.sql<s.films.SQL>`SELECT * FROM ${'films'} WHERE ${'title'} = ${title}`;}If it got past the type (an any, a @ts-ignore), it would still be a name, and the caller would pick it:
- A real column matches every row:
'title'compiles to"title" = "title", true wherevertitleis not null, and so do'films.title'and'"title"'. In aDELETE, that deletes them. - An unknown one fails:
'The Salt Orchard'gives"title" = "The Salt Orchard", a column that does not exist (42703). - It can be any column in scope, a system column such as
xminincluded, and, where a table goes, any relation the role can read,'schema.table'included:FROM ${'pg_catalog.pg_roles'}lists the database's roles. - Any
"in it is doubled, so it can't close its quotes (in Zapatos 6.6.1 it could, Identifiers), but that does not make it a value.
The keys of an interpolated object are names too. The type refuses an unknown key only in an object literal (TS2353): an object built at run time (a Record<string, string>, Object.fromEntries(), a variable with extra keys) passes it with any key.
Security
An interpolated object that is not a param(), a fragment or another of pgcast's values compiles as a Whereable, a class instance included.
A built-in object throws while compiling, before any query: db.sql`DELETE FROM ${'ratings'} WHERE ${when}`, with when a Date typed any, no longer compiles to DELETE FROM "ratings" WHERE TRUE, as in Zapatos 6.6.1, which deleted every rating (Whereable objects). Pass a date as db.param(when).
Any object with no keys is TRUE, a typed Whereable included: const where: s.ratings.Whereable = {}, given a film_id only if (filmId), compiles with no error to the same DELETE. Check for an empty filter before a DELETE or an UPDATE (Whereable objects). The check below catches {}, not a key whose value compiles to true, such as film_id: db.conditions.isNotIn([]):
ts
function removeRatings(filmId?: number) {
const where: s.ratings.Whereable = {};
if (filmId) where.film_id = filmId;
if (Object.keys(where).length === 0) throw new Error('No filter: that would delete every rating');
return db.deletes('ratings', where);
}Cast a parameter Postgres can't type
Postgres gives a parameter a type from where it stands. Compared with a column, it takes the column's type; elsewhere, Postgres may infer another. Give it the type as param()'s second argument:
TSYou write Open in playground
ts
function screeningsWithin(from: Date, hours: number) {
return db.sql<s.screenings.SQL, s.screenings.Selectable[]>`SELECT * FROM ${'screenings'}
WHERE ${'starts_at'} >= ${db.param(from)}
AND ${'starts_at'} < ${db.param(from, 'timestamptz')}
+ make_interval(hours => ${db.param(hours)})`;
}
const evening = await screeningsWithin(new Date('2030-03-01T18:00:00Z'), 4).run(pool);SQLPostgres runs
sql
SELECT *
FROM "screenings"
WHERE "starts_at" >= $1
AND "starts_at" < CAST($2 AS "timestamptz")
+ make_interval(hours => $3)
values ["2030-03-01T18:00:00.000Z", "2030-03-01T18:00:00.000Z", 4]
::TypeScript knows
ts
const evening: s.screenings.Selectable[]
// that is
const evening: {
cinema_id: number;
film_id: number;
id: number;
starts_at: Date;
subtitled: boolean | null;
}[]
$1, compared withstarts_atin"starts_at" >= $1, is atimestamptzalready.$2, inCAST($2 AS "timestamptz") + make_interval(hours => $3), is added to aninterval, so without its cast Postgres reads it as one, and fails:operator does not exist: timestamp with time zone < interval(42883).db.param(from, 'timestamptz')writesCAST($2 AS "timestamptz");${db.param(from)}::timestamptzin the text does the same.- The cast is the type's catalog name, in lower case (
int4,timestamptz,crew_job), each part quoted.'integer'givesCAST($1 AS "integer"), a type that does not exist (42704), as do'int'and'TIMESTAMPTZ'. - Name a schema as
'public.crew_job', which gives"public"."crew_job", as'public"."crew_job'does. An array type keeps its bounds outside the quotes:'int4[]'gives"int4"[]. - With
castArrayParamsToJsonon, an array parameter is sent as JSON,CAST($1 AS "json"), unless its cast ends with[]or[n]:db.param(ids, 'int4[]')givesCAST($1 AS "int4"[])with the array (#36). Any other string cast keeps the JSON, cast to that type,'_int4'included. - With
castObjectParamsToJsonon, a plain object is sent as JSON whatever its cast, cast to the type written.
Every form of the cast: param().
Choose SQL at run time from your own constants
Some choices are syntax, not values: a sort direction, the column to sort on. A parameter can't stand for them, so map the caller's choice to one you wrote:
TSYou write Open in playground
ts
function sortedFilms(by: string, direction: string) {
const sortable = ['title', 'release_year', 'runtime_minutes'] as const;
const column = sortable.find(c => c === by) ?? 'title';
return db.sql<s.films.SQL, s.films.Selectable[]>`SELECT * FROM ${'films'}
ORDER BY ${column} ${db.raw(direction === 'desc' ? 'DESC' : 'ASC')}`;
}
const longestFirst = await sortedFilms('runtime_minutes', 'desc').run(pool);SQLPostgres runs
sql
SELECT *
FROM "films"
ORDER BY "runtime_minutes" DESC
values []
::TypeScript knows
ts
const longestFirst: s.films.Selectable[]
// that is
const longestFirst: {
genres: string[];
id: number;
release_year: number;
runtime_minutes: number;
studio_id: number;
tagline: string | null;
title: string;
}[]
db.raw()writes its string as it is. Here it only ever gets'DESC'or'ASC', whatever the caller sent.- The column is a name: quoted, it can't close its quotes, but the caller's string could name any column in scope, a system column such as
xminincluded, or none (42703, or 42P01 for a table outside the query).find()keeps one of yours, typed as the template wants.
Security
Whatever reaches raw() runs as SQL, and its type takes any string. Give it a caller's text, and the caller writes your query:
ts
const title = "x' OR TRUE --"; // from the caller
db.sql`SELECT ${'title'} FROM ${'films'} WHERE ${'title'} = ${db.raw(`'${title}'`)}`;sql
SELECT "title" FROM "films" WHERE "title" = 'x' OR TRUE --'That returns every film. A value is a param(), always: ${db.param(title)} gives "title" = $1, and no film.
Compare a column with SQL, through self
A fragment as a Whereable value compares its column with SQL of your own, db.self standing for the column:
TSYou write Open in playground
ts
function filmsTitled(title: string) {
return db.sql<s.films.SQL, s.films.Selectable[]>`SELECT * FROM ${'films'}
WHERE ${{ title: db.sql`lower(${db.self}) = lower(${db.param(title)})` }}`;
}
const found = await filmsTitled('the salt orchard').run(pool);SQLPostgres runs
sql
SELECT *
FROM "films"
WHERE ((lower("title") = lower($1)))
values ["the salt orchard"]
::TypeScript knows
ts
const found: s.films.Selectable[]
// that is
const found: {
genres: string[];
id: number;
release_year: number;
runtime_minutes: number;
studio_id: number;
tagline: string | null;
title: string;
}[]
lower(${db.self})islower("title"), and the caller's text is still a parameter,lower($1).- An equality, unlike
ilike(): a_or a%in the caller's text matches only itself. - The same object works as the
whereofselect()and the other shortcuts. - Where it has no column to stand for,
selfthrows "The 'self' column alias has no meaning here", sorun()rejects before any query.
Reach the outer row, through parent()
A hand-written subquery can go in a shortcut's lateral, where db.parent() names a column of the outer query's table:
TSYou write Open in playground
ts
const studios = await db.select('studios', db.all, {
columns: ['name'],
lateral: {
titles: db.sql<s.films.SQL, string | null>`SELECT
string_agg(${'title'}, ', ' ORDER BY ${'release_year'}) AS result
FROM ${'films'} WHERE ${'studio_id'} = ${db.parent('id')}`,
},
}).run(pool);SQLPostgres runs
sql
SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
SELECT jsonb_build_object($1::text, "name") ||
jsonb_build_object($2::text, "lateral_titles".result) AS result
FROM "studios"
LEFT JOIN LATERAL (
SELECT
string_agg("title", ', ' ORDER BY "release_year") AS result
FROM "films"
WHERE "studio_id" = "studios"."id"
) AS "lateral_titles" ON true
) AS "sq_studios"
values ["name", "titles"]
::TypeScript knows
ts
const studios: (db.JSONOnlyColsForTable<"studios", "name"[]> & db.LateralResult<{
titles: db.SQLFragment<string | null, never>;
}>)[]
// that is
const studios: {
name: string;
titles: string | null;
}[]
${db.parent('id')}is"studios"."id", the studio of each row.- The subquery gives one column, named
result: that is whatlateralreads. Its type argument,string | null, is the key's type:string_agg()of no rows isNULL. - With
strictoff, the key is not in the type (lateral). - Outside a
lateral,parent()throws "The 'parent' table alias has no meaning here", sorun()rejects before any query.
Joins through lateral, step by step: Joins.
Check the SQL before you run it
.compile() gives the text and values that run() would send, and sends nothing. Each value from outside belongs in values, with a $n in its place in text. A test that compiles each query with hostile input, expects it in values and freezes text keeps it that way, here for listings() above:
ts
for (const cinema of [`x' OR TRUE --`, `x" --`]) {
test(`listings() keeps ${cinema} a value`, () => {
const { text, values } = listings(cinema).compile();
assert.deepEqual(values, [cinema]);
assert.equal(text.replace(/\s+/g, ' '), 'SELECT "films"."title", "screenings"."starts_at" FROM "screenings" '
+ 'JOIN "films" ON "films"."id" = "screenings"."film_id" JOIN "cinemas" ON "cinemas"."id" = "screenings"."cinema_id" '
+ 'WHERE "cinemas"."name" = $1 ORDER BY "screenings"."starts_at"');
});
}- Here with
testfromnode:testandassertfromnode:assert/strict; any test runner does. textkeeps the line breaks of the template: the test collapses its whitespace, so that only a change to the SQL fails it.- A search of
textis not enough: a name compiled fromx" --is there as"x"" --", its"doubled, and a lone'matches the query's own literals.
See also
- SQL template: every interpolation, and what it compiles to.
- Security: why values are parameters, and what pgcast quotes.
- Joins: subqueries with
parent(), nested.