PagedList: Lazy Paginated Views
PagedList<T> is a facet-based, on-demand paginated list backed by the database. It
targets UI scenarios — data grids, dropdown pickers, search screens — where you want to
expose a potentially very large result set without materializing it in memory, while
still supporting per-facet filtering, sorting, and foreign-key traversal.
It implements kotlin.collections.AbstractList<T> (and therefore Java's List<T>), so
every get(index), size, and iterator() call Just Works — the list loads pages
behind the scenes.
Use PagedQuery for server-side / REST use
PagedList is a stateful UI model. Its per-facet filter/sort are mutable and
shared across callers, its cached page and selected entity have no meaning across
independent REST requests, and its saveState output was never designed as a wire
format — it is an internal round-trip contract.
For server-side use — REST handlers, RPC methods, any stateless request/response
context — use PagedQuery instead. It shares the same underlying
query engine (same FK-aware filtering, sorting, raw facets, constraints) but
exposes a thread-safe, share-once-reuse-many API keyed exclusively by facet
aliases. No schema details or SQL fragments ever leak over the wire.
Quick Start
Construction takes only the entity type. The [Stormify] instance is resolved from
the default instance on first access, so
the same class works identically from Kotlin and Java. If you are juggling multiple
Stormify instances in the same process, use stormify.attach(...) at construction
time to bind the list to a specific one.
import onl.ycode.stormify.biglist.Facet
import onl.ycode.stormify.biglist.PagedList
import db.stormify.Company_ // KSP-generated typed paths
val list = PagedList<Company>()
// val list = stormify.attach(PagedList<Company>()) // when not using a default instance
list.addFacet(Company_.name) // auto-detect TEXT
list.addFacet(Company_.contactPerson.firstName, // OR across FK-traversed fields
Company_.contactPerson.lastName)
list.addSqlFacet("SUM(order_total)", Facet.NUMERIC)
list.getFacet(0).filter = "Acme"
list.getFacet(1).sort = Facet.ASCENDING
// Consume as a normal List
val firstCompany = list[0] // triggers page load
val totalMatches = list.size // triggers COUNT query
for (company in list) println(company.name) // streams page-by-page
import onl.ycode.stormify.biglist.Facet;
import onl.ycode.stormify.biglist.PagedList;
import db.stormify.Company_; // KSP-generated typed paths
PagedList<Company> list = new PagedList<>(Company.class);
// PagedList<Company> list = stormify.attach(new PagedList<>(Company.class)); // when not using a default instance
list.addFacet(Company_.name); // auto-detect TEXT
list.addFacet(Company_.contactPerson().firstName, // OR across FK-traversed fields
Company_.contactPerson().lastName);
list.addSqlFacet("SUM(order_total)", Facet.NUMERIC);
list.getFacet(0).setFilter("Acme");
list.getFacet(1).setSort(Facet.ASCENDING);
// Consume as a normal List
Company firstCompany = list.get(0); // triggers page load
int totalMatches = list.size(); // triggers COUNT query
for (Company c : list) System.out.println(c.getName()); // streams page-by-page
Enabling Type-Safe Paths
Typed paths like Company_.name are generated by the annproc KSP processor. Add it to
your build (once; the same processor also generates entity metadata):
For each entity class Foo the processor emits a Foo_ object under the db.stormify
package, with fields for each scalar property and nested objects for FK references. See
Annotations for the full setup.
On Native/Android/iOS, annproc is required anyway (for entity metadata) — you get
typed paths for free.
Facets
A facet is the unit of filtering and sorting. It can be backed by:
- A single entity field
addFacet(Company_.name)
- A foreign-key path — auto-generates the JOIN
addFacet(Order_.customer.name)
- Any combination of the above — pass any number of paths on the same facet;
they OR-filter together, and you can freely mix scalar fields and FK paths in the
same call:
addFacet(Person_.firstName, Person_.lastName)→ two scalar fieldsaddFacet(Order_.notes, Order_.customer.name)→ scalar field + FK pathaddFacet(Order_.customer.name, Order_.shippingAddress.city)→ two FK paths
- A raw SQL expression — for calculated facets
addSqlFacet("SUM(total)", Facet.NUMERIC)
Filters between different facets use AND semantics; multiple paths within the same facet use OR.
val list = PagedList<Order>()
val nameFacet = list.addFacet(Order_.customer.name) // FK traversal
val statusFacet = list.addFacet(Order_.status) // scalar
val rawFacet = list.addSqlFacet("total * tax_rate", Facet.NUMERIC)
nameFacet.filter = "Acme"
statusFacet.filter = "ACTIVE"
// Both filters active → WHERE customer.name LIKE %Acme% AND status = 'ACTIVE'
PagedList<Order> list = new PagedList<>(Order.class);
Facet nameFacet = list.addFacet(Order_.customer().name); // FK traversal
Facet statusFacet = list.addFacet(Order_.status); // scalar
Facet rawFacet = list.addSqlFacet("total * tax_rate", Facet.NUMERIC);
nameFacet.setFilter("Acme");
statusFacet.setFilter("ACTIVE");
// Both filters active → WHERE customer.name LIKE %Acme% AND status = 'ACTIVE'
String paths with dot notation work too (addFacet("customer.name")) — useful for
cross-cutting code where a field is chosen at runtime. Mix both styles freely.
Facet Types and Filter Syntax
Each facet has a Facet.Type that controls how the filter string is parsed. The type
is auto-detected from the field's Kotlin type, or you can set it explicitly via the
two-argument addFacet(type, ...) overload.
Text (Facet.TEXT)
Case-insensitive by default. Supports Google-like boolean search syntax with wildcards:
Basic patterns
| Filter value | Meaning |
|---|---|
Alice |
Substring match: LIKE %Alice% |
*lice |
Ends with: LIKE %lice |
Ali* |
Starts with: LIKE Ali% |
"foo bar" |
Phrase match: LIKE %foo bar% (treats content as literal substring) |
Boolean operators
Multiple terms are combined with boolean logic following Google search conventions:
| Filter value | Meaning |
|---|---|
foo bar |
AND — both must match (implicit) |
foo OR bar |
OR — either must match (OR must be uppercase) |
-foo |
NOT — exclude matches |
-"foo bar" |
NOT phrase — exclude phrase matches |
(foo OR bar) baz |
Grouping with parentheses |
(Alice OR Bob) -Alicia |
Combined: Alice or Bob, but not Alicia |
Precedence: AND binds tighter than OR (same as Google). A hyphen inside a word
(e.g. part-number) is treated as literal text, not as a negation operator.
NULL matching
The NULL token matches rows where the column is SQL NULL:
| Filter value | Meaning |
|---|---|
NULL |
col IS NULL |
-NULL |
col IS NOT NULL |
foo OR NULL |
contains "foo" OR is null |
NULL works across all facet types (TEXT, NUMERIC, DATE, TIME, TIMESTAMP, ENUM).
Configurable tokens
All syntactic tokens can be overridden via FilterSyntax:
| Property | Default | Role |
|---|---|---|
or |
OR |
OR operator |
not |
- |
NOT operator |
nullToken |
NULL |
NULL match |
phraseDelimiter |
" |
Phrase open/close |
groupOpen |
( |
Grouping open |
groupClose |
) |
Grouping close |
wildcard |
* |
Prefix/suffix wildcard |
Word-like tokens (containing letters, e.g. OR, NULL) require word
boundaries to match — OR inside CONSTRUCTOR is not confused for the
operator. Symbol-like tokens (|, -, --) match at any token boundary,
and longest-match wins (so -- takes precedence over -).
For non-English installations, override the tokens globally on the Stormify
instance:
Set facet.isCaseSensitive = true for a case-sensitive facet.
Numeric (Facet.NUMERIC)
| Filter value | Meaning |
|---|---|
42 |
Equals |
> 10 |
Greater than |
>= 10 |
Greater than or equal |
< 100 |
Less than |
<= 100 |
Less than or equal |
10 ... 20 |
Range (inclusive) |
Temporal (Facet.DATE, Facet.TIME, Facet.TIMESTAMP)
Three separate buckets cover the common wall-clock types:
| Bucket | Kotlin types | Dialect cast (raw facets) |
|---|---|---|
Facet.DATE |
LocalDate, java.sql.Date |
TO_DATE(?, 'YYYY-MM-DD') on Oracle; CAST(? AS DATE) elsewhere |
Facet.TIME |
LocalTime, java.sql.Time |
TO_TIMESTAMP(?, 'HH24:MI:SS') on Oracle (no native TIME); CAST(? AS TIME) elsewhere |
Facet.TIMESTAMP |
LocalDateTime, Instant, OffsetDateTime, ZonedDateTime, Timestamp |
CAST(? AS DATETIME) on MySQL/MariaDB; CAST(? AS DATETIME2) on MSSQL; TO_TIMESTAMP(...) on Oracle; CAST(? AS TIMESTAMP) elsewhere |
All three support the same comparison and range syntax as NUMERIC. Values are parsed via
the active input parser, which runs after the operator/range
AST is resolved.
The bucket for a typed facet is inferred from the field's KClass. For raw facets, the
caller picks the bucket explicitly:
Enum (Facet.ENUM)
For enum-backed fields. Auto-detected when the field's Kotlin type is an enum. The filter
string is matched (case-insensitively, as a substring) against the enum's display names.
Multiple matching display names produce an IN clause of the corresponding DB values.
Supports the full boolean syntax from TEXT facets — active OR pending,
-archived, "active" (exact phrase, does not match INACTIVE), (a OR b) -c —
applied as set operations over the resolved enum values.
To customize the display names (e.g., localized UI strings), implement the
HumanReadable interface on the enum. For non-enum
fields that you want to treat as enums (or to override the auto-built map), pass a
display-to-DB-value Map as the first argument to addFacet — the same method
name, resolved to the enum-facet overload via the first-argument type:
Sorting
Set facet.sort to Facet.ASCENDING or Facet.DESCENDING. Any number of facets can
be active sorts at once; the list sorts by them in the order they were defined:
facet.clearSort() deactivates the sort for a single facet; list.reset() clears
all filters and sorts at once (but keeps constraints — see below).
If no facet has an active sort, rows are ordered by the entity's primary key. This
requires a single-column PK; for composite keys you must explicitly set a sort on at
least one facet, otherwise the list throws IllegalStateException.
Constraints
Constraints are a fixed WHERE clause that is always applied, independent of facet
filters. Use them to scope the list to a sub-query (e.g., "only orders for user 42"):
Constraints and facet filters combine with AND. Unlike filters, constraints are not
cleared by list.reset() — they're considered part of the list's fundamental definition.
Distinct Mode
Set list.isDistinct = true to make the underlying query use SELECT DISTINCT. This is
useful when JOINs cause row duplication. Note that DISTINCT affects both size (which
becomes COUNT(DISTINCT ...)) and page queries. Aggregations are not affected —
getAggregator() ignores the flag.
Pagination and Caching
PagedList keeps one page of rows in memory at a time. pageSize defaults to 15 and
can be changed:
Indexing outside the currently cached page triggers a new page load. The size property
triggers a COUNT(*) query the first time it's read and caches the result. The list
stays in sync with your configuration — filter, sort, constraint, and page-size changes
are picked up automatically on the next access.
Stormify uses the underlying SQL dialect's pagination syntax (LIMIT/OFFSET, FETCH
FIRST, ROWNUM, etc.) automatically.
Selected Entity
You can mark one entity as selected — it appears as the first element of the list regardless of any sort order. This is useful for UI patterns like "show the currently highlighted row at the top":
The helpers list.add(entity) and list.remove(entity) set or clear the selection as a
side effect. Use them when you create/delete an entity and want the list to reflect the
change immediately without resetting the user's filters:
Filter Values (Distinct Per-Facet)
For populating dropdown pickers, call facet.getFilterValues(). This returns a
FilterValues — another lazy paginated list, but of the distinct values of that
facet, filtered by the other facets' active filters. The user only sees values that
would actually return results under the current filter state.
FilterValues stays in sync with the parent list — any change to filters or
constraints is reflected on the next access. For serialization purposes, its
toString() emits a JSON array of the values (["a","b","c"]).
Facet Counts (withCounts)
Some pickers want to show the value plus how many rows it would produce
("Category: Books (12), Movies (4)"). Call withCounts() on the FilterValues to
get a sibling FilterCountedValues — same lazy pagination, but each element is a
FilterCountedValue(value, count):
Repeated calls to withCounts() on the same FilterValues return the same instance,
so you can share it across multiple reads without re-querying. For serialization
purposes, its toString() emits a JSON array of {"value":"X","count":N} objects.
Input Parsing and Locale
Raw user input (e.g., from a text field) often uses locale-specific formats —
numbers like 1.234,56 (comma decimal), dates like 31/12/2026 (day-first).
PagedList applies an input parser — a (String, Facet.Type) -> String function —
that transforms the filter string before it reaches SQL.
Resolution order: facet → list → Stormify instance → identity (no transformation).
// Stormify instance — applies to all PagedList instances that use it
stormify.inputParser = { input, type ->
when (type) {
Facet.NUMERIC -> input.replace(".", "").replace(",", ".")
Facet.DATE, Facet.TIMESTAMP -> flipDayMonthYear(input)
else -> input
}
}
// Per-list override
myList.inputParser = { input, type -> /* ... */ }
// Per-facet override
myList.getFacet(0).inputParser = { input, type -> /* ... */ }
// Stormify instance — applies to all PagedList instances that use it
stormify.setInputParser((input, type) -> {
if (type == Facet.NUMERIC) return input.replace(".", "").replace(",", ".");
if (type == Facet.DATE || type == Facet.TIMESTAMP) return flipDayMonthYear(input);
return input;
});
// Per-list override
myList.setInputParser((input, type) -> /* ... */);
// Per-facet override
myList.getFacet(0).setInputParser((input, type) -> /* ... */);
Set to null (the default) to fall through to the next level.
Raw Facets
A SQL facet is backed by a SQL expression instead of an entity field — useful for calculated values, concatenations, or expressions no entity property maps to. There are two forms:
Simple — just an expression and a type
Pass the expression and the facet type; the default converter decides the WHERE
clause based on the type (LIKE ? for TEXT, = ? / BETWEEN ? AND ? for
NUMERIC, typed comparison with dialect-aware casts for DATE/TIME/TIMESTAMP, mapped equality for ENUM).
This covers the majority of raw-SQL-facet needs.
Custom Converter
When the default converter isn't enough, assign a custom Converter to the facet.
Converter is a function type (column, input, parser, args) -> String — use a
plain lambda. It receives the column expression, the raw user input, an
InputParser, and a SqlArgsCollector, and returns a SQL fragment.
Use it when you need to:
- Translate the input value in a non-trivial way (e.g.
"50k"→50000) - Emit more than one placeholder (e.g.
BETWEEN ? AND ?with a computed upper bound) - Choose different SQL shapes based on the input format
- Implement full-text search (e.g.
MATCH ... AGAINSTfor MySQL)
Custom converters can leverage TextQuery.parse() to get boolean query support
for free (for full-text backends that take a single bound parameter):
facet.converter = converter@{ col, input, _, args ->
if (TextQuery.parse(input) == null) return@converter "1 = 0"
args.add(input) // websearch_to_tsquery / MATCH AGAINST accept Google syntax
"MATCH($col) AGAINST(? IN BOOLEAN MODE)"
}
Table Refs
Raw SQL in setConstraints or in a SQL facet sometimes needs
to reference a joined table by its SQL alias. Engine-assigned aliases (t1,
t2, …) are not stable across configuration changes, so instead of hardcoding
them, register a TableRef with addTableRef(...) and use its
alias (via string interpolation or getAlias()):
Note
FK-path facets like addFacet("parent.name") resolve JOINs automatically —
no TableRef needed. Table refs are only required when you write raw SQL
that references a joined table by alias.
val list = PagedList<Customer>()
val root = list.addTableRef() // the root entity table
val address = list.addTableRef("address") // FK-traversed
val hq = list.addTableRef(Customer_.company.hq) // KSP typed path
list.addFacet(Customer_.name)
list.addSqlFacet(
"CASE WHEN $address.city = $hq.city THEN 1 ELSE 0 END",
Facet.NUMERIC)
list.setConstraints("$root.tenant_id = ?", currentTenantId)
PagedList<Customer> list = new PagedList<>(Customer.class);
TableRef root = list.addTableRef(); // the root entity table
TableRef address = list.addTableRef("address"); // FK-traversed
TableRef hq = list.addTableRef(Customer_.company().hq); // KSP typed path
list.addFacet(Customer_.name);
list.addSqlFacet(
"CASE WHEN " + address.getAlias() + ".city = "
+ hq.getAlias() + ".city THEN 1 ELSE 0 END",
Facet.NUMERIC);
list.setConstraints(root.getAlias() + ".tenant_id = ?", currentTenantId);
Registering a TableRef guarantees the corresponding JOIN is active in every
subsequent SQL build while TableRef.isActive is true (the default). Set
isActive to false to suppress the JOIN without removing the ref — useful
when a ref is referenced only by a facet that may be conditionally inactive.
HumanReadable (Display Names)
Implement HumanReadable on enums (or any class) that participate in enum facets to
provide localized display names:
When the enum facet auto-builds its display-name-to-DB-value map, it uses
displayName instead of name. This is also what FilterValues returns for enum
facets — so your dropdown shows the localized labels instead of ACTIVE/INACTIVE/BANNED.
Refreshing After External Changes
When you mutate data outside of the list's awareness — typically via
stormify.create/update/delete — the list's cached row count and current page are
unaware of the change. Call list.refresh() to force the next read to re-query:
refresh() is pure cache invalidation — there are no listeners to notify. After calling
it, the list itself is ready, and you are responsible for whatever UI refresh your
framework needs.
Streaming with forEachStreaming
A PagedList supports ordinary iteration (for (row in list)), but the index-based
iterator walks page by page and issues one query per page — fine for small result
sets, but wasteful for exports over tens of thousands of rows.
For efficient, one-query streaming use forEachStreaming, which runs a
server-side cursor over the current filter / sort state.
for (row in list) and the plain list.forEach { … } / list.forEach(consumer)
keep their page-by-page semantics — they resolve to the default
kotlin.collections.Iterable.forEach / java.lang.Iterable.forEach and walk the
index-based iterator one page at a time. Pick forEachStreaming when you want
the single-query cursor behaviour.
Sibling-batch friendly. Under the hood, rows are forwarded to your callback in
chunks matching the default sibling-batch size
rather than one at a time. The buffering is invisible to your code (you still see
a stream of single rows), but it guarantees that if your callback touches a
foreign-key reference — row.customer.name, row.category.label — the first
touch triggers a single batched lookup covering the whole chunk of siblings
instead of issuing an individual follow-up query per row. No configuration, no
API change — just the same forEachStreaming call behaving optimally for the
common streaming-with-lazy-FK pattern. See
Sibling Batch Optimization for the
underlying mechanism.
Aggregations
list.getAggregator() returns a PagedAggregator — a fluent builder for SQL aggregate
queries that respect the list's current constraints and per-facet filters. isDistinct
is ignored for aggregates.
The builder methods take a path (as String or typed ScalarPath) plus an optional
alias. When alias is omitted the aggregator generates a unique one by combining the
function name and the path (sum_revenue, count_id, countDistinct_category, …).
Single-Value Aggregation
Chain one method and call execute<R>() (or execute(Class<R>) from Java) to get a
typed scalar:
Multi-Value Aggregation
Keep chaining aggregators to build a MultiAggregator — its execute() runs a single
query and returns a Map<String, Any?> keyed by each aggregator's alias (auto-generated
or explicit):
Map<String, Object> row = list.getAggregator()
.sum(Company_.revenue, "total") // → row.get("total")
.avg(Company_.revenue, "average") // → row.get("average")
.min(Company_.revenue, null) // → row.get("min_revenue")
.max(Company_.revenue, null) // → row.get("max_revenue")
.count("*", "cnt") // → row.get("cnt")
.execute();
Explicit aliases that collide with an already-added entry throw
IllegalArgumentException. Auto-generated aliases never collide — repeating
sum("revenue") gives sum_revenue, sum_revenue_2, sum_revenue_3, …
Raw Expressions
When the built-in functions are not enough, use raw(expression) to emit an arbitrary
SQL fragment. Auto-generated aliases sanitize the expression (operators and parentheses
become underscores, duplicates get a numeric suffix) so you don't have to come up with
one yourself — although supplying an explicit alias is always an option.
// SUM(qty * price) respecting current filters
val revenue = list.getAggregator()
.raw("SUM(order_item.qty * order_item.price)")
.execute<BigDecimal>()
// Multi with a raw alongside a typed aggregator
val row = list.getAggregator()
.sum(Order_.total, "total") // → row["total"]
.raw("SUM(order.total * tax_rate)", "total_with_tax") // → row["total_with_tax"]
.raw("AVG(order.discount)") // → row["AVG_order_discount"]
.execute()
// SUM(qty * price) respecting current filters
BigDecimal revenue = list.getAggregator()
.raw("SUM(order_item.qty * order_item.price)", null)
.execute(BigDecimal.class);
// Multi with a raw alongside a typed aggregator
Map<String, Object> row = list.getAggregator()
.sum(Order_.total, "total") // → row.get("total")
.raw("SUM(order.total * tax_rate)", "total_with_tax") // → row.get("total_with_tax")
.raw("AVG(order.discount)", null) // → row.get("AVG_order_discount")
.execute();
Saving and Restoring State
For navigation flows where the user leaves a grid / picker screen and comes back, use
saveState() / restoreState(). They capture the per-facet filters, sorts,
case-sensitivity flags, the page size, and the distinct flag into a plain
PagedListState data object — easy to persist in a view model or session store:
// Before leaving the screen
PagedListState state = list.saveState();
viewModel.savedListState = state;
// When coming back
PagedList<Company> list = new PagedList<>(Company.class);
list.addFacet(Company_.name);
list.addFacet(Company_.revenue);
if (viewModel.savedListState != null)
list.restoreState(viewModel.savedListState);
restoreState() is permissive: facet keys present in the saved state but missing from
the current list are silently ignored (so your list can add/remove facets between
sessions without blowing up). Constraints, the selected entity, and input parsers are
not part of the state — they are structural or ephemeral, not user-visible choices.
State keys are derived from each facet's paths (raw facets use their SQL expression;
field facets use their paths joined alphabetically), so the order in which paths were
passed to addFacet does not affect the key.