feat(db): support multi-source from queries

[samwillis]

Stack

This PR is stacked on #1536 (caseWhen) and uses case-when as its base branch. Please review/merge #1536 first.

Implementation plan: https://gist.github.com/samwillis/df7e2376c6c2bfd007e53dec797fca12

Summary

This PR adds multi-source from support to the DB query engine. It lets a query union multiple independent sources into one live query result without requiring an explicit join between those sources.

What Changed

• Adds a UnionFrom IR node for from({ a, b, ... }) with multiple sources.
• Updates the query builder so from accepts multiple collection or subquery sources, while joins still accept one source.
• Compiles each source independently, namespaces rows by alias, prefixes row keys by source alias to avoid collisions, and concatenates the source streams.
• Preserves no-select result typing as an exclusive union, so only the active branch is required on each row.
• Supports subquery branches, including pre-filtered and joined subqueries.
• Keeps global where semantics after the union, while allowing users to pre-filter a branch by making it a subquery.
• Updates optimizer/live-query traversal so all union branches are discovered for subscriptions, aliases, docs, and collection extraction.
• Handles joins after multi-source from, including branch-dependent join keys via coalesce.
• Teaches join lazy loading to expand safe union-backed subquery projections into concrete per-branch lazy loads, while deopting computed projections and limited/offset branches.
• Documents multi-source from and ordering behavior in the live queries guide.

Examples

Basic multi-source query:

const feed = createLiveQueryCollection((q) =>
  q
    .from({
      message: messages,
      toolCall: toolCalls,
    })
    .orderBy(({ message, toolCall }) =>
      coalesce(message.timestamp, toolCall.timestamp),
    ),
)

Without select, the result is an exclusive union:

type FeedRow =
  | { message: Message; toolCall?: undefined }
  | { message?: undefined; toolCall: ToolCall }

Pre-filter one branch with a subquery:

const feed = createLiveQueryCollection((q) => {
  const visibleMessages = q
    .from({ message: messages })
    .where(({ message }) => eq(message.kind, `visible`))

  return q.from({
    message: visibleMessages,
    toolCall: toolCalls,
  })
})

Join after a multi-source from with branch-dependent keys:

const feedWithUsers = createLiveQueryCollection((q) =>
  q
    .from({
      message: messages,
      toolCall: toolCalls,
    })
    .join({ user: users }, ({ message, toolCall, user }) =>
      eq(coalesce(message.userId, toolCall.userId), user.id),
    )
    .select(({ message, toolCall, user }) => ({
      userName: user.name,
      timestamp: coalesce(message.timestamp, toolCall.timestamp),
    })),
)

Guard branch-specific projections and includes with caseWhen from #1536:

const feed = createLiveQueryCollection((q) =>
  q
    .from({ message: messages, toolCall: toolCalls })
    .select(({ message, toolCall }) => ({
      message: caseWhen(message.id, {
        id: message.id,
        text: message.text,
        chunks: toArray(
          q
            .from({ chunk: chunks })
            .where(({ chunk }) => eq(chunk.messageId, message.id))
            .select(({ chunk }) => chunk.text),
        ),
      }),
      toolCall: caseWhen(toolCall.id, {
        id: toolCall.id,
        name: toolCall.name,
      }),
      timestamp: coalesce(message.timestamp, toolCall.timestamp),
    })),
)

Test Plan

• pnpm vitest run packages/db/tests/query/multi-source-from.test.ts
• pnpm --filter @tanstack/db build
• Focused type test file passes with pnpm vitest --run --typecheck tests/query/multi-source-from.test-d.ts --coverage.enabled=false from packages/db; Vitest still reports unrelated existing includes.test.ts type errors in the wider typecheck pass.

Made with Cursor

Summary by CodeRabbit

• New Features

  • Added caseWhen for conditional expressions and conditional select projections.
  • Added multi-source from() to build union-style queries with namespaced rows and per-branch optional fields.
  • Includes can be conditionally materialized and written to nested result paths; lazy-loading targets handle multi-branch cases.

• Documentation

  • Updated guides for caseWhen, multi-source from(), multi-source ordering, and include behavior.

• Tests

  • New/expanded test suites covering caseWhen, multi-source from, grouping, ordering, includes, and live/reactivity.

Made with Cursor

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ae7b1fe2-f32c-4532-8f49-1ff12505eb80

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch multi-source-from

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[pkg-pr-new]

More templates

• todos
• offline-first-electron
• tanstack-start-example-basic
• @tanstack/db-example-paced-mutations-demo
• tanstack-start-db-projects-example
• @tanstack/db-example-react-todo
• offline-transactions-react-native
• shopping-list-react-native
• @tanstack/db-example-solid-todo

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1537

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1537

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1537

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1537

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1537

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1537

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1537

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1537

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1537

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1537

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1537

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1537

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1537

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1537

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1537

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1537

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1537

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1537

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1537

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1537

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1537

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1537

commit: 6bb8cc8

[github-actions]

Size Change: +2.52 kB (+2.15%)

Total Size: 119 kB

📦 View Changed

Filename	Size	Change
packages/db/dist/esm/query/builder/index.js	5.67 kB	+136 B (+2.46%)
packages/db/dist/esm/query/compiler/group-by.js	3.44 kB	+92 B (+2.75%)
packages/db/dist/esm/query/compiler/index.js	5.74 kB	+935 B (+19.47%)	⚠️
packages/db/dist/esm/query/compiler/joins.js	2.46 kB	+112 B (+4.78%)	🔍
packages/db/dist/esm/query/compiler/lazy-targets.js	856 B	+856 B (new file)	🆕
packages/db/dist/esm/query/compiler/order-by.js	1.74 kB	+17 B (+0.99%)
packages/db/dist/esm/query/compiler/select.js	1.42 kB	+1 B (+0.07%)
packages/db/dist/esm/query/effect.js	4.77 kB	-16 B (-0.33%)
packages/db/dist/esm/query/ir.js	971 B	+72 B (+8.01%)	🔍
packages/db/dist/esm/query/live/collection-config-builder.js	8.13 kB	+120 B (+1.5%)
packages/db/dist/esm/query/live/collection-subscriber.js	1.93 kB	-16 B (-0.82%)
packages/db/dist/esm/query/live/utils.js	1.78 kB	+36 B (+2.06%)
packages/db/dist/esm/query/optimizer.js	2.79 kB	+172 B (+6.56%)	🔍

ℹ️ View Unchanged

Filename	Size
packages/db/dist/esm/collection/change-events.js	1.39 kB
packages/db/dist/esm/collection/changes.js	1.38 kB
packages/db/dist/esm/collection/cleanup-queue.js	810 B
packages/db/dist/esm/collection/events.js	434 B
packages/db/dist/esm/collection/index.js	3.61 kB
packages/db/dist/esm/collection/indexes.js	1.99 kB
packages/db/dist/esm/collection/lifecycle.js	1.69 kB
packages/db/dist/esm/collection/mutations.js	2.47 kB
packages/db/dist/esm/collection/state.js	5.26 kB
packages/db/dist/esm/collection/subscription.js	3.74 kB
packages/db/dist/esm/collection/sync.js	2.88 kB
packages/db/dist/esm/collection/transaction-metadata.js	144 B
packages/db/dist/esm/deferred.js	207 B
packages/db/dist/esm/errors.js	4.92 kB
packages/db/dist/esm/event-emitter.js	748 B
packages/db/dist/esm/index.js	3.01 kB
packages/db/dist/esm/indexes/auto-index.js	830 B
packages/db/dist/esm/indexes/base-index.js	729 B
packages/db/dist/esm/indexes/basic-index.js	2.05 kB
packages/db/dist/esm/indexes/btree-index.js	2.17 kB
packages/db/dist/esm/indexes/index-registry.js	820 B
packages/db/dist/esm/indexes/reverse-index.js	538 B
packages/db/dist/esm/local-only.js	890 B
packages/db/dist/esm/local-storage.js	2.1 kB
packages/db/dist/esm/optimistic-action.js	359 B
packages/db/dist/esm/paced-mutations.js	496 B
packages/db/dist/esm/proxy.js	3.75 kB
packages/db/dist/esm/query/builder/functions.js	1.33 kB
packages/db/dist/esm/query/builder/ref-proxy.js	1.22 kB
packages/db/dist/esm/query/compiler/evaluators.js	1.81 kB
packages/db/dist/esm/query/compiler/expressions.js	430 B
packages/db/dist/esm/query/expression-helpers.js	1.43 kB
packages/db/dist/esm/query/live-query-collection.js	360 B
packages/db/dist/esm/query/live/collection-registry.js	264 B
packages/db/dist/esm/query/live/internal.js	145 B
packages/db/dist/esm/query/predicate-utils.js	2.97 kB
packages/db/dist/esm/query/query-once.js	359 B
packages/db/dist/esm/query/subset-dedupe.js	960 B
packages/db/dist/esm/scheduler.js	1.3 kB
packages/db/dist/esm/SortedMap.js	1.3 kB
packages/db/dist/esm/strategies/debounceStrategy.js	247 B
packages/db/dist/esm/strategies/queueStrategy.js	428 B
packages/db/dist/esm/strategies/throttleStrategy.js	246 B
packages/db/dist/esm/transactions.js	2.9 kB
packages/db/dist/esm/utils.js	927 B
packages/db/dist/esm/utils/array-utils.js	273 B
packages/db/dist/esm/utils/browser-polyfills.js	304 B
packages/db/dist/esm/utils/btree.js	5.61 kB
packages/db/dist/esm/utils/comparison.js	1.05 kB
packages/db/dist/esm/utils/cursor.js	457 B
packages/db/dist/esm/utils/index-optimization.js	1.54 kB
packages/db/dist/esm/utils/type-guards.js	157 B
packages/db/dist/esm/virtual-props.js	360 B

_{compressed-size-action::db-package-size}

[github-actions]

Size Change: 0 B

Total Size: 4.24 kB

ℹ️ View Unchanged

Filename	Size
packages/react-db/dist/esm/index.js	249 B
packages/react-db/dist/esm/useLiveInfiniteQuery.js	1.32 kB
packages/react-db/dist/esm/useLiveQuery.js	1.34 kB
packages/react-db/dist/esm/useLiveQueryEffect.js	355 B
packages/react-db/dist/esm/useLiveSuspenseQuery.js	567 B
packages/react-db/dist/esm/usePacedMutations.js	401 B

_{compressed-size-action::react-db-package-size}