Alexandre Day / Projects / Publications
LoG 2026 · Learning on Graphs, main track

QUARTET: Quad-branch cross-Attention and Random-walk Traces for Enhancing Transformers on Relational Graphs

Kyaw Hpone Myint, Nan Jiang, Xiang Li, Zhe Wu, Alexandre G.R. Day, Pranab Mohanty, Giri Iyengar

Capital One, AI Foundations

TL;DR

Quartet is a graph transformer for relational deep learning, i.e. learning directly on multi-table databases. It replaces the loose random subgraph sampler of the strongest prior model, RelGT, with a causal random-walk sampler built on recency-truncated Personalized PageRank, and adds four global cross-attention branches (seed features, seed topology, temporal dynamics, collaborative dynamics). On the RelBench classification suite it matches or beats RelGT on 8 of 12 tasks and holds the best test ROC-AUC on 7.

The problem

Relational deep learning treats a relational database as a heterogeneous temporal graph: rows become nodes, primary–foreign key links become edges, and timestamps make the whole thing causal. This is the natural setting for most enterprise data, where the interesting signal lives in how accounts, transactions, events and products relate to one another over time.

Graph transformers currently lead the RelBench benchmark, with RelGT as the strongest baseline. It has two weaknesses. Its local sampler draws neighbours at random, which yields loosely connected subgraphs that hinder message passing. And its global module relies on a single seed-feature memory, which misses the broader temporal and collaborative dynamics around a seed entity.

The approach

Causal Random Walk (CRW) sampling. Instead of uniform one- and two-hop sampling, Quartet ranks a seed's neighbourhood with a recency-truncated Personalized PageRank estimate, keeping only the most recent neighbours at each step (50 by default). This produces compact, densely connected local subgraphs, regularises against high-degree hubs, and never looks past the seed's timestamp, so there is no temporal leakage.

Local module. Each sampled node becomes a token that fuses five encodings (feature, node type, hop distance, time and position). Full self-attention runs over the whole local subgraph.

Quad-branch global module. The seed token cross-attends to four complementary views of global context: its features (a feature codebook), its topology (a structural codebook), temporal dynamics (a Perceiver-style bottleneck over recently active nodes) and collaborative dynamics (a Perceiver-style bottleneck over co-occurring peers).

Quartet pipeline: relational tables become a heterogeneous temporal graph, three causal samplers extract local, temporal and collaborative context, and a transformer with a local module and quad-branch global module produces the prediction.
Figure 1: Overview of the Quartet pipeline. (1) Relational tables are converted into a heterogeneous temporal graph. (2) Three causal samplers extract complementary context: the CRW sampler draws a dense local subgraph, while temporal and collaborative samplers gather globally relevant nodes. (3) A transformer applies full self-attention locally and quad-branch cross-attention globally to produce the seed prediction.

Results

On the twelve RelBench v1 entity-classification tasks, compared against reproduced HGT and RelGT baselines under a shared protocol:

8 / 12
tasks where Quartet matches or beats RelGT
+6.4%
ROC-AUC on rel-trial study-outcome
+5.6%
on rel-f1 driver-top3 and rel-event user-repeat
−6.7%
when the local module and CRW are removed (largest ablation drop)

The ablations are the most informative part. Swapping CRW back to RelGT's uniform sampler costs 1.6% on average, and removing hub truncation or the restart term costs 1.8% and 2.1%. Each of the four global branches contributes on the order of 1.5–2.0%, with the most important branch varying by task. Structurally, CRW subgraphs have higher average degree and a higher seed-connected ratio than the uniform sampler on all twelve tasks, which is exactly the property the method was designed to fix.

Quartet is competitive rather than uniformly dominant: it trails RelGT on a few tasks, notably rel-stack user-badge (−2.7%), and the paper evaluates binary classification only, leaving regression and link prediction to future work.

Why it matters

Two simple ideas carry most of the gain: sample denser, recency-aware local neighbourhoods, and give the seed several distinct views of global context rather than one. Both transfer directly to the kind of large, time-stamped relational data we work with in financial services, and they are a step toward graph and tabular foundation models that can be pretrained once on a database and reused across prediction tasks.