7.1 — Waterfalls & API design

Module 7 · Lesson 1 · 🔴 Advanced · ~40 min

What you'll learn

• The four kinds of waterfall, and how to detect each
• Parallelization, the preload pattern, and React cache()
• BFF design: over‑fetching, N+1, persisted queries, response shaping
• How to give the backend team a specific, actionable ask

---

The four waterfalls

1. SERVER-SIDE SEQUENTIAL     await → await → await             Fix: Promise.all
2. PARENT-CHILD COMPONENT     parent fetches → child fetches     Fix: preload / hoist
3. CLIENT-SIDE FETCH CHAIN    useEffect → fetch → useEffect      Fix: Server Components
4. NETWORK-LEVEL              HTML → JS → API → render           Fix: server-render the data

Each has a distinct signature in a trace. Learn to recognize them.

---

Waterfall 1 — Server‑side sequential

// ❌ 830ms: every await blocks the next
const product   = await getProduct(slug);            // 180ms
const inventory = await getInventory(product.id);    // 140ms  (depends on product)
const price     = await getPrice(product.id);        // 120ms  (depends on product)
const reviews   = await getReviews(product.id);      // 260ms  (depends on product)
const similar   = await getSimilar(product.category);// 130ms  (depends on product)

Only the first is a real dependency. The rest can run together.

// ✅ 320ms
const product = await getProduct(slug);              // 180ms — genuine dependency
const [inventory, price, reviews, similar] = await Promise.all([
  getInventory(product.id),
  getPrice(product.id),
  getReviews(product.id),
  getSimilar(product.category),
]);                                                   // 260ms (the max, not the sum)

Detect it: Server-Timing durations sum to roughly your total server time.

Promise.all vs Promise.allSettled: Promise.all rejects if any promise rejects, which means a failing recommendations service takes down the whole page. For non‑critical data, use allSettled — or better, give each non‑critical call its own Suspense boundary (3.3).

// Critical data: all must succeed
const [product, price] = await Promise.all([getProduct(slug), getPrice(slug)]);

// Non-critical: degrade gracefully
const [reviewsResult, recsResult] = await Promise.allSettled([
  getReviews(product.id),
  getRecommendations(product.id),
]);
const reviews = reviewsResult.status === 'fulfilled' ? reviewsResult.value : null;

---

Waterfall 2 — Parent‑child component fetching

Server Components make this easy to create accidentally, because each component fetches its own data and children can't start until the parent renders.

// ❌ Sequential: Layout resolves before Page even starts
// app/c/[slug]/layout.tsx
export default async function CategoryLayout({ children, params }) {
  const category = await getCategory((await params).slug);   // 120ms
  return <div><CategoryNav category={category} />{children}</div>;
}

// app/c/[slug]/page.tsx — starts at 120ms
export default async function CategoryPage({ params }) {
  const products = await getCategoryProducts((await params).slug);  // 240ms
  return <ProductGrid products={products} />;                        // total 360ms
}

Fix A — the preload pattern

Start the fetch before you await it, so both run in parallel:

// lib/data/category.ts
import { cache } from 'react';
import 'server-only';

export const getCategory = cache(async (slug: string) => { /* … */ });
export const getCategoryProducts = cache(async (slug: string) => { /* … */ });

// Kick off the fetch without awaiting. React's cache() dedupes so the
// later `await` picks up the same in-flight promise.
export function preloadCategoryData(slug: string) {
  void getCategory(slug);
  void getCategoryProducts(slug);
}

// app/c/[slug]/layout.tsx
import { preloadCategoryData, getCategory } from '@/lib/data/category';

export default async function CategoryLayout({ children, params }) {
  const { slug } = await params;
  preloadCategoryData(slug);            // both requests start NOW, in parallel
  const category = await getCategory(slug);
  return <div><CategoryNav category={category} />{children}</div>;
}

// page.tsx's await for products now hits an in-flight (or completed) promise.
// Total: 240ms instead of 360ms.

This pattern is the single most useful trick in RSC data fetching. Any time a parent and child need independent data, preload in the parent.

Fix B — Suspense so the child doesn't block the shell

export default async function CategoryLayout({ children, params }) {
  const { slug } = await params;
  preloadCategoryData(slug);
  return (
    <div>
      <Suspense fallback={<NavSkeleton />}>
        <CategoryNav slug={slug} />       {/* fetches independently */}
      </Suspense>
      {children}
    </div>
  );
}

---

Waterfall 3 — Client‑side fetch chains

The worst kind, because each step includes a full network round trip plus a React render.

// ❌ HTML → JS → render → fetch product → render → fetch reviews → render
//    Four round trips before the user sees reviews.
'use client';
function ProductPage({ slug }) {
  const [product, setProduct] = useState(null);
  const [reviews, setReviews] = useState(null);

  useEffect(() => {
    fetch(`/api/products/${slug}`).then(r => r.json()).then(setProduct);
  }, [slug]);

  useEffect(() => {
    if (!product) return;
    fetch(`/api/reviews/${product.id}`).then(r => r.json()).then(setReviews);
  }, [product]);
  // …
}

// ✅ Zero client round trips: the data is in the HTML
export default async function ProductPage({ params }) {
  const { slug } = await params;
  const product = await getProduct(slug);
  return (
    <>
      <ProductView product={product} />
      <Suspense fallback={<ReviewsSkeleton />}>
        <Reviews productId={product.id} />
      </Suspense>
    </>
  );
}

Find them:

rg -n 'useEffect' app components | rg -i 'fetch|axios|swr|useQuery' 

Every hit is a candidate for a Server Component. Keep the client fetch only where the data is genuinely client‑only (live stock polling, user‑triggered search).

---

Waterfall 4 — Network‑level

Visible in the Network panel as a staircase: HTML → JS chunk → API call → render.

HTML       ├────┤
main.js         ├──────┤
chunk.js               ├────┤
/api/product                ├──────┤
paint                              ▲ 2,100ms

This is Waterfall 3's signature at the network layer. Same fix: server‑render the data.

---

BFF design

Your GraphQL/REST layer between Next.js and the microservices is where a lot of latency hides.

Over‑fetching

# ❌ The PDP query returns 340 fields; the page uses 28.
#    Serialization, transfer, and parsing all scale with this.
query ProductPage($slug: String!) {
  product(slug: $slug) { ...ProductFullDetails }   # 340 fields, 180KB response
}

# ✅ Ask for what you render
query ProductPage($slug: String!) {
  product(slug: $slug) {
    id name slug descriptionHtml
    images(first: 8) { url alt width height }
    listPrice { amount currency }
    variants(first: 50) { id sku size color inStock }
    category { id name slug }
    rating { average count }
  }                                                # 28 fields, 14KB response
}

Measure your response sizes. A 180 KB JSON response takes ~90 ms to parse on a mid‑tier device if it reaches the client, and even server‑side it's serialization cost on both ends.

N+1 in the BFF

// ❌ 1 query for the category + 48 queries for prices
const products = await db.product.findMany({ where: { categoryId } });
const withPrices = await Promise.all(
  products.map(async (p) => ({ ...p, price: await pricingService.get(p.id) })),
);
// 48 parallel calls is better than 48 sequential, but it's still 48 calls
// and it will melt the pricing service under load.

// ✅ Batch
const products = await db.product.findMany({ where: { categoryId } });
const prices = await pricingService.getMany(products.map((p) => p.id));   // 1 call
const priceById = new Map(prices.map((p) => [p.productId, p]));
const withPrices = products.map((p) => ({ ...p, price: priceById.get(p.id) }));

DataLoader generalizes this — it collects calls made within a tick and issues one batched request:

// lib/loaders.ts
import DataLoader from 'dataloader';
import { cache } from 'react';

// cache() gives one loader instance PER REQUEST — critical, or you'd share
// a cache across users.
export const getLoaders = cache(() => ({
  price: new DataLoader<string, Price>(async (ids) => {
    const prices = await pricingService.getMany([...ids]);
    const byId = new Map(prices.map((p) => [p.productId, p]));
    return ids.map((id) => byId.get(id) ?? new Error(`No price for ${id}`));
  }, { maxBatchSize: 100 }),

  inventory: new DataLoader<string, Inventory>(async (ids) => { /* … */ }),
}));

// Components call it individually; the loader batches automatically
const { price } = getLoaders();
const p = await price.load(productId);      // one network call for all callers in this tick

The cache() wrapper is a correctness requirement, not an optimization. A module‑level DataLoader would leak one user's data into another user's request. This is a real bug class in RSC codebases.

Persisted queries

Sending a 4 KB GraphQL query string on every request wastes bytes and CPU, and it lets clients send arbitrary expensive queries.

// Client sends a hash; the server looks up the query.
const response = await fetch(GRAPHQL_ENDPOINT, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    // Query text never crosses the wire after the first registration
    extensions: { persistedQuery: { version: 1, sha256Hash: PDP_QUERY_HASH } },
    variables: { slug },
  }),
});

Benefits: smaller requests, GET‑able (so CDN‑cacheable), and an allowlist that prevents arbitrary query execution. For a public‑facing commerce BFF, the security benefit alone justifies it.

Response shaping at the edge of your app

Even with a good BFF, shape the data before it crosses into React:

// lib/data/product.ts
import 'server-only';
import { cache } from 'react';

export const getProduct = cache(async (slug: string): Promise<Product | null> => {
  const raw = await bff.query(PDP_QUERY, { slug });
  if (!raw.product) return null;

  // Explicit mapping: what the UI needs, nothing else.
  // Keeps the RSC payload small (see 3.2) and makes prop changes visible in diffs.
  return {
    id: raw.product.id,
    name: raw.product.name,
    descriptionHtml: raw.product.descriptionHtml,
    images: raw.product.images.map((i) => ({
      url: i.url, alt: i.alt, width: i.width, height: i.height,
    })),
    listPrice: raw.product.listPrice.amount,
    currency: raw.product.listPrice.currency,
    variants: raw.product.variants.map((v) => ({
      id: v.id, size: v.size, color: v.color, inStock: v.inStock,
    })),
  };
});

---

Timeouts and resilience

Every external call needs a timeout, or one slow service holds your whole page.

// lib/fetch-json.ts
import 'server-only';

type Options = RequestInit & { timeoutMs?: number; retries?: number };

export async function fetchJson<T>(url: string, opts: Options = {}): Promise<T> {
  const { timeoutMs = 1000, retries = 1, ...init } = opts;

  for (let attempt = 0; attempt <= retries; attempt++) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), timeoutMs);
    try {
      const res = await fetch(url, { ...init, signal: controller.signal });
      if (!res.ok) {
        // Don't retry client errors — they won't succeed the second time
        if (res.status < 500) throw new Error(`${res.status} ${res.statusText}`);
        throw new RetryableError(`${res.status}`);
      }
      return (await res.json()) as T;
    } catch (err) {
      const retryable = err instanceof RetryableError || (err as Error).name === 'AbortError';
      if (attempt === retries || !retryable) throw err;
      // Exponential backoff with jitter — never retry in lockstep
      await new Promise((r) => setTimeout(r, 2 ** attempt * 100 + Math.random() * 100));
    } finally {
      clearTimeout(timer);
    }
  }
  throw new Error('unreachable');
}

Timeout budget for a PDP with a 400 ms TTFB target:

Call	Timeout	Retries	On failure
Product (shell‑critical)	800 ms	1	500 page
Price (dynamic hole)	600 ms	1	"See price in bag"
Inventory (dynamic hole)	500 ms	0	Assume available, verify at checkout
Reviews (streamed)	800 ms	0	Omit section
Recommendations (streamed)	600 ms	0	Omit section

Retries multiply your timeout. A 1‑second timeout with 2 retries is a 3‑second worst case, and under load retries amplify the outage that caused them. Use retries only on shell‑critical calls, and add a circuit breaker for anything that fails often.

// lib/circuit-breaker.ts — stop hammering a service that's already down
class CircuitBreaker {
  private failures = 0;
  private openUntil = 0;

  constructor(private threshold = 5, private cooldownMs = 30_000) {}

  async run<T>(fn: () => Promise<T>, fallback: () => T): Promise<T> {
    if (Date.now() < this.openUntil) return fallback();
    try {
      const result = await fn();
      this.failures = 0;
      return result;
    } catch (err) {
      if (++this.failures >= this.threshold) {
        this.openUntil = Date.now() + this.cooldownMs;
        logWarn('circuit_open', { failures: this.failures });
      }
      return fallback();
    }
  }
}

export const recsBreaker = new CircuitBreaker();

---

Giving the backend team an actionable ask

Vague asks ("the API is slow") get ignored. Specific ones get fixed.

## Request: batch endpoint for pricing

**Problem.** Rendering a 48-product PLP makes 48 calls to `GET /pricing/{id}`.
p75 per call is 42ms; with a concurrency limit of 12, that's ~170ms of the PLP's
420ms server time, and it generates 1.8M calls/hour at peak.

**Evidence.** Server-Timing breakdown attached; APM trace `abc-123`.

**Ask.** `POST /pricing/batch` accepting up to 100 IDs, returning the same shape.
Target p75 ≤ 60ms for a 48-ID batch.

**Impact.** PLP server time 420ms → ~250ms. Pricing service request volume
−97%. Removes the concurrency-limit contention we hit at peak.

**Alternative if that's not feasible.** We cache pricing for 60s in our Data Cache,
accepting 60s of price staleness — but we'd rather not, because promo prices
change during flash sales.

Attach the trace, name the numbers, propose the alternative. This gets scheduled.

---

Aurora's data layer results

Change	PDP server time
Baseline (6 sequential calls)	830 ms
Parallelized	380 ms
Streaming (shell awaits product only)	180 ms
Response shaping (180 KB → 14 KB)	155 ms
Batch pricing endpoint	140 ms
Keep‑alive on the BFF client	118 ms
DataLoader for inventory	96 ms

---

Common mistakes

Mistake	Cost
Sequential awaits for independent data	Server time = sum, not max
Promise.all on non‑critical data	One flaky service breaks the page
Module‑level DataLoader	🚨 Cross‑user data leak
No timeouts	One hung service holds every request
Retries on everything	Amplifies outages; multiplies worst‑case latency
Over‑fetching from the BFF	Serialization and parse cost on both ends
Client useEffect fetching	Four round trips instead of zero
No cache() on data accessors	Duplicate queries per render
No keep‑alive	50–150 ms TLS handshake per backend call

---

Lab 7.1 — Kill your waterfalls

1. Instrument with Server-Timing for every backend call on your slowest page.
2. Compare sum vs total. If they're close, you have a sequential waterfall.
3. Parallelize everything without a genuine dependency. Re‑measure.
4. Apply the preload pattern to any layout/page data split.
5. Find client fetch chains: rg -n 'useEffect' | rg -i 'fetch|axios'. Convert the top three to Server Components.
6. Measure BFF response sizes. Anything over 30 KB, shape it down.
7. Count backend calls per page render. Anything scaling with item count is an N+1 — batch it.
8. Add timeouts with a budget table, and verify by injecting a delay in a staging service.

---

Checklist

• No sequential awaits for independent data
• Preload pattern used across layout/page boundaries
• cache() on every data accessor; DataLoaders created per request
• Non‑critical data in Suspense boundaries with graceful degradation
• Every external call has a timeout, with a documented budget
• Retries only on shell‑critical calls, with backoff and jitter
• Circuit breakers on flaky services
• BFF responses shaped to what the UI renders
• No N+1 patterns; batch endpoints where item counts scale
• Keep‑alive configured for backend HTTP

Next: 7.2 Caching architecture