Sitecore Content Hub Bulk Read SDK vs. GraphQL: Performance and Use Cases

Sooner or later every Sitecore Content Hub project gets a request like this: "Can you give us a spreadsheet of every usage right, with its status and expiry dates?" or "Find all assets whose rights expired more than six months ago." In other words, you have to read every entity of one definition. That can be tens of thousands of records, and you need them reliably, without duplicates or gaps.

I recently built exactly this for the M.DRM.RightsProfile definition, which holds usage rights. There are two very different APIs for it, and the SDK itself offers two ways to page:

The Content Hub Web SDK (Stylelabs.M.Sdk.WebClient), using either SearchAfterQuery or the Entity Scroller.
Sitecore Experience Edge GraphQL, which can return up to 1,000 records per call.

I put both behind a single switch in the same helper, so they write an identical CSV and I could time them against each other. This post covers how each approach works, the traps I hit, and when I'd pick one over the other.

Figure 1: One exporter, interchangeable sources, identical output.

The goal

Export every M.DRM.RightsProfile to CSV with these columns:

Id, Name, UsageRightsStatus
StartDate, ExpirationDate, CalculatedExpirationDate (all normalised to UTC ISO 8601)
IsActive: the effective expiry (ExpirationDate, falling back to CalculatedExpirationDate) is within the last 6 months, or there's no date at all

The export had three requirements:

Complete: every record is exported exactly once.
Resumable: if a page fails, it can be retried without restarting the whole export.
Memory-friendly: each page is written and flushed to disk before the next page is fetched.

Approach 1: Web SDK

Option A: SearchAfterQuery

Classic Skip/Take paging gets slow, and eventually runs into limits, as you go deeper into a large result set. SearchAfterQuery avoids that. Each page returns LastHitData, and you pass it back as SearchAfter to get the next page.

The important detail is the sort. Sort only by created_on and any two entities created in the same millisecond can be skipped or duplicated at a page boundary. Always add a unique tie-breaker, such as identifier:

var query = new SearchAfterQuery(
    QueryFilterBuilder.Definition.Name == "M.DRM.RightsProfile",
    new List<Sorting>
    {
        new() { Field = "created_on", Order = QuerySortOrder.Asc },
        new() { Field = "identifier", Order = QuerySortOrder.Asc }, // unique tie-breaker
    })
{
    Take = 100,
    LoadConfiguration = new EntityLoadConfiguration(
        new CultureLoadOption(new CultureInfo("en-US")),
        new PropertyLoadOption(
            "DRM.RightsProfile.Name",
            "UsageRightsStatus",
            "DRM.RightsProfile.StartDate",
            "DRM.RightsProfile.ExpirationDate",
            "DRM.RightsProfile.CalculatedExpirationDate"),
        RelationLoadOption.None),   // don't pay for relations you don't need
};

long fetched = 0;
while (true)
{
    var result = await client.Querying.SearchAfterAsync(query);
    if (result.Items == null || result.Items.Count == 0) break;

    // map + write this page to CSV, then flush ...
    fetched += result.Items.Count;

    // Don't stop just because a page is short: the server may cap Take.
    if (result.LastHitData == null || result.LastHitData.Count == 0
        || fetched >= result.TotalItems) break;

    query.SearchAfter = result.LastHitData;
}

Figure 2: Why the sort needs a unique tie-breaker at page boundaries.

Two more tips:

Load only the properties you need, and set RelationLoadOption.None. That cuts the payload and the server time noticeably.
Handle culture-sensitive and culture-insensitive properties. Check for ICultureSensitiveProperty and ICultureInsensitiveProperty when you read values, or a multi-language property will silently come back as null.

Option B: the Entity Scroller

The SDK also has a scroller, which is the older way to walk a large result set. You give it a normal Query and a scroll time, then call MoveNextAsync() until it returns false. You don't have to manage any sort or cursor yourself:

var query = Query.CreateQuery(entities =>
    from e in entities
    where e.DefinitionName == "M.DRM.RightsProfile"
    select e);
query.Take = 100;                                  // batch size per MoveNextAsync

var scroller = client.Querying.CreateEntityScroller(
    query,
    TimeSpan.FromSeconds(60),                      // how long the server keeps the scroll alive between calls
    loadConfig);                                   // same trimmed EntityLoadConfiguration as above

while (await scroller.MoveNextAsync())
{
    var items = scroller.Current.Items;
    // map + write this batch to CSV, then flush ...
}

If you only need IDs, for example to drive a bulk update, client.Querying.CreateEntityIdScroller(query, scrollTime) returns IDs alone. You can then load them in batches with client.Entities.GetManyAsync(ids, loadConfig).

The difference from SearchAfter is where the paging state lives. A scroller opens a scroll context on the server, which is a snapshot of the result set taken when the scroll starts. That has some useful consequences and some painful ones.

Figure 3: Where the paging state lives decides how failures behave.

	Entity Scroller	SearchAfterQuery
Paging state	Held on the server (scroll context)	Held by you (`LastHitData`); stateless
Consistency	Snapshot: entities created or changed mid-run don't move pages around	Live index: new entities can appear in later pages; a stable sort keeps it gap-free
Slow processing	If one batch takes longer than the scroll time, the context expires and the scroll fails	No timeout between pages; take as long as you need
Retry / resume	A failed batch can't be safely retried; usually you restart from the beginning	Retry the same page with the same `SearchAfter`; you can even save it and resume later
Sorting	Not needed	Required, and it must include a unique tie-breaker
Server cost	Each open scroll holds resources until it ends or expires; avoid many in parallel	Nothing held between calls
Code	Simplest loop of the three	A few more lines for the cursor and stop conditions

I went with SearchAfter, because I wanted page-level retries in a long-running export. The scroller is still a good fit for quick, one-off scripts, or when you need a frozen snapshot while other processes are changing the data. Just keep each batch's work short, or raise the scroll time.

Approach 2: Experience Edge GraphQL

If your Content Hub entities are published to Sitecore Experience Edge, you can query them with GraphQL at https://edge.sitecorecloud.io/api/graphql/v1, authenticated with an X-GQL-Token header. Each definition gets a generated all<Definition> query that supports first (up to 1,000), after, orderBy and where.

query Run($first: Int, $after: String) {
  allM_DRM_RightsProfile(
    first: $first,
    after: $after,
    orderBy: DRM_RIGHTSPROFILE_CALCULATEDEXPIRATIONDATE_DESC
  ) {
    total
    results {
      entityID
      dRM_RightsProfile_Name
      usageRightsStatus
      dRM_RightsProfile_StartDate
      dRM_RightsProfile_ExpirationDate
      dRM_RightsProfile_CalculatedExpirationDate
    }
    pageInfo { endCursor hasNext }
  }
}

The client side is a plain HttpClient loop that follows the cursor:

using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(100) };
http.DefaultRequestHeaders.Add("X-GQL-Token", gqlToken);   // keep this out of source control!

var after = string.Empty;
while (true)
{
    var body = new { query = GqlQuery, variables = new { first = 1000, after } };
    using var response = await http.PostAsJsonAsync(endpoint, body, ct);
    response.EnsureSuccessStatusCode();

    var payload = await response.Content.ReadFromJsonAsync<GqlResponse>(cancellationToken: ct);
    if (payload?.Errors is { Count: > 0 })            // GraphQL errors arrive with HTTP 200!
        throw new InvalidOperationException(string.Join(" | ", payload.Errors.Select(e => e.Message)));

    var page = payload!.Data!.Connection!;
    if (page.Results is not { Count: > 0 }) break;

    // map + write this page to CSV, then flush ...

    if (page.PageInfo is not { HasNext: true } || string.IsNullOrEmpty(page.PageInfo.EndCursor)) break;
    if (page.PageInfo.EndCursor == after) break;      // defensive: never loop forever
    after = page.PageInfo.EndCursor;
}

Is `orderBy` safe for paging?

I wanted the CSV sorted by expiry date, so I added orderBy: ..._CALCULATEDEXPIRATIONDATE_DESC. Thousands of records share the same "no expiry" date (9999-12-31), so I worried about rows going missing or repeating across pages.

To check, I decoded the cursor. It's just Base64 JSON:

{"searchAfter":[253402300799999,"<unique-id>"],"count":1000}

Edge applies the same search-after technique internally, and it adds a unique id as a tie-breaker to the cursor automatically, so paging stays consistent even with lots of duplicate sort values. The cursor is also stateless, which means a failed page can be retried with the same cursor.

orderBy takes a list, so you can combine sort keys, for example orderBy: [USAGERIGHTSSTATUS_ASC, DRM_RIGHTSPROFILE_CALCULATEDEXPIRATIONDATE_DESC]. You can look up the allowed values with an introspection query on the <Definition>Sorts enum.

One helper, one switch

Both sources are written as an IAsyncEnumerable that yields one page of rows at a time. The rest of the exporter, which handles CSV writing, counting and logging, doesn't know or care where the rows came from:

public enum UsageRightsSource { Sdk, GraphQl }

var pages = options.Source == UsageRightsSource.GraphQl
    ? FetchPagesViaGqlAsync(cutoff, ct)
    : FetchPagesViaSdkAsync(client, cutoff, ct);

await foreach (var (rows, total) in pages)
{
    foreach (var row in rows) { csv.WriteRecord(row); csv.NextRecord(); }
    writer.Flush();                       // at most one page in memory
}

Both paths call the same BuildRow(...) function to compute dates and IsActive, so the CSV contents are identical. Only the row order differs, because each source sorts differently.

Retries that don't lie about timing

Every call goes through a small retry wrapper that uses linear backoff. It fails fast on 401/403, because a bad token won't fix itself. It also records API time per attempt, excluding the backoff delays, so the comparison measures the API rather than my retry policy:

for (var attempt = 1; ; attempt++)
{
    var callSw = Stopwatch.StartNew();
    try
    {
        try   { return await action(); }
        finally { _requests++; _apiTime += callSw.Elapsed; }   // delays not counted
    }
    catch (Exception ex) when (attempt < MaxRetries
                               && ex is not OperationCanceledException
                               && !IsAuthFailure(ex))
    {
        await Task.Delay(RetryBaseDelayMs * attempt, ct);
    }
}

Results

Both exports read the same definition of roughly 36,000 records and wrote to the same CSV format.

Figure 4: Ten times fewer round trips with GraphQL.

Metric	Web SDK (100/page)	Edge GraphQL (1000/page)
API requests	~360	~36
Total time	[[ 5 to 6 mins depends on Server load ]]	[[ within 1 minute ]]
Rows exported	[[ 35807 ]]	[[ 35807 ]]

Pros and cons

	Web SDK (SearchAfter)	Experience Edge GraphQL
Page size	Small pages, so many more round trips	Up to 1,000 per call
Data freshness	Live authoring data	Published data only; can lag behind authoring (You can use preview endpoint too)
Coverage	Any definition, property or relation your user can see	Only definitions and fields that are published to Edge
Payload	Full entity objects; trim with load configurations	Exactly the fields you ask for
Write back	Yes: read, then update in the same client	No, read-only delivery API
Auth	OAuth client and user; respects user permissions	One delivery token; treat it like a secret
Load on Content Hub	Hits your instance directly	Served by Edge; your instance isn't touched
Developer experience	Strongly typed SDK; familiar property names	Generated field names (`dRM_RightsProfile_Name`); write your own DTOs
Error handling	Exceptions from the SDK	Errors can arrive with HTTP 200, so always check `errors`

When to use which

Figure 5: Picking an approach.

Reporting and read-only exports of published data: use GraphQL. You get far fewer calls and no load on your Content Hub instance.
Bulk updates (read, decide, then write), data that must be up to date to the second, or definitions that aren't on Edge: use the Web SDK.
Within the SDK: use SearchAfter for long or resumable jobs. Use the Entity Scroller (or the ID scroller) for quick scripts, or when you need a snapshot that won't shift while you read it.
Large updates where the selection is read-only: combine them. Use GraphQL to find the IDs quickly, then use the SDK to update only those entities.

Gotchas checklist

Always use a unique tie-breaker in SearchAfter sorts. Edge adds one for you; with the SDK you add it yourself.
With a scroller, the scroll time is per batch, not for the whole run. Keep work inside the loop short (write, flush, move on), or the context expires partway through.
Don't stop paging on a short page. Stop when the cursor is empty or you've fetched total records.
Guard against a repeated cursor, so a bug can never cause an infinite loop.
Normalise every date to UTC before comparing or writing it. ISO 8601 strings like yyyy-MM-ddTHH:mm:ssZ also sort correctly as text in Excel.
Compare the exported count with total at the end, and log a warning if they differ.
Keep tokens and secrets in environment variables, not in source code.
Look at the edges of your data. When I sorted ascending, the very first record had an expiry date in the year 0027, which was clearly a typo for 2027. A bulk export is also a free data-quality audit.

Wrapping up

Bulk reading in Content Hub is mostly about paging correctly: use search-after with a stable sort and stream each page to disk. Once that's in place, the choice of API is a trade-off. The Web SDK is complete and live, and it lets you write back. Experience Edge GraphQL is much faster for read-only work on published data.

Putting both behind one switch costs about a hundred lines. In return you can compare them on your own data, and you can move to the other API later without rewriting your exporter.

Have you hit other limits with Edge or the SDK on large definitions? Let me know in the comments.

Thanks for reading and keep learning!!

You can check my other blogs too if interested. Blog Website

Search Blog

Sitecore