Shiny.Data.Sync — Offline-First Record Sync, Built on Jobs & HTTP Transfers


For years Shiny has had two solid answers for “do work when the app isn’t in the foreground”:

  • Shiny Jobs runs periodic background tasks — WorkManager on Android, BGTaskScheduler on iOS, an in-process timer everywhere else.
  • Shiny.Net.Http moves files in the background — NSURLSession on iOS, a foreground service on Android, a connectivity-driven HttpClient loop elsewhere.

But there’s always been a gap in the middle: records. Not a 40 MB video, not a timer that fires every hour — the dozen Create / Update / Delete operations a user racks up offline that need to reach a REST API reliably, survive an app kill, and come back carrying the server’s changes.

Shiny.Data.Sync fills that gap. And the thing I want to call out in this post isn’t “look, a new library” — it’s that I went out of my way not to reinvent background execution. Data Sync rides the exact same OS playbook that Jobs and HTTP Transfers already proved out in production.

dotnet add package Shiny.Data.Sync

Three libraries, one background playbook

iOS and Android take cross-platform “background” promises away from you. My answer has always been to match what each OS actually allows instead of pretending one mechanism works everywhere. All three libraries land on the same per-platform tiers:

PlatformJobsHTTP TransfersData Sync
iOS / Mac CatalystBGTaskSchedulerBackground NSURLSessionBackground NSURLSession (upload + download tasks)
AndroidWorkManagerForeground service + HttpClientForeground service + HttpClient
Windows / Linux / macOSIn-process timerHttpClient + connectivity loopHttpClient + connectivity loop
Blazor WASMIn-process (tab alive)Service Worker Background SyncHttpClient + LocalStorage (tab alive)

If you’ve shipped a background download with Shiny.Net.Http, you already understand Data Sync’s runtime model — because it is the same model. Where transfers move files, sync moves records, and the two deliberately share a playbook because the OS guarantees are identical.


How Data Sync uses Jobs

You don’t wire up a background pull yourself. AddDataSync<TDelegate> registers a SyncJob with the Shiny Jobs scheduler for you — this is effectively what runs under the hood:

services.AddJob<SyncJob>(r => r.WithInternet(InternetAccess.Any));

So periodic inbox pulls keep happening on whatever background cadence the OS allows — WorkManager on Android, BGTaskScheduler on iOS — through the same IJobManager you’d use for any other job. The job respects each endpoint’s MinPullInterval so it never hammers your server, and because it’s just a normal job, you cancel it through the normal job API when your pulls are push-triggered instead:

var jobs = host.Services.GetRequiredService<IJobManager>();
await jobs.Cancel(nameof(Shiny.Data.Sync.SyncJob));

That’s the whole point of building on Jobs instead of beside it. The scheduler, the runtime criteria (WithInternet, charging, battery level), the platform background hooks — all already solved. Data Sync registers a job and inherits the lot.


How Data Sync mirrors HTTP Transfers

The architectural heart of Shiny.Net.Http is a persistent queue drained by a platform-tiered transport. A transfer is written to disk before any network call, so a process kill mid-transfer leaves the work intact and the next launch (or the OS itself, on iOS) resumes it.

Data Sync uses the identical pattern for its outbox:

public class TodosService(IDataSyncManager sync)
{
    public Task Create(TodoItem item) => sync.Queue(SyncVerb.Create, item);
    public Task Update(TodoItem item) => sync.Queue(SyncVerb.Update, item);
    public Task Delete(TodoItem item) => sync.Queue(SyncVerb.Delete, item);
}

Queue<T> writes a durable SyncOperation to the Shiny repository before touching the network, then returns immediately — the caller never blocks on the round-trip. From there it’s pure HTTP Transfers thinking:

  • On iOS / Mac Catalyst, queued ops drive NSURLSession upload tasks. Even if Shiny’s in-process queue dies, the OS keeps its own queue, drives the upload to completion, and wakes the app to dispatch the result — exactly how background file uploads survive suspension.
  • On Android, ops drain inside a foreground service that spawns on Queue<T> and dies when the queue empties — the same foreground-service contract transfers use to stay alive while work is pending.
  • On Windows / Linux / macOS / Blazor, an in-process HttpClient loop drains the queue, woken by IConnectivity.Changed, app startup, and Queue<T> itself — the same connectivity loop that drives transfers off-Apple.

Attempts and NextAttemptAt are persisted next to each op, so even the exponential-backoff window survives a restart. None of that is new machinery — it’s the transfers playbook applied to records.

Where it goes beyond a file transfer is the second direction: an inbox that pulls server deltas keyed by an opaque cursor, draining pages until the server says hasMore: false. A file transfer is one-way; a record sync is two-way, so Data Sync layers the inbox, tombstone streams, conflict resolution, and an operation coalescer on top of the shared foundation.


Setup, end to end

// 1. Entity — one property
public record TodoItem(string Identifier, string Title, bool Completed) : ISyncEntity;

// 2. AOT-safe JSON, once per app
[ShinyJsonContext]
[JsonSerializable(typeof(TodoItem))]
public partial class AppJsonContext : JsonSerializerContext;

// 3. Register — picks the transport for the TFM AND auto-registers SyncJob
builder.Services.AddDataSync<MyDataSyncDelegate>(opts =>
{
    opts.RegisterEndpoint<TodoItem>("https://api.example.com/todos", ep =>
    {
        ep.Direction = SyncDirection.Both;            // PullOnly / PushOnly also valid
        ep.Batch = true;                              // coalesce redundant ops per round-trip
        ep.MinPullInterval = TimeSpan.FromMinutes(5); // throttle the scheduled SyncJob
        ep.MaxAttempts = 8;
        ep.DefaultConflictPolicy = ConflictPolicy.ServerWins;
    });
});

Your one IDataSyncDelegate is the integration seam — OnSent, OnError, OnReceived, OnConflict. Received items arrive already deserialized and strongly typed; you apply them to whatever local store you like. Data Sync is a transport, not a database — I pair it with DocumentDb inside OnReceived when I want local query, but that’s your call.


So which one do I reach for?

This is the question the three libraries answer together:

You need to…UseWhy
Run a periodic background task (cleanup, refresh, telemetry flush)JobsA scheduler with runtime criteria. No queue, no HTTP shape.
Move a large file up or down, resumable, in the backgroundHTTP TransfersRange-aware resume for multi-megabyte blobs. One-way.
Reliably push record CRUD and pull deltas, offline-firstData SyncPersistent outbox + cursor inbox, drain-on-reconnect, conflict handling.

They compose rather than compete. A real offline-first app often uses all three: Jobs for the periodic housekeeping, HTTP Transfers for the user’s photo attachments, and Data Sync for the records those photos belong to — every one of them riding the same NSURLSession / foreground-service / connectivity-loop tiering underneath.

And I kept the boundary explicit. Large blobs go to HTTP Transfers — sync moves JSON records, transfers move multi-megabyte files with range-aware resume. Realtime streams go to SignalR or Push. A client-of-record backup is a file push, not a sync. Moving records — Create / Update / Delete queued on failure, drained on reconnect, pulled back as deltas — is the one lane Data Sync is built for, and it doesn’t try to be more than that.


Get started

dotnet add package Shiny.Data.Sync

If you already know how Shiny runs work in the background, you already know how Data Sync runs. It just moves records. Pull it down, point it at your API, and tell me what breaks.


comments powered by Disqus