What this page shows
A live view of files moving out of an old database into new storage, with the text read out of each file along the way.
- Click Add 10 files. Watch them move left to right across the pipeline below.
- Each file stops at every step: picked up, queued, claimed by a worker, read, then stored.
- Switch Vendor mode to offline. Text extraction goes down. Work pauses instead of failing, nothing is lost.
- Switch it back to working. Everything that piled up finishes on its own.
- Restart demo wipes everything and reloads the original 500 files.
Controls
Pipeline
Chip movement is paced so each step is visible. Real timings are shown on each stored file below, in Recently stored.
Files still sitting in the old database, waiting to be picked up by the migration.
Noticed by the migration, either by the live lane watching the database's change log (change data capture, CDC) for new rows, or by the backfill lane scanning the source table for rows it has not reached yet. Counted since this page loaded.
Ready to be worked on, but no worker has taken it yet.
A worker has taken this file and is about to send it out for text extraction.
Waiting for text extraction. These files have been sent to the text extraction service, and they stay here until it returns the words it found.
Safely saved in the new home: the file's bytes in object storage, and its details, including the extracted text, in the new database.
These files hit repeated errors and were set aside. This is not necessarily final: a later update to the same file can pick it back up and move it forward again. Hidden while empty, since a vendor mode of working or offline never counts a vendor failure toward this; it appears the moment a file actually lands here, for example one the vendor rejects as unreadable.
Freshness (service level agreement, SLA)
0s outstandingHow long the oldest unfinished live-lane file has been waiting. Green means on track, amber means it is approaching the target, red means it has missed it.
Recently stored
The most recently completed files, newest first. Each duration below is the file's real, measured end-to-end time, not the paced speed of the chips above.