Skip to main content
The Token Migrator is a tool, deployed with the Helm chart, that migrates tokens from Authlete 2.3 to 3.0 in a self-managed environment. To set it up, see Migration Phase (Optional) in the Kubernetes Installation Guide. The Token Migrator starts migrating automatically when its container starts. This happens whether or not it is the first start, and whether or not the previous stop was intentional. The Token Migrator saves its progress on the persistent volume that the Helm chart mounts. If saved progress exists on that volume, the Token Migrator resumes from the last saved point. To resume where it left off, the Token Migrator must start with the same volume. It may reprocess some tokens, but it never skips a token.

​
Token ID mapping files (mapping-output)

The mapping-output directory holds one file per client. The Token Migrator generates these files as part of migration processing. Each file maps each source token ID to its destination token ID. The Token Migrator uses these files to find the destination token to update when a source token changes. Each file is named <service ID>:<client ID>.json. If the mapping files are lost, the Token Migrator regenerates them, but some tokens may then be created as new tokens in the destination instead of updating the existing mapped tokens. The previously mapped tokens (stale tokens) remain in the destination, so a token that was refreshed or revoked in the source may still be valid there. In this situation, we recommend completely restarting the whole token migration:
  1. Stop the Token Migrator.
  2. Remove the migrated services and clients from the destination (keep the service and client that the Token Migrator uses).
  3. Delete all files on the Token Migrator’s persistent volume.
  4. Migrate the services and clients again.
  5. Start the Token Migrator.

​
Recovery point

​
How the last-processed timestamp is saved

During migration, the Token Migrator tracks a last-processed timestamp, which records the time up to which it has processed all token changes (log messages call it the moving timestamp). After it has processed all token changes up to the current last-processed timestamp, it writes this value to timestamp.txt as a Unix epoch time in milliseconds (for example 1784401524667). This file is the recovery point in case the Token Migrator stops. On the next start, the Token Migrator continues from the time in timestamp.txt. Because the Token Migrator processes all token changes before it writes timestamp.txt, restarting from that point may reprocess tokens that were already created or updated, but it does not miss any token changes.

​
If timestamp.txt is lost

If timestamp.txt is lost, the Token Migrator cannot read it on the next start and logs Setting moving timestamp to the provided initial timestamp [<value>]ms instead of Read in moving timestamp initial value (see Last-processed timestamp confirmation). To recover, follow these steps:
  1. Stop the Token Migrator (for example, set tokenmigrator.enabled to false and run helm upgrade).
  2. Find the last processed timestamp in the shutdown log: Shutting down sync migration task, latest timestamp [<value>]ms (see Shutdown logs).
  3. Create <mount path>/timestamp.txt containing only that value, in milliseconds.
  4. Start the Token Migrator again.
If the container was killed and no shutdown log was written, use a timestamp that you are sure is no later than the lost last-processed timestamp, such as the value in the last Read in moving timestamp initial value log. Starting from such a timestamp only causes some tokens to be reprocessed.