[community-group] Pull Request: Proposal: breaking changes to Aliases (JSON Schema $ref)s

drwpow has just submitted a new pull request for https://github.com/design-tokens/community-group:

== Proposal: breaking changes to Aliases (JSON Schema $ref)s ==
## Summary

This makes a breaking change to aliases, changing the syntax:

```diff
- { "$value": "{color.blue.01}" }
+ { "$value": { "$ref": "#/color/blue/01" } }
```

This provides a solution for #166 as well as makes the implementation for the upcoming Resolver Spec that addresses #210.

## Reasoning

DTCG users have kept their files separately for a long time, and have asked for the ability to reference tokens in other files (#166). Since we were already borrowing from [JSON Schema](https://json-schema.org/draft/2020-12/json-schema-core#name-schema-references) in places (primarily the `$` character to mark reserved keys), this brings JSON Schema’s `$ref` keyword to the concept of token aliasing.

The `$ref` keyword comes both from JSON Schema and the [OpenAPI specification](https://spec.openapis.org/oas/latest.html) (previously _Swagger_), and has been in use for over a decade.

For the DTCG format, it is a way to keep all the functionality of aliases while extending it for future needs, borrowing from prior art where this syntax is known, used, and well-defined. For simplicity, this proposal aims to **replace** the previous alias syntax (which will be covered below).

### Pros

- This greatly expands what’s possible with the DTCG format, now that any part of the document may be reused.
  - For example, aliased tokens could now even reuse parts of `$extensions` if they wanted to, while selectively applying overrides
  - Because it allows overrides, you can reuse more DTCG syntax than you could before, even sub-schemas in remote documents (e.g. extending public design systems)
- While this is flagged as a breaking _syntax_ change, it’s actually a fully-backwards-compatible upgrade to aliases. There is no known functionality lost from this change.
- Figma Styles and Variables use the `/` character in names, so now token names will map 1:1
  - _Are there other cases where this is true?_
- This is backwards-compatible for token names. Tokens can still use `/` and `#` characters in their names; they just have to be escaped with `~0` and `~1` respectively, according to the [spec](https://datatracker.ietf.org/doc/html/rfc6901#section-3):

   ```json
   {
     "my/token": { "$type": "number", "$value": 5 },
     "other-token": { "$ref": "#/my~1token" }
   }
   ```
- For DTCG parsers, this simplifies parsing/handling of aliases. Consider the old syntax:
   ```json
   {
     "font-1": { "$type": "fontFamily", "$value": "Inter" },
     "font-2": { "$type": "fontFamily", "$value": "{font-1}" }
   }
   ```

   A tool maker would have to do additional work figuring out if `$value` was an alias or not, because both present as strings. But with the `$ref` syntax, if it’s an object with a `$ref` key, it is only an alias, no string parsing necessary.
- Tool makers also benefit from over a decade of prior art and tooling for handling this syntax (even if they may not have used it directly before)

### Cons

- Given that aliases are core syntax, this is a disruptive change, not just for toolmakers but for consumers of the DTCG spec. We likely would want to talk about a deprecation strategy
- It does change the structure, converting a string node (`"{my.token.alias"}`) to an object (`{ "$ref": "#/my/token/alias" }`). Again, though long-term it will be easier to work with, short-term will impose migration pains.

### Alternatives

- Those familiar with JSON Schema will also be familiar with its counterpart [`$defs`](https://json-schema.org/draft/2020-12/json-schema-core#name-schema-re-use-with-defs), which allows you to declare reusable parts of a document that can be `$ref`’d anywhere (but by default are ignored/not parsed). $defs are NOT being proposed here, because it doesn’t solve an immediate problem, and introduces more complexity than necessary (but could be an additive followup)
- An alternate idea is to keep _aliases_ the same `{color.blue.05}`, and introduce this new concept as a _reference_ (and distinguish between the terms). Aliases would just be the “legacy” way to declare token aliases. I didn’t initially propose this because:
  - There were no advantages I could find. Even the automatic `$type` inheritance is possible with `$ref`, but `$ref` carries more benefits
  - I couldn’t think of a sensible usecase of distinguishing “aliases” from “references” after reading this note in the [JSON Schema spec (2019-09](https://json-schema.org/draft/2019-09/draft-handrews-json-schema-02#rfc.appendix.B.2):
  
    > Attempting to remove all references and produce a single schema document does not, in all cases, produce a schema with identical behavior to the original form.

    In other words, even the concept of `$ref` feels spiritally identical to the existing DTCG alias syntax—there are times when you _do_ want to preserve and reference those values later, and they are significant in some ways (even including overrides).
    
  However, I think we could define a some sort of deprecation strategy where the old syntax is supported for some time before switching over.

## Notes

- This does introduce some confusion in syntax. For example, is the “official” token ID now `color.blue.05` or `color/blue/05`?
  - Further, does this mean `.` is allowed in token names again?
- As someone that’s worked extensively with JSON Schema syntax, it’s often not enough to say “follow the spec” because there are, like, dozens of conflicting versions that all have breaking changes. So I’m proposing specifically the [2020-12](https://json-schema.org/draft/2020-12/json-schema-core) version, in the case that tool makers run into one of these conflicts
  - Specifically, `2020-12` DOES allow `$ref` to have sibling keys (“overrides”), which is **IMPORTANT**! Without this, I believe this would lose some functionality of aliases—specifically their ability to inherit `$type` automatically.
  - 99.9% of the time it’s not an issue; this is just coverage for that one weird edge case 😅 

### Resolver Spec problem

Though this proposal does NOT relate to the upcoming Resolver Spec proposal, I’ll share partial syntax that posed a problem:

```json
{
  "sets": [
    { "values": ["foundation.json"] },
    { "values": ["components/button.json"] }
  ]
}
```

Without going too much into detail, part of the resolver spec relied around declaring token “overrides” in individual files (modes/themes/what-have-you). However, the files would have had to be complete, full, valid DTCG format documents. Further, the references MUST be separate documents, otherwise the overrides wouldn’t work.

I believe this would lead to issues where the Resolver Spec would overreach in requiring what tokens went in what files. The DTCG format would swing drastically from “no opinion on files” to “extremely-detailed opinion about files and relative relationships.” At best, it would be a timesink shuffling tokens around between files and file boundaries. At worst, the Resolver Spec wouldn’t work if you declared your files in some magically-incorrect way (especially if you tried to reuse the same token files and apply them in different orders for different themes). And also, within the files, you have no way to reuse parts of other files.

In short, it was not only a user problem; it was an infrastructure problem having to worry about whether local or remote files would be supported, and for design system teams worrying about how to deploy and/or maintain these files in the perfect way.

Instead, `$ref` is a happy compromise because **it allows complete flexibility in file structure.** Because it allows referencing sub-documents anywhere—locally, remotely, even _partial_ documents—you could simply organize tokens in separate files how you wanted to first (which is how all DTCG consumers operate today—being in full control of their file separation), while still being able to opt in to the Resolver Spec. So **this provides a clear upgrade path with more flexibility,** without asking anything in return.

See https://github.com/design-tokens/community-group/pull/259


-- 
Sent via github-notify-ml as configured in https://github.com/w3c/github-notify-ml-config

Received on Saturday, 11 January 2025 00:01:39 UTC