Registry versioning for backward compatibility

Protocol & Infrastructure
1

TLDR:

  • Currently, breaking changes to the registry’s module schema, like removing an existing property, results in breaking changes for the registry consumers.
  • To prevent that, we need to introduce a versioning solution to the registry, so that consumers will be able to rely on their consumed version for not introducing such breaking changes.
  • The proposed solution for now is to maintain major versions, each in a separate GitHub branch.

Background

The Hats Modules Registry is a curated modules database. Tools, apps, and platforms in the Hats ecosystem — including the Modules SDK — use it as the source of truth for safe, high quality modules they can serve to users and other developers.

It is currently hosted in GitHub, on this repository, on the main branch. For each module, there’s a JSON file that contains the module’s metadata. It enables consumers to automatically create new instances of the module, interact with these instances and in addition contains useful information for their users. These files have a predefined structure, specified by the module schema.

The Problem

Currently, consumers of the registry always fetch its current state. The consequence is that changes to the schema may introduce breaking changes to consumers (e.g. removing certain properties from the registry).

Ideally, consumers will be able to have certainty about the anticipated structure of the modules schema.

Proposed Solution

Maintain major versions of the registry. For each new breaking change in the schema, there will be a new version, while keeping all previous versions still available for usage. We chose to use a separate GitHub branch for each version:

  • The default branch will be called submissions - will be the target branch for new PRs
  • Production branches, v-[version number] for each major version

Once a PR is merged to submissions, it will be automatically merged to the relevant production branch.

Considerations / Tradeoffs

The vision for the registry is to serve as a schelling point which connects between module builders and their users. Ideally, the registry will be hosted via decentralized infrastructure and governed by a broad community.

At this point, using GitHub as the registry’s host is optimizing for simplicity, allowing for the registry to evolve with less friction. In parallel, our goal is that the current architecture will enable the transition to a decentralized infrastructure and governance to be as smooth as possible for all users of the registry.

With these considerations in mind, we find GitHub hosting to be a good middle ground, essentially serving as one place for both the DB itself, the Devops to automate/manage certain operation on it and the governance/source control over it.

Please feel free to provide any feedback, ideas or general thoughts!

1 Like
2

This approach makes good sense to me as it gives more transparency and control to Modules Registry consumers. The only question I’d pose is: how can the versioning schema be designed to best promote the vision of the registry (e.g. promoting the ideal of hosting it via decentralized infra and governed by a broad community)? fwiw I don’t think this needs to be answered immediately given GitHub is still hosts the registry.

3

This feature will definitely need to happen at some point, but my only question is whether it needs to happen now since it could be a significant maitenance burden. Is the lack of versioning support a problem now or is it an anticipated problem?

4

Great question @tsully. @gershido.eth can probably speak to this in more detail than I can, but from my perspective it’s a problem now because we need to make a breaking change to the registry schema: adding tags and removing the deprecated key.

1 Like
5

Thanks @tsully, yes it’s currently a problem because of everything that @spengrah mentioned above and also the removal of the factory metadata from the registry.

1 Like
6

Thank you @moreReese! It’s a question that’s pretty hard to confidently answer right now as I think the process of figuring out the exact end goal and the road there is still ongoing (and maybe it’s always ongoing :slight_smile: ).

But, I think we have at least enough clarity right now to make good decisions, in the sense that they have a high chance of being aligned with the future. Generally, I think that we should keep evaluating our ongoing solutions in terms of how well can they be generalized, in the sense that they are not too tied/dependent on a single framework/platform/app. Specifically aspects like:

  1. The registry’s state representation (including the schema).
  2. How consumers interact with the registry.
  3. The registry’s governance process.

And so I can imagine for example switching from the current solution into one that’s hosted on a decentralized infra (onchain + ipfs), consumers fetch the registry from ipfs instead of Github and the registry is governed onchain by the DAO.

2 Likes
7

The v0 schema can be frozen and archived statically. The branching structure feels confusing as the current branch is moving and the old branches will never be updated.